Emphasize development and debug utility of ipy command

[tleonhardt]

The utility of the optional ipy command combined with self.self_in_py = True for developers is truly phenomenal. We don't do these amazing features justice in our documentation and I'm not sure how discoverable they are for new cmd2 developers. These features open the door to an enhanced development experience that allows easy experimenting as well as the ability to do some after-the-fact debugging when things don't behave like you think they should.

We should add a section on "Developer Tips" or something like that to our documentation.

[KyleKing]

I found some documentation on ipy in embedded_python_shells.html#ipython-optional. I think I'm missing something obvious, but how do you access self from within the ipy shell?

❯ poetry run python main.py
(Cmd) ipy
Entering an embedded IPython shell. Type quit or <Ctrl>-d to exit.
Run Python code from external files with: run filename.py

In [1]: self
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
~/miniconda3/envs/py385/lib/python3.8/site-packages/cmd2/cmd2.py in <module>
----> 1 self

[kmvanbrunt]

Set self.self_in_py = True. We default this to False so normal users can't mess with self in pyscripts or IPython shells.
Typically in apps that I write, I include a developer mode where I set that flag to True. That way I can easily make changes to the application state within ipy for debugging purposes.

[KyleKing]

Oh 🤦 , I missed the information in your first comment and at the top of the page for embedded_python_shells

Maybe an example under the ipython-optional header would help?

class PyApp(Cmd):

    def __init__(self):
        """Initialize cmd2 application with ipy enabled."""
        super().__init__(use_ipython=True)
        self.self_in_py = True  # Note: when enabled, user can access `self` in ipy shell, which is not recommended for production

[tleonhardt]

@kmvanbrunt What other changes do you typically make in your developer mode?

[kmvanbrunt]

In developer mode, I typically do the following:

• self.debug = True
• self.self_in_py = True

If not in developer mode:

• self.self_in_py = False
• Add ipy and py to self.hidden_commands

[tleonhardt]

@kmvanbrunt Do you think that is a generic enough functionality that we should add it into cmd2, perhaps as an opt-in feature supported by an optional flag provided to the initializer?

[jayrod]

@kmvanbrunt Please tell me more how you enable 'developer mode'. Is this something that you toggle with a general command after release or do you hide it somewhere and only use it when needed. So normal users could be developers if they wanted.

This sounds like a good idea for my cmd2 templates.

[kmvanbrunt]

@jayrod I just make developer mode a command line option when opening the app. I'm not worried if my users are aware of it, but other applications may want to hide it depending on the intended users.

[jayrod]

I've been using cmd2 for a long time and it is a great framework to begin any CLI application, the examples are great for familiarization but more complicated things like "developer mode" are a bit more difficult to flesh out. I think adding a developer mode to my cookiecutter template would mean all would get this functionality by default.

I just want to make sure I do it the right way ;)

[jayrod]

Maybe we can flesh out what "developer mode" really means. I know why self_in_py is used for but why is that a bad thing for normal user mode? I also saw that there was a nice auto-doc script in the example folder to help spit out help documentation for all commands but it required self_in_py mode. Perhaps developer mode could auto set everything and offer this sort of command.

Are there other developer only commands settings?

[tleonhardt]

I like the idea of adding a "developer" mode option to the cookie cutter template and possibly adding some kind of support for it to cmd2 itself.

At a minimum, during development of cmd2 applications I always do the following:

• Set use_ipython=True when calling cmd2.Cmd.__init__() to enable ipy command for embedded IPython shell
• Set self.self_in_py = True within __init__()
• Set self.debug = True within __init__()

The first two allow me to do complete realtime troubleshooting of application state within an IPython shell embedded in the application. The last allows me to see a full traceback if/when there is an unhandled exception. Often I have altered at least the self_in_py and self.debug setting before shipping the applications to customers, but that depends on the technical sophistication of the intended user audience. I also frequently hide commands which might confuse or distract end users from the main point of the application, with py being the most commonly hidden.

I could see adding an optional argument to __init__() called something like developer_mode which is set to None by default and if it's that, current default behavior applies. Then if set to True or False we could take different actions.

I could also see taking it to the next level and accepting a --dev command-line flag to set this behavior instead.

@kmvanbrunt Any thoughts?

[jayrod]

@kmvanbrunt could you perhaps share a snippet of how you enable dev mode from the command line..

create a do_dev_mode function that "toggles" settings on and off?

[tleonhardt]

I'm pretty sure @kmvanbrunt uses argparse to parse various command line arguments when his cmd2-based application launches and then based on them passes various arguments to the __init__ method of the custom class derived from cmd2.Cmd.

We used to have a good example showing how to do this and I'm not sure what happened to it. The decorator_example.py example still shows the basics of how to do it - just not in a way that is useful, but the important mechanics are there.

So I think in his app, he would launch his CLI like so to launch it in dev mode:

% ./custom_cmd2_app.py --dev

[tleonhardt]

I like the concept of putting this in the basic oookiecutter template: jayrod/cookiecutter-python-cmd2#10

We can probably also update our documentation a little to emphasize just how darn useful ipy is for debugging.

[jayrod]

I've got some time and now some experience but I admit I have not used ipy or even debug during my development of cmd2 apps. I played around with self and tried to perform a command in ipython but building a Statement object is a bit weird.

So my question is what use cases are really useful for ipython usage?

when I run into issues I simply set a debug point inside of a command and create a command to duplicate the issue.

ex.

app.py "command1" "command2 -with args" "quit"

[kmvanbrunt]

In both py and ipy shells, you can do everything that's possible in a pyscript. Therefore, you don't need to call self.do_foo(). Just use app('foo arg1 arg2').

These shells offer a lot for debugging. I commonly do the following operations:

1. Check application state via self object.
2. Alter application state via self object to simulate various conditions.
3. Run commands via app command to see what's returned in the CommandResult object.

I also use run_pyscript for debugging. Recently I wrote a pyscript which rewrites a do_* command function while the application is running. That way I don't have to keep closing and reopening the application to test my code changes.