How To Make Changes And Rerun Python

jurigged

Jurigged lets y'all update your code while information technology runs. Using information technology is trivial:

1. jurigged your_script.py
2. Alter some function or method with your favorite editor and relieve the file
3. Jurigged will hot patch the new function into the running script

Jurigged updates live code smartly: irresolute a function or method will fudge lawmaking pointers then that all existing instances are simultaneously modified to implement the new beliefs. When modifying a module, only changed lines will be re-run.

Y'all can also optionally install the develoop, a concluding-based alive development environment:

Equally seen above, jurigged --loop <function_name> script.py will "loop" on a particular role of the script. That funtion will exist re-run every time the source lawmaking is modified, with changes hot-patched into the running process. The residuum of the program is non re-run, so preprocessing is preserved and heavy modules do not have to be reloaded!

Install

Jurigged requires Python version >= iii.eight.

To also install the develoop characteristic, which lets yous interactively develop functions:

pip install jurigged[develoop]

Command line

The simplest manner to use jurigged is to add -chiliad jurigged to your script invocation, or to use jurigged instead of python. You can use -five to go feedback virtually what files are watched and what happens when you lot modify a file.

python -m jurigged -v script.py  OR  jurigged -five script.py

With no arguments given, it volition get-go a live REPL:

python -yard jurigged  OR  jurigged

Full help:

              usage: jurigged [-h] [--interactive] [--spotter PATH] [--debounce DEBOUNCE] [--poll POLL] [-m MODULE] [--dev] [--verbose] [--version]                 [SCRIPT] ...  Run a Python script then that information technology is alive-editable.  positional arguments:   SCRIPT                Path to the script to run   ...                   Script arguments  optional arguments:   -h, --aid            show this help bulletin and get out   --interactive, -i     Run an interactive session afterwards the program ends   --watch PATH, -w PATH                         Wildcard path/directory for which files to watch   --debounce DEBOUNCE, -d DEBOUNCE                         Interval to await for to refresh a modified file, in seconds   --poll POLL           Poll for changes using the given interval   -m MODULE             Module or module:part to run   --dev                 Inject jurigged.loop.__ in builtins   --verbose, -v         Show watched files and changes as they happen   --version             Print version                          

Develoop

Usage:

                              #                Loop over a function              jurigged --loop function_name script.py jurigged --loop module_name:function_name script.py                              #                Simply stop on exceptions              jurigged --xloop function_name script.py

The "develoop" is an optional characteristic of Jurigged that provides a sort of alive development environs for a function. If you run jurigged --loop <function_name> <script>, the function of that proper noun in the script will exist role of the "develoop". When information technology is entered, it will be run, its output volition be captured and displayed, and the program volition wait for input. If the source code is changed, the function will run again.

The --xloop or -x flag works the same, but the loop is only done if the function raises an exception. If it does not raise an exception, information technology volition run similar normal. Both --loop and --xloop can exist used multiple times, if you want to loop over multiple functions.

The default interface allows a few commands:

• r to manually re-run the loop. This can exist done in the eye of a run.
• a to abort the current run (e.g. if you lot get stuck in an infinite loop).
• c to leave the loop and continue the programme ordinarily.
• q to quit the program birthday.

Using with stdin

The default develoop interface does not play well with stdin. If you want to read from stdin or set a breakpoint(), apply the decorator @__.loop(interface="basic"). The interface will be cruder, but stdin/pdb will work.

Troubleshooting

First, if at that place'due south a trouble, use the verbose flag (jurigged -v) to get more data. It will output a Picket <file> argument for every file that it watches and Update/Add/Delete <part> statements when you update, add or delete a function in the original file and so relieve it.

The file is not being watched.

Past default, scripts are watched in the current working directory. Endeavor jurigged -westward <file> to sentinel a specific file, or jurigged -w / to lookout all files.

The file is watched, but nothing happens when I modify the function.

You tin can try using the --poll <INTERVAL> flag to utilise polling instead of the OS'due south native mechanisms. If that doesn't work, endeavour and see if it works with a different editor: it might have to do with the way the editor saves. For example, some editors such equally vi save into a temporary swap file and moves it into place, which used to crusade issues (this should be fixed starting with v0.3.five).

Jurigged said information technology updated the part but it's still running the old code.

If you are editing the torso of a for loop within a function that'southward currently running, the changes will only be in effect the next time that function is chosen. A workaround is to extract the body of the for loop into its ain helper function, which you can then edit. Alternatively, you tin can use reloading alongside Jurigged.

Similarly, updating a generator or async function will not change the behavior of generators or async functions that are already running.

I can update some functions but non others.

There may be issues updating some functions when they are busy or stashed in some data construction that Jurigged does not understand. Jurigged does have to detect them to update them, unfortunately.

API

You can call jurigged.sentry() to programmatically first watching for changes. This should also work within IPython or Jupyter as an alternative to the %autoreload magic.

              import              jurigged              jurigged.watch()

By default all files in the current directory will be watched, but you tin use jurigged.picket("script.py") to only watch a single file, or jurigged.scout("/") to sentinel all modules.

Recoders

Functions tin can be programmatically changed using a Recoder. Brand one with jurigged.make_recoder. This can be used to implement hot patching or mocking. The changes can also be written back to the filesystem.

              from              jurigged              import              make_recoder              def              f(10):              render              x              *              ten              assert              f(ii)              ==              iv              # Change the behavior of the role, simply not in the original file              recoder              =              make_recoder(f)              recoder.patch("def f(x): return x * x * x")              assert              f(2)              ==              8              # Revert changes              recoder.revert()              affirm              f(2)              ==              4              # OR: write the patch to the original file itself              recoder.commit()

revert will only revert up to the final commit, or to the original contents if in that location was no commit.

A recoder also allows yous to add imports, helper functions and the like to a patch, but you have to use recoder.patch_module(...) in that case.

Caveats

Jurigged works in a surprisingly large number of situations, just there are several cases where it won't work, or where problems may ascend:

• Functions that are already running will keep running with the existing code. Only the next invocations will use the new code.
  • When debugging with a breakpoint, functions currently on the stack can't be changed.
  • A running generator or async part won't modify.
  • You tin can use reloading in improver to Jurigged if you desire to be able to modify a running for loop.
• Changing initializers or attribute names may cause errors on existing instances.
  • Jurigged modifies all existing instances of a class, merely it will not re-run __init__ or rename attributes on existing instances, and so you tin can easily end up with cleaved objects (new methods, merely sometime information).
• Updating the code of a decorator or a closure may or may not piece of work. Jurigged will do its best, just it is possible that some closures volition exist updated simply not others.
• Decorators that look at/tweak function code will probably not update properly.
  • Wrappers that endeavour to compile/JIT Python code won't know about jurigged and won't exist able to redo their work for the new code.
  • They can be made to work using a __conform__ method (see below).

Customizing beliefs

In order to update a transform of a Python function, for example a transform that generates a new code object based on the original source lawmaking, you need to do something like this:

              form              Custom:              __slots__              =              ("lawmaking",)              def              __init__(self,              transformed_fn,              code):              self.code              =              code              cocky.transformed_fn              =              transformed_fn              def              __conform__(self,              new_code):              if              new_code              is              None:              # Function is being deleted              ...              if              isinstance(new_code,              types.FunctionType):              new_code              =              new_code.__code__              do_something(new_code)              self.code              =              new_code              ...              transformed_fn.somefield              =              Custom(transformed_fn,              orig_fn.__code__)

Basically, when the original code is changed, jurigged will apply the gc module to find objects that betoken to it, and if they have a __conform__ method it will be chosen with the new code so that the transformed function can be updated. The original code must be in a slot on that object (it is of import that information technology is in __slots__, otherwise the referrer is a dictionary). Multiple transformed functions may exist.

How it works

In a nutshell, jurigged works every bit follows:

1. Inventory existing modules and functions: a. Insert an import hook that collects and watches source files. b. Look at all existing functions using gc.get_objects(). c. Add an audit claw that watches calls to exec in guild to inventory any new functions.
2. Parse source files into sets of definitions.
3. When a file is modified, re-parse it into a set of definitions and match them against the original, yielding a ready of changes, additions and deletions.
4. When a office's code has inverse: a. Strip out the decorators b. Execute the new code c. Use gc.get_referrers() to find all functions that use the sometime code d. Supersede their internal __code__ pointers
5. If the replacement fails or if brand new code is added, execute the new code in the module namespace.

Comparison

The two nigh comparable implementations of Jurigged'southward feature set that I could find (but it tin exist a bit difficult to find everything comparable) are %autoreload in IPython and limeade. Here are the cardinal differences:

• They both re-execute the entire module when its code is changed. Jurigged, by dissimilarity, surgically extracts changed functions from the parse tree and only replaces these. It simply executes new or changed statements in a module.

  Which is better is somewhat situation-dependent: on i hand, re-executing the module will pick up more changes. On the other mitt, it volition reinitialize module variables and state, and then sure things might break. Jurigged's approach is more than bourgeois and will only pick up on modified functions, but it will not touch on anything else, so I believe it is less likely to have unintended side furnishings. Information technology also tells you what it is doing :)

• They volition re-execute decorators, whereas Jurigged will instead dig into them and update the functions information technology finds inside.

  Over again, there'southward no objectively superior arroyo. %autoreload will properly re-execute changed decorators, but these decorators will return new objects, so if a module imports an already decorated function, it won't update to the new version. If y'all only modify the function's code and non the decorators, however, Jurigged will unremarkably exist able to change information technology inside the decorator, so all the old instances volition use the new behavior.

• They rely on synchronization points, whereas Jurigged tin can be run in its own thread.

  This is a double-edged sword, because even though Jurigged can add alive updates to existing scripts with zilch lines of additional lawmaking, information technology is not thread safe at all (code could be executed in the center of an update, which is perchance an inconsistent land).

Other similar efforts:

• reloading can wrap an iterator to make modifiable for loops. Jurigged cannot practice that, but you can use both packages at the aforementioned time.

Source: https://github.com/breuleux/jurigged

Posted by: grimexpearre.blogspot.com

Share this post