Daemon mode to speed up startup?

[gvanrossum]

One idea that I've heard a few times is a "daemon" mode where there's a server that can execute general Python requests, and a tiny client that just passes arguments to the server. There are lots of problems with this (e.g. dribbling stdout/stderr, tty access, etc.) and it's probably better done as a 3rd party package that certain apps can use. Does anyone know of a general solution like this? (Mypy has a daemon mode that works like this, but the client is still a Python script, so it wouldn't help startup time.)

[markshannon]

For this to work the daemon would need to have a pool of VMs that have already started and are ready to execute arbitrary Python code. This implies that a significant amount of time is spent readying the VM before it can start executing. If that is the case, then we should implement some sort of snapshot of an initialized VM #32 (comment)

The difference between the daemon and the snapshot at startup would most likely be insignificant.
If Python is executed frequently, the snapshotted executable will be in the O/S disk cache and loading it will be fast.
If Python is executed infrequently, then the in-memory VM held by the daemon is likely to have been paged out and will be as slow of the snapshotted executable.

[gvanrossum]

The daemon could presumably pre-fork a bunch of processes after importing a bunch of popular modules, a la Instagram (at least on UNIX). Then no snapshot feature would be required. (Not sure what to do for Windows.)

[markshannon]

My point was that he daemon process offers no performance advantages over the snapshot approach, but is less portable.

I suspect which is simpler to implement depends on your skill set. Personally, I would find the snapshot approach easier to implement.

[gvanrossum]

Maybe snapshots can be implemented using the approach from the Alibaba Compiler team (#86).

[anthonyGosme]

On the question in the opening post, five years on: I wrote one, speedrunner, along the pre-fork route suggested above, though with one daemon per CLI tool rather than a pool for arbitrary code. A daemon imports the tool, then forks for each call.

On the two problems named above:

stdout/stderr: the client is a small Rust binary that passes argv, cwd, env, umask and its own fds 0, 1 and 2 over a Unix socket. Output is never relayed, and isatty() answers as it would natively.
tty access is the real gap: the fork has no controlling terminal. The tool’s own Python gets /dev/tty through a passed fd, but a subprocess that opens /dev/tty (ssh, sudo) fails.

Over 96 commands the median gain is x4.8, and 69 of them only print a version or help. No Windows. Code, measurements and known limits: