Add support for `pex3 run`.

[jsirois]

Pex has long supported (since 2010!) an immediate hermetic run mode for tools like so:

:; pex cowsay -ccowsay -- -tMoo!
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

This is akin to pipx run:

:; pipx run cowsay -tMoo!
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

And uvx:

:; uvx cowsay -tMoo!
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||

Pex has two disadvantages in use compared to pipx run and uvx:

1. You must name the entry point, even when it matches the name of the project supplying the entry point - which is common.
2. The PEX environment created to run the tool is ephemeral - it uses a temp directory that is destroyed; so subsequent runs are not as fast as you'd hope since they must re-create the PEX and re-bootstrap.

A new pex3 run command can fix both of these issues for more convenient ad-hoc tool use. In addition, pex3 run can support the proposal in https://discuss.python.org/t/pre-pep-add-ability-to-install-a-package-with-reproducible-dependencies by @CarliJoy since it aligns so well with Pex goals of hermeticity and reproducibility. Towards that end, I've published version 0.32.0 of dev-cmd which embeds a pylock.toml in its sdist at the top level and a dev_cmd-0.32.0.dist-info/pylock/pylock.toml as per the spec proposal. I'd like to have pex3 run dev-cmd --locked ... look for and use this metadata to create a tool environment known-matching with the environment dev-cmd was tested against when it was released at 0.32.0. There's alot of kerfuffle in the PyPA discourse thread(s) but it all makes perfect sense to me. Applications (wheel console script entry points are these!) are better served with lock files. Full stop!

[jsirois]

I'm still not sure about requiring --locked. The idea is to only take the perf hit of 1st downloading the project with --no-deps, checking for an embedded pylock.toml and then using that to resolve when --locked is requested since dists containing locks is so rare right now (dev-cmd may be ~the only one!). Perhaps I'll default to --locked and provide an --unlocked to skip looking for locks. Maybe --locked {ignore,require,auto} with a default of auto is the way to go. In this scenario auto would use a lock if found and otherwise proceed without. The ignore option would never use locks and the require option would require a lock or error out noting the project has no embedded lock.

[jsirois]

Early results have pex3 run still lagging, but not too bad and as expected:

:; hyperfine -w2 'pex cowsay -ccowsay -- -tMoo!' 'pipx run cowsay -tMoo!' 'uvx cowsay -tMoo!' 'python -mpex.cli run cowsay -- -tMoo!'
Benchmark 1: pex cowsay -ccowsay -- -tMoo!
  Time (mean ± σ):      1.107 s ±  0.008 s    [User: 0.937 s, System: 0.100 s]
  Range (min … max):    1.097 s …  1.125 s    10 runs
 
Benchmark 2: pipx run cowsay -tMoo!
  Time (mean ± σ):     130.1 ms ±   1.0 ms    [User: 113.2 ms, System: 16.7 ms]
  Range (min … max):   128.8 ms … 132.3 ms    22 runs
 
Benchmark 3: uvx cowsay -tMoo!
  Time (mean ± σ):      26.0 ms ±   0.5 ms    [User: 17.4 ms, System: 8.9 ms]
  Range (min … max):    24.7 ms …  27.9 ms    113 runs
 
Benchmark 4: python -mpex.cli run cowsay -- -tMoo!
  Time (mean ± σ):     287.5 ms ±   4.3 ms    [User: 251.5 ms, System: 37.0 ms]
  Range (min … max):   282.2 ms … 297.3 ms    10 runs
 
Summary
  uvx cowsay -tMoo! ran
    5.01 ± 0.11 times faster than pipx run cowsay -tMoo!
   11.07 ± 0.28 times faster than python -mpex.cli run cowsay -- -tMoo!
   42.62 ± 0.94 times faster than pex cowsay -ccowsay -- -tMoo!

So of the Pex 288ms, most is from import time (~210ms):

:; PYTHONPROFILEIMPORTTIME=1 python -mpex.cli -V 2>&1 | tail
import time:      1595 |       1595 |       pex.cli.commands.pip.core
import time:       145 |       1828 |     pex.cli.commands.pip.download
import time:       122 |        122 |     pex.cli.commands.pip.wheel
import time:       717 |        717 |     pex.cli.commands.run
import time:       886 |        886 |         pex.venv.installer_configuration
import time:        90 |        975 |       pex.venv.installer_options
import time:       244 |       1218 |     pex.cli.commands.venv
import time:       146 |     204230 |   pex.cli.commands
import time:        95 |     209818 | pex.cli.pex
2.47.0

There is not much to be done about that until the https://github.com/pex-tool/pexcz project expands from a PEX runtime replacement to a full Pex build-time replacement as well.

[jsirois]

Alright, @CarliJoy with Pex 2.48.0 released you can now install at least 1 known tool on PyPI using its embedded lock (Run with some extra debug output via -v to show what's going on a bit):

:; pex3 run -v dev-cmd -- -h
...
pex: Using lock in dev_cmd-0.32.0-py3-none-any.whl at dev_cmd-0.32.0.dist-info/pylock/pylock.toml.
pex: Resolving urls to fetch for 1 requirements from lock /tmp/pex3-run.id3b99_q.dev-cmd-locked/dev_cmd-0.32.0.dist-info/pylock/pylock.toml created by uv: 1.4ms
pex: Downloading ansicolors from https://files.pythonhosted.org/packages/53/18/a56e2fe47b259bb52201093a3a9d4a32014f9d85071ad07e9d60600890ca/ansicolors-1.1.8-py2.py3-none-any.whl: 586.7ms
pex: Downloading filelock from https://files.pythonhosted.org/packages/4d/36/2a115987e2d8c300a974597416d9de88f2444426de9571f4b59b2cca3acc/filelock-3.18.0-py3-none-any.whl: 596.8ms
pex: Downloading typing-extensions from https://files.pythonhosted.org/packages/df/c5/e7a0b0f5ed69f94c8ab7379c599e6036886bffcde609969a5325f47f1332/typing_extensions-4.13.1-py3-none-any.whl: 597.7ms
pex: Downloading packaging from https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl: 626.5ms
pex: Downloading aioconsole from https://files.pythonhosted.org/packages/fa/ea/23e756ec1fea0c685149304dda954b3b3932d6d06afbf42a66a2e6dc2184/aioconsole-0.8.1-py3-none-any.whl: 629.8ms
pex: Downloading pex from https://files.pythonhosted.org/packages/af/da/1d91d20d3e56a0f65d56106e2b56ae3e5a863e43a3f32ffbda1c3c7fa698/pex-2.33.7-py2.py3-none-any.whl: 923.0ms
pex: Downloading 7 distributions to satisfy 1 requirements: 935.3ms
...
pex:   Installing 7 distributions: 166.4ms
pex:     Using 2 parallel jobs to install 7 wheels: 166.1ms
...
pex: Running: /home/jsirois/.cache/pex/venvs/1/dba00c5aef6218f9181a632a5bed43cd749242de/5ab92962fe5cd2c4e599156529051a701c5984d2/bin/dev-cmd -h
usage: dev-cmd [-h] [-V] [-l] [-q] [-s SKIPS] [-p] [-t] [--hashseed HASHSEED] [--py PYTHON] [-k | -X {after-step,immediate,end}] [--grace-period GRACE_PERIOD]
               [--color {always,auto,never}]
               [cmd|task ...]

A simple command runner to help running development tools easily and consistently.

positional arguments:
  cmd|task              One or more names of `commands` or `tasks` to run that are defined in the [tool.dev-cmd] section of `pyproject.toml`. If no command or task names are passed
                        and a [tool.dev-cmd] `default` is defined or there is only one command defined, that is run.

options:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
  -l, --list            List the commands and tasks that can be run.
  -q, --quiet           Do not output information about the commands `dev-cmd` is running; just show output from the commands run themselves.
  -s, --skip SKIPS      After calculating all steps to run given the command line args, remove these steps, whether they be command names or task names from that list.
  -p, --parallel        Run all the top level command and task names passed in parallel. Has no effect unless there are two or more top level commands or tasks requested.
  -t, --timings         Emit timing information for each command run.
  --hashseed HASHSEED   Set the {--hashseed} command placeholder value.
  --py, --python PYTHON
                        Select an older python to run `dev-cmd` against. The value can be a full path to the exact CPython or PyPy binary to use or just the binary name; e.g.:
                        pypy3.10,in which case the interpreter will be looked up on the PATH. You can omit the python prefix for a PATH search and it will be filled in; i.e.: 3.12
                        will be expanded to python3.12 and then looked up on the PATH. Finally, you can even omit the dot in the version; i.e. 38 will expand to python3.8 and then
                        looked up on the PATH.
  -k, --keep-going      Normally, `dev-cmd` exits with an error code after the first task step with an errored command completes. You can choose to `-k` / `--keep-going` to run all
                        steps to the end before exiting. This option is mutually exclusive with `-X` / `--exit-style`.
  -X, --exit-style {after-step,immediate,end}
                        When to exit a `dev-cmd` invocation that encounters command errors. Normally, `dev-cmd` exits with an error code after the first task step with an errored
                        command completes. This option is mutually exclusive with `-k` / `--keep-going`.
  --grace-period GRACE_PERIOD
                        The amount of time in fractional seconds to wait for terminated commands to exit before killing them forcefully: 5.0 seconds by default. If set to zero or a
                        negative value, commands are always killed forcefully with no grace period. This setting comes into play when the `--exit-style` is either 'after-step' or
                        'immediate'.
  --color {always,auto,never}
                        When to color `dev-cmd` output. By default an appropriate color mode is 'auto'-detected, but color use can be forced 'always' on or 'never' on.

I did find an issue just after release with transitioning from pex3 run --locked require ... to pex3 run ... (which is --locked auto); so the 2.48.1 release should be out soon with, I think, proper support for experimenting with your idea: #2843