pip install -U brish
Or install the latest master (recommended, as I might have forgotten to push a new versioned update):
pip install git+https://github.com/NightMachinery/brish
You need a recent Python version, as Brish uses some of the newer metaprogramming APIs. Obviously, you also need zsh installed.
Brish needs Python 3.10 or newer. Working on Brish itself is described in docs/development.org: the setup, the tests and the dependency audit.
from brish import z, zp, Brishname="A$ron"
z("echo Hello {name}")Hello A$ron
z automatically converts Python lists to shell lists:
alist = ["# Fruits", "1. Orange", "2. Rambutan", "3. Strawberry"]
z("for i in {alist} ; do echo $i ; done")# Fruits 1. Orange 2. Rambutan 3. Strawberry
z returns a CmdResult (more about which later):
res = z("date +%Y")
repr(res)CmdResult(retcode=0, out='2021\n', err='', cmd=' date +%Y ', cmd_stdin='')
You can use zp as a shorthand for print(z(...).outerr, end=''):
for i in range(10):
cmd = "(( {i} % 2 == 0 )) && echo {i} || {{ echo Bad Odds'!' >&2 }}" # Using {{ and }} as escapes for { and }
zp(cmd)
print(f"Same thing: {z(cmd).outerr}", end='')0 Same thing: 0 Bad Odds! Same thing: Bad Odds! 2 Same thing: 2 Bad Odds! Same thing: Bad Odds! 4 Same thing: 4 Bad Odds! Same thing: Bad Odds! 6 Same thing: 6 Bad Odds! Same thing: Bad Odds! 8 Same thing: 8 Bad Odds! Same thing: Bad Odds!
CmdResult is true if its return code is zero:
if z("test -e ~/"):
print("HOME exists!")
else:
print("We're homeless :(")HOME exists!
CmdResult is smart about iterating:
for path in z("command ls ~/tmp/"): # `command` helps bypass potential aliases defined on `ls`
zp("du -h ~/tmp/{path}") # zp prints the result524K /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/c01ed1a32d65c8d4ecb9095509e61f97 524K /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/36b77e6b3b7fde31f2fc4f182c0ecf82 1.3M /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/tumblr/dreamcorp420 1.3M /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/tumblr 0B /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/34cc02221710caf309bff5ca96808d7a 520K /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/6cc3d153426e2b6d1ac0f3736aaf74a1 2.9M /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43 0B /Users/evar/tmp/8826215ac0ed61d617906f658322fce7 348K /Users/evar/tmp/IMG_0396.PNG 44K /Users/evar/tmp/a2.jpg 40K /Users/evar/tmp/a4.jpg 8.0K /Users/evar/tmp/bills 0B /Users/evar/tmp/garden 468K /Users/evar/tmp/image-14000213234237913.png 40K /Users/evar/tmp/photo_2021-05-08_00-35-24.jpg 152K /Users/evar/tmp/photo_2021-05-08_00-55-29.jpg 8.0K /Users/evar/tmp/tumblr 4.0M /Users/evar/tmp/tumblr_2c0ad7a3fba563996c9abaedc5e8d4f7_356ef3d9_1280.gif 576K /Users/evar/tmp/tumblr_5a2868650b058c42a7d141b8a2f474bc_eac04dc0_1280.jpg 976K /Users/evar/tmp/tumblr_5cc2e0e48418ec3c9eb200d151daf647_e44e419b_1280.jpg 44K /Users/evar/tmp/tumblr_6c90d77a676cf20fc096cc19220af4ab_e124dbec_540.gif.mp4 0B /Users/evar/tmp/tumblr_70675efa5303a58292957ac942663309_f48499c2_1280.jpg 4.0K /Users/evar/tmp/tumblr_70675efa5303a58292957ac942663309_f48499c2_1280.jpg.aria2 656K /Users/evar/tmp/tumblr_bc2259b471f792065eb6707b7c29d27e_97f97d94_1280.jpg 1.0M /Users/evar/tmp/tumblr_ec36fde70ee0264cdc2f61f394181c61_575227e7_1280.jpg 84K /Users/evar/tmp/view.php
res = z("""echo This is stdout
echo This is stderr >&2
(exit 6) # this is the return code""")
repr(res.out)This is stdout\n
CmdResult.outrs strips the final newlines:
repr(res.outrs)This is stdout
repr(res.err)This is stderr\n
res.retcode6
res.longstrcmd: echo This is stdout
echo This is stderr >&2
(exit 6) # this is the return code
stdout:
This is stdout
stderr:
This is stderr
return code: 6
By default, z doesn’t fork. So we can use it to change the state of the running zsh session:
z("""
(($+commands[imdbpy])) || pip install -U imdbpy
imdb() imdbpy search movie --first "$*"
""")
z("imdb Into the Woods 2014")Movie ===== Title: Into the Woods (2014) Genres: Adventure, Comedy, Drama, Fantasy, Musical. Director: Rob Marshall. Writer: James Lapine, James Lapine. Cast: Anna Kendrick (Cinderella), Daniel Huttlestone (Jack), James Corden (Baker / Narrator), Emily Blunt (Baker's Wife), Christine Baranski (Stepmother). Runtime: 125. Country: United States. Language: English. Rating: 5.9 (134093 votes). Plot: A witch tasks a childless baker and his wife with procuring magical items from classic fairy tales to reverse the curse put on their family tree.
We can force a fork. This is useful to make your scripts more robust.
print(z("exit 7", fork=True).retcode)
zp("echo 'Still alive!'")7 Still alive!
Working with stdin:
# the intuitive way
a="""1
2
3
4
5
"""
z("<<<{a} wc -l")6
z("wc -l", cmd_stdin=a)5
The stdin will by default be set to the empty string:
zp("cat")
zp("echo 'As you see, the previous command produced no output. It also did not block.'")as you see, the previous command produced no output. It also did not block.
z escapes your Python variables automagically:
python_var = "$HOME"
z("echo {python_var}")$HOME
Turning off the auto-escape:
z("echo {python_var:e}")/Users/evar
Working with Python bools from the shell:
z("test -n {True:bool}").retcode0
z("test -n {False:bool}").retcode1
Working with NUL-terminated output:
for f in z("fd -0 . ~/tmp").iter0():
zp("echo {f}")/Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43 /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/34cc02221710caf309bff5ca96808d7a /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/36b77e6b3b7fde31f2fc4f182c0ecf82 /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/36b77e6b3b7fde31f2fc4f182c0ecf82/tumblr_9527f4f6d2f1a39ef2b839780831f38f_859e5e2b_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/36b77e6b3b7fde31f2fc4f182c0ecf82/tumblr_dd64a6ced93d19ffe78b47cf3439373d_e8e18fb0_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/6cc3d153426e2b6d1ac0f3736aaf74a1 /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/6cc3d153426e2b6d1ac0f3736aaf74a1/tumblr_9527f4f6d2f1a39ef2b839780831f38f_859e5e2b_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/6cc3d153426e2b6d1ac0f3736aaf74a1/tumblr_dd64a6ced93d19ffe78b47cf3439373d_e8e18fb0_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/c01ed1a32d65c8d4ecb9095509e61f97 /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/c01ed1a32d65c8d4ecb9095509e61f97/tumblr_9527f4f6d2f1a39ef2b839780831f38f_859e5e2b_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/c01ed1a32d65c8d4ecb9095509e61f97/tumblr_dd64a6ced93d19ffe78b47cf3439373d_e8e18fb0_2048.jpg /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/tumblr /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/tumblr/dreamcorp420 /Users/evar/tmp/83e93d36396014e0cd979ddcad2d9d43/tumblr/dreamcorp420/tumblr_dreamcorp420_650543836474589184_01.gif /Users/evar/tmp/8826215ac0ed61d617906f658322fce7 /Users/evar/tmp/IMG_0396.PNG /Users/evar/tmp/a2.jpg /Users/evar/tmp/a4.jpg /Users/evar/tmp/bills /Users/evar/tmp/garden /Users/evar/tmp/image-14000213234237913.png /Users/evar/tmp/photo_2021-05-08_00-35-24.jpg /Users/evar/tmp/photo_2021-05-08_00-55-29.jpg /Users/evar/tmp/tumblr /Users/evar/tmp/tumblr_2c0ad7a3fba563996c9abaedc5e8d4f7_356ef3d9_1280.gif /Users/evar/tmp/tumblr_5a2868650b058c42a7d141b8a2f474bc_eac04dc0_1280.jpg /Users/evar/tmp/tumblr_5cc2e0e48418ec3c9eb200d151daf647_e44e419b_1280.jpg /Users/evar/tmp/tumblr_6c90d77a676cf20fc096cc19220af4ab_e124dbec_540.gif.mp4 /Users/evar/tmp/tumblr_70675efa5303a58292957ac942663309_f48499c2_1280.jpg /Users/evar/tmp/tumblr_70675efa5303a58292957ac942663309_f48499c2_1280.jpg.aria2 /Users/evar/tmp/tumblr_bc2259b471f792065eb6707b7c29d27e_97f97d94_1280.jpg /Users/evar/tmp/tumblr_ec36fde70ee0264cdc2f61f394181c61_575227e7_1280.jpg /Users/evar/tmp/view.php
You can bypass the automatic iterable conversion by converting the iterable to a string first:
z("echo {' '.join(map(str,alist))}")# Fruits 1. Orange 2. Rambutan 3. Strawberry
Normal Python formatting syntax works as expected:
z("echo {67:f}")67.0
z("echo {[11, 45]!s}")[11, 45]
You can obviously nest your z calls:
z("""echo monkey$'\n'{z("curl -s https://www.poemist.com/api/v1/randompoems | jq --raw-output '.[0].content'")}$'\n'end | sed -e 's/monkey/Random Poem:/'""")Random Poem: ’Tis said that the Passion Flower, With its figures of spear and sword And hammer and nails, is a symbol Of the Woe of our Blessed Lord. So still in the Heart of Beauty Has been hidden, since Life drew breath, The sword and the spear of Anguish, And the hammer and nails of Death. end
z and zp are just convenience methods:
bsh = Brish() z = bsh.z zp = bsh.zp zq = bsh.zsh_quote zs = bsh.zstring
You can use Brish instances yourself (all arguments to it are optional). The boot command boot_cmd allows you to easily initialize the zsh session:
my_own_brish = Brish(boot_cmd="mkdir -p ~/tmp ; cd ~/tmp")
my_own_brish.z("echo $PWD")/Users/evar/tmp
Interpolation uses the caller’s globals and locals by default. Pass locals_ to supply an explicit namespace to zstring, z, zpopen or z_background. It accepts a dict or another mapping, including sys._getframe().f_locals on Python 3.13 and later. Non-dict mappings are copied to a dict before evaluation, so nested expressions can resolve their names and evaluation does not write into the original mapping. The copy is shallow. An explicit dict is used as supplied, preserving evaluation side effects; an empty namespace never falls back to the caller’s variables.
Brish.z itself is sugar around Brish.zstring and Brish.send_cmd:
cmd_str = my_own_brish.zstring("echo zstring constructs the command string that will be sent to zsh. It interpolates the Pythonic variables: {python_var} {alist}")
cmd_strecho zstring constructs the command string that will be sent to zsh. It interpolates the Pythonic variables: '$HOME' '# Fruits' '1. Orange' '2. Rambutan' '3. Strawberry'
my_own_brish.send_cmd(cmd_str)zstring constructs the command string that will be sent to zsh. It interpolates the Pythonic variables: $HOME # Fruits 1. Orange 2. Rambutan 3. Strawberry
You can restart a Brish instance:
my_own_brish.z("a=56")
my_own_brish.zp("echo Before restart: $a")
my_own_brish.restart()
my_own_brish.zp("echo After restart: $a")
my_own_brish.zp("echo But the boot_cmd has run in the restarted instance, too: $PWD")Before restart: 56 After restart: But the boot_cmd has run in the restarted instance, too: /Users/evar/tmp
restart() never waits for a worker that is in use. It starts a new set of workers (each runs boot_cmd), and returns once they are up: by then every worker whose lock was free has been replaced by a new one. A worker that a thread holds (with acquire_lock, or a running popen, such as an endless tail -f) keeps serving that thread, with its state, and is replaced once the thread releases it, when the next call takes it. restart() returns True if every worker was replaced, and False if some wait for their release. A thread that holds a worker lock may call it too. A call made while another restart is under way waits for it and then starts its own set, since that one may have read the startup files before a change the caller made; calls that queue up meanwhile share that next set. Sending the command %BRISH_RESTART does the same, and its output names the workers that wait.
Brish is threadsafe. I have built BrishGarden on top of Brish to provide an HTTP REST API for executing zsh code (if wanted, in sessions). Using BrishGarden, you can embed zsh in pretty much any programming language, and pay no cost whatsoever for its startup. It can also function as a remote code executor.
server_count allows the underlying zsh instance of a Brish object to fork that many times, and so serve that many clients in parallel. This will not increase the startup time, as the forking happens after loading the zsh interpreter completely.
I have combined this with GNU parallel to easily parallelize my zsh functions.
n = 32
my_parallel_brish = Brish(server_count=n)
import logging
import threading
import time
def thread_function(name):
logging.info("Thread %s: starting", name)
my_parallel_brish.zp("echo Started {name} at $EPOCHREALTIME ; sleep 10 ; echo Finished {name} at $EPOCHREALTIME")
logging.info("Thread %s: finishing", name)
if __name__ == "__main__":
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO,
datefmt="%H:%M:%S")
threads = list()
now = float(z("echo $EPOCHREALTIME").outrs)
for index in range(32):
logging.info("Main : create and start thread %d.", index)
x = threading.Thread(target=thread_function, args=(index,))
threads.append(x)
x.start()
for index, thread in enumerate(threads):
logging.info("Main : before joining thread %d.", index)
thread.join()
logging.info("Main : thread %d done", index)
end = float(z("echo $EPOCHREALTIME").outrs)
print(f"Took {(end - now)}")
17:25:26: Main : create and start thread 0. 17:25:26: Thread 0: starting 17:25:26: Main : create and start thread 1. 17:25:26: Thread 1: starting 17:25:26: Main : create and start thread 2. 17:25:26: Thread 2: starting 17:25:26: Main : create and start thread 3. 17:25:26: Thread 3: starting 17:25:26: Main : create and start thread 4. 17:25:26: Thread 4: starting 17:25:26: Main : create and start thread 5. 17:25:26: Thread 5: starting 17:25:26: Main : create and start thread 6. 17:25:26: Thread 6: starting 17:25:26: Main : create and start thread 7. 17:25:26: Thread 7: starting 17:25:26: Main : create and start thread 8. 17:25:26: Thread 8: starting 17:25:26: Main : create and start thread 9. 17:25:26: Thread 9: starting 17:25:26: Main : create and start thread 10. 17:25:26: Thread 10: starting 17:25:26: Main : create and start thread 11. 17:25:26: Thread 11: starting 17:25:26: Main : create and start thread 12. 17:25:26: Thread 12: starting 17:25:26: Main : create and start thread 13. 17:25:26: Thread 13: starting 17:25:26: Main : create and start thread 14. 17:25:26: Thread 14: starting 17:25:26: Main : create and start thread 15. 17:25:26: Thread 15: starting 17:25:26: Main : create and start thread 16. 17:25:26: Thread 16: starting 17:25:26: Main : create and start thread 17. 17:25:26: Thread 17: starting 17:25:26: Main : create and start thread 18. 17:25:26: Thread 18: starting 17:25:26: Main : create and start thread 19. 17:25:26: Thread 19: starting 17:25:26: Main : create and start thread 20. 17:25:26: Thread 20: starting 17:25:26: Main : create and start thread 21. 17:25:26: Thread 21: starting 17:25:26: Main : create and start thread 22. 17:25:26: Thread 22: starting 17:25:26: Main : create and start thread 23. 17:25:26: Thread 23: starting 17:25:26: Main : create and start thread 24. 17:25:26: Thread 24: starting 17:25:26: Main : create and start thread 25. 17:25:26: Thread 25: starting 17:25:26: Main : create and start thread 26. 17:25:26: Thread 26: starting 17:25:26: Main : create and start thread 27. 17:25:26: Thread 27: starting 17:25:26: Main : create and start thread 28. 17:25:26: Thread 28: starting 17:25:26: Main : create and start thread 29. 17:25:26: Thread 29: starting 17:25:26: Main : create and start thread 30. 17:25:26: Thread 30: starting 17:25:26: Main : create and start thread 31. 17:25:26: Thread 31: starting 17:25:26: Main : before joining thread 0. Started 0 at 1620651326.2126729488 Finished 0 at 1620651336.2229239941 17:25:36: Thread 0: finishing 17:25:36: Main : thread 0 done 17:25:36: Main : before joining thread 1. Started 1 at 1620651327.2022259235 Finished 1 at 1620651337.2120540142 17:25:37: Thread 1: finishing 17:25:37: Main : thread 1 done 17:25:37: Main : before joining thread 2. Started 30 at 1620651327.2101778984 Finished 30 at 1620651337.2140960693 17:25:37: Thread 30: finishing Started 2 at 1620651328.2068090439 Finished 2 at 1620651338.2182691097 17:25:38: Thread 2: finishing 17:25:38: Main : thread 2 done 17:25:38: Main : before joining thread 3. Started 31 at 1620651328.2222359180 Finished 31 at 1620651338.2338199615 17:25:38: Thread 31: finishing Started 7 at 1620651329.2063989639 Finished 7 at 1620651339.2115590572 17:25:39: Thread 7: finishing Started 15 at 1620651330.2087130547 Finished 15 at 1620651340.2192440033 17:25:40: Thread 15: finishing Started 21 at 1620651331.2160348892 Finished 21 at 1620651341.2246019840 17:25:41: Thread 21: finishing Started 23 at 1620651332.2160398960 Finished 23 at 1620651342.2200219631 17:25:42: Thread 23: finishing Started 9 at 1620651333.2236700058 Finished 9 at 1620651343.2359619141 17:25:43: Thread 9: finishing Started 18 at 1620651334.2257950306 Finished 18 at 1620651344.2365601063 17:25:44: Thread 18: finishing Started 12 at 1620651335.2241439819 Finished 12 at 1620651345.2335329056 17:25:45: Thread 12: finishing Started 20 at 1620651336.2342200279 Finished 20 at 1620651346.2429049015 17:25:46: Thread 20: finishing Started 16 at 1620651337.4859669209 Finished 16 at 1620651347.4899230003 17:25:47: Thread 16: finishing Started 22 at 1620651338.2339038849 Finished 22 at 1620651348.2375440598 17:25:48: Thread 22: finishing Started 19 at 1620651339.2459530830 Finished 19 at 1620651349.2504169941 17:25:49: Thread 19: finishing Started 13 at 1620651340.2416980267 Finished 13 at 1620651350.2485001087 17:25:50: Thread 13: finishing Started 10 at 1620651340.2490129471 Finished 10 at 1620651350.2568130493 17:25:50: Thread 10: finishing Started 29 at 1620651341.2439520359 Finished 29 at 1620651351.2504179478 17:25:51: Thread 29: finishing Started 25 at 1620651342.2465701103 Finished 25 at 1620651352.2498950958 17:25:52: Thread 25: finishing Started 17 at 1620651343.2493131161 Finished 17 at 1620651353.2571830750 17:25:53: Thread 17: finishing Started 28 at 1620651344.2550890446 Finished 28 at 1620651354.2586359978 17:25:54: Thread 28: finishing Started 14 at 1620651345.2569661140 Finished 14 at 1620651355.2659308910 17:25:55: Thread 14: finishing Started 5 at 1620651346.2559928894 Finished 5 at 1620651356.2631940842 17:25:56: Thread 5: finishing Started 4 at 1620651347.2538421154 Finished 4 at 1620651357.2619009018 17:25:57: Thread 4: finishing Started 3 at 1620651347.2638580799 Finished 3 at 1620651357.2686970234 17:25:57: Thread 3: finishing 17:25:57: Main : thread 3 done 17:25:57: Main : before joining thread 4. 17:25:57: Main : thread 4 done 17:25:57: Main : before joining thread 5. 17:25:57: Main : thread 5 done 17:25:57: Main : before joining thread 6. Started 26 at 1620651348.2553079128 Finished 26 at 1620651358.2628009319 17:25:58: Thread 26: finishing Started 27 at 1620651348.2706210613 Finished 27 at 1620651358.2781529427 17:25:58: Thread 27: finishing Started 24 at 1620651349.2586579323 Finished 24 at 1620651359.2646100521 17:25:59: Thread 24: finishing Started 11 at 1620651350.2648739815 Finished 11 at 1620651360.2702779770 17:26:00: Thread 11: finishing Started 8 at 1620651351.2621378899 Finished 8 at 1620651361.2658278942 17:26:01: Thread 8: finishing Started 6 at 1620651352.4786870480 Finished 6 at 1620651362.4896230698 17:26:02: Thread 6: finishing 17:26:02: Main : thread 6 done 17:26:02: Main : before joining thread 7. 17:26:02: Main : thread 7 done 17:26:02: Main : before joining thread 8. 17:26:02: Main : thread 8 done 17:26:02: Main : before joining thread 9. 17:26:02: Main : thread 9 done 17:26:02: Main : before joining thread 10. 17:26:02: Main : thread 10 done 17:26:02: Main : before joining thread 11. 17:26:02: Main : thread 11 done 17:26:02: Main : before joining thread 12. 17:26:02: Main : thread 12 done 17:26:02: Main : before joining thread 13. 17:26:02: Main : thread 13 done 17:26:02: Main : before joining thread 14. 17:26:02: Main : thread 14 done 17:26:02: Main : before joining thread 15. 17:26:02: Main : thread 15 done 17:26:02: Main : before joining thread 16. 17:26:02: Main : thread 16 done 17:26:02: Main : before joining thread 17. 17:26:02: Main : thread 17 done 17:26:02: Main : before joining thread 18. 17:26:02: Main : thread 18 done 17:26:02: Main : before joining thread 19. 17:26:02: Main : thread 19 done 17:26:02: Main : before joining thread 20. 17:26:02: Main : thread 20 done 17:26:02: Main : before joining thread 21. 17:26:02: Main : thread 21 done 17:26:02: Main : before joining thread 22. 17:26:02: Main : thread 22 done 17:26:02: Main : before joining thread 23. 17:26:02: Main : thread 23 done 17:26:02: Main : before joining thread 24. 17:26:02: Main : thread 24 done 17:26:02: Main : before joining thread 25. 17:26:02: Main : thread 25 done 17:26:02: Main : before joining thread 26. 17:26:02: Main : thread 26 done 17:26:02: Main : before joining thread 27. 17:26:02: Main : thread 27 done 17:26:02: Main : before joining thread 28. 17:26:02: Main : thread 28 done 17:26:02: Main : before joining thread 29. 17:26:02: Main : thread 29 done 17:26:02: Main : before joining thread 30. 17:26:02: Main : thread 30 done 17:26:02: Main : before joining thread 31. 17:26:02: Main : thread 31 done Took 36.33210492134094
The z_background function allows you to execute shell commands asynchronously in a new thread. It starts a new Zsh instance and runs the given command without blocking your main Python thread. This is particularly useful when you want to perform non-blocking operations or execute long-running shell commands without interrupting your Python program’s flow.
from brish import z_background
msg = "You can’t make an omelet without breaking a few eggs."
result_future = z_background(
"say {msg}",
# Needs the `say` command, available by default on macOS
)
result_future
<Future at 0x1435b51e0 state=running>
The z_background function returns a Future object.
result_future.result()
CmdResult(retcode=0, out='', err='', cmd=" say 'You can’t make an omelet without breaking a few eggs.' ", cmd_stdin='')
Here is another example:
from brish import z_background
import concurrent.futures
# Define multiple commands
commands = [
"sleep 5 && echo 'First command completed.'",
"sleep 3 && echo 'Second command completed.'",
"sleep 1 && echo 'Third command completed.'",
]
# Execute all commands asynchronously
futures = [z_background(cmd) for cmd in commands]
for future in concurrent.futures.as_completed(futures):
result = future.result()
print(result.out)
Third command completed. Second command completed. First command completed.
Brish.popen runs a command and hands you its output while it runs, as raw bytes:
from brish import Brish
b = Brish(binary=True)
with b.popen("for i in 1 2 3; do print -r -- step $i; sleep 0.5; done; print -ru2 -- done; return 3") as p:
for stream, chunk in p:
print(stream, chunk)
print(p.retcode)
popen(cmd, cmd_stdin””, fork=False, server_index=None, lock_sleep=1, buffer=False, cancelled=None)= takes the arguments of send_cmd and returns a BrishPopen:
popenreturns once the worker has read the whole request and is starting the command (binary mode), or once the request is written, while the worker may still be reading it (legacy mode). In legacy modekill()needs the worker’s PID, which the firstpopenon a new worker (afterinit(), a restart or a replacement) reads from the bootstrap:brish2.zshreports every worker’s PID on its stdout when it starts. A customshellthat does not is asked by the firstpopenon each worker, with one internal request, and the answer is checked with onepsrun: a few tens of milliseconds, more on a loaded machine.- Iterating it yields
(stream, chunk)pairs:streamis"out"or"err", andchunkis non-emptybytes, as read, so progress bars (\r) and partial lines show at once. With2>&1in the command, everything arrives as"out", in order. p.retcodeisNoneuntil the command has ended, and then its status, assend_cmdwould report it.p.wait()reads the rest and returns the retcode.p.close()kills the command if it still runs, reads the rest and frees the worker; leaving thewithblock does that, also onbreakor an exception.- With
buffer=True, the chunks are also kept, andp.resultis theCmdResultthatsend_cmdwould have returned (Noneuntil the end). b.zpopen(template, ...)ispopenofzstring(template), aszissend_cmdof it.cancelled(also an argument ofsend_cmd,zandacquire_lock) is a callable without arguments, for a caller whose client may go away while the call waits for a worker: a free one, a slot’s lock that another thread holds, or a dead worker’s replacement. Brish calls it every 0.05 seconds while it waits for a lock or for a replacement starting in the background (also withinlock_sleep), and once more after it holds the worker, just before it writes the request (acquire_lock: before it returns). When it returns true, the call frees the worker and raisesBrishCancelledException(aRuntimeError): nothing ran. An exception that the callable raises propagates the same way. A replacement that the call starts itself (witheager_replacementoff) is not interrupted, and stays in the slot for the next call, as does one starting in the background. In binary mode a stale worker (see Threads) takes the request at once and runs it when its abandoned command ends, socancelledis not asked after that. With%BRISH_RESTARTas the command, it is asked once, before the restart starts; a restart under way is not called off.- Brish holds back only bytes that could be the start of the reply’s end marker. In binary mode that is a suffix starting at a NUL byte, so output without NUL is never delayed. In legacy mode it is a trailing newline (or newline and NUL), so each line’s newline arrives with the next chunk. How output splits into chunks depends on timing in both modes; one run of the example above in legacy mode (
Brish()) gave:out b'step 1' out b'\nstep 2' out b'\nstep 3' err b'done\n' out b'\n'(
err b'done\n'arrives whole when the end of the stderr reply is read together with it; otherwise aserr b'done'anderr b'\n'.) - In legacy mode, output containing newline, NUL, newline ends the stream early, exactly as it ends a
send_cmdreply (see Known Issues). - When you stop reading, the command blocks once Brish and the pipes hold what it wrote. In binary mode Brish holds at most one read per stream (up to 64 KiB each), and each pipe holds up to 64 KiB more. In legacy mode two reader threads, one per reply FIFO, queue at most 64 KiB for both streams together, and each FIFO holds a little more (8 KiB on macOS). A reader that falls behind gets fewer, larger chunks, up to 64 KiB each, as the pipes would merge them.
Threads. A BrishPopen holds its worker’s lock from popen until the output has been read to the end, so read it, wait for it and close it in the thread that created it, its owner. A read (also next() on an iterator that the owner made), wait() or close() from another thread raises RuntimeError and changes nothing: the command runs on, and the owner reads on. kill() works from any thread.
Calls that the owner makes while it streams (send_cmd, another popen, acquire_lock, inside the for loop say) never land on that busy worker. With server_index=None they take another worker that is free at once, and when no other worker is free, they raise BrishWorkerBusyException (a RuntimeError) at once instead of waiting for one, and nothing is restarted: such a wait would hold the streaming worker while waiting for another, and two threads that streamed and waited that way would wait for each other for good. A call that names the busy worker raises it too. A call that names another worker waits for that worker’s lock like any call, so two streaming threads that name each other’s workers do wait for each other for good, as two acquire_lock holders would. And in binary mode, a worker that is free can be stale (it still runs a command whose reply was abandoned, see “When a Worker Dies”): a call that takes it, with any server_index, first waits for that command to end. So make such calls after the loop. A thread that already holds the lock (from acquire_lock) passes its server_index, and can go on using that worker afterwards:
lock, server_index = b.acquire_lock()
try:
b.send_cmd('cd "$dir"', server_index=server_index)
with b.popen("{ make; } 2>&1", server_index=server_index) as p:
for stream, chunk in p: # meanwhile another thread may call p.kill()
show(chunk)
rc = p.retcode
finally:
lock.release()
Use with, or call close() in the owner thread. A BrishPopen that is dropped unclosed is closed when it is collected. In the owner thread that happens at once: the command gets one SIGINT, and the worker is freed; legacy mode then replaces the worker, unless Brish had already read the whole reply. In binary mode a command that survives the SIGINT runs on, and the next command on that worker waits for it to end. One that is collected in another thread (an orphan) is stopped by a helper thread with every step of kill(), and its output is read to the end and dropped, but only the owner can free the worker’s lock. The owner does so at its next call to the instance (send_cmd, popen, acquire_lock or restart, and what is built on them), which first waits for the helper thread to finish (for every step of kill() that the command needs: seconds for one that ignores SIGINT); until then the worker stays taken. The worker keeps its state, unless the kill had to reach step 4 (then it is replaced). An orphan that is collected while Python ends gets no helper thread, since none can start then: Python’s end stops its command (see “When a Worker Dies”).
A BrishPopen holds its worker’s lock until its output has been read to the end, but neither restart() nor the replacement of another, dead worker waits for it: an endless stream such as tail -f keeps its worker, and the other workers are replaced at once (see “When a Worker Dies”). cleanup() does wait for every BrishPopen in flight, so an endless stream holds it up until the stream ends or is killed. It does not wait for an orphan whose output has been read to the end, unless its owner also holds that worker’s lock for its own reasons (it took it with acquire_lock and has not released it yet), and restart() then gives that worker’s slot a new lock and worker at once; the owner frees the old lock at its next call. When the owner thread has ended, the slot gets a new lock and a new worker, so the worker is not lost for good: once the helper thread is done, or at the next call that wants that worker.
Killing. p.kill() (or p.terminate()) interrupts the command, not the worker, so the worker keeps its state. It is the way to interrupt a command: the workers run in a session of their own, which no terminal signal reaches (see “When a Worker Dies”). Besides kill(), Brish sends one SIGINT only to the command of a BrishPopen that is abandoned in its owner thread: dropped unclosed (see Threads), or interrupted by a KeyboardInterrupt inside a read. It is idempotent, returns at once, and the reader sees the rest of the output and the end as usual. The first signal goes out from a helper thread, after one ps run. Until it is out, a read (iteration, next(), wait(), close()) waits for it, for at most p.kill_grace seconds, and reads nothing: draining the pipes earlier would let a command that a full pipe holds up run on before the signal lands (head -c 20000000 /dev/zero to its end, say), and the statements after it with it. p.signalled says whether the first signal is out, and p.wait_signalled(timeout=None) waits for it, with the same bound, and returns p.signalled; it returns at once when there is nothing to wait for (kill() was not called, or the command had already ended). Any thread may use both. The steps stop at the first one after which the command has ended. Each lists the processes below the worker with one ps run, which can take a second or more on a loaded machine, and comes p.kill_grace seconds (2 by default) after the previous step’s signals went out when the command has gone quiet, or two graces after them while its output keeps coming:
- SIGINT to the worker, then to every process below it. The worker treats it like Ctrl-C in an interactive shell: the command is aborted and the retcode is 130, unless the command traps INT itself (
trap 'print caught' INT), and then the trap runs and the command goes on. One difference: an interactive shell lets a command swallow the interrupt withTRY_BLOCK_INTERRUPT=0in analwaysblock of its own and go on after that block. In the worker, a non-fork command’s block runs, and then the command ends with 130; a fork command’s subshell exits with 130 at once, without running the block. - SIGINT to the worker again, then SIGTERM to the processes below it.
- SIGKILL to the processes below the worker. If there are none, a non-fork command runs in the worker itself (a loop that ignores INT, say), and step 4 comes at once. For a fork command, none means that its subshell has exited and the worker has only the end of the reply left to write, so step 4 comes a grace later, and only if that end has not shown up by then.
- SIGKILL to the worker. Brish first reads what the pipes already hold, up to 128 KiB beyond what it has read ahead and for at most half a second, and looks for the end of the reply: a command that ended just after the last step can have its end waiting there behind output you have not read yet, and then the worker is spared. The retcode is 9001 (also when the command happened to end just before and its end was not found that way), and the last chunk is
("err", b"brish: worker died during this command\n"), that isWORKER_DIED_NOTEand a newline, as a chunk of its own; when the stderr so far does not end with a newline, an("err", b"\n")chunk comes before it. The worker is replaced (see “When a Worker Dies”); the other workers keep their state, and nothing waits for them.
The steps go by what the command does, not by how fast you read. Once the first signal is out, Brish reads up to 256 KiB further ahead of you, so it notices at your next read that the command has ended, and you still get the rest of its output. Legacy mode’s reader threads read ahead by themselves; binary mode reads ahead inside your reads, and when a step is due, first for as long as the command keeps writing (up to half a second). When you read slower than the command writes after a signal, Brish takes the next step only once it has read more than 128 KiB since that signal, more than the pipes could hold before it. So a command that ends at a signal is not killed further because you read slowly, as long as it writes less than about 256 KiB after the signal: one that writes more while you read slowly (a long report on its way out, say) looks like a command that ignores the signal and goes on writing, and gets the next step.
So the retcode after a kill is 130 for a command that dies of the interrupt; the command’s own status when it traps INT with a trap string (trap 'print caught; return 7' INT gives 7), or when one of its processes was stopped by step 2 or 3; 1 for a non-fork command whose own TRAPINT function returns non-zero, which is zsh’s status for an interrupted function (with fork=True, the function’s own status); 143 or 137 for a fork command whose subshell step 2 or 3 stopped; and 9001 only when the worker had to go. Background jobs that earlier commands left running on that worker are below it too: step 1 does not affect them (they ignore SIGINT), but steps 2 to 4 stop them.
Step 1 also works for a non-fork command that runs under set -e, set -euo pipefail, setopt err_return or emulate sh: options with which zsh would exit instead of unwinding are turned off for the rest of the interrupted command, and posix_builtins (part of emulate sh) is set back to what it was before the command (see docs/protocol.org, Interrupts). Some cases still cost the worker. As a rule, an interrupt costs the worker when it unwinds out of a scope that restores options (a function with setopt local_options or emulate -L, or emulate ... -c) into code where err_exit is on (in legacy mode also err_return), or into a special builtin such as eval or . while posix_builtins is on. For example: set -e; f() { emulate -L zsh; sleep 100 }; f, set -e; emulate zsh -c 'sleep 100', and emulate sh; eval 'emulate zsh -c "sleep 100"'. A SIGINT also kills the worker while a non-fork command has removed the worker’s trap (trap - INT, unfunction TRAPINT). The retcode is then 9001, as in step 4, and the worker is replaced. In a fork command only the subshell dies. Rarely, an interrupt hangs the worker instead: zsh runs the trap from its signal handler, and when the signal lands while a builtin of the command (print, echo) is inside the C library’s lock of stdout, the trap’s own builtins wait for that lock forever. kill() still ends the command, at step 4 (9001, and the worker is replaced; for a fork command, whose subshell hangs, at step 2 with 143; see docs/protocol.org, Interrupts). Since only Brish sends SIGINT (see above), this needs one of its SIGINTs to land at that moment. The same interplay can also make the interrupted print write its line a second time before the command ends with 130 (seen once in about 125 floods of SIGINTs at a worker that ran print back to back).
Cost. Supporting kill() makes the worker do a little more for every command: it keeps the per-command state of its SIGINT trap. Measured against the version before streaming (commit a9661d1), with the calls of both interleaved in one process, a trivial send_cmd (true) takes more at a load average of about 2: about 8 µs (8 percent) in binary mode and about 10 µs (11 to 12 percent) in legacy mode; at a load average of 4 to 7 it takes 9 to 15 µs (8 to 14 percent) more in both modes; a fork=True call takes 1 to 25 µs (at most 5 percent) more. Calls that carry stdin got faster (a 1 KiB cmd_stdin: 60 to 130 µs, 3 to 6 percent), and so did large output in legacy mode (2,000 lines: about 7 percent), while in binary mode it got about 2 percent slower. Worker slots (see “When a Worker Dies”) add nothing measurable: against release 0.4.0, interleaved the same way (60 rounds of 300 calls, several runs per mode, at a load average of 3 to 7), a trivial send_cmd differed by -27 to +16 µs in legacy mode and -31 to +14 µs in binary mode, with no consistent sign. tests/test_slots.py repeats a shorter version of that comparison.
Brish has two transports. Binary mode carries exact bytes in every direction. Legacy mode, the default, does too, except where its frozen wire format cannot: a NUL in a command or its stdin is refused with retcode 9000, and an output line that holds only a NUL ends the reply early (see Known Issues). Turn binary mode on per instance, or for a whole process with the env var BRISH_BINARY (read when each Brish is created, including the module-level z, zp and friends at import):
from brish import Brish b = Brish(binary=True) # or, for everything in the process: # BRISH_BINARY=1 python3 my_script.py
Every CmdResult has byte views next to its text fields: outb, errb, outrsb (outb without trailing newlines), outerrb, iterb() (lines) and iter0b() (NUL-separated items). They hold exactly what the command wrote, and the text views are decoded from those bytes without any newline translation:
r = b.send_cmd("print -rn -- $'caf\\xc3\\xa9\\r\\n\\xff\\0'")
(r.outb, r.out)(b'caf\xc3\xa9\r\n\xff\x00', 'café\r\n\\xff\x00')
stdin and commands can be bytes. A str is encoded with the instance encoding (UTF-8 by default) and surrogateescape, so file names from os.listdir round-trip. In binary mode, cmd_stdin=None gives the command /dev/null instead of an empty pipe (legacy mode cannot tell the two apart):
(b.send_cmd("cat", cmd_stdin=b"\xff\xfe").outb,
b.send_cmd("[[ -c /dev/stdin ]] && print /dev/null", cmd_stdin=None).out)(b'\xff\xfe', '/dev/null\n')
Interpolated bytes, bytearray, memoryview and path-like values (including os.DirEntry) are quoted byte-exactly, also inside lists, and {value:e} with bytes inserts them unquoted:
name = b"report \xff.txt"
(b.zstring("print -rn -- {name}"), b.z("print -rn -- {name}").outb)(" print -rn -- $'report \\xff.txt' ", b'report \xff.txt')
zp and zpe write the bytes to sys.stdout.buffer and sys.stderr.buffer (flushing the text layer first, so output stays in order). A script that runs zp("cat some.png") with its stdout redirected to a file makes an identical copy. For a target without a binary buffer, such as io.StringIO, they print the text views.
Non-fork commands run in a protected loop, in both modes (legacy mode got it later, so older releases behave differently):
exit Nreturns a normal result with retcode N, and that worker is replaced (see “When a Worker Dies”);- NOMATCH and other shell errors return retcode 1 instead of killing the worker;
- a bare
breakorcontinueends the command with status 0; setopt errexitanderrreturndo not persist to the next command;- a syntax error no longer re-runs the previous command.
In both modes a command without stdin forks nothing. Binary mode is faster with large data: it reads stdin in 64 KiB chunks, where legacy mode reads one byte per syscall (about 1.5 seconds per MiB), and it parses large many-line output much faster. The wire protocols, including what legacy mode’s frozen format rules out and why, are documented in docs/protocol.org, and the design in docs/binary-transport.org.
If the command ran and killed its shell (with exit, errexit, ${x:?}, a signal, …), you get a CmdResult like from subprocess.run: the command’s status if the worker reported it (exit N reports N, in both modes), otherwise the reserved retcode 9001 with brish: worker died during this command appended to .err. A background job that keeps the dead shell’s output open does not delay the result. That worker alone is replaced; the others keep their state. If a worker died before it could run the command (killed between two commands, say), Brish replaces it and retries once; only when that fails too does it raise BrishWorkerDiedException.
Each worker index (0 to server_count - 1) is a slot: a lock, and the worker that serves it now. A dead worker’s slot gets a new worker of its own, which runs boot_cmd and has the same $brish_server_index, and nothing waits for the other slots, busy or not. By default the new worker starts in the background as soon as Brish sees the death, so the next call that takes the slot finds it ready. Set Brish.eager_replacement (on the class or an instance) to False to start it only when the slot is next taken: that call then waits for it, about as long as a Brish takes to start, and saves a process when the slot is not used again. Either way, a call that takes the slot and finds no new worker ready starts one itself. With server_index=None, calls prefer slots whose worker is ready. A slot changes its worker only when a thread takes its lock afresh, never under a thread that holds it (see below). A thread that ends while it holds a slot’s lock (an acquire_lock never released, a popen left running) can never release it: the slot then gets a new lock and a new worker.
A KeyboardInterrupt in send_cmd, or in popen before it returns, abandons the reply but does not stop the shell command (one while you read a BrishPopen sends the command one SIGINT). In binary mode the next command on that worker waits for it and then gets its own result, and other idle workers are preferred meanwhile. If the interrupt came while the request was still being written, the worker may hold part of it: in binary mode it then takes no request again, and it is replaced. In legacy mode the worker is replaced either way.
Brish starts its workers in a session of their own, so no signal from a terminal reaches them or their commands: not Ctrl-C (SIGINT), Ctrl-\ (SIGQUIT) or Ctrl-Z (SIGTSTP), nor the SIGHUP of a closing terminal, nor any other signal to Python’s process group. A Ctrl-C interrupts Python alone, as a KeyboardInterrupt, and the paragraph above applies. Only the main thread gets it, so a program whose other threads stream with popen (the jobs of a ThreadPoolExecutor, say) does not end at a Ctrl-C while they do: Python waits for those threads before it ends, and their commands run on (a second Ctrl-C interrupts that wait). Have the main thread kill() every running BrishPopen when it gets the KeyboardInterrupt, before it exits or shuts the executor down. To stop a command from Python, run it with popen and call kill() (see Streaming output). Commands have no controlling terminal either. tty prints not a tty, stty fails (also with </dev/tty), and opening /dev/tty fails at once (ENXIO, “Device not configured”), so a program that prompts on the terminal (a password prompt of ssh or sudo, say) fails or falls back to whatever else it can use (Python’s getpass reads stdin then), instead of waiting for a terminal no one may be watching. Background jobs that commands start are in the same session, so a closing terminal does not hang them up either. Earlier versions started the workers in Python’s process group: a terminal Ctrl-C then also ended a fork=True command (retcode 130, while a non-fork command ran on), and commands could open the terminal.
No command outlives Python. When Python ends, however it ends (normally, by an exception, by SIGTERM or SIGKILL, or because its terminal closed), the workers’ bootstrap sees its stdin close: idle workers exit, and a worker that still runs a command a second later is stopped with every process below it (SIGTERM, and SIGKILL a second later). A request that Python’s end cut short (a large cmd_stdin still being written, say) is not run at all. cleanup(), restart() and the replacement of a dead worker stop at once, the same way, a worker whose command may still run because its reply was abandoned (after a KeyboardInterrupt in send_cmd, say). Background jobs that commands left on idle workers keep running (as they did before at a normal exit; a closing terminal could hang them up), and so does anything that detached itself (a double fork); a busy worker’s background jobs are below it, and are stopped with it.
A thread that holds a worker lock (from acquire_lock, or inside a popen) keeps that worker until it releases the lock: a restart or a replacement never swaps it under the thread, and never waits for it. A call that finds its held worker dead in the meantime, or unusable after an interrupt (see above), raises BrishWorkerDiedException at once; release the lock, and the next call that takes the worker gets a new one. The thread may call restart(): the workers it does not hold are replaced at once, and its own once it releases them.
Interpolated values are quoted in pure Python (see brish/quoting.py), and zsh reads each quoted word back as exactly the original bytes. So a value from an untrusted source is safe to interpolate as {value}, as long as nothing evaluates it as code a second time. The template’s literal text is code, and so are values inserted with :e.
This holds in both modes, for every byte: in legacy mode a NUL inside a value is written as $'\x00', so it reaches the command intact.
Earlier versions quoted values with zsh’s ${(q+)...} flag, which mis-escapes some bytes and code points. Several of them are valid UTF-8 (U+001C, U+0080 to U+009F and U+00AD), and the broken escape can end the quoted word early, so the rest of the value ran as shell code. Upgrade if you interpolate untrusted text.
I am not a security expert, and feedback by security experts will be appreciated.
In legacy mode, a command’s output can forge the end of its reply: an output line holding only a NUL ends the reply there, and the line after it is read as the retcode. So a command that prints untrusted data (the contents of a downloaded file, say) can make its result report any status and shift the results of later commands by one. Use binary mode for such commands, whose replies are framed with a fresh random nonce the command never sees.
Note that you can create security holes for yourself, by, e.g., running eval on user input:
untrusted_input = " ; echo do evil | cat"
z("eval {untrusted_input}") # unsafedo evil
z("echo {untrusted_input}") # safe; echo do evil | cat
- Legacy mode (the default) keeps a wire format that older Brish code still speaks, so it cannot carry everything (details in docs/protocol.org). Binary mode has none of these limits:
- a NUL in a command or in its stdin gives retcode 9000 (a NUL inside an interpolated value works);
- an output line that holds only a NUL ends the reply early. On stdout the line after it is read as the retcode (a line that is not a number gives 9001 and a new worker), and the rest of the output as the next command’s reply on that worker, so results lag one behind (see Security Considerations); on stderr the retcode stays the command’s own, and the rest of its stderr joins the next reply’s;
- output that a background job writes between two commands joins the next command’s result;
- large stdin is slow, about 1.5 seconds per MiB;
cmd_stdin=Noneis the same as an empty stdin;- a
KeyboardInterruptinsend_cmd, or inpopenbefore it returns, costs that worker, which is replaced (kill()does not).
- Nonstandard encodings (non UTF-8) are corrupted
z("echo 'sth × another (ver.-)'")
sth Ã\xb7 another (ver.-) - A command’s standard input is always a pipe (an empty one by default). This can alter the behavior of some commands such as
ripgrep; Using</dev/nullor<&-can be a suitable workaround. In binary mode,cmd_stdin=Nonegives the command/dev/null. - A SIGINT that Brish sends (from
kill(), or the one for an abandonedBrishPopen) can, rarely, hang the worker, or make an interruptedprintwrite its line twice (see “Streaming output”).kill()still ends such a command: at step 4 with 9001 and a new worker, or, for a fork command, at step 2 with 143. After an abandonedBrishPopen’s single SIGINT nothing more is sent, and in binary mode the next command on that worker waits for good. - A
KeyboardInterruptinsend_cmddoes not stop the shell command (no terminal signal reaches the workers), and there is no per-command timeout.popenandkill()stop a command, and a timer that callskill()gives a time limit (threading.Timer(60, p.kill).start()). - Output that background jobs write while a later command runs is part of that command’s result.
- A
KeyboardInterruptthat arrives in the few bytecodes betweensend_cmdorpopentaking a worker’s lock and thetrythat releases it on an exception keeps that lock while the thread lives. The thread (usually the main thread) then holds that worker: other threads never get it (withserver_index=Nonethey use the others),restart()cannot replace it, andcleanup()from another thread waits for good. If the thread ends, the slot gets a new lock and a new worker.send_cmdhas always had this window.
I like to add a mode where the zsh session inherits the stderr from the parent Python process. This allows usage of interactive programs like fzf.
If you have any good design ideas, create an issue!
- pysh uses comments in bash scripts to switch the interpreter to Python, allowing variable reuse between the two.
- plumbum is a small yet feature-rich library for shell script-like programs in Python. It attempts to mimic the shell syntax (“shell combinators”) where it makes sense, while keeping it all Pythonic and cross-platform. I personally like this one a lot. A robust option that is also easy-to-use.
- shellfuncs: Python API to execute shell functions as they would be Python functions. (Last commit is in 2017.)
- xonsh is a superset of Python 3.5+ with additional shell primitives.
- daudin tries to eval your code as Python, falling back to the shell if that fails. It does not currently reuse a shell session, thus incurring large overhead. I think it can use Brish to solve this, but someone needs to contribute the support.
- duct.py is a library for running child processes. It’s quite low-level compared to the other projects in this list.
python -ccan also be powerful, especially if you write yourself a helper library in Python and some wrappers in your shell dotfiles. An example:alias x='noglob calc-raw' calc-raw () { python3 -c "from math import *; print($*)" }- Z shell kernel for Jupyter Notebook allows you to do all sorts of stuff if you spend the time implementing your usecase; See emacs-jupyter to get a taste of what’s possible. Jupyter Kernel Gateway also sounds promising, but I haven’t tried it out yet. Beware the completion support in this kernel though. It uses a pre-alpha proof of concept thingy that was very buggy when I tried it.
- Finally, if you’re feeling adventurous, try Rust’s rust_cmd_lib. It’s quite beautiful.
Dual-licensed under MIT and GPL v3 or later.