ubelt.util_cmd module¶
This module exposes the ubelt.cmd()
command, which provides a simple
means for interacting with the commandline. This uses
subprocess.Popen
under the hood, but improves upon existing
subprocess
functionality by:
(1) Adding the option to “tee” the output, i.e. simultaniously capture and write to stdout and stderr.
(2) Always specify the command as a string. The subprocess
module
expects the command as either a List[str]
if shell=False
and str
if shell=True
. If necessary, ubelt.util_cmd.cmd()
will automatically
convert from one format to the other, so passing in either case will work.
(3) Specificy if the process blocks or not by setting detach
. Note: when
detach is True
it is not possible to tee the output.
Example
>>> import ubelt as ub
>>> # Running with verbose=1 will write to stdout in real time
>>> info = ub.cmd('echo "write your command naturally"', verbose=1)
write your command naturally
>>> # Unless `detach=True`, `cmd` always returns an info dict.
>>> print('info = ' + ub.repr2(info))
info = {
'command': 'echo "write your command naturally"',
'cwd': None,
'err': '',
'out': 'write your command naturally\n',
'proc': <...Popen...>,
'ret': 0,
}
- ubelt.util_cmd.cmd(command, shell=False, detach=False, verbose=0, tee=None, cwd=None, env=None, tee_backend='auto', check=False, system=False, timeout=None)[source]¶
Executes a command in a subprocess.
The advantage of this wrapper around subprocess is that (1) you control if the subprocess prints to stdout, (2) the text written to stdout and stderr is returned for parsing, (3) cross platform behavior that lets you specify the command as a string or tuple regardless of whether or not shell=True. (4) ability to detach, return the process object and allow the process to run in the background (eventually we may return a Future object instead).
- Parameters
command (str | List[str]) – command string, tuple of executable and args, or shell command.
shell (bool, default=False) – if True, process is run in shell.
detach (bool, default=False) – if True, process is detached and run in background.
verbose (int, default=0) – verbosity mode. Can be 0, 1, 2, or 3.
tee (bool | None) – if True, simultaneously writes to stdout while capturing output from the command. If not specified, defaults to True if verbose > 0. If detach is True, then this argument is ignored.
cwd (str | PathLike | None) – Path to run command. Defaults to current working directory if unspecified.
env (Dict[str, str] | None) – environment passed to Popen
tee_backend (str, default=’auto’) – backend for tee output. Valid choices are: “auto”, “select” (POSIX only), and “thread”.
check (bool, default=False) – if True, check that the return code was zero before returning, otherwise raise a
subprocess.CalledProcessError
. Does nothing if detach is True.system (bool, default=False) – if True, most other considerations are dropped, and
os.system()
is used to execute the command in a platform dependant way. Other arguments such as env, tee, timeout, and shell are all ignored. (new in version 1.1.0)timeout (float) – If the process does not complete in timeout seconds, raises a
subprocess.TimeoutExpired
. (new in version 1.1.0) Currently unhandled when tee is True.log (Callable | None) – If specified, verbose output is written using this function, otherwise the builtin print function is used.
- Returns
info - information about command status. if detach is False
info
contains captured standard out, standard error, and the return code if detach is Trueinfo
contains a reference to the process.- Return type
- Raises
ValueError - on an invalid configuration –
subprocess.TimeoutExpired - if the timeout limit is exceeded –
subprocess.CalledProcessError - if check and the return value is non zero –
Note
Inputs can either be text or tuple based. On UNIX we ensure conversion to text if shell=True, and to tuple if shell=False. On windows, the input is always text based. See [SO_33560364] for a potential cross-platform shlex solution for windows.
When using the tee output, the stdout and stderr may be shuffled from what they would be on the command line.
- Related Work:
https://github.com/pycontribs/subprocess-tee https://github.com/mortoray/shelljob https://github.com/netinvent/command_runner
References
- SO_11495783
https://stackoverflow.com/questions/11495783/redirect-subprocess-stderr-to-stdout
- SO_7729336
- SO_33560364
https://stackoverflow.com/questions/33560364/python-windows-parsing-command-lines-with-shlex
CommandLine
xdoctest -m ubelt.util_cmd cmd:6 python -c "import ubelt as ub; ub.cmd('ping localhost -c 2', verbose=2)" pytest "$(python -c 'import ubelt; print(ubelt.util_cmd.__file__)')" -sv --xdoctest-verbose 2
Example
>>> import ubelt as ub >>> info = ub.cmd(('echo', 'simple cmdline interface'), verbose=1) simple cmdline interface >>> assert info['ret'] == 0 >>> assert info['out'].strip() == 'simple cmdline interface' >>> assert info['err'].strip() == ''
Example
>>> import ubelt as ub >>> info = ub.cmd('echo str noshell', verbose=0) >>> assert info['out'].strip() == 'str noshell'
Example
>>> # windows echo will output extra single quotes >>> import ubelt as ub >>> info = ub.cmd(('echo', 'tuple noshell'), verbose=0) >>> assert info['out'].strip().strip("'") == 'tuple noshell'
Example
>>> # Note this command is formatted to work on win32 and unix >>> import ubelt as ub >>> info = ub.cmd('echo str&&echo shell', verbose=0, shell=True) >>> assert info['out'].strip() == 'str' + chr(10) + 'shell'
Example
>>> import ubelt as ub >>> info = ub.cmd(('echo', 'tuple shell'), verbose=0, shell=True) >>> assert info['out'].strip().strip("'") == 'tuple shell'
Example
>>> import pytest >>> import ubelt as ub >>> info = ub.cmd('echo hi', check=True) >>> import subprocess >>> with pytest.raises(subprocess.CalledProcessError): >>> ub.cmd('exit 1', check=True, shell=True)
Example
>>> import ubelt as ub >>> from os.path import join, exists >>> fpath1 = join(ub.get_app_cache_dir('ubelt'), 'cmdout1.txt') >>> fpath2 = join(ub.get_app_cache_dir('ubelt'), 'cmdout2.txt') >>> ub.delete(fpath1) >>> ub.delete(fpath2) >>> # Start up two processes that run simultaneously in the background >>> info1 = ub.cmd(('touch', fpath1), detach=True) >>> info2 = ub.cmd('echo writing2 > ' + fpath2, shell=True, detach=True) >>> # Detached processes are running in the background >>> # We can run other code while we wait for them. >>> while not exists(fpath1): ... pass >>> while not exists(fpath2): ... pass >>> # communicate with the process before you finish >>> # (otherwise you may leak a text wrapper) >>> info1['proc'].communicate() >>> info2['proc'].communicate() >>> # Check that the process actually did finish >>> assert (info1['proc'].wait()) == 0 >>> assert (info2['proc'].wait()) == 0 >>> # Check that the process did what we expect >>> assert ub.readfrom(fpath1) == '' >>> assert ub.readfrom(fpath2).strip() == 'writing2'
Example
>>> # Can also use ub.cmd to call os.system >>> import pytest >>> import ubelt as ub >>> import subprocess >>> info = ub.cmd('echo hi', check=True, system=True) >>> with pytest.raises(subprocess.CalledProcessError): >>> ub.cmd('exit 1', check=True, shell=True)