gevent.subprocess – Cooperative subprocess module

Caution

On POSIX platforms, this module is not usable from native threads other than the main thread; attempting to do so will raise a TypeError. This module depends on libev’s fork watchers. On POSIX systems, fork watchers are implemented using signals, and the thread to which process-directed signals are delivered is not defined. Because each native thread has its own gevent/libev loop, this means that a fork watcher registered with one loop (thread) may never see the signal about a child it spawned if the signal is sent to a different thread.

Note

The interface of this module is intended to match that of the standard library subprocess module (with many backwards compatible extensions from Python 3 backported to Python 2). There are some small differences between the Python 2 and Python 3 versions of that module (the Python 2 TimeoutExpired exception, notably, extends Timeout and there is no SubprocessError) and between the POSIX and Windows versions. The HTML documentation here can only describe one version; for definitive documentation, see the standard library or the source code.

class Popen(args, bufsize=None, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=<object object>, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, threadpool=None, **kwargs)

Bases: object

The underlying process creation and management in this module is handled by the Popen class. It offers a lot of flexibility so that developers are able to handle the less common cases not covered by the convenience functions.

See also

subprocess.Popen This class should have the same interface as the standard library class.

Changed in version 1.2a1: Instances can now be used as context managers under Python 2.7. Previously this was restricted to Python 3.

Changed in version 1.2a1: Instances now save the args attribute under Python 2.7. Previously this was restricted to Python 3.

Create new Popen instance.

Parameters:kwargsOnly allowed under Python 3; under Python 2, any unrecognized keyword arguments will result in a TypeError. Under Python 3, keyword arguments can include pass_fds, start_new_session, restore_signals, encoding and errors

Changed in version 1.2b1: Add the encoding and errors parameters for Python 3.

communicate(input=None, timeout=None)

Interact with process: Send data to stdin. Read data from stdout and stderr, until end-of-file is reached. Wait for process to terminate. The optional input argument should be a string to be sent to the child process, or None, if no data should be sent to the child.

communicate() returns a tuple (stdout, stderr).

Parameters:timeout – Under Python 2, this is a gevent extension; if given and it expires, we will raise TimeoutExpired, which extends gevent.timeout.Timeout (note that this only extends BaseException, not Exception) Under Python 3, this raises the standard TimeoutExpired exception.

Changed in version 1.1a2: Under Python 2, if the timeout elapses, raise the gevent.timeout.Timeout exception. Previously, we silently returned.

Changed in version 1.1b5: Honor a timeout even if there’s no way to communicate with the child (stdin, stdout, and stderr are not pipes).

kill()

Kill the process with SIGKILL

pipe_cloexec()

Create a pipe with FDs set CLOEXEC.

poll()

Check if child process has terminated. Set and return returncode attribute.

send_signal(sig)

Send a signal to the process

terminate()

Terminate the process with SIGTERM

wait(timeout=None, _raise_exc=True)

Wait for child process to terminate. Returns returncode attribute.

Parameters:timeout – The floating point number of seconds to wait. Under Python 2, this is a gevent extension, and we simply return if it expires. Under Python 3, if this time elapses without finishing the process, TimeoutExpired is raised.
call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) → returncode

Run command with arguments. Wait for command to complete or timeout, then return the returncode attribute.

The arguments are the same as for the Popen constructor. Example:

retcode = call(["ls", "-l"])

Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported versions of Python (not just Python 3) and if it expires will raise a TimeoutExpired exception (under Python 2 this is a subclass of Timeout).

check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None) → 0

Run command with arguments. Wait for command to complete. If the exit code was zero then return, otherwise raise CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute.

The arguments are the same as for the Popen constructor. Example:

retcode = check_call(["ls", "-l"])
check_output(args, *, input=None, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None) → output

Run command with arguments and return its output.

If the exit code was non-zero it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute and output in the output attribute.

The arguments are the same as for the Popen constructor. Example:

>>> check_output(["ls", "-1", "/dev/null"])
'/dev/null\n'

The stdout argument is not allowed as it is used internally.

To capture standard error in the result, use stderr=STDOUT:

>>> check_output(["/bin/sh", "-c",
...               "ls -l non_existent_file ; exit 0"],
...              stderr=STDOUT)
'ls: non_existent_file: No such file or directory\n'

There is an additional optional argument, “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it too will be used internally. Example:

>>> check_output(["sed", "-e", "s/foo/bar/"],
...              input=b"when in the course of fooman events\n")
'when in the course of barman events\n'

If universal_newlines=True is passed, the return value will be a string rather than bytes.

Changed in version 1.2a1: The timeout keyword argument is now accepted on all supported versions of Python (not just Python 3) and if it expires will raise a TimeoutExpired exception (under Python 2 this is a subclass of Timeout).

Changed in version 1.2a1: The input keyword argument is now accepted on all supported versions of Python, not just Python 3

run(args, *, stdin=None, input=None, stdout=None, stderr=None, shell=False, timeout=None, check=False) → CompletedProcess

Run command with arguments and return a CompletedProcess instance.

The returned instance will have attributes args, returncode, stdout and stderr. By default, stdout and stderr are not captured, and those attributes will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them. If check is True and the exit code was non-zero, it raises a CalledProcessError. The CalledProcessError object will have the return code in the returncode attribute, and output & stderr attributes if those streams were captured.

If timeout is given, and the process takes too long, a TimeoutExpired exception will be raised.

There is an optional argument “input”, allowing you to pass a string to the subprocess’s stdin. If you use this argument you may not also use the Popen constructor’s “stdin” argument, as it will be used internally. The other arguments are the same as for the Popen constructor. If universal_newlines=True is passed, the “input” argument must be a string and stdout/stderr in the returned object will be strings rather than bytes.

New in version 1.2a1: This function first appeared in Python 3.5. It is available on all Python versions gevent supports.

class CompletedProcess(args, returncode, stdout=None, stderr=None)

Bases: object

A process that has finished running.

This is returned by run().

Attributes:
  • args: The list or str args passed to run().
  • returncode: The exit code of the process, negative for signals.
  • stdout: The standard output (None if not captured).
  • stderr: The standard error (None if not captured).

New in version 1.2a1: This first appeared in Python 3.5 and is available to all Python versions in gevent.

check_returncode()

Raise CalledProcessError if the exit code is non-zero.

exception CalledProcessError(returncode, cmd, output=None, stderr=None)

Bases: subprocess.SubprocessError

This exception is raised when a process run by check_call() or check_output() returns a non-zero exit status. The exit status will be stored in the returncode attribute; check_output() will also store the output in the output attribute.

stdout

Alias for output attribute, to match stderr

getstatusoutput(cmd)

Return (status, output) of executing cmd in a shell.

Execute the string ‘cmd’ in a shell with ‘check_output’ and return a 2-tuple (status, output). Universal newlines mode is used, meaning that the result with be decoded to a string.

A trailing newline is stripped from the output. The exit status for the command can be interpreted according to the rules for the function ‘wait’. Example:

>>> import subprocess
>>> subprocess.getstatusoutput('ls /bin/ls')
(0, '/bin/ls')
>>> subprocess.getstatusoutput('cat /bin/junk')
(256, 'cat: /bin/junk: No such file or directory')
>>> subprocess.getstatusoutput('/bin/junk')
(256, 'sh: /bin/junk: not found')
getoutput(cmd)

Return output (stdout or stderr) of executing cmd in a shell.

Like getstatusoutput(), except the exit status is ignored and the return value is a string containing the command’s output. Example:

>>> import subprocess
>>> subprocess.getoutput('ls /bin/ls')
'/bin/ls'
exception TimeoutExpired(cmd, timeout, output=None, stderr=None)

Bases: subprocess.SubprocessError

This exception is raised when the timeout expires while waiting for a child process.

Next page: gevent.thread – Implementation of the standard thread module that spawns greenlets