Introduction to Python Subprocess
The process is defined as a running program, each process has its own different states like its memory, list of open files, etc. As we know that every process executes one statement at a time. In Python versions 2 and 3 there are several different ways to run the process externally to interact with the operating system. Earlier versions had os modules to do this but now in the latest versions, these os modules are replaced by subprocess modules to call external commands from Python code. Subprocess modules are independent because they run completely independent entities in their private system state and execute the main thread concurrently with the original process.
Working of Python Subprocess with Syntax and Example
In the latest versions of Python as it supports the subprocess module instead of the old os modules. The process that creates the subprocess work with other things where the subprocess carries out its work even though the process is working with something else. The subprocess module is used to run new programs through Python code by creating a new process that helps to gain input or output or error pipes and also helps to gain exit codes of various commands.
The Subprocess provides a high-level interface than the older os modules like os.system(), os.spawn(), os.popen(), etc these all are replaced with a new spawn process, connect to their input or output or error pipes and also to gain their return codes.
subprocess.call (args, *, stdin=None, stdout=None, stderr=None, shell=False)
- args: this is used when you want the commands to be executed.
- stdin: it carries the value of standard input to be passed.
- stdout: it carries the value of output which is obtained by standard output stream.
- stderr: this carries the error obtained from the standard error stream.
- shell: It returns a Boolean value. True when commands get executed.
So you can run the command using args parameter, wait until the command gets executed and then it will return the returncode attribute if its zero then it returns the function else it raises CalledProcessError.
Let us take that we want to execute echo command which is UNIX command in the Python script, but this will give us an error saying its syntax error as the echo is not a built-in command in Python. Echo is a UNIX command to print the value or statement whereas in Python we have printed as a command to print the value or statement.
So to run these UNIX commands in Python we have to create a new process that can be done using subprocess modules. This was earlier done using os.system() but this was only for the older versions instead we can use subprocess.call() which executes a program as a child process in which we cannot complete our needs. So we need to use open in advanced cases as popen is a class and not just a method.
It is as shown below:
subprocess.Popen(‘ls –la’, shell = True)
The above program can also use a subprocess.call() but this executes the program as a child process which is executed by the operating system as a separate process so as per the Python documentation it is recommended to use popen class. It can be written as
subprocess.call(‘ls –la’, shell =True)
In the above both the programs shell argument will take ‘false’ as default but we want the commands to be executed then we have to assign shell argument as ‘True’.
If the subprocess is not imported then as discussed earlier no commands can be executed externally in Python codes so to do this we need to use subprocess modules. To do this we need to first import the subprocess else you will get the error saying no such file is available which means the ls command cannot be executed in Python as its UNIX command. It is shown in the below screenshot.
In subprocess documentation, you get a list of methods of popen class few of them are popen.wait() which will wait for the child process to get terminate, popen.kill() to kill the child process, popen.terminate() stops the child process, popen.communicate() this is the most commonly used method which helps to interact with the process. Let us consider about communicate() method which allows reading and sending data from and to the standard input and standard output respectively.
Let us take an example to execute windows commands like dir and sort. So there are two commands to execute these commands we have to create two subprocesses as we want the directories to be sorted in reverse order in which we use ‘\R’ option in sort call. In the below program we use stdout of p1 as input to p2 so we declare PIPE in stdout for p1. So we have to close p1 as we have used it as input to p2. And as usual, the communication is done using the communicate() method.
p1 = subprocess.Popen(‘dir’, shell =True, stdin = None, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
p2 = subprocess.Popen('sort /R', shell=True, stdin=p1.stdout)
out, err = p2.communicate()
From the above example, it has two subprocess dir and sort commands to be executed through Python script as given above and the dir command process is an input to the sort command process to sort the obtained directories in reverse order.
In Python, versions 2 and 3 have separated ways of calling commands externally through the Python code. In Python 2 we had operating system (os) methods as a better way to call the child process of other languages through the Python script. However, in the present version, it uses the subprocess module which has several methods. Popen is one of the most used classes in both versions but in the latest versions, it again has many other methods in it to make it very efficient. Subprocess also has a call(), check_stdout(), check_stdin() are also some of the methods of a subprocess which are used instead of Popen class’s method communicate().
This is a guide to Python Subprocess. Here we discuss the Working of Python Subprocess with appropriate syntax and respective example. You may also have a look at the following articles to learn more –