Everything about the python subprocess module

Official Documentation Link: https://docs.python.org/3/library/subprocess.html

Use case scenario:

1. When you write your logics in a python package/module, which required to grab the outputs from another program (or just the Linux command).

TL;DR summary:

1. Two ways to run, first is the recommended approach: subprocess.run() will return a CompletedProcess instance. Second way for more advanced use cases is to use the underlying Popen interface directly.

Understanding the args option:

1. capture_output=True is equivalent to setting stdout=PIPE and stderr=PIPE. You cannot supply both at the same time.
2. check=true will raised a CalledProcessError exception when the process exits with a non-zero exit code
3. timeout can be specified in seconds to kill the child process and raise the TimeoutExpired exception after the timeout expires. Good safeguard for potential bugs (like endless loop) of the child process
4. text=True and/or encoding=’utf-8' . for opening file objects for stdin, stdout and stderr as string mode (default is byte)
5. shell=True. Will run the command in shell environment. One significant effect it has is how you construct the arguments.

Passing the args correctly:

1. You can pass the whole args as a string. This will required setting shell=True (which use default shell as /bin/sh). Then, the string must be formatted exactly as it would be when typed at shell prompt.
2. You can split the args into a sequence of arguments. For correct tokenization for args you will need to use the shlex.split() method. Note if you use str.split() method unexpected tokens may be produced when trying to parse args with quotation (‘ or “). This may cause a problem in running the cmd if the program you called is sensitive to the string literal quote.

#Example, assuming you are running in linux os environment
import subprocess
import shlex

def test_diff_splits(name: str):
    try:
        cmd = f'''echo "test-splits-'affect',-hello-{name}"'''
        # set shell=True
        result = subprocess.run(cmd, shell=True,  check=True, 
                capture_output=True, text=True)
        print(f'args for shell=true: {result.args}')
        print(f'output for shell=true: {result.stdout}')

        # use str.split()
        result1 = subprocess.run(cmd.split(' '),  check=True, 
                capture_output=True, text=True)
        print(f'args for str.split(): {result1.args}')
        print(f'output for str.split(): {result1.stdout}')

        # use shlex.split()
        result2 = subprocess.run(shlex.split(cmd),  check=True, 
                capture_output=True, text=True)
        print(f'args for shlex.split(): {result2.args}')
        print(f'output for shlex.split(): {result2.stdout}')

    except Exception as es:
        print(f'error:{es}')

Example console outputs. note the difference in string literal (‘ and “) in args and output