How to run terminal commands in Python

You can run terminal commands from a Python script to automate system tasks and integrate external tools. Python provides several modules that execute shell commands directly from your code.

In this article, you'll learn several techniques to run commands, with practical tips for real-world applications. You'll also get debugging advice to help you write robust and efficient scripts.

Using os.system() for basic command execution

import os
os.system('echo "Hello from the terminal!"')--OUTPUT--Hello from the terminal!
0

The os.system() function provides a direct way to execute a shell command. It passes the command string to your system's standard shell, which runs it in a subshell. The command's output, in this case, "Hello from the terminal!", is printed directly to your console.

After the command finishes, os.system() returns the command's exit code. An exit code of 0 typically signifies success. While this method is simple, it doesn't capture the command's output into a variable, making it less flexible for scripts that need to process the results.

Using the subprocess module

The subprocess module offers a more robust alternative to os.system(), giving you greater control over how commands are executed and their results are handled.

Running commands with subprocess.run()

import subprocess
result = subprocess.run(['ls', '-l'], capture_output=True, text=True)
print(result.stdout)--OUTPUT--total 8
-rw-r--r-- 1 user user 123 Jul 10 14:23 example.py
drwxr-xr-x 2 user user 4096 Jul 10 14:22 test_folder

The subprocess.run() function gives you more control over command execution. Notice the command and its arguments are passed as a list, like ['ls', '-l']. This approach is more secure because it helps prevent shell injection vulnerabilities.

• The capture_output=True argument is key—it tells Python to grab the command's output so you can use it in your script.
• Using text=True ensures the captured output is a regular string instead of bytes, which makes it easier to work with.

The function returns a CompletedProcess object, which holds details about the finished command. You can then access the captured output through the stdout attribute.

Capturing output with subprocess.check_output()

import subprocess
output = subprocess.check_output(['date'], text=True)
print(f"Current date and time: {output.strip()}")--OUTPUT--Current date and time: Thu Jul 10 14:25:30 UTC 2023

For situations where you only need a command’s output, subprocess.check_output() offers a more direct approach. It simplifies your code by returning the output string directly, so you don't need to access it through an object attribute like result.stdout.

• The main distinction is its built-in error handling. If the command fails by returning a non-zero exit code, check_output() automatically raises a CalledProcessError.
• This makes it ideal for scripts where you expect a command to succeed and want to halt execution immediately if it doesn't.

Executing commands with arguments using subprocess.Popen()

import subprocess
process = subprocess.Popen(['find', '.', '-name', '*.py'], stdout=subprocess.PIPE, text=True)
stdout, stderr = process.communicate()
print(stdout.split('\n')[:3]) # Print first three results--OUTPUT--['./example.py', './test_folder/test.py', '']

For complex interactions, subprocess.Popen() offers maximum flexibility. It launches a command in a new process and continues your script's execution without waiting for it to finish. This non-blocking approach is ideal for managing long-running tasks or complex I/O pipelines.

• Use stdout=subprocess.PIPE to create a channel for capturing the command's output.
• Call process.communicate() to read from the output stream and wait for the process to terminate, ensuring your script gets the results when they're ready.

Advanced techniques

With the fundamentals of the subprocess module covered, you can now handle more advanced scenarios like asynchronous execution, custom environments, and robust error handling.

Executing commands asynchronously with asyncio

import asyncio

async def run_command():
proc = await asyncio.create_subprocess_shell('echo "Async command execution"', stdout=asyncio.subprocess.PIPE)
stdout, _ = await proc.communicate()
print(stdout.decode().strip())

asyncio.run(run_command())--OUTPUT--Async command execution

The asyncio module lets you run commands non-blockingly, which is ideal for I/O-heavy tasks. Your script can continue running other operations while it waits for a command to complete, making your code more efficient.

• The async and await syntax is central to this process. You use await asyncio.create_subprocess_shell() to start the command without halting your program.
• To retrieve the results, you await proc.communicate(), which pauses execution only within the async function until the command is finished and its output is ready.

Passing environment variables to commands

import subprocess
import os

env = os.environ.copy()
env['CUSTOM_VAR'] = 'Hello from environment variable'
result = subprocess.run('echo $CUSTOM_VAR', shell=True, env=env, capture_output=True, text=True)
print(result.stdout.strip())--OUTPUT--Hello from environment variable

You can pass specific settings or secrets to a command by creating a custom environment for it. This isolates the variables to just that command, which avoids altering your main script's environment.

• First, make a copy of the current environment with os.environ.copy().
• Next, add or modify variables in that copy, like setting CUSTOM_VAR.
• Finally, use the env parameter in subprocess.run() to execute your command within this custom environment, making your variables accessible to it.

Handling errors and timeouts with subprocess

import subprocess

try:
result = subprocess.run(['ls', 'nonexistent_directory'], check=True, capture_output=True, text=True, timeout=5)
except subprocess.TimeoutExpired:
print("Command timed out")
except subprocess.CalledProcessError as e:
print(f"Command failed with error code {e.returncode}: {e.stderr.strip()}")--OUTPUT--Command failed with error code 2: ls: cannot access 'nonexistent_directory': No such file or directory

Robust scripts anticipate problems. You can gracefully handle command failures and prevent your script from hanging by wrapping subprocess.run() in a try...except block. This lets you catch specific issues that might come up during execution.

• Setting check=True is crucial—it makes subprocess.run() raise a CalledProcessError if the command fails. You can then inspect the error object for details like the returncode and stderr.
• The timeout parameter sets a time limit in seconds. If the command exceeds it, a TimeoutExpired exception is raised, stopping your script from waiting indefinitely.

Move faster with Replit

Replit is an AI-powered development platform that comes with all Python dependencies pre-installed, so you can skip setup and start coding instantly. Instead of piecing together individual techniques, you can use Agent 4 to build complete applications from a simple description.

Describe the app you want to build, and the Agent will take it from an idea to a working product. It can write the code, connect to databases, integrate APIs, and handle deployment. For example, you could ask it to build:

• A file organizer that uses shell commands to find all images in a directory, sorts them by date created, and moves them into corresponding monthly folders.
• A system monitoring utility that runs commands like df -h and free -m to generate a daily report on server disk and memory usage.
• An automated deployment script that executes a series of build commands and then uses rsync or scp to push the files to a production server.

Simply describe your app, and Replit will write the code, test it, and fix issues automatically, all within your browser.

Common errors and challenges

Even with the right tools, you might run into a few common issues when executing terminal commands in your Python scripts.

Handling FileNotFoundError when commands don't exist

A FileNotFoundError usually means the command you're trying to run isn't installed or can't be found in your system's PATH. The PATH is a list of directories your shell searches for executables. To fix this, ensure the command is installed and its location is in the PATH, or provide the full path to the executable in your script.

Avoiding command injection with shell=True

Using shell=True with external input creates a security risk known as command injection, where malicious commands can be passed into your script. It's much safer to pass the command and its arguments as a list of strings. This method bypasses the shell's interpretation, preventing unintended commands from running.

Resolving UnicodeDecodeError with proper encoding

A UnicodeDecodeError occurs when a command's output uses an encoding that Python can't read with its default—usually UTF-8. This is common with tools that produce output in different character sets. You can solve this by specifying the correct format, like encoding='latin-1', in your subprocess call to ensure the text is decoded properly.

Handling FileNotFoundError when commands don't exist

You'll likely encounter a FileNotFoundError if the command you're calling doesn't exist on your system or isn't in the shell's PATH. Python raises this error because it can't find the specified executable. See what happens when you try it.

import subprocess

# Will raise FileNotFoundError if 'nonexistent-command' is not installed
result = subprocess.run(['nonexistent-command', '--version'])

Because nonexistent-command isn't a real executable, subprocess.run() can't find it and raises the error. You can adapt your script to handle this possibility gracefully. The following example shows you how.

import subprocess
import shutil

# Check if command exists before running it
command = 'nonexistent-command'
if shutil.which(command):
result = subprocess.run([command, '--version'])
else:
print(f"{command} is not installed or not in PATH")

You can prevent a FileNotFoundError by checking if a command exists before you try to run it. The shutil.which() function searches the system's PATH for the executable. If it returns a path, you know the command is available. If it returns None, you can print a helpful message instead of letting the script crash. This is a robust way to handle dependencies on external tools that may not be installed on a user's system.

Avoiding command injection with shell=True

Using shell=True with user input is a major security risk because it allows external data to be interpreted as part of the command. This vulnerability, known as command injection, can be exploited by malicious actors to run unintended, harmful commands on your system.

The following code demonstrates this dangerous practice in action.

import subprocess

# DANGEROUS: User input directly in command with shell=True
filename = input("Enter filename to list: ")
subprocess.run(f"ls -l {filename}", shell=True)

If a user provides input like my_file.txt; rm -rf /, the shell will run the destructive rm command. This happens because shell=True processes the entire string, including command separators. The following example demonstrates a safer implementation.

import subprocess

# SAFE: Arguments passed as list elements
filename = input("Enter filename to list: ")
subprocess.run(["ls", "-l", filename])

By passing the command and its arguments as a list, like ["ls", "-l", filename], you ensure the user's input is treated as a single, literal argument rather than executable code. This prevents the shell from interpreting any malicious commands hidden in the input string. You should always use this list-based technique whenever your script incorporates external data to completely avoid command injection vulnerabilities and keep your application secure.

Resolving UnicodeDecodeError with proper encoding

Resolving UnicodeDecodeError with proper encoding

A UnicodeDecodeError can pop up when your script tries to interpret a command's output using the wrong character encoding. This often happens because Python defaults to UTF-8, but some commands might return text in a different format. The following code shows how this error can occur when you try to decode the output without specifying an encoding.

import subprocess

# May fail with UnicodeDecodeError on systems with non-UTF-8 locales
result = subprocess.run(['ls', '-la'], capture_output=True)
output = result.stdout.decode() # No encoding specified
print(output)

The error happens because result.stdout.decode() tries to read the output using the default system encoding. If the command's output uses a different format, Python can't interpret it correctly. The following code demonstrates the proper fix.

import subprocess
import locale

# Get system's preferred encoding
system_encoding = locale.getpreferredencoding()
result = subprocess.run(['ls', '-la'], capture_output=True)
output = result.stdout.decode(system_encoding)
print(output)

To fix this, you can dynamically find the correct encoding instead of guessing. The locale.getpreferredencoding() function gets your system's default character set. By using this encoding when you call result.stdout.decode(), you ensure the output is read correctly. This is especially useful on systems with non-English language settings, where command outputs might not use the standard UTF-8 format. This approach prevents your script from crashing due to mismatched encodings.

Real-world applications

Now that you can confidently run commands and handle errors, you can apply these skills to automate powerful, real-world workflows through vibe coding.

Using Python to automate Git operations with subprocess

The subprocess module lets you integrate Git directly into your Python scripts, making it easy to automate version control tasks like checking the repository status.

import subprocess

def get_git_status():
result = subprocess.run(['git', 'status', '--porcelain'], capture_output=True, text=True)
if result.stdout.strip():
print("Changes detected in repository:")
print(result.stdout)
else:
print("Repository is clean, no changes detected.")

get_git_status()

This function automates checking your Git repository's status. It uses subprocess.run() to execute git status with the --porcelain flag, which formats the output specifically for easy parsing by scripts.

• The script captures the command's output into the result.stdout attribute.
• An if statement then checks if the output string is empty after removing whitespace with .strip().
• If the string contains any text, it means there are pending changes, and they're printed to the console. Otherwise, it confirms the repository is clean.

Monitoring log files with subprocess

You can use subprocess to keep an eye on application activity by running commands like tail, which reads the most recent entries from log files.

import subprocess
import os

def check_for_new_logs(log_file):
if not os.path.exists(log_file):
return "Log file does not exist"

# Get last 3 lines from log file
result = subprocess.run(['tail', '-n', '3', log_file], capture_output=True, text=True)
return result.stdout.strip()

# Check latest logs
print("Latest log entries:")
print(check_for_new_logs("server.log"))

This function, check_for_new_logs, offers a quick way to view the latest entries in a log file. It starts by using os.path.exists() to confirm the file is present, preventing potential errors if it's missing. This initial check makes the script more reliable.

• The core of the function uses subprocess.run() to execute the command tail -n 3, which fetches the last three lines of the specified file.
• It captures the command's output and returns it as a clean string, thanks to .strip() removing any leading or trailing whitespace.

Get started with Replit

Put your knowledge into practice and create a real tool. Describe what you want to Replit Agent, like "build a script that uses git log to generate a weekly changelog" or "create a utility that pings a list of servers."

Replit Agent will write the code, test for errors, and deploy your application, turning your description into a finished product. Start building with Replit.