Python Copy Files from a Directory

Copying a file from one directory to another is a common programming task. Python has many modules (like os, subprocess, and sh-util) to support file copy. From this post, you get nine unique ways to copy a file to another directory in Python. So, let’s get started.

Why the Need for Copy File Operation?

Before you begin, you must understand why is it so important to know which copy file method in Python is best for you. This is because file I/O operations are performance-intensive and often lead to bottlenecks. That’s why you should opt for the best possible approach as per the design of your application.

Some programs which use shared resources would prefer to copy a file in blocking mode whereas some may want to do it asynchronously. For example – using a thread to copy a file or starting a separate process to do it. Another point to consider is the platform portability. It means you should know the target OS (Windows/Linux/Mac OS X etc.) wherever you’ll run the program.

With the Shutil module, you can automate copying both the files and directories. This module follows an optimized design. It saves you from doing time-intensive operations like the opening, reading, writing, and closing of a file when there is no real processing need. It is full of utility functions and methods that can let you do tasks like copying, moving, or removing files and folders.

9 Unique Ways to Copy a File in Python

Here are the nine methods to demonstrate “How to copy a file in Python?”.

Shutil’s CopyFile() Method

This method copies the content of the source to the destination only if the target is writable. If you don’t have the right permissions, then it will raise an IOError.

It works by opening the input file for reading while ignoring its file type.

Next, it doesn’t treat special files any differently and won’t create their clones.

The copyfile() makes use of the lower-level function copyfileobj() underneath. It takes file names as arguments, opens them, and passes file handles to Python copyfileobj(). There is one optional third argument in this method which you can use to specify the buffer length. It’ll then open the file for reading in chunks of the specified buffer size. However, the default behavior is to read the entire file in one go.

Check out the syntax for the Python copy file method.

# Syntax
copyfile(source_file, destination_file)

Following are the points to know about the copyfile() method.

• It copies the contents of the source_file to file named destination_file.
• If the destination isn’t writable, then the copy operation would result in an IOError exception.
• It will return the SameFileError if both the source and destination files are the same.
• However, if the destination pre-exists with a different name, then the copy will overwrite its content.
• Error 13 will occur if the destination is a directory which means this method won’t copy to a folder.
• It doesn’t support copying files such as characters or block devices and pipes.

In order to learn how to use the copy file method, review and run the below Python example.

# Copy a File using Shutil copyfile() method
        
from shutil import copyfile
from sys import exit

with open("hello.txt", mode="w") as file:
    file.write("Hello, World!")

source = "hello.txt"
target = "welcome.txt"

# adding exception handling
try:
    copyfile(source, target)
except IOError as e:
    print("Unable to copy file. %s" % e)
    exit(1)
except:
    print("Unexpected error:", sys.exc_info())
    exit(1)

print("\nFile copy done!\n")

while True:
    print("Do you like to print the file ? (y/n): ")
    check = input()
    if check == 'n':
        break
    elif check == 'y':
        file = open(target, "r")
        print("\nHere follows the file content:\n")
        print(file.read())
        file.close()
        print()
        break
    else:
        continue

Shutil’s Copy() Method

# Syntax
copyfile(source_file, [destination_file or dest_dir])

In Python, if you want to copy files to another directory, use this method.

The copy() method functions like the “cp” command in Unix. It means if the target is a directory, then it’ll copy the file inside it with the same name (basename) as the source file. Also, this method will sync the permissions of the target file with the source after copying its content. It too throws the SameFileError if you are copying the same file.

Copy/paste the following Python example and see how to copy a file to another directory.

# Python Copy File using shutil.copy() method

import os
import shutil

assert not os.path.isabs(source)
target = os.path.join(target, os.path.dirname(source))

# create the folders if not already exists
os.makedirs(target)

# adding exception handling
try:
    shutil.copy(source, target)
except IOError as e:
    print("Unable to copy file. %s" % e)
except:
    print("Unexpected error:", sys.exc_info())

We hope it is now clear how to call the shutil's copy() file method in a Python program.

Some notes on copy() vs copyfile() –

1. The copy() also sets the permission bits while copying the contents whereas the copyfile() only copies the data.
2. The copy() will copy a file if the destination is a directory whereas the copyfile() will fail with error 13.
3. Interestingly, the copyfile() utilizes the copyfileobj() in its implementation, whereas the copy() method makes use of the copyfile() and copymode() functions in turn.
4. Point 3 makes it apparent that copyfile() would be a bit faster than the copy() as the latter has an additional task (preserving the permissions) at hand.

Shutil’s CopyFileObj() Method

This method copies the file to a target path or file object. If the target is a file object, then you need to close it explicitly after calling the copyfileobj(). It assumes an optional argument (the buffer size) which you can use to supply the buffer length. It is the number of bytes kept in memory during the copy process. The default size that the system uses is 16KB.

Let’s now test the shutil's copy file object method with the help of this simple Python code.

# Python Copy File using shutil.copyfileobj()

from shutil import copyfileobj
status = False
if isinstance(target, string_types):
    target = open(target, 'wb')
    status = True
try:
    copyfileobj(self.stream, target, buffer_size)
finally:
    if status:
        target.close()

Shutil’s Copy2() Method

The copy2() method works almost similarly to copy(). But, in addition, it also gets the access and modification times added to the meta-data while copying the data. Copying the same file would result in SameFileError.

Let’s prepare a short Python script to demonstrate the use of the copy2() method in file copy.

# Copy a File using shutil.copy2() method

from shutil import *
import os 
import time
from os.path import basename

def displayFileStats(filename):
    file_stats = os.stat(basename(filename))
    print('\tMode    :', file_stats.st_mode)
    print('\tCreated :', time.ctime(file_stats.st_ctime))
    print('\tAccessed:', time.ctime(file_stats.st_atime))
    print('\tModified:', time.ctime(file_stats.st_mtime))

os.mkdir('test')

print('SOURCE:')
displayFileStats(__file__)

copy2(__file__, 'testfile')

print('TARGET:')
displayFileStats(os.path.realpath(os.getcwd() + './test/testfile'))

Some notes on copy() vs copy2()

1. The copy() function only sets permission bits whereas copy2() also updates the file metadata with timestamps.
2. The copy() method calls copyfile() and copymode() internally whereas copy2() replaces the call to copymode() with copystat().

Follow differences between copymode() and copystat()

shutil.copymode()

# Syntax
copymode(source, target, *, follow_symlinks=True)

• It intends to copy the permission bits from the source to the target files.
• The file contents, owner, and group remain unchanged. The arguments passed are strings.
• If the follow_symlinks arg is false and the first two args are symlinks, then copymode() will try to update the target link, not the actual file it is pointing to.

shutil.copystat()

# Syntax
copystat(source, target, *, follow_symlinks=True)

• It attempts to preserve the permission bits, last used time/update time, and flags of the target file.
• The copystat() includes the extended attributes while copying on Linux. The file contents/owner/group remain unchanged.
• If the follow_symlinks arg is false and the first two args are symlinks, then copystat() will update them, not the files they point to.

Please note if you want good results from a tutorial, then not only read or see its code but also modify the code and run it with different cases. After that, you’ll surely see positive improvements.

Os’s Popen() Method

This method creates a pipe to or from the command. It returns an open file object which connects to a pipe. You can use it for reading or writing according to the file mode, i.e., ‘r’ (default) or ‘w.’

# Syntax
os.popen(command[, mode[, bufsize]])

• mode – It can be ‘r’ (default) or ‘w’.
• bufsize – If its value is 0, then no buffering will occur. If the bufsize is 1, then line buffering will take place while accessing the file. If you provide a value greater than 1, then buffering will occur with the specified buffer size. However, for a negative value, the system will assume the default buffer size.

With the popen(), it is just one line of Python code you need to copy a file from one location to another. Here, we bring you two versions of the code, one for Windows and the second for Linux.

For Windows OS.

import os

os.popen('copy 1.txt.py 2.txt.py')

For Linux OS.

import os

os.popen('cp 1.txt.py 2.txt.py')

Os’s System() Method

The system() method allows you to instantly execute any OS command or script in the subshell.

You need to pass the command or the script as an argument to the system() call. Internally, this method calls the standard C library function.

Its return value is the exit status of the command. You can see that it works in a similar fashion as does the popen() method.

For Windows OS.

import os

os.system('copy 1.txt.py 2.txt.py')

For Linux OS.

import os

os.system('cp 1.txt.py 2.txt.py')

Copy File using Thread Lib

If you want to copy a file asynchronously, then use the below method. In this, we’ve used Python’s threading module to run the copy operation in the background.

While using this method, please make sure to employ locking to avoid deadlocks. You may face it if your application is using multiple threads reading/writing a file.

import shutil
from threading import Thread

src="1.txt.py"
dst="3.txt.py"

Thread(target=shutil.copy, args=[src, dst]).start()

Subprocess’s Call() Method

The subprocess module gives a simple interface to work with child processes. It enables us to launch subprocesses, attach them to their input/output/error pipes, and retrieve the return values.

The subprocess module aims to replace the legacy modules and functions like – os.system, os.spawn*, os.popen*, popen2.*.

It exposes a call() method to invoke system commands to execute user tasks. Here is an example illustrating the use of the subprocess module and its call() method.

import subprocess

src="1.txt.py"
dst="2.txt.py"
cmd='copy "%s" "%s"' % (src, dst)

status = subprocess.call(cmd, shell=True)

 if status != 0:
     if status < 0:
         print("Killed by signal", status)
     else:
         print("Command failed with return code - ", status)
else:
     print('Execution of %s passed!\n' % cmd)

Subprocess’s Check_OutPut() Method

With the subprocess’s check_output() method, you can run an external command or a program and capture its output. It also supports pipes.

Here is another example demonstrating how to use both OS and Subprocess modules for copying a file in Python.

import os, subprocess

src=os.path.realpath(os.getcwd() + "./1.txt.py")
dst=os.path.realpath(os.getcwd() + "./2.txt.py")
cmd='copy "%s" "%s"' % (src, dst)

status = subprocess.check_output(['copy', src, dst], shell=True)

print("status: ", status.decode('utf-8'))

Hello all, it is important to realize the advantage of self-learning. But, it works best when you just not only follow the guidance but also try to improvise. From our end, we made sure that we give you the best and fully working code. But it doesn’t mean you limit yourself to the given coding help. Instead, use this knowledge to gain more skills and experience.

Before You Leave

Even if you are a seasoned Python programmer, you might have liked the different methods to copy a file to another directory. Python is so rich in features that one can quickly utilize them to perform a single task in many ways.

The os, subprocess, and shutil modules in Python bear tremendous support for file I/O operations. However, here, we’ve only covered the Python file copy methods. But you’ll see a lot of new and intuitive ways to do other things as well in our future posts.

Lastly, our site needs your support to remain free. Share this post on social media (Linkedin/Twitter) if you gained some knowledge from this tutorial.

Enjoy coding,
TechBeamers.