In this tutorial, you’ll learn 3 easy ways to read a file line by line in Python, including examples using readlines(), context managers, and while loops. By the end, you’ll be able to choose the best method for your specific project needs and implement it with confidence.
Python has made File I/O super easy for programmers. However, it is for you to decide what’s the most efficient technique for your situation. It would depend on many parameters, such as the frequency of such an operation, the size of the file, etc.
Let’s assume we have a logs.txt file that resides in the same folder along with the Python script.
3 Ways to Read a File Line by Line in Python
We’ll now go over each method to read a file line by line.
Read All Lines at Once with readlines()
We recommend this solution for files with a smaller size. If the file size is large, it becomes inefficient as it loads the entire file in memory.
However, when the file is small, it is easier to load and parse the content line by line.
The readlines() returns a sequence of all lines from the file each containing a newline char except the last one.
We’ve demonstrated the use of readlines() function in the below example. Here, you can see that we are also using the Python while loop to traverse the lines.
Example
"""
Example:
Using readlines() to read all lines in a file
"""
ListOfLines = ["Python", "CSharp", "PHP", "JavaScript", "AngularJS"]
# Function to create our test file
def createFile():
wr = open("Logs.txt", "w")
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
wr.close()
# Function to demo the readlines() function
def readFile():
rd = open ("Logs.txt", "r")
# Read list of lines
out = rd.readlines()
# Close file
rd.close()
return out
# Main test
def main():
# create Logs.txt
createFile()
# read lines from Logs.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()The output is as follows:
Python CSharp PHP JavaScript AngularJS
On the other hand, the above solution will cause high memory usage for large files. So, you should choose a different approach.
For instance, you may like to try this one.
Check this: Loop Through Files in a Directory
Read Line by Line with readline()
When the file size reaches MBs or GB, the right idea is to fetch one line at a time. Python readline() method does this job efficiently. It does not load all data in one go.
The readline() reads the text until the newline character and returns the line. It handles the EOF (end of file) by returning a blank string.
Example
"""
Example:
Using readline() to read a file line by line in Python
"""
ListOfLines = ["Tesla", "Ram", "GMC", "Chrysler", "Chevrolet"]
# Function to create our test file
def createFile():
wr = open("Logs.txt", "w")
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
wr.close()
# Function to demo the readlines() function
def readFile():
rd = open ("Logs.txt", "r")
# Read list of lines
out = [] # list to save lines
while True:
# Read next line
line = rd.readline()
# If line is blank, then you struck the EOF
if not line :
break;
out.append(line.strip())
# Close file
rd.close()
return out
# Main test
def main():
# create Logs.txt
createFile()
# read lines from Logs.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()After execution, the output is:
Tesla Ram GMC Chrysler Chevrolet
Efficient File Handling Using Context Managers
Python provides a concept of context managers. It involves using the “with” clause with the File I/O functions. It keeps track of the open file and closes it automatically after the file operation ends.
Hence, we can confirm that you never miss closing a file handle using the context manager. It performs cleanup tasks like closing a file accordingly.
In the below example, you can see that we are using the context manager (with) along with the Python for loop to first write and then read lines.
Example
"""
Example:
Using context manager and for loop read a file line by line
"""
ListOfLines = ["NumPy", "Theano", "Keras", "PyTorch", "SciPy"]
# Function to create test log using context manager
def createFile():
with open ("testLog.txt", "w") as wr:
for line in ListOfLines:
# write all lines
wr.write(line)
wr.write("\n")
# Function to read test log using context manager
def readFile():
rd = open ("testLog.txt", "r")
# Read list of lines
out = [] # list to save lines
with open ("testLog.txt", "r") as rd:
# Read lines in loop
for line in rd:
# All lines (besides the last) will include newline, so strip it
out.append(line.strip())
return out
# Main test
def main():
# create testLog.txt
createFile()
# read lines from testLog.txt
outList = readFile()
# Iterate over the lines
for line in outList:
print(line.strip())
# Run Test
if __name__ == "__main__":
main()
After running the code snippet, the output is:
NumPy Theano Keras PyTorch SciPy
📚 Want to dive deeper into File I/O? Check out our comprehensive tutorial on file handling in Python for more tips and examples!
Summary: Which Method Should You Use?
In this tutorial, we explored three efficient approaches for handling files in Python: using readlines(), context managers, and while loops. Each technique has its strengths, making it suitable for different scenarios:
- readlines() is perfect for quick and simple file reading.
- Context managers ensure clean and resource-efficient code.
- while loops offer flexibility, especially for processing large files.
Now that you’ve mastered these methods, it’s time to put them into practice!
If you found this guide helpful:
1️⃣ Share it on LinkedIn or Twitter to help others discover it.
2️⃣ Subscribe to our YouTube channel for more Python tutorials and coding tips!
Enjoy Coding,
TechBeamers
