How to Read File Line by Line in Python – Definitive Guide

Python provides inbuilt libraries to handle files operation such as create, read, update, delete from the Python application.

You can read file line by line in python using the readlines() method.

In this tutorial, you’ll learn how to open and read files line by line in python using different methods.

If You’re in Hurry…

Here is the example to read file line by line into the list.

with open("full_file_path_With_extenstion",  'r', encoding='utf8' ) as f:
    content = f.readlines()
print(content)

If You Want to Understand Details, Read on…

Before you start reading the file, you need to open the file in Python. You can use open() method to open the file.

Files can be opened in different modes by specifying the parameters.

"r" – Opens the file in read Mode.

"a" -Opens file in Append mode. Creates a file if it’s not existing

"w" – Opens file in write mode. Creates a file if it’s not existing

"x" – Creates the file and opens it for processing. throws an error if a file with the same name already exists.

You can also specify the encoding of the file while opening it.

Encoding is a representation of a set of characters which can be very useful when you handle files with special characters like Ä in German language or something other language characters.

Sample File contents

This is first line
this is second line

This is third line after empty line

This is fourth line after empty line

This is fifth line with special character Ä

#this is a commented line which should be ignored while reading

This is sixth line after a commented line

Open File Syntax

open("file_name_with_extension", 'r', encoding='utf8') as f:

where

  • file_name_with_Extension – Full path of the file name
  • r – To open the file in the read mode. This is optional and file will be opened in read mode by default.
  • encoding – Specify the encoding of the file which can be used to read the file with correct charsets. utf8 is the most commonly used encoding.

Now the file will be opened and stored in the f file object.

Using ReadLines()

readlines() method is used to read one complete line from the file. It appends \n character at the end of each line read.

Syntax

file.readlines(sizehint)

Parameters

It accepts an optional parameter sizehint.

If you specify sizehint, whole lines totaling to sizehint bytes will be read instead of reading up to the end of the file.

Read file line by line into list

In this section, you’ll learn how to read the file line by line into a list with open file statement and readlines().

First, open the file with an open statement and create a file object.

Then use readlines() method in the file object to read the complete lines of the file.

It’ll read the file line by line and return a list as shown in the below example.

Example

with open("c:\temp\Sample_File.txt",  'r', encoding='utf8' ) as f:
    content = f.readlines()
print(content)

Output

['This is first line\n', 'this is second line\n', '\n', 'This is third line after empty line\n', '\n', 'This is fourth line after empty line\n', '\n', 'This is fifth line with special character Ä\n', '\n', '#this is a commented line which should be ignored while reading\n', '\n', 'This is sixth line after a commented line']

Read file line by line into array

In this section, you’ll learn how to read the file line by line into an array with open file statement and readlines().

Python doesn’t have inbuilt support for arrays. But the lists can be used instead, which is similar to the array.

It can be done using the read() method and assigning it into an array as shown in the below example.

Example

with open("c:\temp\Sample_File.txt",  'r', encoding='utf8' ) as f:
    filecontentasarray = f.read()
print(filecontentasarray)

Output

This is first line
this is second line

This is third line after empty line

This is fourth line after empty line

This is fifth line with special character Ä

#this is a commented line which should be ignored while reading

This is sixth line after a commented line

Read file line by line into tuple

In this section, you’ll learn how to read files line by line into tuples.

You can pass the file object directly into the tuple constructor while creating a file object using the open statement.

With this, it’ll yield a tuple with the lines from the file and you need not use the readlines() or read() method explicitly.

When to use

Tuples are unchangeable and ordered.

You can use it when you want to create an unchangeable record from the file and the order of the lines must be maintained.

Example

linesastuple = tuple(open("c:\temp\Sample_File.txt", 'r'))
print(linesastuple)

Output

('This is first line\n', 'this is second line\n', '\n', 'This is third line after empty line\n', '\n', 'This is fourth line after empty line\n', '\n', 'This is fifth line with special character Ä\n', '\n', '#this is a commented line which should be ignored while reading\n', '\n', 'This is sixth line after a commented line')

Read file line by line into set

In this section, you’ll learn how to read file line by line into Set. This is useful to remove duplicate lines in a file while reading it.

For this, you’ll use read() method available in the file object. Because readlines() method returns the lines appended with \n character making it difficult to identify the duplicate lines.

Hence, use the read() method and splitlines() that splits each line into a separate item in a list.

Then this list can be used to create a Set to avoid duplicate lines automatically.

When to use

sets are unordered and unindexed and it’ll not allow duplicate values.

You can use it when you want to create a unique collection of lines from a file by eliminating the duplicate lines.

Example

with open("c:\temp\Sample_File.txt",  'r', encoding='utf8' ) as f:
    listcontent = f.read().splitlines()

thisset = set (listcontent)

print(thisset)

Output

    {'This is fifth line with special character Ä\n', 'This is first line\n', 'This is sixth line after a commented line', '\n', 'This is fourth line after empty line\n', 'this is second line\n', 'This is third line after empty line\n', '#this is a commented line which should be ignored while reading\n'}

Read file line by line into DataFrame

In this section, you’ll learn how to read a file line by line into pandas DataFrame.

Pandas DataFrame is a two-dimensional data structure used to store the data in the tabular format.

You can use read_csv() method available in pandas to read a text file as well.

Syntax

pd.read_csv('file_name', encoding='utf8',header=None)

where

  • File_name – Full path of the file name to be read.
  • encoding=’utf8′ – Specify the encoding of the file which can be used to read the file with correct charsets. utf8 is the most commonly used encoding.
  • header=None – To specify to pandas that the header should not be created with the first row of the file.

This will read the file line by line and create a dataframe with each line in the first column and the line numbers as the indexes of the DataFrame. You can refer rename column of pandas dataframe if you want to set the column for the Pandas DataFrame.

Example

import pandas as pd
df = pd.read_csv('c:\temp\Sample_File.txt', encoding='utf8',header=None)
print(df)

Output

                                                   0
0                                 This is first line
1                                this is second line
2                This is third line after empty line
3               This is fourth line after empty line
4        This is fifth line with special character Ä
5  #this is a commented line which should be igno...
6          This is sixth line after a commented line
7          This is sixth line after a commented line

Read file line by line ignoring comments

In this section, you’ll learn how to read file line by line in a loop and ignore the comments in the file.

Comments lines are denoted by using the # character at the beginning of each file.

You’ll use the startswith() method to check if the line is having the # character at the beginning.

Before using the startswith() method, it is also advisable to strip() the string to remove unwanted space characters from the beginning and end of the string.

Example

for line in open("c:\temp\Sample_File.txt", encoding='utf8'):
    li=line.strip()
    if not li.startswith("#"):
        print(line.rstrip())

You can see that the commented line is not printed unlike the output in the previous section.

Output

This is first line
this is second line

This is third line after empty line

This is fourth line after empty line

This is fifth line with special character Ä

This is sixth line after a commented line

This is how you can read the file line by line and ignore the comments in the file.

Read file line by line with index

In this section, you’ll learn how to read specific lines from a file by using line numbers. This is also called reading files line by line using an index.

For this, first, you’ll read the file with the file object and the readlines() method. It’ll return the list where each line will be available in each index.

You can read that specific line with index by passing the line number to the list as shown below.

The index is 0 based, and use 0 if you want to read the first line.

Example

f=open('c:\temp\Sample_File.txt', encoding='utf8')
lines=f.readlines()
print(lines[1])
print(lines[3])

Output

this is second line

This is third line after empty line


You can also use the below method if you want to read any specific line directly without reading and holding the complete lines in the memory.

This will read only the second line from the file and store it in the line a variable that can be used in your program.

Example

line = open('c:\temp\Sample_File.txt', encoding='utf8').readlines()[1]
print(line)

Output

this is second line

This is how you can read file line by line using its index.

Read file line by line backwards

In this section, you’ll learn how to read files line by line backward or in reverse order.

First, you’ll read the file into the list and use the reversed method available in the list to iterate the list items in the reverse order.

reversed() method will return a revered iterator object which can be iterated using the for loop and access the file contents from the reverse order.

Example

for line in reversed(list(open('c:\temp\Sample_File.txt', encoding='utf8'))):
    print(line.rstrip())

Output

This is sixth line after a commented line

#this is a commented line which should be ignored while reading

This is fifth line with special character Ä

This is fourth line after empty line

This is third line after empty line

this is second line
This is first line

This is how you can read a file line by line in python backward or read a file from the end of the file. There is no specific method available to read files in the reverse order directly.

Conclusion

In this tutorial, you’ve learned how to read files line by line in python in different available ways.

If you know a different way other than the information available here, feel free to comment below.

You May Also Like

How To Read Excel With Multiple Sheets In Pandas?

Leave a Comment