How To Append a Pandas Dataframe Into An Existing CSV File – Definitive Guide

CSV files store values in a comma-separated format.

You can append the pandas dataframe into an existing CSV file using the to_csv(‘file_name’, ‘a’) method in the append mode.

This tutorial teaches you how to append a pandas dataframe into an existing CSV file and handle headers, indexes and separators while appending.

If you’re in Hurry

Use the following code to append a pandas dataframe df into an existing csv file called addresses.csv.

df.to_csv('addresses.csv', mode='a')

If You Want to Understand Details, Read on…

Create a sample dataframe and append it to an existing CSV file using the various methods explained below.

Creating Dataframe To Append

Create a sample dataframe that needs to be appended to the existing CSV file.

• Ensure that the dataframe has the same column headers as in the existing CSV file to avoid incorrect data.

Code

import pandas as pd

df = pd.DataFrame({'First Name': ['Vikram'],
                   'Last Name': ['Aruchamy'],
                   'Address:': ['Chepauk, second street'],
                   'City': ['Coimbatore'],
                   'State': ['TamilNadu'],
                   'Zip Code': ['600100']})
df

DataFrame Will look like

	First Name	Last Name	Address:	City	State	Zip Code
0	Vikram	Aruchamy	Chepauk, second street	Coimbatore	TamilNadu	600100

Using to_CSV in Append Mode

The to_csv() method writes the pandas dataframe to a CSV file.

To append the dataframe to an existing file,

• Use the parameter mode=‘a’, which means append.
• This ensures the data is appended to an existing CSV file.

Code

df.to_csv('addresses.csv', mode='a')

CSV File Will Look Like

The data is appended with the header information and the index information.

First Name,Last Name,Address,City,State,Zip Code
John,Doe,120 jefferson st.,Riverside, NJ, 08075
,First Name,Last Name,Address,City,State,Zip Code
0,Vikram,Aruchamy,"Chepauk, second street",Coimbatore,TamilNadu,600100

Handling Header While Appending

The to_CSV() method adds the pandas dataframe header to the CSV file while using the append mode.

To append the dataframe to CSV without header information,

• Use the parameter header=None.
• This ignores the headers from the dataframe and appends only the data.

Code

df.to_csv('addresses.csv', mode='a', header=None)

CSV File Will Look Like

No headers are appended. Only the existing headers of the file are available.

First Name,Last Name,Address,City,State,Zip Code
John,Doe,120 jefferson st.,Riverside, NJ, 08075
0,Vikram,Aruchamy,"Chepauk, second street",Coimbatore,TamilNadu,600100

Handling Index While Appending

The to_CSV() method adds the pandas dataframe indexes to the CSV file while using the append mode.

To append the dataframe to CSV without index information,

• Use the parameter index =None.
• This ignores the indexes from the dataframe and appends only the column values.

Code

df.to_csv('addresses.csv', mode='a', index=None, header=None)

CSV File Will Look Like

Index is not appended in the new line.

First Name,Last Name,Address,City,State,Zip Code
John,Doe,120 jefferson st.,Riverside, NJ, 08075
Vikram,Aruchamy,"Chepauk, second street",Coimbatore,TamilNadu,600100

Handling Separator

The to_CSV() method uses , as the default separator while writing/appending to a CSV file.

To use a different separator,

• Use the parameter sep = ';'

Code

df.to_csv('addresses.csv', mode='a', header=No‘;’ sep = ';')

CSV File Will Look Like

Data is appended with the ; separator.

First Name,Last Name,Address,City,State,Zip Code
John,Doe,120 jefferson st.,Riverside, NJ, 08075
0;Vikram;Aruchamy;Chepauk, second street;Coimbatore;TamilNadu;600100

Pandas To CSV Mode Options

There are different modes supported in the to_csf() method.

• ‘w’ – open for writing, truncating the file first
• ‘x’ – open for exclusive creation, failing if the file already exists
• ‘a’ – open for writing, appending to the end of the file if it exists
• ‘b’ – binary mode
• ‘t’ – text mode (default)

View CSV after Appending

Additional Resources

• How To Read a CSV file Without a Header in Pandas
• How to write pandas dataframe to CSV