Skip to content

Instantly share code, notes, and snippets.

@wikytam

wikytam/merge_csv.py

Last active July 14, 2023 02:45
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save wikytam/c1073eac1a4e92379075b7dfee4b4200 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save wikytam/c1073eac1a4e92379075b7dfee4b4200 to your computer and use it in GitHub Desktop.
Download ZIP
Merge CSV file using Python
Raw
merge_csv.py
import os
import pandas as pd
if __name__ == '__main__':
csv_files = [file for file in os.listdir('excel') if file.endswith('.csv')]
dfs = []
for file in csv_files:
file_path = os.path.join('excel', file)
df = pd.read_csv(file_path)
dfs.append(df)
merged_df = pd.concat(dfs, axis=0, join='outer', ignore_index=True, keys=None, levels=None, names=None,
verify_integrity=False, copy=True)
merged_df.to_csv('merged_file.csv', index=False)
@wikytam
Copy link
Copy Markdown
Author

wikytam commented Jul 14, 2023 •
edited
Loading

Merge xlsx file using Python

Requirements:

  • The script requires that all input Excel files (.xlsx) be located in a folder named excel.

  • The output file should be named output_file.xlsx.

  • The script will merge all worksheets with the same name across all input Excel files into a single worksheet in the output file.

import os
import pandas as pd
from glob import glob

if __name__ == '__main__':
    folder_path = 'excel'

    file_list = glob(os.path.join(folder_path, '*.xlsx'))

    sheet_dict = {}
    for file_name in file_list:
        with pd.ExcelFile(file_name) as xls:
            for sheet_name in xls.sheet_names:
                df = pd.read_excel(xls, sheet_name=sheet_name)
                if sheet_name in sheet_dict:
                    sheet_dict[sheet_name].append(df)
                else:
                    sheet_dict[sheet_name] = [df]

    with pd.ExcelWriter('output_file.xlsx') as writer:
        for sheet_name in sheet_dict:
            merged_df = pd.concat(sheet_dict[sheet_name], sort=False)
            merged_df.to_excel(writer, sheet_name=sheet_name, index=False)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment