Skip to content

Instantly share code, notes, and snippets.

@taniomi

taniomi/transform_template.py

Last active March 14, 2025 14:35
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save taniomi/64d468cf96dbf0dcd8498e9a9a9104c4 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save taniomi/64d468cf96dbf0dcd8498e9a9a9104c4 to your computer and use it in GitHub Desktop.
Download ZIP
Transform: define data type template
Raw
transform_template.py
bit_cols = [
'active',
'anonymousResponse',
'deleted',
'isAdditionalQuestion'
]
datetime2_cols = [
'date',
'answerDate'
]
decimal_cols = [
'review'
]
int_cols = [
'quarter'
]
varchar_cols = [
'_id_answer',
'companyId'
]
df[bit_cols] = df[bit_cols].astype(bool)
df[datetime2_cols] = df[datetime2_cols].apply(lambda x: pd.to_datetime(x).dt.floor('s'))
df[decimal_cols] = df[decimal_cols].astype(float)
df[int_cols] = df[int_cols].apply(lambda col: col.astype('Int64')) # 'Int64' to allow NaN
df[varchar_cols] = df[varchar_cols].astype(str)
df.replace(to_replace='[]', value=None, inplace=True) # replace empty strings
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment