How to rename nested json fields in Dataframe
I have json events of the following structure:
{ 'atlas_ts': '', 'ev': { '_a': { 'build': '', 'name': '', 'version': '' },
'_abtest': '',
'_c': {
'referrer': '',
'utm_campaign': '',
'utm_content': '',
'utm_medium': '',
'utm_source': '',
'utm_term': ''
}
}
}
I want to rename the columns of the dataframe created. I want to replace ev._a to ev.app; ev._c to ev.campaign; ev._a.name to ev.app_name. How can I do this.
You specify a schema when created the DataFrame if you want to retain the structure of the json records. If you're planning to flatten the records, you can use the .withColumnRenamed() api.
9 People are following this question.
