This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import sys | |
| from awsglue.transforms import * | |
| from awsglue.utils import getResolvedOptions | |
| from pyspark.context import SparkContext | |
| from awsglue.context import GlueContext | |
| from awsglue.dynamicframe import DynamicFrame | |
| from awsglue.job import Job | |
| args = getResolvedOptions(sys.argv, ['JOB_NAME']) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| clear-host; | |
| #Remove-Variable * -ErrorAction SilentlyContinue | |
| #get-item Variable:* | |
| #Get-Variable | Select-Object -ExpandProperty Name | |
| . C:\Users\linda_l\Desktop\PowerShell\GoogleOauth.ps1 | |
| Add-Type -Path "C:\Users\linda_l\Documents\visual studio 2015\Projects\TestingLibrary\packages\AE.Net.Mail.1.7.10.0\lib\net45\AE.Net.Mail.dll" | |
| Add-Type -AssemblyName System.IO | |
| Add-Type -AssemblyName System.Text.Encoding |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| //var csv is the CSV file with headers | |
| function csvJSON(csv){ | |
| var lines=csv.split("\n"); | |
| var result = []; | |
| var headers=lines[0].split(","); | |
| for(var i=1;i<lines.length;i++){ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| db.collection.find({ field : { $exists : true } }).forEach( function (x) { | |
| var temp = {} | |
| var count = 0 | |
| for (var i in x.field) { | |
| if (typeof x.field[i] == "string") { | |
| // make sure index is a string | |
| temp[x.field[i]] = x.date; | |
| count++; | |
| } |