Created
March 23, 2015 21:34
-
-
Save amZotti/201248290a9c3e678c86 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| { | |
| "metadata": {}, | |
| "nbformat": 3, | |
| "nbformat_minor": 0, | |
| "worksheets": [ | |
| { | |
| "cells": [ | |
| { | |
| "cell_type": "html", | |
| "metadata": {}, | |
| "source": [ "<h1>Using Plotly, Node.js, and MongoDB together to Create TimeSeries Graphs</h1>\n", | |
| "I recently learned about Plotly and used it to make a <a href='https://github.com/amZotti/TweetTracker.js'>awesome data visualizations of Twitter data</a>.\n", | |
| "Plotly is a free web-based platform for making graphs. You can keep graphs private, make them public, and run <a href= 'https://plot.ly/product/enterprise/'>Plotly on your own servers.</a>\n", | |
| "Plotly integrates very well with Node.js and MongoDB. I will take this time to discuss how I\n", | |
| "configured MongoDB and Plotly together, and how those two entities interacted with each\n", | |
| "other. I will be going step by step so you can follow along with me!</p>\n", | |
| "<h2>Step 1: Install MongoDB</h2>\n", | |
| "<p>Installing MongoDB is refreshingly easy. Here I will outline the steps neccessarry to get MongoDB up and\n", | |
| "running. <b>Note</b>: <i>This guide assumes you are running OSX with <a href='http://brew.sh/'>Homebrew</a>\n", | |
| "installed. If difficulties arise please refer to <a href='http://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/'>the\n", | |
| "Mongo documentation.</a></i></p>\n", | |
| "<ol>\n", | |
| " <li>\n", | |
| " <i>Update homebrew</i>\n", | |
| " <pre>brew update</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>Install MongoDB</i>\n", | |
| " <pre>brew install mongodb</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " <a href='http://docs.mongodb.org/manual/tutorial/install-mongodb-on-os-x/'>\n", | |
| " Specify a directory for datastore:\n", | |
| " </a>\n", | |
| " </i> \n", | |
| " <pre>mkdir -p /data/db</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i> \n", | |
| " <a href='http://docs.mongodb.org/manual/reference/program/mongod/#bin.mongod'>\n", | |
| " Run MongoDB\n", | |
| " </a> \n", | |
| " </i>\n", | |
| " <pre>mongod</pre></li>\n", | |
| "</ol>\n", | |
| "<h2>Step 2: Configure MongoDB for Node.js</h2>\n", | |
| "<p>If you do not already have <a href=https://nodejs.org/'>Node.js</a>installed, then the first thing you are\n", | |
| "going to want to do is install it via homebrew by simply typing <code>brew install\n", | |
| "node</code> in terminal. Assuming you have Node.js successfully installed on your\n", | |
| "machine, all we need to do is simply create a new Node.js app and download the\n", | |
| "MongoDB drivers for Node via <a href='https://www.npmjs.com/'>npm</a>.</p>\n", | |
| "<ol>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " Create a new directory for our awesome Node.js app\n", | |
| " </i>\n", | |
| " <pre>mkdir appName</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " cd into directory\n", | |
| " </i>\n", | |
| " <pre>cd appName</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " Initialize npm\n", | |
| " </i>\n", | |
| " <pre>npm init</pre> \n", | |
| " After you will receive a series of prompts asking you for additional information for setup. Just keep pressing\n", | |
| " enter.\n", | |
| " </li>\n", | |
| " <img src='http://i.imgur.com/9V1T0dB.png' />\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " Install mongoDB client for Node\n", | |
| " </i>\n", | |
| " <pre>npm install mongodb --save</pre>\n", | |
| " </li>\n", | |
| " <img src='http://i.imgur.com/8l5vdyn.png'/>\n", | |
| "</ol>\n", | |
| "<h2>Step 3: Configure Plotly for Node.js</h2>\n", | |
| "<ol>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " Install plotly adapter for Node.js from root directory of app\n", | |
| " </i>\n", | |
| " <pre>npm install plotly</pre>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " Create a new file in app directory named <code>app.js</code>\n", | |
| " </i>\n", | |
| " <pre>touch app.js</pre>\n", | |
| " <p>Since we are going to be using Plotly API to create awesome graphs, now\n", | |
| "would be a good time to create a <a href='https://plot.ly/ssi/'>free plotly\n", | |
| "account</a>. After logging into your plotly account navigate to <a\n", | |
| "href='https://plot.ly/settings/api'>API settings</a>. You will need the provided\n", | |
| "Username and API Key in the next step. From there open the <code>app.js</code>\n", | |
| "file we previously created in your favorite editor of choice.</p>\n", | |
| " </li>\n", | |
| " <li>\n", | |
| " <i>\n", | |
| " require plotly package and fill in private API information\n", | |
| " </i>\n", | |
| " <pre>var plotly = require('plotly')(<i>your API Username here</i>,Your API Key here</i> );</pre>\n", | |
| " </li>\n", | |
| "</ol>\n", | |
| "<h2>Step 4: Generating Sample Data with MongoDB</h2>\n", | |
| "<p>We now have a file named <code>app.js</code>; this is going to be where we\n", | |
| "pull data from MongoDB and use it to create awesome graphs in real time with\n", | |
| "Plotly API. But first we need to fill MongoDB with some sample data to make our\n", | |
| "awesome graphs with. Create a new file in the app directory named\n", | |
| "<code>testFill.js</code> and include the following code</p>\n", | |
| "<b>testFill.js</b>\n", | |
| "<pre>\n", | |
| "var MongoClient = require('mongodb').MongoClient;\n", | |
| "var tweets = [ \n", | |
| " {'message': 'I love barbeque', 'time': '10:44'},\n", | |
| " {'message': 'When I had it, so I was drunk last night I bought a lizard.', 'time': '10:44'},\n", | |
| " {'message': 'Finale - thoughts? Check out - an amazing project. and Ed Sheerans latest album please', 'time': '10:44'},\n", | |
| " {'message': 'A surface preparation? During smoking, Collagen gets turned into Gelatin creating a rub into Gelatin.', 'time': '10:45'},\n", | |
| " {'message': 'Please sign up at to use the free Twitter Calls service Please sign up at to use the free Twitter Calls.', 'time': '10:46'},\n", | |
| " {'message': 'Sunday and my happy times.', 'time': '10:46'},\n", | |
| " {'message': 'Daaaaaaang work to tease me = Omg = do kids hang out digits', 'time': '10:46'},\n", | |
| " {'message': 'Payday loans in ga loan today all 191 sourcesRelated', 'time': '10:47'}\n","]\n", | |
| "MongoClient.connect('mongodb://127.0.0.1:27017/test10', function(err, db) {\n", | |
| " var data = db.collection('data');\n", | |
| " for (var i = 0;i < tweets.length;i++) {\n", | |
| " data.insert(tweets[i], function(err, success) {\n", | |
| " if (err)\n", | |
| " console.log(err);\n", | |
| " else\n", | |
| " console.log(success);\n", | |
| " });\n", | |
| " }\n", | |
| "});\n", | |
| "</pre>\n", | |
| "<p>After running this code, if you were to go into mongoDB at this point and check the database you\n", | |
| "persisted the <code>data</code> collection into, you would see the list of tweet\n", | |
| "objects we created.</p>\n", | |
| "<img src='http://i.imgur.com/FK9DVUA.png'/>\n", | |
| "<p><i>Note: In the examples I will be showing you we will be using database <code>test10</code></i></p>\n", | |
| "<h2>Step 5: Integrating Plotly and MongoDB</h2>\n", | |
| "<p>Now, let's get back into <code>app.js</code> and use this new Tweet data to\n", | |
| "make an awesome <a href='https://plot.ly/nodejs/time-series/'>time series\n", | |
| "graph</a></p>\n", | |
| "<b>app.js</b>\n", | |
| "<pre>\n", | |
| "var plotly = require('plotly')(<i>your API Username here</i>,Your API Key here</i> );\n", | |
| "var MongoClient = require('mongodb').MongoClient;\n", | |
| "MongoClient.connect('mongodb://127.0.0.1:27017/test10', function(err, db) {\n", | |
| " var data = db.collection('data')\n", | |
| " data.find().toArray(function(err, tweets) {\n", | |
| " var tweetData = formatTweetData(tweets);\n", | |
| " var data = [{x: tweetData.timeDate, y: tweetData.frequency, type: 'scatter'}];\n", | |
| " var graphOptions = {\n", | |
| " 'filename': 'TweetTrackr', \n", | |
| " 'fileopt': 'overwrite', \n", | |
| " 'layout': {\n", | |
| " 'title': 'Ratio of tweets per minute'\n", | |
| " },\n", | |
| " 'world_readable': true\n", | |
| " };\n", | |
| " plotly.plot(data, graphOptions, function (err, msg) {\n", | |
| " if (err) return console.log(err);\n", | |
| " console.log(msg);\n", | |
| " });\n", | |
| " });\n", | |
| "});\n", | |
| "function formatTweetData(tweets) {\n", | |
| " var timeCount = {};\n", | |
| " var frequency = [];\n", | |
| " var timeDate = [];\n", | |
| " for (var i = 0;i < tweets.length;i++) {\n", | |
| " var time = tweets[i].time;\n", | |
| " timeCount[time] = timeCount[time] || 0;\n", | |
| " timeCount[time]++;\n", | |
| " }\n", | |
| " for (var key in timeCount) {\n", | |
| " frequency.push(timeCount[key]);\n", | |
| " timeDate.push(key);\n", | |
| " }\n", | |
| " return {frequency: frequency, timeDate: timeDate};\n", | |
| "}\n", | |
| "</pre>\n", | |
| "<p>The above code will retrieve all entries in our MongoDB <code>data</code>\n", | |
| "collection and convert all that data into an array. After that we call\n", | |
| "the <code>formatTweetData</code> helper function on our array of tweet objects.\n", | |
| "We need the function <code>formatTweetData</code> because the Plotly API\n", | |
| "requires us to provide two arrays. The first array represents all the x coordinates,\n", | |
| "and the second array represents all the y coordinates. There is a one-to-one\n", | |
| "mapping between each element in the two arrays. The first element in the array\n", | |
| "which represents the x coordinate should correspond to the first element in the\n", | |
| "array which represents the y coordinate. This is the line of code where this\n", | |
| "magic happens:\n", | |
| "<pre>\n", | |
| " var data = [{x: tweetData.timeDate, y: tweetData.frequency, type: 'scatter'}];\n", | |
| "</pre>\n", | |
| "As you can see, we are also including <code>type: 'scatter'</code>. This is\n", | |
| "where we specify the type of graph we want. After we put our graph data in the\n", | |
| "correct format all you need to do is create our <code>graphOptions</code>object\n", | |
| "and pass it all to <code>plotly.plot</code> along with a callback. The callback\n", | |
| "will return an object with a <code>url</code> property.</p> \n", | |
| "<img src='http://i.imgur.com/g3rrD5O.png?1'/>\n", | |
| "<p>This property contains the url where the graph will be accessible.</p>\n", | |
| "<img src='http://imgur.com/FET85TO.png'/>\n" | |
| ] | |
| }] | |
| }] | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment