Skip to content

Instantly share code, notes, and snippets.

@bradybd

bradybd/engineering.md

Forked from matflores/engineering.md
Last active August 29, 2015 14:19
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save bradybd/2e86d934ed4faffa507c to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save bradybd/2e86d934ed4faffa507c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
engineering.md

Engineering: Redesign Elasticsearch/Redis health checks to improve performance of all M2X requests

Engineering: Design & Implement robust & reliable messaging solution between data centers using RabbitMQ

Engineering: Redesign workers messaging and queuing (possibly migrating them from Ost-based queues to RabbitMQ queues)

Engineering: Unify datapoint writer and trigger evaluator queues (avoid queueing each datapoint twice).

Engineering: Optimize datapoint writer worker

Engineering: Adjust number of workers and background processes based on load testing results.

Engineering: Bootstrap logging management system

Engineering: Review logging needs for all app and system components

Engineering: Review & improve logging generation and management

Engineering: Add metrics generation & management for all application & system components (find new alternative to NewRelic)

Engineering: Synchronize developer portal sessions across datacenters

Engineering: Multi-stream data modelling

Engineering: Jobs API

Engineering: Background jobs monitoring interface

Engineering: Refactor all datapoint writing methods to avoid hitting ES (it requires that users, keys and device IDs are all cached in Redis, and therefore it requires RabbitMQ to be able to invalidate caches stored in a different DC). This prioritizes access to writing time series data over any other API methods (performance-wise).

Engineering: Refactor all datapoint writing methods to give the option to the user of checking the status of this operation (i.e. return a Location header along with the 202 response pointing to a url which users can ping to check the status of this transaction).

Engineering: Continue improving deployment workflow

Engineering: Improve device's last activity storage

Engineering: Redesign app settings configuration

Engineering: Automate Elasticsearch templates configuration

Engineering: Cache frequent responses in Redis to improve performance

Engineering: Automate Elasticsearch mappings configuration

Engineering: Extract datapoint writing routes into its own router/microservice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment