This gist is a collection of my rough notes from Strange Loop 2012.

Follow me on Twitter to get updates as they're posted.

I'm posting these notes immediately after each talk. Expect typos, formatting glitches, incomplete thoughts, and ...

Michael Stonebraker

Abstract

--- rough notes ---

• "not your father's transaction processing"
• how does this fit into big data?
  • big volume - I have too much data
  • big velocity - data is coming at me too fast
  • big variety
• focusing on high velocity today
• in 1985, 1,000 transactions/second seemed like an amazing stretch goal
• today, you can do that on your iPhone
• TP (Transaction Processing) is now a much broader problem (New TP)
  • massively multiplayer games
  • social networking
  • real time ad placement (i.e., you have 1 millisecond to decide which ad to show me)
  • real time couponing
  • etc.
  • sensor tagging generates new TP applications
    • marathon runners
    • taxicab
    • dynamic traffic routing
    • car insurance "by the drink"
    • mobile social networking
    • ...
    • and TP volumes are ginormous!!
    • serious need for speed and scalability
  • wall street electronic trading
  • real-time fraud detection
  • micro transactions (e.g., buying a soda with your iPhone)
• In all cases
  • workload is a mix of updates and queries
  • coming at you like a firehose
  • still and ACID problem
    • don't lose my data
    • make sure it's correct
  • tends to break traditional solutions
• put differently
  • you need to ingest a firehose in real-time
  • you need to process, validate, enrich and respond in real-time (i.e., update)
  • you often need real-time analytics
• if your data doesn't fit in main memory now, then wait a couple of years and it will
  • yes, Facebook is an exception; you are not Facebook
  • main memory is going down in price faster than the size of TP data is growing
• 2007 paper "Through the OLTP Looking Glass" found that traditional databases (e.g., Oracle, DB2, etc.) spend less than 10% of their time doing "useful work." The rest is the overhead of record-level locking, latching, recovery, and buffer pool management.
• How do we go faster?
  • Main memory deployment gets rid of buffer pool (which eliminates 25% of the overhead); leaves other 75% of overhead intact
• Solution Choices
  • OldSQL - legacy RDBMS vendors
    • Code lines date from the 1980s
    • 30 years worth of "bloatware"
    • Mediocre performance on New TP
    • Slow because they spend all of their time on overhead
    • Would have to re-architect their legacy code to do better
    • They all face the "The Innovators Dilemma"
    • They'll ultimately drift off into the sunset
  • NoSQL - Give up SQL and ACID for performance
    • Give up SQL? That's throwing away 30 years of RDBMS experience.
    • "Stored procedures are good! One round trip from app to DBMS rather than one round trip per record. Move the code to the data, not the other way around." (Editor's note: Hmm. Datomic anyone?)
    • Give up ACID? Can you guarantee that you won't need it tomorrow?
    • Eventual consistency does not mean "eventual consistency"; eventual consistency means "creates garbage"
    • Eventual consistency only works if the changes are allowed to happen in any order (i.e., commutative)
    • Appropriate for:
      • non-transactional systems
      • single record transactions that are commutative
  • NewSQL
    • Preserve SQL and ACID
    • Get performance from a new architecture
    • Scale by running on a cluster of nodes
    • Automatic sharding (parallelism)
    • Focus on OLTP workload
      • a few high volume transaction signatures; implement as stored procedures
      • occasional ad-hoc transactions
    • Issue #1: Buffer pool; Solution: Run in memory
    • Issue #2: Write ahead log; Solution: replication and tandem-style failover (and fail back); You need HA anyway for New TP
    • Issue #3: Multithreading; Solution: Don't do it, or get rid of all shared data structures
    • Issue #4: Record-level locking; Solution: run to completion in timestamp order => no locking
• VoltDB is NewSQL
  • Open source
  • 70x faster than OldSQL, running on the same hardware
  • 5x faster than Cassandra on VoltDB key-value layer
  • Runs a subset of SQL
  • Scales linearly to 384 cores

Stuart Sierra

Abstract

--- rough notes ---

• The concept of design patterns have been largely overlooked in dynamic languages.
  • Norvig (1998) - 16 of the 23 patterns in the GoF book have qualitatively simpler implementation in Lip or Dylan than in C++ for at least some uses of each pattern
• "Are there any Haskell programmers in the audience? Yeah: you're gonna be pissed." (i.e., not gonna talk about monads)
• Pattern-Oriented Software Architecture
  • categories
    • architectural patterns
    • design patterns
    • idioms (specific to a single programming language)
  • we're going to focus somewhere in the middle between design patterns and idioms

• State/event pattern
  • about
    • state is derived from previous state + input
    • many divers inputs
    • need to recover past states
    • need to visualize intermediate states
  • implementation
    • update-state function takes the current state and an event and returns the new state
  • can recreate any past state by reducing over events
  • great for logging (i.e., you store every event that happened in the system)
  • offers great flexibility for how you will store state in your system
    • one extreme: never store state; you can always rebuild it from events
• Consequences pattern
  • about
    • each event can trigger multiple events
    • generated events cause state changes
    • need to visualize intermediate states
  • implementation
    • function takes the current state and an event, and returns a sequence of consequences
    • the consequences might be more events
  • consequence functions do not compose naturally; you have to update state in between

• Accumulator pattern
  • large collection of inputs; maybe larger than memory
  • small or scalar result
  • lazy sequences (i.e., map, mapcat, filter, etc.)
  • reduce is the universal accumulator
• Map-reduce pattern
  • input is linear; maybe larger than one disk
  • motivated by things that were historically true, but are becoming less so:
    • disks are (were?) slow and local to one machine
    • networks are (were?) slow
  • prediction that map-reduce will become less important as the constraints above become less and less true
  • now quite map and reduce
• Reduce/combine Pattern
  • example:
    • Clojure reducers
      • Reducers - A Library and Model for Collection Processing)
      • Anatomy of a Reducer
  • operates in parallel
• Recursive expansion pattern
  • build up result out of primitives
  • build abstractions in layers
  • recuse until no more work left to do
  • e.g., macroexpansion, Dataomic transaction functions

• Pipeline pattern

  • process with many discrete steps

  • similar "shape" of data at each step; usually a map or record

  • only one execution path

  • example:

      (defn large-process [input]
        (-> input
            subprocess-a
            subprocess-ab
            subprocess-c))
    
      (defn subprocess-a [data]
        ; ...
        )
    

  • useful to be able to easily the exact order of the steps; show it to business users to vet that you've ordered it correctly?

• Wrapper pattern
  • about
    • process with many discrete steps
    • one main execution path
    • possible branch at each step
  • Clojure's Ring library is an example
  • implementation
    • input: a function
    • output: a function that does something before and/or after the given function; might not call the given function at all
• Token pattern
  • about
    • may need to cancel an operation, but ...
    • the operation itself is not an identity
  • implementation
    • create a fn that performs the operation
    • returns a fn that allows you to cancel (i.e., undo) the operation
  • examples:
    • the scheduled thread pool in Java
    • Clojure watches
• Observer pattern
  • yes, it's a GoF pattern
  • register an observer fn with stateful container
  • examples:
    • Clojure watches
• Strategy pattern
  • yes, another GoF pattern
  • many processes with a similar structure
  • need extension points for future variations
  • examples:
    • Clojure protocols; very similar to the original OO approach
    • Clojure multimethods; dispatch on input

Gary Bernhardt

Abstract

--- rough notes ---

Gary introduces a side project that Gary has been working on part-time for a little over a year, including ...

• modal (like Vim)
• terminal only
• "much more powerful than Vim"
• not an IDE
• layers
  • overlay an orthogonal "layer" on top of the source code
  • examples:
    • diff layer
    • crash layer => overlay backtrace onto the source code
    • performance layer => overlay time profiling information onto the source code
• interactions
  • the user experience:
    • one keystroke shows you all the code that the current line interacts with
    • renders a graphical tree-like display in the terminal
    • can navigate through the graph
    • uses GraphViz
  • Useful questions you can answer
    • what code does this test hit?
    • what code does this request hit?
    • what code might hit this crash point?

• raster graphics
• 24-bit color
• italics, bold, underline
• momentary keypresses

And, we've all been punk'd. All lies. None of the things above actually exist.

• We were surprised (right?) when Gary said that he wrote his own terminal. Why? Why is it surprising that someone would write a terminal?
  1. shipping culture => our "shipping culture" is poisonous to infrastructure replacement, even when the infrastructure sucks
  2. legacy and paralysis
     • we overlook the things that have existed our entire programming careers (e.g., terminals)
     • we overlook their limitations and whether those limitations make sense in today's world
• Advocating for thinking, hammock time, prototyping
• Our "shipping culture", incremental changes don't allow for fundamental improvements (i.e., rethinkings) like the ones Gary described above

Editor's note: With Light Table, Catnip, aneditor/anterminal, and Bret Victor's upcoming talk, it seems like "Re-imagining Your Development Environment" will be one of the themes of Strange Loop 2012.

Michael Feathers

Abstract

--- rough notes ---

• Revised talk title? "Using Haskell Type Signatures as a Functional Design Notation"

• Haskell type signatures useful for thinking about design

• Even in Ruby code, Michael finds himself adding Haskell-esque type signatures as comments (i.e., documentation for methods)

     String -> [String] -> [[String]] -> [String] -> String
  

• Look at the shape of your data and how each function transforms the data; doing so will influence how you want to name your functions

  • Example 1
  • Example 2

• Focuses on data

• Think of problems as transformations from one thing to another thing

Robert Pitts

Abstract

--- rough notes ---

• Ruby gives us many of the tools we need to write code in a more functional style
  • Enumerable includes most (all?) of your favorite HOFs (e.g., #map, #reduce, #filter, etc.)
  • First(ish) class functions
    • blocks, procs, lambdas, methods
    • Currying and partial application (via Proc.curry added in 1.9)
  • Continuations (via callcc)
  • Tail call optimization (as of Ruby 1.9)
    • not enabled by default
    • enabled by passing args when launching Ruby
  • Facets (added in 1.9)
• hamster - persistent, efficient, immutable, lazy data structures in Ruby
• stunted - not yet ready for production use
• celluloid - actor-based concurrent object framework for Ruby
• monads (e..g, ick)

Editor's note: If this topic interests you, you might also enjoy Alan Dipert's "Functional Programming in Ruby" slides from Ruby Hoedown 2011.

David Nolen

Abstract

--- rough notes ---

• Reiterating one of the key motivations for ClojureScript: JavaScript has reach.
• Arms race among Google, Apple, Mozilla to have the fastest JS runtime. As a result, we (developers) win.
• Argues that no browser (other than Chrome) will ever adopt Dart. [A]
• "Lisp programmers know the value of everything and the cost of nothing." — Alan Perlis
  • And that's unfortunate, because efficiency is important (because performance is important)
• ClojureScript is expression oriented (i.e., not a mix of statements and expressions)
• ClojureScript's advantages over vanilla JavaScript
  • persistent data structures
  • functions are values
  • protocols
  • namespacing
  • browser-connected REPL
  • macros
• Translations from JavaScript to ClojureScript
• Admittedly, adopting ClojureScript does have some costs
  • debugging [B]
  • a lot of code generated for trivial programs (e.g., "Hello World")
  • some components of Clojure not yet present in ClojureScript [C]
  • not bootstrapped; requires Clojure
  • uses JavaScript arithmetic underneath (which is inferior); does not use Clojure numerics
  • performance enhancements are not evenly distributed (e.g., keywords, multimethods)
• Live demo of spectralnorm benchmark
  • Java: 9 seconds
  • JavaScript (on V8): 12 seconds
  • Dart-optimized JavaScript (on V8): 13 seconds
  • ClojureScript-generated JavaScript (on V8): 13 seconds
• Live demo of using core.logic in ClojureScript
• Q & A
  • Is ClojureScript in use at the New York Times? No.
  • Source maps are coming along. Stay tuned.
  • What helps your dev workflow? lein-cljsbuild

[A] Editor's note: However, Dart can compile to JavaScript. (See notes from Lars Bak's talk.)

[B] Editor's note: Debugging is probably better than you expect. It's better than I expected.

[C] Editor's note: Check out the "Differences from Clojure" wiki page for a good breakdown.

Amanda Laucher and Paul Snively

Abstract

--- rough notes ---

• Logic programming is in the air
  • Datalog
  • MiniKanren
  • core.logic
• Assumptions for this talk
  • you've used a REPL
  • logic programming is upon us; see above
  • some languages are better suited for correct code; admittedly a potentially controversial assumption
  • we spend way more time reading code than writing it
  • we are lazy bastards!
• Quotes
  • "When it doubt, create a type." — Martin Fowler
  • "Make illegal states unrepresentable." — Yaron Minsky
  • Michael Feathers defines legacy code as "code without automated tests"
  • "In 5 years, we'll view compilation as the weakest form of unit testing." — Stuart Halloway
  • "Given a good test suite, the return on investment simply does not justify the use of static typing." — Jay Fields
• Stereotypes
  • It's easier to refactor with tests that types
  • No tests = no trust
  • Tests take a long time to run and types to compile
  • Property-based testing can replace unit testing
  • Modular design only occurs with TDD
  • I don't get errors than can be prevented by types
  • Ivory tower vs. hippies
  • 100% test coverage or bust [A]
  • Typed code is verbose
  • Types take too long
  • Testing is for QA
• They individually worked through code katas. In an effort to see the other side of the "tests vs types" debate, they each approached the kata in ways that were different than their usual approach.
• Amanda coming from an F# background (strong emphasis on testing than types); Paul coming from a Scala background (stronger emphasis on types)
• Amanda's approach:
  • F#
  • Signatures first
  • Types first (as opposed to tests first)
  • All algorithmic development via REPL experimentation
  • Tests for validation
• Amanda's lessons learned:
  • types saved me from even having to think about certain categories of tests
  • tests help me out when I get stuck, but I mostly run them in the REPL and delete
  • structured her code differently to make it more friendly to work with in the REPL
  • most modern languages don't have a strong enough type system to make illegal state unrepresentable
  • it's easy to get lost in a space where you never deliver
  • tests verify when types can't prove
• Paul's approach:
  • Scala
  • Tests without types (as opposed to preferring types first)
  • Property-based testing
  • Types
  • Delete some tests
• Paul's lessons learned
  • even for such a small problem, spelling out unit tests made me want to gouge out my eyeballs with a rusty spoon; 32 unit tests for just 4 small user stories
  • when developing property-based tests, every forAll made me think, "could/should that be a type?"
  • for some use cases, having examples of correct input/output gave no real guidance whatsoever
  • test code is still code, with its own correctness, maintenance, etc. burden
• Amanda's and Paul's discussion (i.e., things they seemed to agree upon) as they worked through these katas:
  • small codebase = little value for the type system
  • types scale better than tests (i.e., scaling as the code base grows)
  • types have little value when talking with non-technical users
  • the hardest part is understanding the requirements
  • we are rarely handed sample input/output as part of the requirements (which makes it harder to write tests) [B]
  • tests can be good for forming ideas, but then can be deleted
• Paul's and Amanda's kata solutions will be up on GitHub soon

[A] Editor's note: Testing Anti-Patterns — How to Fail With 100% Test Coverage

[B] Editors note: This might be a problem if you're coding in a cave. But you're not. So, go talk to your customer. Just sayin'.

Rich Hickey

Abstract

--- rough notes ---

• What is Datomic?
  • a new database
  • a sound model of information, with time
  • provides database as a value to applications
• We program in a functional style to help get a grip on complexity
  • Paper: Out of the Tar Pit - Mosely and Marks (2006) — TODO LINK
  • Complexity caused by state and control
• Database is a giant, shared global variable
• DB Complexity
  • stateful, inherently
  • same query, different results (over time)
    • no basis
  • the data is "over there" (i.e., in some far off place away from the code, outside of the application)
  • "update" poorly defined
    • current databases are place-oriented (regardless of whether your database has "rows" or "documents")
• What does "update" mean?
  • Does new replace the old?
  • What is the granularity? Record? Document?
  • Who sees the update? (If you've ever used SQL, you've seen a bug related to this question.)
• Terms
  • Value - immutable
  • Identity - a putative entity we associate with a series of casually related (states) over time
  • State - value of an identity at a moment in time
  • Time - relative before/after ordering of casual values
• Implementing values (efficiently)
  • persistent data structure
  • via trees with structural sharing
• On existing databases being "place-oriented"
  • "I don't mean to disparage any particular database. Let's just disparage them all equally."
  • This is the same problem we see with object-orientation: conflating identity with value
• Perception should not require permission (i.e., you shouldn't need a transaction in order to read)
• Database state
  • the database has an expanding value
    • an accretion of facts
    • the past doesn't change — immutable
  • new information needs new space
  • fundamental move away from places
• We're stuck in a decades-old model that was designed to fit a relational model into a tiny machine
• Facts
  • Fact is "an event or thing known to have happened or existed"
    • derived from: "factum" with means "something done"
    • must include time
  • Atomic Datom => Entity/Attribute/Value/Transaction (i.e., RDF, plus time)
• If you've ever looked at the database and wondered, "Wow! I wonder how it got like that." ... then you know (one reason) why this is important.
• Deconstructing the things we associate with databases
  • Process
    • Transactions
    • Indexing
    • Output
  • Perception/Reaction
    • Queries
    • Indexes
    • Input
• Diagrams of Datomic architecture (i.e., things that are difficult to capture in text ;-)

Lars Bak

Abstract

--- rough notes ---

• Lars has spent the last 26 years optimizing implementations of OO runtimes
• Experience includes work on Strongtalk VM, Hotspot VM, OOVM Smalltalk, V8, Dart, and more
• Motivation for VMs
  • platform-independent execution
  • sandboxing
  • optimizations can take place at runtime (e.g., you can decide what to inline at runtime; you're not forced to decide at compile time)
  • debugging is possible in production
  • loading of third-party code at runtime
• When the VM gets faster ...
  • existing programs run faster
  • programmers get room for software innovation
• OH "Hands-up: who knows what inline caching is? Ooh. That's not many people."
• Overview of the journey from Self to Strongtalk to Hotspot
• Strategy for Chrome: simple, secure, and fast
• Speed enables new kinds of applications, and you cannot predict the kinds of apps you'll get
• Original V8 goals
  • make JavaScript 10 times faster
  • enable JavaScript programs to scale
  • open source the project
  • and raise the industry performance bar as a result
• After the first four months, V8 was 24 times faster than code running inside Firefox
• Making V8 a separate project led to unexpected uses (e.g., node.js)
• JavaScript is now faster but ...
  • promotes spaghetti style programming
  • OO programming is hard
  • objects can change on-the-fly
  • no support for libraries
  • tool support is weak
  • slow application startup
  • runtime performance is unpredictable
• The web is great
  • developing small applications is easy
  • platform independence
  • no installation of applications
  • supports incremental development
  • ... but innovation is crucial for survival
• Goals for a new web platform (i.e. Dart)
  • support for programming in the large
  • compared to V8 performance
    • ultra-fast startup => 10 times faster
    • predictable performance => 2 time faster
  • avoid fragmentation of the web
• simple and unsurprising OO language
  • class-based single inheritance
  • interfaces with default implementation
  • optional static types
  • real lexical scoping
  • single-threaded
  • compact and readable
• Inspiration for Dart
  • object model inspired by Smalltalk
  • compilation strategy inspired by Self
  • optional types inspired by Strongtalk
  • ... and more
• Dart was designed for a VM
  • straightforward semantics
  • simple object model
  • no class initialization
  • applications are declared
• OH (while showing some Dart sample code) "I guess you all can read that. No monads!"
• Dart's optional type system
  • not your traditional static type system
  • used for specifying programmer intent
  • types can be introduced gradually
  • adding types to fields, variables, and method signatures does not change behavior
  • allows implicit downcasting
  • types are only verified in checked mode
  • type checker only issues warnings
• Platform independence
  • Dart can run on the Dart VM in Chrome
  • Dart can also compile down to JavaScript to run on other browsers
• Can write server code in Dart as well
• Dart SDK coming soon, including ...
  • language spec
  • libraries
  • VM
  • translator to JavaScript
  • IDE
• Things Lars has learned
  • always start small with a small team
  • focus on solving the hardest problem first
  • competitive situation fuels motivation
  • only people that are smarter than you
  • open source projects are great
    • helps industry
    • keep your work honest
    • not forced to wait for your vendor's release cycle in order to get a bug fixed
  • when building for performance, be sure to track performance from day one
    • run all benchmarks
    • on all revisions
    • on all platforms
    • and do it automatically (and have your build show you performance over time)
    • ... or don't, and performance will deteriorate
• Keep in mind
  • Speed will continue to drive innovation for web apps
  • Programmer productivity will be key as web applications get larger

Jeff Hawkins

Abstract

--- rough notes ---

• Jeff's book: On Intelligence
• Approach to the goal of "computing like the brain"
  1. discover operating principles of neocortex
  2. build systems based on these principles
• Or ...
  • start with anatomy and physiology of the brain
  • identify theoretical principles
  • build software
• Neocortex is a predictive modeling system
  • all high-level intelligence occurs in the neocortex
  • retina, cochlea, and somatic feed a high-velocity data stream into the neocortex
  • neocortex builds online models from streaming data
    • makes predictions
    • detects anomalies
    • generates actions (e.g., speech)
• "The brain doesn't actually compute; it's a memory system."
• Neocortex uses ...
  • a hierarchy of similar memory regions
  • sequence memory in each region
  • sparse distributed representations (SDRs)
• Dense representations (used in computers)
  • few bits (8 to 128)
  • all combinations of 1s and 0s
  • example: 8-bit ASCII
  • individual bits have no inherent meaning; representation is assigned by programmer
• Sparse distributed representations (used in the neocortex)
  • many bits (thousands
  • few 1s and mostly 0s
  • example: 2,000 bits, 2% active
  • each it has semantic meaning
  • meaning of each bit is learned, not assigned
• SDR properties
  • similarity: shared bits = semantic similarity
  • store and compare
    • store indices of active bits
    • subsampling is OK
  • union membership
• Intelligent machines will be built on SDRs
• Online learning
  • train on every new input
  • if pattern does not repeat, forget it
  • if pattern repeats, reinforce it
• Predictive analytics
  • Today
    • data comes in
    • we store it
    • we visualize it
    • we make predictive models
  • Tomorrow
    • data streams directly to online models
    • generate actions from online models acting on data
• Grok: An engine for acting on data streams
  • Runs on Amazon AWS
  • Example application: energy demand/response
    • Each night at midnight, Grok predicts the factory's energy use for the next 24 hours (based on patterns from recent time periods)
    • Predictions used to bid on energy at optimal prices
  • Example application: managing server capacity
    • Grok used to predict server demand
    • Used to provision instances ahead of time
    • Results show approximately 15% reduction in AWS cost
  • Example application: predictively and proactively optimize maintenance schedule for windmills
• Future of machine intelligence
  • more advanced than the (admittedly impressive) things we're seeing now (e.g., Watson beating Jeopardy pros, Google's self-driving cars)
  • a lot more theory still to be developed
  • embodiment
    • today: cloud-based
    • eventually ...
      • embedded
      • distributed (benefiting from billions of distributed sensors)
  • advances in hardware
  • applications
    • today: prediction, anomaly detection
    • classics: vision, language, speech
    • the big wins will be way more impressive than these things; the big wins will be things we cannot possibly imagine today
    • things that can benefit from systems that are thousands of times faster than the human brain, and that don't get tired