Skip to content

Instantly share code, notes, and snippets.

@jamestalmage

jamestalmage/profiling.md

Created December 17, 2015 07:31
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jamestalmage/758b09c42069ad96c792 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jamestalmage/758b09c42069ad96c792 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. jamestalmage created this gist Dec 17, 2015.
    26 changes: 26 additions & 0 deletions profiling.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,26 @@
    @bcoe @novemberborn The self-coverage stuff is actually a pretty interesting profiling tool when used in conjunction with `npm link`. Since we automatically use `index.covered.js` if it exists, we can get a better idea how our code behaves IRL. All these screenshots were generated running the AVA test suite:

    ### Comparing runs with an empty cache, vs ones with a complete cache

    *First run, no cache hits:*

    We can see that out of 50 forked processes, it only becomes necessary to create an instrumenter in 3 of those processes (meaning that 47 forked processes simply pulled from the cache - even on the first run). This makes sense with the speedups I am seeing. You still get the majority of the benefit from caching even on your first run. The second run may be faster, but imperceptibly so.

    <img width="677" alt="screenshot 2015-12-16 23 11 05" src="https://cloud.githubusercontent.com/assets/4082216/11861175/978adb30-a44b-11e5-949d-d0839554c83d.png">

    Also notice that the instrumenter is only accessed 14 times. There are 14 files instrumented in AVA. This means we never had a race condition for the cache (two threads seeing a cache miss, instrumenting and then racing to write the result back). Since `tap` executes tests synchronously and serially, this makes sense. However, it means we aren't exercising a potential failure mode here. We will definitely need to find a way to simulate race conditions as part of our test suite.

    *Second run, no cache misses:*

    We can see that the instrumenter is never created. Just a validation that two subsequent runs with identical content never has a cache miss.

    <img width="668" alt="screenshot 2015-12-16 23 15 32" src="https://cloud.githubusercontent.com/assets/4082216/11861207/dd347e98-a44b-11e5-8b3d-375c0d993834.png">

    ### Optimization

    Also, using self coverage on a real world test suite helps create a heat map and identify the best places to focus optimization efforts.

    <img width="735" alt="screenshot 2015-12-16 23 53 00" src="https://cloud.githubusercontent.com/assets/4082216/11861625/369f209c-a450-11e5-8777-67d7af5dda53.png">


    <img width="651" alt="screenshot 2015-12-16 23 47 32" src="https://cloud.githubusercontent.com/assets/4082216/11861558/9e7178f6-a44f-11e5-9031-7bd9e7f09f7f.png">