Skip to content

Instantly share code, notes, and snippets.

@ryansmith3136
Created January 23, 2012 05:03
Show Gist options
  • Select an option

  • Save ryansmith3136/1660752 to your computer and use it in GitHub Desktop.

Select an option

Save ryansmith3136/1660752 to your computer and use it in GitHub Desktop.

Revisions

  1. Ryan Smith (ace hacker) revised this gist Apr 12, 2012. 1 changed file with 9 additions and 12 deletions.
    21 changes: 9 additions & 12 deletions worker-pattern.md
    Original file line number Diff line number Diff line change
    @@ -10,23 +10,20 @@
    ## Introduction

    The modern application developer has undoubtedly dealt with the problem of
    process execution locality. More precisely, when responding to HTTP requests,
    developers must carefully balance what work is to be done in the process
    handling the request and the work that can be done in a process outside of the
    HTTP request. The Worker Pattern exists to help strike such a balance.
    process execution locality. More precisely, when responding to HTTP
    requests, developers must carefully balance what work is to be done in
    and outside the process handling the request. The Worker Pattern exists
    to help provide a framework for thinking about the balance.

    In this article, we will define the pattern and look at several
    applications of the pattern. However, before we dive in, a quick note on how to
    successfully grok this material. The fundamental idea of the Worker
    Pattern is and should always be simple. Much like mathematical techniques, the
    definition is quite simple and the application is not alway clear. However, with
    sufficient exposure to examples and with enough practice, the pattern becomes
    a reflex for the application developer.
    applications of the pattern. Similar to advanced mathematical techniques,
    the Worker Pattern is simple yet the application is not alway clear.
    However, with sufficient exposure to examples and with enough practice,
    the pattern becomes a reflex for the application developer.

    ## Definition

    **Processor:** Something that can execute instructions. Not a physical
    processor, thread, coroutine, etc...
    **Processor:** Something that can execute instructions.

    **Group of Computation:** One or many atomic instructions.

  2. Ryan Smith (ace hacker) revised this gist Apr 12, 2012. 1 changed file with 2 additions and 1 deletion.
    3 changes: 2 additions & 1 deletion worker-pattern.md
    Original file line number Diff line number Diff line change
    @@ -81,4 +81,5 @@ end

    ## Links

    [RailsConf 2010 Slide Deck](https://s3.amazonaws.com/ryandotsmith/deck.pdf)
    * [rack-worker](https://github.com/csquared/rack-worker)
    * [RailsConf 2010 Slide Deck](https://s3.amazonaws.com/ryandotsmith/deck.pdf)
  3. Ryan Smith (ace hacker) revised this gist Apr 12, 2012. 1 changed file with 5 additions and 1 deletion.
    6 changes: 5 additions & 1 deletion worker-pattern.md
    Original file line number Diff line number Diff line change
    @@ -5,7 +5,7 @@
    * Introduction
    * Definition
    * Examples
    * Credits
    * Links

    ## Introduction

    @@ -78,3 +78,7 @@ if work
    set cache *key=request_signature* *value=process_result*
    end
    ```

    ## Links

    [RailsConf 2010 Slide Deck](https://s3.amazonaws.com/ryandotsmith/deck.pdf)
  4. Ryan Smith (ace hacker) revised this gist Apr 12, 2012. 2 changed files with 80 additions and 42 deletions.
    80 changes: 80 additions & 0 deletions worker-pattern.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,80 @@
    # The Worker Pattern

    ## Contents

    * Introduction
    * Definition
    * Examples
    * Credits

    ## Introduction

    The modern application developer has undoubtedly dealt with the problem of
    process execution locality. More precisely, when responding to HTTP requests,
    developers must carefully balance what work is to be done in the process
    handling the request and the work that can be done in a process outside of the
    HTTP request. The Worker Pattern exists to help strike such a balance.

    In this article, we will define the pattern and look at several
    applications of the pattern. However, before we dive in, a quick note on how to
    successfully grok this material. The fundamental idea of the Worker
    Pattern is and should always be simple. Much like mathematical techniques, the
    definition is quite simple and the application is not alway clear. However, with
    sufficient exposure to examples and with enough practice, the pattern becomes
    a reflex for the application developer.

    ## Definition

    **Processor:** Something that can execute instructions. Not a physical
    processor, thread, coroutine, etc...

    **Group of Computation:** One or many atomic instructions.

    **The Worker Pattern:** To divide a group of computations amongst a set of
    processors.

    ## Example: Processing HTTP Requests

    A web service that requires high throughput will undoubtedly need to ensure
    low latency while processing requests. In other words, the process that
    is serving HTTP requests should spend the least amount of time possible
    to serve the request. Subsequently if the server does not have all of the
    data necessary to properly respond to the request, it must not wait until
    the data is found. Instead it must let the client know that it is working
    on the fulfilment of the request and that the client should check back
    later. Such an arrangement will guarantee that our web servers are always
    available to respond to requests with low latency.

    The application of the Worker Pattern in this case involves moving the
    fulfilment of the request to another process. Leaving the server process
    free to respond to other requests. Let us know explore the algorithm:

    **HTTP Server**

    ```
    receive request
    look in cache for data to satisfy request
    if data in cache
    respond to request with cached data
    else
    if cache contains key=request_signature
    respond to request with nothing. client should retry request
    else
    set cache with *key=request_signature* *value=NULL*
    enqueue a job to fetch data
    respond to request with nothing. client should retry request
    end
    end
    ```

    **Worker**

    ```
    look in queue for work
    if work
    process work
    save process result
    decode request_signature
    set cache *key=request_signature* *value=process_result*
    end
    ```
    42 changes: 0 additions & 42 deletions worker_pattern.md
    Original file line number Diff line number Diff line change
    @@ -1,42 +0,0 @@
    # The Worker Pattern

    ## Contents

    * Introduction
    * Definition
    * Examples
    * Credits

    ## Introduction

    The modern application developer has undoubtedly dealt with the problem of
    process execution locality. More precisely, when responding to HTTP requests,
    developers must carefully balance what work is to be done in the process
    handling the request and the work that can be done in a process outside of the
    HTTP request. The Worker Pattern exists to help strike such a balance.

    In this article, we will define the pattern and look at several
    applications of the pattern. However, before we dive in, a quick note on how to
    successfully grok this material. The fundamental idea of the Worker
    Pattern is and should always be simple. Much like mathematical techniques, the
    definition is quite simple and the application is not alway clear. However, with
    sufficient exposure to examples and with enough practice, the pattern becomes
    a reflex for the application developer.

    ## Definition

    Processor: Something that can execute instructions. Not a physical processor,
    thread, coroutine, etc...

    Group of Computation: One or many atomic instructions.

    The Worker Pattern: To divide a group of computations amongst a set of
    processors.

    ## Examples

    ### Dispatch

    ### Append Only Log

    ### Fork
  5. Ryan R. Smith (ace hacker) revised this gist Jan 23, 2012. 1 changed file with 10 additions and 0 deletions.
    10 changes: 10 additions & 0 deletions worker_pattern.md
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,7 @@
    # The Worker Pattern

    ## Contents

    * Introduction
    * Definition
    * Examples
    @@ -23,6 +25,14 @@ a reflex for the application developer.

    ## Definition

    Processor: Something that can execute instructions. Not a physical processor,
    thread, coroutine, etc...

    Group of Computation: One or many atomic instructions.

    The Worker Pattern: To divide a group of computations amongst a set of
    processors.

    ## Examples

    ### Dispatch
  6. Ryan R. Smith (ace hacker) revised this gist Jan 23, 2012. 1 changed file with 9 additions and 3 deletions.
    12 changes: 9 additions & 3 deletions worker_pattern.md
    Original file line number Diff line number Diff line change
    @@ -3,9 +3,6 @@
    * Introduction
    * Definition
    * Examples
    ** Dispatch
    ** AOL (append only log)
    ** Fork
    * Credits

    ## Introduction
    @@ -24,3 +21,12 @@ definition is quite simple and the application is not alway clear. However, with
    sufficient exposure to examples and with enough practice, the pattern becomes
    a reflex for the application developer.

    ## Definition

    ## Examples

    ### Dispatch

    ### Append Only Log

    ### Fork
  7. Ryan R. Smith (ace hacker) revised this gist Jan 23, 2012. 1 changed file with 10 additions and 0 deletions.
    10 changes: 10 additions & 0 deletions worker_pattern.md
    Original file line number Diff line number Diff line change
    @@ -1,5 +1,15 @@
    # The Worker Pattern

    * Introduction
    * Definition
    * Examples
    ** Dispatch
    ** AOL (append only log)
    ** Fork
    * Credits

    ## Introduction

    The modern application developer has undoubtedly dealt with the problem of
    process execution locality. More precisely, when responding to HTTP requests,
    developers must carefully balance what work is to be done in the process
  8. Ryan R. Smith (ace hacker) revised this gist Jan 23, 2012. 1 changed file with 16 additions and 1 deletion.
    17 changes: 16 additions & 1 deletion worker_pattern.md
    Original file line number Diff line number Diff line change
    @@ -1 +1,16 @@
    # Worker Pattern
    # The Worker Pattern

    The modern application developer has undoubtedly dealt with the problem of
    process execution locality. More precisely, when responding to HTTP requests,
    developers must carefully balance what work is to be done in the process
    handling the request and the work that can be done in a process outside of the
    HTTP request. The Worker Pattern exists to help strike such a balance.

    In this article, we will define the pattern and look at several
    applications of the pattern. However, before we dive in, a quick note on how to
    successfully grok this material. The fundamental idea of the Worker
    Pattern is and should always be simple. Much like mathematical techniques, the
    definition is quite simple and the application is not alway clear. However, with
    sufficient exposure to examples and with enough practice, the pattern becomes
    a reflex for the application developer.

  9. Ryan Smith (ace hacker) renamed this gist Jan 23, 2012. 1 changed file with 0 additions and 0 deletions.
    File renamed without changes.
  10. Ryan Smith (ace hacker) created this gist Jan 23, 2012.
    1 change: 1 addition & 0 deletions gistfile1.
    Original file line number Diff line number Diff line change
    @@ -0,0 +1 @@
    # Worker Pattern