Created
January 23, 2012 05:03
-
-
Save ryansmith3136/1660752 to your computer and use it in GitHub Desktop.
Revisions
-
Ryan Smith (ace hacker) revised this gist
Apr 12, 2012 . 1 changed file with 9 additions and 12 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,23 +10,20 @@ ## Introduction The modern application developer has undoubtedly dealt with the problem of process execution locality. More precisely, when responding to HTTP requests, developers must carefully balance what work is to be done in and outside the process handling the request. The Worker Pattern exists to help provide a framework for thinking about the balance. In this article, we will define the pattern and look at several applications of the pattern. Similar to advanced mathematical techniques, the Worker Pattern is simple yet the application is not alway clear. However, with sufficient exposure to examples and with enough practice, the pattern becomes a reflex for the application developer. ## Definition **Processor:** Something that can execute instructions. **Group of Computation:** One or many atomic instructions. -
Ryan Smith (ace hacker) revised this gist
Apr 12, 2012 . 1 changed file with 2 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -81,4 +81,5 @@ end ## Links * [rack-worker](https://github.com/csquared/rack-worker) * [RailsConf 2010 Slide Deck](https://s3.amazonaws.com/ryandotsmith/deck.pdf) -
Ryan Smith (ace hacker) revised this gist
Apr 12, 2012 . 1 changed file with 5 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,7 +5,7 @@ * Introduction * Definition * Examples * Links ## Introduction @@ -78,3 +78,7 @@ if work set cache *key=request_signature* *value=process_result* end ``` ## Links [RailsConf 2010 Slide Deck](https://s3.amazonaws.com/ryandotsmith/deck.pdf) -
Ryan Smith (ace hacker) revised this gist
Apr 12, 2012 . 2 changed files with 80 additions and 42 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,80 @@ # The Worker Pattern ## Contents * Introduction * Definition * Examples * Credits ## Introduction The modern application developer has undoubtedly dealt with the problem of process execution locality. More precisely, when responding to HTTP requests, developers must carefully balance what work is to be done in the process handling the request and the work that can be done in a process outside of the HTTP request. The Worker Pattern exists to help strike such a balance. In this article, we will define the pattern and look at several applications of the pattern. However, before we dive in, a quick note on how to successfully grok this material. The fundamental idea of the Worker Pattern is and should always be simple. Much like mathematical techniques, the definition is quite simple and the application is not alway clear. However, with sufficient exposure to examples and with enough practice, the pattern becomes a reflex for the application developer. ## Definition **Processor:** Something that can execute instructions. Not a physical processor, thread, coroutine, etc... **Group of Computation:** One or many atomic instructions. **The Worker Pattern:** To divide a group of computations amongst a set of processors. ## Example: Processing HTTP Requests A web service that requires high throughput will undoubtedly need to ensure low latency while processing requests. In other words, the process that is serving HTTP requests should spend the least amount of time possible to serve the request. Subsequently if the server does not have all of the data necessary to properly respond to the request, it must not wait until the data is found. Instead it must let the client know that it is working on the fulfilment of the request and that the client should check back later. Such an arrangement will guarantee that our web servers are always available to respond to requests with low latency. The application of the Worker Pattern in this case involves moving the fulfilment of the request to another process. Leaving the server process free to respond to other requests. Let us know explore the algorithm: **HTTP Server** ``` receive request look in cache for data to satisfy request if data in cache respond to request with cached data else if cache contains key=request_signature respond to request with nothing. client should retry request else set cache with *key=request_signature* *value=NULL* enqueue a job to fetch data respond to request with nothing. client should retry request end end ``` **Worker** ``` look in queue for work if work process work save process result decode request_signature set cache *key=request_signature* *value=process_result* end ``` This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,42 +0,0 @@ -
Ryan R. Smith (ace hacker) revised this gist
Jan 23, 2012 . 1 changed file with 10 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,7 @@ # The Worker Pattern ## Contents * Introduction * Definition * Examples @@ -23,6 +25,14 @@ a reflex for the application developer. ## Definition Processor: Something that can execute instructions. Not a physical processor, thread, coroutine, etc... Group of Computation: One or many atomic instructions. The Worker Pattern: To divide a group of computations amongst a set of processors. ## Examples ### Dispatch -
Ryan R. Smith (ace hacker) revised this gist
Jan 23, 2012 . 1 changed file with 9 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -3,9 +3,6 @@ * Introduction * Definition * Examples * Credits ## Introduction @@ -24,3 +21,12 @@ definition is quite simple and the application is not alway clear. However, with sufficient exposure to examples and with enough practice, the pattern becomes a reflex for the application developer. ## Definition ## Examples ### Dispatch ### Append Only Log ### Fork -
Ryan R. Smith (ace hacker) revised this gist
Jan 23, 2012 . 1 changed file with 10 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,15 @@ # The Worker Pattern * Introduction * Definition * Examples ** Dispatch ** AOL (append only log) ** Fork * Credits ## Introduction The modern application developer has undoubtedly dealt with the problem of process execution locality. More precisely, when responding to HTTP requests, developers must carefully balance what work is to be done in the process -
Ryan R. Smith (ace hacker) revised this gist
Jan 23, 2012 . 1 changed file with 16 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1 +1,16 @@ # The Worker Pattern The modern application developer has undoubtedly dealt with the problem of process execution locality. More precisely, when responding to HTTP requests, developers must carefully balance what work is to be done in the process handling the request and the work that can be done in a process outside of the HTTP request. The Worker Pattern exists to help strike such a balance. In this article, we will define the pattern and look at several applications of the pattern. However, before we dive in, a quick note on how to successfully grok this material. The fundamental idea of the Worker Pattern is and should always be simple. Much like mathematical techniques, the definition is quite simple and the application is not alway clear. However, with sufficient exposure to examples and with enough practice, the pattern becomes a reflex for the application developer. -
Ryan Smith (ace hacker) renamed this gist
Jan 23, 2012 . 1 changed file with 0 additions and 0 deletions.There are no files selected for viewing
File renamed without changes. -
Ryan Smith (ace hacker) created this gist
Jan 23, 2012 .There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1 @@ # Worker Pattern