Skip to content

Instantly share code, notes, and snippets.

@ma2gedev

ma2gedev/isucon4-qual-phoenix.md

Last active September 12, 2015 00:45
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ma2gedev/00e079836d2a7379b81f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ma2gedev/00e079836d2a7379b81f to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. ma2gedev revised this gist Sep 12, 2015. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,8 @@
    追記: @sonots さんに AMI 作って検証をしていただいた結果は以下にあります。
    https://gist.github.com/sonots/62179703d429d36b1a57

    ----

    全部ローカルの MacBook Air で計測
    - 1.8 GHz Intel Core i5 (デュアルコアっぽい)

  2. ma2gedev revised this gist Sep 12, 2015. 1 changed file with 3 additions and 0 deletions.
    3 changes: 3 additions & 0 deletions isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,3 +1,6 @@
    追記: @sonots さんに AMI 作って検証をしていただいた結果は以下にあります。
    https://gist.github.com/sonots/62179703d429d36b1a57

    全部ローカルの MacBook Air で計測
    - 1.8 GHz Intel Core i5 (デュアルコアっぽい)

  3. ma2gedev revised this gist Sep 9, 2015. 1 changed file with 8 additions and 0 deletions.
    8 changes: 8 additions & 0 deletions isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,6 +1,14 @@
    全部ローカルの MacBook Air で計測
    - 1.8 GHz Intel Core i5 (デュアルコアっぽい)

    やったことは
    - table index 追加
    - my.cnf への設定追加
    - sysctl の設定をいくつか(FreeBSD なのであまり良くわかっておらず)

    途中で golang を追加したり統一した感じで出来ていない。。
    golang はそのままでもかなり高パフォーマンスで動く。Phoenix はパラメータ調整することで同じか超えるくらいのパフォーマンスを出せる。

    ## デフォルト

    ```
  4. ma2gedev revised this gist Sep 9, 2015. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,5 @@
    全部ローカルの MacBook Air で計測
    - 1.8 GHz Intel Core i5 (デュアルコアっぽい)

    ## デフォルト

  5. ma2gedev revised this gist Sep 9, 2015. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -218,6 +218,7 @@ $ ./benchmarker bench --workload 2 --host=localhost:8080
    23:32:49 type:info message:Result not sent to server because API key is not set
    23:32:49 type:score success:47260 fail:0 score:10210
    # ruby production mode
    $ ./benchmarker bench --workload 4 --host=localhost:8080
    23:38:35 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    23:38:35 type:info message:launch benchmarker
  6. ma2gedev renamed this gist Sep 9, 2015. 1 changed file with 98 additions and 4 deletions.
    102 changes: 98 additions & 4 deletions isucon4 qual phoenix → isucon4-qual-phoenix.md
    View file
    Original file line number Diff line number Diff line change
    @@ -34,9 +34,7 @@ $ ./benchmarker bench --host=localhost:4000
    00:11:14 type:debug incomplete_ip:127.1.1.37 reported:true
    00:11:14 type:info message:Result not sent to server because API key is not set
    00:11:14 type:score success:43050 fail:0 score:9299
    ```
    ```
    # ruby production mode
    $ ./benchmarker bench --host=localhost:8080
    23:39:30 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    @@ -78,11 +76,27 @@ $ ./benchmarker bench --host=localhost:4000
    00:39:35 type:score success:46950 fail:0 score:10142
    ```

    ```
    # golang MARTINI_ENV=production GOMAXPROCS=4 ./golang-webapp
    $ ./benchmarker bench --host=localhost:8080
    22:01:56 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    22:01:56 type:info message:launch benchmarker
    22:01:56 type:warning message:Result not sent to server because API key is not set
    22:01:56 type:info message:init environment
    22:02:03 type:info message:run benchmark workload: 1
    22:03:03 type:info message:finish benchmark workload: 1
    22:03:08 type:info message:check banned ips and locked users report
    22:03:10 type:report count:banned ips value:202
    22:03:10 type:report count:locked users value:3043
    22:03:10 type:info message:Result not sent to server because API key is not set
    22:03:10 type:score success:52130 fail:0 score:11261
    ```

    ## workload 増やす

    workload=2でもエラーが出るようになったので以下を設定した。

    ```
    ```shell
    $ sudo sysctl -w net.inet.ip.portrange.first=10000
    net.inet.ip.portrange.first: 32768 -> 10000
    $ sudo sysctl -w net.inet.ip.portrange.hifirst=10000
    @@ -135,6 +149,86 @@ $ ./benchmarker bench --workload 4 --host=localhost:4000
    01:11:37 type:info message:Result not sent to server because API key is not set
    01:11:37 type:score success:69420 fail:2 score:14996
    # ruby は workload=2 でエラーだらけになる(理由は Mysql2::Error - Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2):)
    # golang は安定している
    $ ./benchmarker bench --workload 4 --host=localhost:8080
    22:07:21 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    22:07:21 type:info message:launch benchmarker
    22:07:21 type:warning message:Result not sent to server because API key is not set
    22:07:21 type:info message:init environment
    22:07:27 type:info message:run benchmark workload: 4
    22:08:31 type:info message:finish benchmark workload: 4
    22:08:36 type:info message:check banned ips and locked users report
    22:08:38 type:report count:banned ips value:267
    22:08:38 type:report count:locked users value:3320
    22:08:38 type:debug incomplete_user:kristofer_jaskolski reported:true
    22:08:39 type:info message:Result not sent to server because API key is not set
    22:08:39 type:score success:82010 fail:0 score:17716
    ```

    ## さらに以下を設定

    ```shell
    $ sudo sysctl -w kern.ipc.somaxconn=32768
    kern.ipc.somaxconn: 128 -> 32768
    $ sudo sysctl -w net.inet.tcp.msl=1000
    net.inet.tcp.msl: 15000 -> 1000
    ```

    ```
    # phoenix production
    $ ./benchmarker bench --workload 4 --host=localhost:4000
    22:45:08 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    22:45:08 type:info message:launch benchmarker
    22:45:08 type:warning message:Result not sent to server because API key is not set
    22:45:08 type:info message:init environment
    22:45:14 type:info message:run benchmark workload: 4
    22:46:14 type:info message:finish benchmark workload: 4
    22:46:19 type:info message:check banned ips and locked users report
    22:46:21 type:report count:banned ips value:267
    22:46:21 type:report count:locked users value:3370
    22:46:21 type:info message:Result not sent to server because API key is not set
    22:46:21 type:score success:83810 fail:0 score:18105
    # golang は何故かサーバが動かなくなった。
    # sudo sysctl -w kern.ipc.somaxconn=30000 とした動いたが、結果は悪くなった
    $ ./benchmarker bench --workload 4 --host=localhost:8080
    22:58:03 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    22:58:03 type:info message:launch benchmarker
    22:58:03 type:warning message:Result not sent to server because API key is not set
    22:58:03 type:info message:init environment
    22:58:10 type:info message:run benchmark workload: 4
    22:59:10 type:info message:finish benchmark workload: 4
    22:59:15 type:info message:check banned ips and locked users report
    22:59:18 type:report count:banned ips value:258
    22:59:18 type:report count:locked users value:3147
    22:59:18 type:fail reason:Missmatch banned IPs message:Report checking is failed. Do not send score.
    22:59:18 type:score success:65530 fail:0 score:14157
    # ruby production mode
    $ ./benchmarker bench --workload 2 --host=localhost:8080
    23:31:35 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    23:31:35 type:info message:launch benchmarker
    23:31:35 type:warning message:Result not sent to server because API key is not set
    23:31:35 type:info message:init environment
    23:31:43 type:info message:run benchmark workload: 2
    23:32:43 type:info message:finish benchmark workload: 2
    23:32:48 type:info message:check banned ips and locked users report
    23:32:49 type:report count:banned ips value:153
    23:32:49 type:report count:locked users value:2959
    23:32:49 type:info message:Result not sent to server because API key is not set
    23:32:49 type:score success:47260 fail:0 score:10210
    $ ./benchmarker bench --workload 4 --host=localhost:8080
    23:38:35 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    23:38:35 type:info message:launch benchmarker
    23:38:35 type:warning message:Result not sent to server because API key is not set
    23:38:35 type:info message:init environment
    23:38:43 type:info message:run benchmark workload: 4
    23:39:43 type:info message:finish benchmark workload: 4
    23:39:48 type:info message:check banned ips and locked users report
    23:39:49 type:report count:banned ips value:185
    23:39:49 type:report count:locked users value:3031
    23:39:49 type:debug incomplete_user:dudley.gleason reported:true
    23:39:49 type:info message:Result not sent to server because API key is not set
    23:39:49 type:score success:50820 fail:0 score:10978
    ```
  7. ma2gedev revised this gist Sep 8, 2015. 1 changed file with 47 additions and 0 deletions.
    47 changes: 47 additions & 0 deletions isucon4 qual phoenix
    View file
    Original file line number Diff line number Diff line change
    @@ -76,7 +76,20 @@ $ ./benchmarker bench --host=localhost:4000
    00:39:35 type:report count:locked users value:2987
    00:39:35 type:info message:Result not sent to server because API key is not set
    00:39:35 type:score success:46950 fail:0 score:10142
    ```

    ## workload 増やす

    workload=2でもエラーが出るようになったので以下を設定した。

    ```
    $ sudo sysctl -w net.inet.ip.portrange.first=10000
    net.inet.ip.portrange.first: 32768 -> 10000
    $ sudo sysctl -w net.inet.ip.portrange.hifirst=10000
    net.inet.ip.portrange.hifirst: 32768 -> 10000
    ```

    ```
    # phoenix production mode workload 2
    $ ./benchmarker bench --workload 2 --host=localhost:4000
    00:55:51 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    @@ -90,4 +103,38 @@ $ ./benchmarker bench --workload 2 --host=localhost:4000
    00:57:05 type:report count:locked users value:3130
    00:57:05 type:info message:Result not sent to server because API key is not set
    00:57:05 type:score success:63850 fail:0 score:13793

    # phoenix production mode workload 3
    $ ./benchmarker bench --workload 3 --host=localhost:4000
    01:04:11 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    01:04:11 type:info message:launch benchmarker
    01:04:11 type:warning message:Result not sent to server because API key is not set
    01:04:11 type:info message:init environment
    01:04:17 type:info message:run benchmark workload: 3
    01:05:20 type:info message:finish benchmark workload: 3
    01:05:25 type:info message:check banned ips and locked users report
    01:05:27 type:report count:banned ips value:206
    01:05:27 type:report count:locked users value:3188
    01:05:27 type:debug incomplete_user:timmy_powlowski reported:true
    01:05:27 type:info message:Result not sent to server because API key is not set
    01:05:27 type:score success:67790 fail:0 score:14645

    # phoenix production mode workload 4 エラーが出るようになる
    $ ./benchmarker bench --workload 4 --host=localhost:4000
    01:10:20 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    01:10:20 type:info message:launch benchmarker
    01:10:20 type:warning message:Result not sent to server because API key is not set
    01:10:20 type:info message:init environment
    01:10:26 type:info message:run benchmark workload: 4
    01:11:18 type:fail reason:Response code should be 200, got 500 method:POST uri:/login
    01:11:29 type:fail reason:Response code should be 200, got 500 method:POST uri:/login
    01:11:29 type:info message:finish benchmark workload: 4
    01:11:34 type:info message:check banned ips and locked users report
    01:11:37 type:report count:banned ips value:257
    01:11:37 type:report count:locked users value:3196
    01:11:37 type:info message:Result not sent to server because API key is not set
    01:11:37 type:score success:69420 fail:2 score:14996

    # ruby は workload=2 でエラーだらけになる(理由は Mysql2::Error - Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2):)

    ```
  8. ma2gedev created this gist Sep 8, 2015.
    93 changes: 93 additions & 0 deletions isucon4 qual phoenix
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,93 @@
    全部ローカルの MacBook Air で計測

    ## デフォルト

    ```
    $ ./benchmarker bench --host=localhost:4000
    23:19:39 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    23:19:39 type:info message:launch benchmarker
    23:19:39 type:warning message:Result not sent to server because API key is not set
    23:19:39 type:info message:init environment
    23:19:44 type:info message:run benchmark workload: 1
    23:20:44 type:info message:finish benchmark workload: 1
    23:20:49 type:info message:check banned ips and locked users report
    23:21:28 type:report count:banned ips value:0
    23:21:28 type:report count:locked users value:2543
    23:21:28 type:info message:Result not sent to server because API key is not set
    23:21:28 type:score success:4720 fail:0 score:1020
    ```

    ## login_log table へ index 追加

    ```
    # phoenix production mode
    $ ./benchmarker bench --host=localhost:4000
    00:09:59 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    00:09:59 type:info message:launch benchmarker
    00:09:59 type:warning message:Result not sent to server because API key is not set
    00:09:59 type:info message:init environment
    00:10:06 type:info message:run benchmark workload: 1
    00:11:06 type:info message:finish benchmark workload: 1
    00:11:11 type:info message:check banned ips and locked users report
    00:11:14 type:report count:banned ips value:173
    00:11:14 type:report count:locked users value:2944
    00:11:14 type:debug incomplete_ip:127.1.1.37 reported:true
    00:11:14 type:info message:Result not sent to server because API key is not set
    00:11:14 type:score success:43050 fail:0 score:9299
    ```

    ```
    # ruby production mode
    $ ./benchmarker bench --host=localhost:8080
    23:39:30 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    23:39:30 type:info message:launch benchmarker
    23:39:30 type:warning message:Result not sent to server because API key is not set
    23:39:30 type:info message:init environment
    23:39:35 type:info message:run benchmark workload: 1
    23:40:10 type:fail reason:Connection timeout method:GET uri:/stylesheets/isucon-bank.css
    23:40:35 type:info message:finish benchmark workload: 1
    23:40:40 type:info message:check banned ips and locked users report
    23:40:42 type:report count:banned ips value:99
    23:40:42 type:report count:locked users value:2768
    23:40:42 type:debug incomplete_ip:127.1.0.202 reported:true
    23:40:42 type:info message:Result not sent to server because API key is not set
    23:40:42 type:score success:26009 fail:1 score:5619
    ```

    ## mysql my.cnf 設定追加

    ```
    innodb_buffer_pool_size = 1G
    innodb_flush_log_at_trx_commit = 0
    innodb_flush_method=O_DIRECT
    ```

    ```
    # phoenix production mode
    $ ./benchmarker bench --host=localhost:4000
    00:38:21 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    00:38:21 type:info message:launch benchmarker
    00:38:21 type:warning message:Result not sent to server because API key is not set
    00:38:21 type:info message:init environment
    00:38:28 type:info message:run benchmark workload: 1
    00:39:28 type:info message:finish benchmark workload: 1
    00:39:33 type:info message:check banned ips and locked users report
    00:39:35 type:report count:banned ips value:192
    00:39:35 type:report count:locked users value:2987
    00:39:35 type:info message:Result not sent to server because API key is not set
    00:39:35 type:score success:46950 fail:0 score:10142

    # phoenix production mode workload 2
    $ ./benchmarker bench --workload 2 --host=localhost:4000
    00:55:51 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
    00:55:51 type:info message:launch benchmarker
    00:55:51 type:warning message:Result not sent to server because API key is not set
    00:55:51 type:info message:init environment
    00:55:57 type:info message:run benchmark workload: 2
    00:56:57 type:info message:finish benchmark workload: 2
    00:57:02 type:info message:check banned ips and locked users report
    00:57:05 type:report count:banned ips value:257
    00:57:05 type:report count:locked users value:3130
    00:57:05 type:info message:Result not sent to server because API key is not set
    00:57:05 type:score success:63850 fail:0 score:13793
    ```