追記: @sonots さんに AMI 作って検証をしていただいた結果は以下にあります。 https://gist.github.com/sonots/62179703d429d36b1a57
全部ローカルの MacBook Air で計測
- 1.8 GHz Intel Core i5 (デュアルコアっぽい)
やったことは
- table index 追加
- my.cnf への設定追加
- sysctl の設定をいくつか(FreeBSD なのであまり良くわかっておらず)
途中で golang を追加したり統一した感じで出来ていない。。 golang はそのままでもかなり高パフォーマンスで動く。Phoenix はパラメータ調整することで同じか超えるくらいのパフォーマンスを出せる。
$ ./benchmarker bench --host=localhost:4000
23:19:39 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
23:19:39 type:info message:launch benchmarker
23:19:39 type:warning message:Result not sent to server because API key is not set
23:19:39 type:info message:init environment
23:19:44 type:info message:run benchmark workload: 1
23:20:44 type:info message:finish benchmark workload: 1
23:20:49 type:info message:check banned ips and locked users report
23:21:28 type:report count:banned ips value:0
23:21:28 type:report count:locked users value:2543
23:21:28 type:info message:Result not sent to server because API key is not set
23:21:28 type:score success:4720 fail:0 score:1020
# phoenix production mode
$ ./benchmarker bench --host=localhost:4000
00:09:59 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
00:09:59 type:info message:launch benchmarker
00:09:59 type:warning message:Result not sent to server because API key is not set
00:09:59 type:info message:init environment
00:10:06 type:info message:run benchmark workload: 1
00:11:06 type:info message:finish benchmark workload: 1
00:11:11 type:info message:check banned ips and locked users report
00:11:14 type:report count:banned ips value:173
00:11:14 type:report count:locked users value:2944
00:11:14 type:debug incomplete_ip:127.1.1.37 reported:true
00:11:14 type:info message:Result not sent to server because API key is not set
00:11:14 type:score success:43050 fail:0 score:9299
# ruby production mode
$ ./benchmarker bench --host=localhost:8080
23:39:30 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
23:39:30 type:info message:launch benchmarker
23:39:30 type:warning message:Result not sent to server because API key is not set
23:39:30 type:info message:init environment
23:39:35 type:info message:run benchmark workload: 1
23:40:10 type:fail reason:Connection timeout method:GET uri:/stylesheets/isucon-bank.css
23:40:35 type:info message:finish benchmark workload: 1
23:40:40 type:info message:check banned ips and locked users report
23:40:42 type:report count:banned ips value:99
23:40:42 type:report count:locked users value:2768
23:40:42 type:debug incomplete_ip:127.1.0.202 reported:true
23:40:42 type:info message:Result not sent to server because API key is not set
23:40:42 type:score success:26009 fail:1 score:5619
innodb_buffer_pool_size = 1G
innodb_flush_log_at_trx_commit = 0
innodb_flush_method=O_DIRECT
# phoenix production mode
$ ./benchmarker bench --host=localhost:4000
00:38:21 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
00:38:21 type:info message:launch benchmarker
00:38:21 type:warning message:Result not sent to server because API key is not set
00:38:21 type:info message:init environment
00:38:28 type:info message:run benchmark workload: 1
00:39:28 type:info message:finish benchmark workload: 1
00:39:33 type:info message:check banned ips and locked users report
00:39:35 type:report count:banned ips value:192
00:39:35 type:report count:locked users value:2987
00:39:35 type:info message:Result not sent to server because API key is not set
00:39:35 type:score success:46950 fail:0 score:10142
# golang MARTINI_ENV=production GOMAXPROCS=4 ./golang-webapp
$ ./benchmarker bench --host=localhost:8080
22:01:56 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
22:01:56 type:info message:launch benchmarker
22:01:56 type:warning message:Result not sent to server because API key is not set
22:01:56 type:info message:init environment
22:02:03 type:info message:run benchmark workload: 1
22:03:03 type:info message:finish benchmark workload: 1
22:03:08 type:info message:check banned ips and locked users report
22:03:10 type:report count:banned ips value:202
22:03:10 type:report count:locked users value:3043
22:03:10 type:info message:Result not sent to server because API key is not set
22:03:10 type:score success:52130 fail:0 score:11261
workload=2でもエラーが出るようになったので以下を設定した。
$ sudo sysctl -w net.inet.ip.portrange.first=10000
net.inet.ip.portrange.first: 32768 -> 10000
$ sudo sysctl -w net.inet.ip.portrange.hifirst=10000
net.inet.ip.portrange.hifirst: 32768 -> 10000# phoenix production mode workload 2
$ ./benchmarker bench --workload 2 --host=localhost:4000
00:55:51 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
00:55:51 type:info message:launch benchmarker
00:55:51 type:warning message:Result not sent to server because API key is not set
00:55:51 type:info message:init environment
00:55:57 type:info message:run benchmark workload: 2
00:56:57 type:info message:finish benchmark workload: 2
00:57:02 type:info message:check banned ips and locked users report
00:57:05 type:report count:banned ips value:257
00:57:05 type:report count:locked users value:3130
00:57:05 type:info message:Result not sent to server because API key is not set
00:57:05 type:score success:63850 fail:0 score:13793
# phoenix production mode workload 3
$ ./benchmarker bench --workload 3 --host=localhost:4000
01:04:11 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
01:04:11 type:info message:launch benchmarker
01:04:11 type:warning message:Result not sent to server because API key is not set
01:04:11 type:info message:init environment
01:04:17 type:info message:run benchmark workload: 3
01:05:20 type:info message:finish benchmark workload: 3
01:05:25 type:info message:check banned ips and locked users report
01:05:27 type:report count:banned ips value:206
01:05:27 type:report count:locked users value:3188
01:05:27 type:debug incomplete_user:timmy_powlowski reported:true
01:05:27 type:info message:Result not sent to server because API key is not set
01:05:27 type:score success:67790 fail:0 score:14645
# phoenix production mode workload 4 エラーが出るようになる
$ ./benchmarker bench --workload 4 --host=localhost:4000
01:10:20 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
01:10:20 type:info message:launch benchmarker
01:10:20 type:warning message:Result not sent to server because API key is not set
01:10:20 type:info message:init environment
01:10:26 type:info message:run benchmark workload: 4
01:11:18 type:fail reason:Response code should be 200, got 500 method:POST uri:/login
01:11:29 type:fail reason:Response code should be 200, got 500 method:POST uri:/login
01:11:29 type:info message:finish benchmark workload: 4
01:11:34 type:info message:check banned ips and locked users report
01:11:37 type:report count:banned ips value:257
01:11:37 type:report count:locked users value:3196
01:11:37 type:info message:Result not sent to server because API key is not set
01:11:37 type:score success:69420 fail:2 score:14996
# golang は安定している
$ ./benchmarker bench --workload 4 --host=localhost:8080
22:07:21 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
22:07:21 type:info message:launch benchmarker
22:07:21 type:warning message:Result not sent to server because API key is not set
22:07:21 type:info message:init environment
22:07:27 type:info message:run benchmark workload: 4
22:08:31 type:info message:finish benchmark workload: 4
22:08:36 type:info message:check banned ips and locked users report
22:08:38 type:report count:banned ips value:267
22:08:38 type:report count:locked users value:3320
22:08:38 type:debug incomplete_user:kristofer_jaskolski reported:true
22:08:39 type:info message:Result not sent to server because API key is not set
22:08:39 type:score success:82010 fail:0 score:17716
$ sudo sysctl -w kern.ipc.somaxconn=32768
kern.ipc.somaxconn: 128 -> 32768
$ sudo sysctl -w net.inet.tcp.msl=1000
net.inet.tcp.msl: 15000 -> 1000# phoenix production
$ ./benchmarker bench --workload 4 --host=localhost:4000
22:45:08 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
22:45:08 type:info message:launch benchmarker
22:45:08 type:warning message:Result not sent to server because API key is not set
22:45:08 type:info message:init environment
22:45:14 type:info message:run benchmark workload: 4
22:46:14 type:info message:finish benchmark workload: 4
22:46:19 type:info message:check banned ips and locked users report
22:46:21 type:report count:banned ips value:267
22:46:21 type:report count:locked users value:3370
22:46:21 type:info message:Result not sent to server because API key is not set
22:46:21 type:score success:83810 fail:0 score:18105
# golang は何故かサーバが動かなくなった。
# sudo sysctl -w kern.ipc.somaxconn=30000 とした動いたが、結果は悪くなった
$ ./benchmarker bench --workload 4 --host=localhost:8080
22:58:03 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
22:58:03 type:info message:launch benchmarker
22:58:03 type:warning message:Result not sent to server because API key is not set
22:58:03 type:info message:init environment
22:58:10 type:info message:run benchmark workload: 4
22:59:10 type:info message:finish benchmark workload: 4
22:59:15 type:info message:check banned ips and locked users report
22:59:18 type:report count:banned ips value:258
22:59:18 type:report count:locked users value:3147
22:59:18 type:fail reason:Missmatch banned IPs message:Report checking is failed. Do not send score.
22:59:18 type:score success:65530 fail:0 score:14157
# ruby production mode
$ ./benchmarker bench --workload 2 --host=localhost:8080
23:31:35 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
23:31:35 type:info message:launch benchmarker
23:31:35 type:warning message:Result not sent to server because API key is not set
23:31:35 type:info message:init environment
23:31:43 type:info message:run benchmark workload: 2
23:32:43 type:info message:finish benchmark workload: 2
23:32:48 type:info message:check banned ips and locked users report
23:32:49 type:report count:banned ips value:153
23:32:49 type:report count:locked users value:2959
23:32:49 type:info message:Result not sent to server because API key is not set
23:32:49 type:score success:47260 fail:0 score:10210
# ruby production mode
$ ./benchmarker bench --workload 4 --host=localhost:8080
23:38:35 type:info message:!!! DEBUG MODE !!! DEBUGE MODE !!!
23:38:35 type:info message:launch benchmarker
23:38:35 type:warning message:Result not sent to server because API key is not set
23:38:35 type:info message:init environment
23:38:43 type:info message:run benchmark workload: 4
23:39:43 type:info message:finish benchmark workload: 4
23:39:48 type:info message:check banned ips and locked users report
23:39:49 type:report count:banned ips value:185
23:39:49 type:report count:locked users value:3031
23:39:49 type:debug incomplete_user:dudley.gleason reported:true
23:39:49 type:info message:Result not sent to server because API key is not set
23:39:49 type:score success:50820 fail:0 score:10978