kaitas/crawler-strategy.md

Created January 10, 2026 15:19

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/kaitas/4016312685637e57df7a2dcd30c8cada.js"></script>
Save kaitas/4016312685637e57df7a2dcd30c8cada to your computer and use it in GitHub Desktop.

Download ZIP

AI経済新聞各社クローリング戦略比較

Raw

crawler-strategy.md

企業	リクエスト数	IP数	1IPあたり	アプローチ
OpenAI (GPTBot)	44	16	2.75	分散・ステルス型
Anthropic (ClaudeBot)	19	1	19.0	集中・透明型
Google (Googlebot)	26	10	2.6	分散・安定型

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment