Skip to content

Instantly share code, notes, and snippets.

@ogibayashi

ogibayashi/fluentd_pidstat.rb

Created August 3, 2015 05:57
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save ogibayashi/31efe86fcb167ccda1a1 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save ogibayashi/31efe86fcb167ccda1a1 to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. Hironori Ogibayashi created this gist Aug 3, 2015.
    68 changes: 68 additions & 0 deletions fluentd_pidstat.rb
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,68 @@
    #!/usr/bin/env ruby

    # Send process cpu usage for each fluentd process to fluentd.
    # Each fluentd must be exected with -c (config) option
    # This script might be executed periodically by cron.

    require 'set'
    require 'optparse'
    require 'fluent-logger'

    PIDSTAT_INTERVAL=30

    opts = {}
    ARGV.options {|opt|
    opt.on('-h HOST', 'fluentd host'){|v| opts[:host] = v }
    opt.on('-p PORT', 'fluentd port'){|v| opts[:port] = v }
    opt.on('-t tag', 'fluentd tag'){|v| opts[:tag] = v }
    opt.on('-i string', 'host identifier'){|v| opts[:hostid] = v }
    opt.parse!
    }

    opts = {
    :host => "localhost",
    :port => 24224,
    :tag => "fluentd.processcpu",
    :hostid => `hostname`.strip
    }.merge(opts)

    log = Fluent::Logger::FluentLogger.new(nil, :host=>opts[:host], :port=>opts[:port])

    # list of fluentd process
    fluentd_processes = `ps -C ruby -f | grep fluentd`

    # get conf name (assumes fluentd is started with -c option)
    fluentd_conf_names = Set.new
    fluentd_processes.split("\n").each do |l|
    if /\W(\w+\.conf)/ =~ l
    fluentd_conf_names << $1
    end
    end

    # start pidstat process in background
    pipes = []
    fluentd_conf_names.each do |f|
    # get pid of latest process. (not fluentd supervisor, but actual engine process)
    pid = `pgrep -n -f #{f}`.chomp
    i, o = IO.pipe
    puts "pidstat -p #{pid} 1 1"
    Process.spawn("LANG=C pidstat -h -p #{pid} #{PIDSTAT_INTERVAL} 1", :out => o)
    pipes << {:name => f.gsub(/\.conf$/,""), :in => i, :out => o }
    end

    Process.waitall
    puts "Process finished"

    # read result
    pipes.each do |p|
    p[:out].close
    while l = p[:in].gets
    if /^\s+\d+\s/ =~ l
    # ["", "1438580253", "2778", "0.00", "0.00", "0.00", "0.00", "0", "ruby"] blank, timestamp, pid, %user, %system, %guest, %CPU, CPU, command
    pidstat_out = (l.split(/\s+/))
    puts l.split(/\s+/).to_s
    puts "#{p[:name]} " + pidstat_out[6]
    log.post(opts[:tag],{ "name" => p[:name], "pid" => pidstat_out[2], "percent_cpu" => pidstat_out[6].to_f , "hostname" => opts[:hostid]})
    end
    end
    end