15 OTTL transform statements (set, ToLowerCase, Len, conditional set, Concat) run through
the transform processor as a black box (NewFactory() → CreateLogs() → ConsumeLogs()),
comparing attributes["key"] vs body["key"] access across varying map sizes and nesting depths.
Machine: Apple M4 Pro, darwin/arm64
Go benchmark flags: -benchmem -count=5
| # Create an OpenTelemetry Collector configuration that processes log signals and performs the following 4 alterations: | |
| # | |
| # 1. Add a new attribute called env with the value production, but only if it doesn't already exist. | |
| # 2. Remove the attribute internal.debug_id. | |
| # 3. Change the value of the log.level attribute to "INFO" on records where it already exists. | |
| # 4. Copy the value of the host.name attribute into a new attribute called hostname. | |
| receivers: | |
| otlp: | |
| protocols: |
| bash-5.1$ JRUBY_OPTS="-Xcompile.invokedynamic=true -Xjit.threshold=0 -Xjit.debug=true -Xjit.logging=true " bundle exec rake test | |
| 2025-09-08T17:26:17.477Z [main] INFO Ruby : done compiling target script: /usr/share/logstash/vendor/jruby/bin/bundle 2025-09-08T17:26:17.493Z [main] INFO JITCompiler : method done jitting: Kernel require at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/core_ext/kernel_require.rb:35 | |
| 2025-09-08T17:26:17.505Z [main] INFO JITCompiler : method done jitting: #<Class:0x63f259c3> find_default_spec at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems.rb:1261 2025-09-08T17:26:17.509Z [main] INFO JITCompiler : method done jitting: #<Class:0x1869fbd2> unresolved_deps at /usr/share/logstash/vendor/jruby/lib/ruby/stdlib/rubygems/specification.rb:1237 |
| require 'manticore' | |
| require 'json' | |
| HOST = "http://localhost:9200" | |
| JSON_HEADERS = {"Content-Type" => "application/json"} | |
| #client = Manticore::Client.new(:auth => {:user => 'elastic', :password => 'changeme' }, :ssl => { :verify => :none }) | |
| client = Manticore::Client.new |
| /tmp/sandbox | |
| ❯ jruby -v | |
| jruby 9.4.7.0 (3.1.4) 2024-04-29 597ff08ac1 OpenJDK 64-Bit Server VM 17.0.9+9 on 17.0.9+9 +jit [arm64-darwin] | |
| /tmp/sandbox | |
| ❯ git clone git@github.com:logstash-plugins/logstash-input-salesforce.git | |
| Cloning into 'logstash-input-salesforce'... | |
| remote: Enumerating objects: 331, done. | |
| remote: Counting objects: 100% (57/57), done. | |
| remote: Compressing objects: 100% (21/21), done. |
| # env var: | |
| # keystore: | |
| # pipeline.workers = default to cpu cores | |
| ❯ docker run docker.elastic.co/logstash/logstash-full:8.16.0-SNAPSHOT sh -c 'rm config/logstash.keystore && echo 1 | bin/logstash -e "" | grep pipeline.workers' | |
| [2024-08-09T16:06:29,979][INFO ][logstash.javapipeline][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>10, [...] | |
| # env var: 2 | |
| # keystore: | |
| # pipeline.workers = env var | |
| ❯ docker run -e PIPELINE_WORKERS="2" docker.elastic.co/logstash/logstash-full:8.16.0-SNAPSHOT sh -c 'rm config/logstash.keystore && echo 1 | bin/logstash -e "" | grep pipeline.workers' |
| ❯ cat with_copy/Dockerfile | |
| FROM centos:7 | |
| COPY logstash-8.13.2-linux-aarch64.tar.gz /tmp/logstash.tar.gz | |
| RUN tar -zxf /tmp/logstash.tar.gz && rm /tmp/logstash.tar.gz | |
| ❯ docker build -q with_copy | |
| sha256:fce85351f818038d93afbd4005ff4fe10a2d7771ce1e603a54f91300e4032d1e | |
| ❯ docker run -it fce85351f818038d93afbd4005ff4fe10a2d7771ce1e603a54f91300e4032d1e /logstash-8.13.2/bin/logstash -V |
| ❯ for type in inputs filters codecs outputs; do echo "List of $type"; cat ./logstash_node_stats.json | jq -r ".pipelines | map(.plugins.$type)[] | map(.name)[]" | sort | uniq -c | sort -k2; done | |
| List of inputs | |
| 1 beats | |
| 2 jdbc | |
| 1 syslog | |
| 1 tcp | |
| List of filters | |
| 1 clone | |
| 4 date | |
| 1 dissect |
| ❯ | |
| /tmp/logstash-8.12.2 | |
| ❯ LS_JAVA_OPTS="-verbose:class" bin/logstash -e "input { generator { count => 1 } } output { stdout {} }" | |
| Using bundled JDK: /tmp/logstash-8.12.2/jdk.app/Contents/Home | |
| exec "/tmp/logstash-8.12.2/jdk.app/Contents/Home/bin/java" -cp "/tmp/logstash-8.12.2/vendor/jruby/lib/jruby.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/checker-qual-3.33.0.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/commons-codec-1.15.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/commons-compiler-3.1.0.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/commons-logging-1.2.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/error_prone_annotations-2.18.0.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/failureaccess-1.0.1.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/google-java-format-1.15.0.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/guava-32.1.2-jre.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/httpclient-4.5.13.jar:/tmp/logstash-8.12.2/logstash-core/lib/jars/httpcore-4.4.14.jar:/tmp/logstash-8.12.2/logstash |
| require 'socket' | |
| HOST = 'localhost' | |
| PORT = 5555 | |
| def connect_and_close | |
| socket = TCPSocket.new(HOST, PORT) | |
| linger = [1,0].pack('ii') | |
| socket.setsockopt(Socket::SOL_SOCKET, Socket::SO_LINGER, linger) | |
| socket.close |