Leslie A asamoal

AdVEGAnt Calendar

This example dashboard contains a Vega visualization, which renders a Christmas tree! When each node in the tree is clicked, a corresponding Kibana filter is created, so that the advent calendar blog post can be seen on the TSVB Markdown on the right hand side.

I was planning to write a post for Japanese Elastic Stack (Elasticsearch) Advent Calendar 2022, but every slot is filled. Nice! So I'm sharing it here... :)

This flow using Wait processor to wait for the all split FlowFiles to be processed. When it runs, following log messages can be seen:

2019-04-05 10:02:58,752 INFO [Timer-Driven Process Thread-7] o.a.nifi.processors.standard.LogMessage LogMessage[id=eafe4890-0169-1000-9927-a81fbbafafc6] Processing 1/5 message for 875bd765-209d-4136-a75d-5d14241384fe
2019-04-05 10:02:58,769 INFO [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.LogMessage LogMessage[id=eafe4890-0169-1000-9927-a81fbbafafc6] Processing 2/5 message for 875bd765-209d-4136-a75d-5d14241384fe
2019-04-05 10:02:58,770 INFO [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.LogMessage LogMessage[id=eafe4890-0169-1000-9927-a81fbbafafc6] Processing 3/5 message for 875bd765-209d-4136-a75d-5d14241384fe
2019-04-05 10:02:58,770 INFO [Timer-Driven Process Thread-3] o.a.nifi.processors.standard.LogMessage LogMessage[id=eafe4890-0169-1000-9927-a81fbbafafc6] Processing 4/5 message for 875bd765-209d-4136-a75d-5d14241384fe
2019-04-05 10:02:5

	{
	"flowContents": {
	"identifier": "a226434e-78d6-3846-bb1f-55680679a3b3",
	"instanceIdentifier": "710e0519-018a-1000-072b-2e0e390412d3",
	"name": "NiFi Flow",
	"comments": "",
	"position": {
	"x": 0.0,
	"y": 0.0
	},

	#!/bin/bash

	LOG="/hadoopfs/fs1/automation/nifi-cli.log"

	echo $@ >> $LOG

	EVENT=$1
	BUCKET=$2
	FLOW=$3
	VERSION=$4

	DROP TABLE if exists d_date;

	CREATE TABLE d_date
	(
	date_dim_id INT NOT NULL,
	date_actual DATE NOT NULL,
	epoch BIGINT NOT NULL,
	day_suffix VARCHAR(4) NOT NULL,
	day_name VARCHAR(9) NOT NULL,
	day_of_week INT NOT NULL,

	{
	"FunctionName": "NiFi_Function_Quickstart",
	"Description": "NiFi Function that uses the AWS S3 Lambda trigger to move telemetry data from landing zone S3 folder into processed S3 folder",
	"Timeout": 300,
	"MemorySize": 4096,
	"Environment": {
	"Variables": {
	"DF_ACCESS_KEY": "<<REPLACE>>",
	"DF_PRIVATE_KEY": "<<REPLACE>>",
	"FLOW_CRN": "<<REPLACE>>",

	version: "3"
	services:
	# configuration manager for NiFi
	zookeeper:
	hostname: myzookeeper
	container_name: zookeeper_container_persistent
	image: 'bitnami/zookeeper:3.7.0' # latest image as of 2021-11-09.
	restart: on-failure
	environment:
	- ALLOW_ANONYMOUS_LOGIN=yes

	// Licence: Robert Koch-Institut (RKI), dl-de/by-2-0

	class IncidenceWidget {

	constructor() {
	this.previousDaysToShow = 31;
	this.apiUrlDistricts = (location) => `https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_Landkreisdaten/FeatureServer/0/query?where=1%3D1&outFields=RS,GEN,cases7_bl_per_100k,cases7_per_100k,BL&geometry=${location.longitude.toFixed(3)}%2C${location.latitude.toFixed(3)}&geometryType=esriGeometryPoint&inSR=4326&spatialRel=esriSpatialRelWithin&returnGeometry=false&outSR=4326&f=json`
	this.apiUrlDistrictsHistory = (districtId) => `https://services7.arcgis.com/mOBPykOjAyBO2ZKk/ArcGIS/rest/services/Covid19_hubv/FeatureServer/0/query?where=IdLandkreis%20%3D%20%27${districtId}%27%20AND%20Meldedatum%20%3E%3D%20TIMESTAMP%20%27${this.getDateString(-this.previousDaysToShow)}%2000%3A00%3A00%27%20AND%20Meldedatum%20%3C%3D%20TIMESTAMP%20%27${this.getDateString(1)}%2000%3A00%3A00%27&outFields=Landkreis,Meldedatum,AnzahlFall&outSR=4326&f=json`
	this.stateToAbbr = {

	<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
	<template encoding-version="1.2">
	<description>This template gives an example of using a ScriptedLookupService to provide sequence numbers from a database.</description>
	<groupId>5d7e55f8-0165-1000-1170-d42741815ddb</groupId>
	<name>SequenceLookupExample</name>
	<snippet>
	<connections>
	<id>0cf6aae2-8e5a-361f-0000-000000000000</id>
	<parentGroupId>cedd92a0-903e-3f68-0000-000000000000</parentGroupId>
	<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>

	<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
	<template encoding-version="1.2">
	<description></description>
	<groupId>2e083aef-0162-1000-80bc-8c6bbfc00ca8</groupId>
	<name>wait-notify</name>
	<snippet>
	<processGroups>
	<id>582ee2ef-8a17-3830-0000-000000000000</id>
	<parentGroupId>16e377d4-8369-3464-0000-000000000000</parentGroupId>
	<position>