Bogdan Lashkov b09dan

The behavior when dbt issuing numerous SHOW OBJECTS IN <database>.<schema> queries when interfacing with Snowflake—is a recognized concern, especially in projects with a large number of schemas. This approach can introduce significant overhead due to the sequential execution of these commands, each requiring a separate connection to Snowflake.

Optimization Strategies:

Aggregate Object Retrieval:
- Current Behavior: dbt executes individual SHOW OBJECTS commands for each schema, which can be time-consuming in environments with many schemas.
- Proposed Improvement: Modify dbt to retrieve all objects at the database level using a single SHOW OBJECTS IN DATABASE command, then filter the results for the relevant schemas locally. This method reduces the number of queries sent to Snowflake, thereby decreasing latency and computational overhead.

Considerations: The SHOW OBJECTS command has a limit of returning a maximum of 10,000 records. In databases with a vast number of o

	from snowflake.snowpark import Session, DataFrame
	from snowflake.snowpark.functions import col, concat
	import snowflake.snowpark.exceptions

	from libs.support.snowflake_connection import get_snowflake_spark_connection, get_snowflake_sql_connection, \
	execute_snowflake_query


	def create_tables():
	# Create dimension tables

	#
	# This constraints file was automatically generated on 2022-03-22T09:30:15Z
	# via "eager-upgrade" mechanism of PIP. For the "main" branch of Airflow.
	# This variant of constraints install uses the HEAD of the branch version for 'apache-airflow' but installs
	# the providers from PIP-released packages at the moment of the constraint generation.
	#
	# Those constraints are actually those that regular users use to install released version of Airflow.
	# We also use those constraints after "apache-airflow" is released and the constraints are tagged with
	# "constraints-X.Y.Z" tag to build the production image for that version.
	#

	{
	"data": [{
	"stock_id": "337",
	"adj_close": "58.33659744262695",
	"volume": "6910000.0",
	"timestamp_pr": "2019-12-31 00:00:00"
	},
	{
	"stock_id": "337",
	"adj_close": "57.48381042480469",

	#!/bin/bash
	#
	# This script configures WordPress file permissions based on recommendations
	# from http://codex.wordpress.org/Hardening_WordPress#File_permissions
	#
	# Author: Michael Conigliaro
	#
	WP_OWNER=changeme # <-- wordpress owner
	WP_GROUP=changeme # <-- wordpress group
	WP_ROOT=/home/changeme # <-- wordpress root directory

	# coding: utf8

	xml = '''<?xml version="1.0" encoding="UTF-8"?>
	<soft>
	<os>
	<item name="linux" dist="ubuntu">
	This text about linux
	</item>
	<item name="mac os">
	Apple company