Skip to content

Instantly share code, notes, and snippets.

@esciara

esciara/README-metabase-inactive-tables-issue.md

Created April 1, 2026 13:02
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save esciara/2c99df5b96f6e3ff0555bd38a74c47b7 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save esciara/2c99df5b96f6e3ff0555bd38a74c47b7 to your computer and use it in GitHub Desktop.
Download ZIP
Reproduce Metabase 0.57 issue: GET /api/database/:id/metadata never returns inactive tables despite remove_inactive=false
Raw
README-metabase-inactive-tables-issue.md

To spin the docker containers and run the tests:

just docker-reset      # only if the containers have been created before
just docker-setup
just docker-test-inactive-tables
Raw
docker-compose.yml
services:
# Metabase application database (stores cards, dashboards, settings)
postgres:
image: postgres:16-alpine
environment:
POSTGRES_DB: metabase
POSTGRES_USER: metabase
POSTGRES_PASSWORD: metabase
ports:
- "5432:5432"
volumes:
- metabase-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U metabase"]
interval: 5s
retries: 5
# Sample database to create cards against (simulates BigQuery tables)
sample-db:
image: postgres:16-alpine
environment:
POSTGRES_DB: sample
POSTGRES_USER: sample
POSTGRES_PASSWORD: sample
ports:
- "5433:5432"
volumes:
- sample-data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U sample"]
interval: 5s
retries: 5
metabase:
image: metabase/metabase:v0.57.16
ports:
- "3000:3000"
environment:
MB_DB_TYPE: postgres
MB_DB_HOST: postgres
MB_DB_PORT: 5432
MB_DB_DBNAME: metabase
MB_DB_USER: metabase
MB_DB_PASS: metabase
depends_on:
postgres:
condition: service_healthy
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/api/health"]
interval: 10s
timeout: 5s
retries: 20
start_period: 30s
volumes:
metabase-data:
sample-data:
Raw
docker-setup.sh
#!/usr/bin/env bash
# docker-setup.sh — Idempotent bootstrap for local Metabase development environment.
#
# Takes running Docker containers to a fully seeded Metabase instance in 6 phases:
# 1. Wait for Metabase to be ready
# 2. Admin account setup (or login if already set up)
# 3. Connect sample-db as a Metabase database
# 4. Create test tables in sample-db
# 5. Sync Metabase and wait for tables to appear
# 6. Create test cards (native SQL + query builder)
#
# Usage: bash scripts/docker-setup.sh
set -euo pipefail
# ---------------------------------------------------------------------------
# Configuration
# ---------------------------------------------------------------------------
METABASE_URL="http://localhost:3000"
ADMIN_EMAIL="admin@test.com"
ADMIN_PASSWORD="Testpass1"
ADMIN_FIRST_NAME="Admin"
ADMIN_LAST_NAME="User"
COMPOSE_FILE="docker-compose.yml"
MAX_WAIT_METABASE=90 # seconds to wait for Metabase readiness
MAX_WAIT_SYNC=30 # seconds to wait for table sync
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
info() { echo "==> $*"; }
ok() { echo " ✓ $*"; }
skip() { echo " • $* (already exists)"; }
fail() { echo " ✗ $*" >&2; exit 1; }
# Parse JSON with Python (guaranteed available in this project)
json_get() {
python3 -c "import sys, json; data=json.load(sys.stdin); print($1)"
}
# ---------------------------------------------------------------------------
# Phase 1: Wait for Metabase
# ---------------------------------------------------------------------------
info "Phase 1: Waiting for Metabase to be ready..."
elapsed=0
while [ $elapsed -lt $MAX_WAIT_METABASE ]; do
if curl -sf "${METABASE_URL}/api/session/properties" > /dev/null 2>&1; then
ok "Metabase is ready (${elapsed}s)"
break
fi
sleep 2
elapsed=$((elapsed + 2))
done
if [ $elapsed -ge $MAX_WAIT_METABASE ]; then
fail "Metabase did not become ready within ${MAX_WAIT_METABASE}s"
fi
# ---------------------------------------------------------------------------
# Phase 2: Admin setup (idempotent)
# ---------------------------------------------------------------------------
info "Phase 2: Setting up admin account..."
SETUP_TOKEN=$(curl -sf "${METABASE_URL}/api/session/properties" \
| json_get "data.get('setup-token', '')")
if [ -n "$SETUP_TOKEN" ] && [ "$SETUP_TOKEN" != "None" ]; then
# First-time setup
RESPONSE=$(curl -sf "${METABASE_URL}/api/setup" \
-H "Content-Type: application/json" \
-d "{
\"token\": \"${SETUP_TOKEN}\",
\"user\": {
\"email\": \"${ADMIN_EMAIL}\",
\"password\": \"${ADMIN_PASSWORD}\",
\"first_name\": \"${ADMIN_FIRST_NAME}\",
\"last_name\": \"${ADMIN_LAST_NAME}\",
\"site_name\": \"Metabase Dev\"
},
\"prefs\": {
\"site_name\": \"Metabase Dev\",
\"site_locale\": \"en\",
\"allow_tracking\": false
}
}")
TOKEN=$(echo "$RESPONSE" | json_get "data.get('id', '')")
ok "Admin account created (${ADMIN_EMAIL})"
else
# Already set up — log in
RESPONSE=$(curl -sf "${METABASE_URL}/api/session" \
-H "Content-Type: application/json" \
-d "{
\"username\": \"${ADMIN_EMAIL}\",
\"password\": \"${ADMIN_PASSWORD}\"
}")
TOKEN=$(echo "$RESPONSE" | json_get "data['id']")
skip "Admin account"
fi
if [ -z "$TOKEN" ] || [ "$TOKEN" = "None" ]; then
fail "Could not obtain session token"
fi
AUTH_HEADER="X-Metabase-Session: ${TOKEN}"
# ---------------------------------------------------------------------------
# Phase 3: Connect sample-db (idempotent)
# ---------------------------------------------------------------------------
info "Phase 3: Connecting sample database..."
EXISTING_DB=$(curl -sf "${METABASE_URL}/api/database" \
-H "$AUTH_HEADER" \
| json_get "[d['id'] for d in data.get('data', data) if d['name'] == 'Sample DB']")
if [ "$EXISTING_DB" = "[]" ]; then
curl -sf "${METABASE_URL}/api/database" \
-H "$AUTH_HEADER" \
-H "Content-Type: application/json" \
-d '{
"engine": "postgres",
"name": "Sample DB",
"details": {
"host": "sample-db",
"port": 5432,
"dbname": "sample",
"user": "sample",
"password": "sample"
}
}' > /dev/null
ok "Sample DB connected"
else
skip "Sample DB connection"
fi
# Get the database ID for later use
DB_ID=$(curl -sf "${METABASE_URL}/api/database" \
-H "$AUTH_HEADER" \
| json_get "[d['id'] for d in data.get('data', data) if d['name'] == 'Sample DB'][0]")
if [ -z "$DB_ID" ] || [ "$DB_ID" = "None" ]; then
fail "Could not find Sample DB ID"
fi
# ---------------------------------------------------------------------------
# Phase 4: Create test tables (idempotent via IF NOT EXISTS / ON CONFLICT)
# ---------------------------------------------------------------------------
info "Phase 4: Creating test tables..."
docker compose -f "$COMPOSE_FILE" exec -T sample-db \
psql -U sample -d sample -q <<'SQL'
CREATE SCHEMA IF NOT EXISTS analytics;
CREATE TABLE IF NOT EXISTS analytics.old_user_stats (
user_id INTEGER PRIMARY KEY,
booking_cnt INTEGER,
total_amount DECIMAL(10,2),
last_booking_date DATE,
user_type VARCHAR(50)
);
INSERT INTO analytics.old_user_stats (user_id, booking_cnt, total_amount, last_booking_date, user_type)
VALUES
(1, 5, 150.00, '2026-01-15', 'regular'),
(2, 12, 480.50, '2026-02-20', 'power'),
(3, 1, 25.00, '2026-03-01', 'new')
ON CONFLICT (user_id) DO NOTHING;
SQL
ok "Test tables and data created"
# ---------------------------------------------------------------------------
# Phase 5: Sync Metabase and wait for tables to appear
# ---------------------------------------------------------------------------
info "Phase 5: Triggering Metabase sync..."
curl -sf -X POST "${METABASE_URL}/api/database/${DB_ID}/sync_schema" \
-H "$AUTH_HEADER" > /dev/null
# Wait for old_user_stats to appear in Metabase table list
elapsed=0
TABLE_ID=""
while [ $elapsed -lt $MAX_WAIT_SYNC ]; do
TABLE_ID=$(curl -sf "${METABASE_URL}/api/table" \
-H "$AUTH_HEADER" \
| json_get "str([t['id'] for t in data if t.get('name') == 'old_user_stats' and t.get('db_id') == ${DB_ID}][0])" 2>/dev/null || echo "")
if [ -n "$TABLE_ID" ] && [ "$TABLE_ID" != "" ] && [ "$TABLE_ID" != "None" ]; then
break
fi
sleep 2
elapsed=$((elapsed + 2))
done
if [ -z "$TABLE_ID" ] || [ "$TABLE_ID" = "None" ]; then
fail "Table old_user_stats did not appear in Metabase within ${MAX_WAIT_SYNC}s"
fi
ok "Sync complete — old_user_stats is table ID ${TABLE_ID}"
# ---------------------------------------------------------------------------
# Phase 6: Create test cards (idempotent — check by name)
# ---------------------------------------------------------------------------
info "Phase 6: Creating test cards..."
EXISTING_CARDS=$(curl -sf "${METABASE_URL}/api/card" \
-H "$AUTH_HEADER" \
| json_get "[c['name'] for c in data]")
# --- Native SQL card ---
if echo "$EXISTING_CARDS" | grep -q "Test Native Card"; then
skip "Test Native Card"
else
NATIVE_CARD=$(curl -sf "${METABASE_URL}/api/card" \
-H "$AUTH_HEADER" \
-H "Content-Type: application/json" \
-d "{
\"name\": \"Test Native Card\",
\"display\": \"table\",
\"visualization_settings\": {},
\"dataset_query\": {
\"type\": \"native\",
\"native\": {
\"query\": \"SELECT user_id, booking_cnt, total_amount FROM analytics.old_user_stats WHERE user_type = {{user_type}}\",
\"template-tags\": {
\"user_type\": {
\"id\": \"b0a3d8a0-1234-5678-9abc-def012345678\",
\"type\": \"text\",
\"name\": \"user_type\",
\"display-name\": \"User Type\"
}
}
},
\"database\": ${DB_ID}
}
}")
NATIVE_CARD_ID=$(echo "$NATIVE_CARD" | json_get "data['id']")
ok "Test Native Card created (ID: ${NATIVE_CARD_ID})"
fi
# --- Query Builder card ---
if echo "$EXISTING_CARDS" | grep -q "Test Query Builder Card"; then
skip "Test Query Builder Card"
else
QB_CARD=$(curl -sf "${METABASE_URL}/api/card" \
-H "$AUTH_HEADER" \
-H "Content-Type: application/json" \
-d "{
\"name\": \"Test Query Builder Card\",
\"display\": \"table\",
\"visualization_settings\": {},
\"dataset_query\": {
\"type\": \"query\",
\"query\": {
\"source-table\": ${TABLE_ID}
},
\"database\": ${DB_ID}
}
}")
QB_CARD_ID=$(echo "$QB_CARD" | json_get "data['id']")
ok "Test Query Builder Card created (ID: ${QB_CARD_ID})"
fi
# ---------------------------------------------------------------------------
# Summary
# ---------------------------------------------------------------------------
# Fetch final card IDs for summary
ALL_CARD_IDS=$(curl -sf "${METABASE_URL}/api/card" \
-H "$AUTH_HEADER" \
| json_get "','.join(str(c['id']) for c in data)")
echo ""
echo "============================================================"
echo " Metabase Local Environment Ready!"
echo "============================================================"
echo ""
echo " URL: ${METABASE_URL}"
echo " Login: ${ADMIN_EMAIL} / ${ADMIN_PASSWORD}"
echo " Database: Sample DB (ID: ${DB_ID})"
echo " Table: analytics.old_user_stats (ID: ${TABLE_ID})"
echo " Cards: ${ALL_CARD_IDS}"
echo ""
echo " Next steps:"
echo " just docker-test-inactive-tables # test inactive tables API behavior"
echo "============================================================"
Raw
docker-test-inactive-tables.sh
#!/usr/bin/env bash
# docker-test-inactive-tables.sh — Integration test proving that deleted tables
# appear as inactive via the Metabase metadata API (remove_inactive=false).
#
# Phases:
# 1. Verify preconditions (table exists and is active)
# 2. Drop the table from the sample database
# 3. Sync Metabase and wait for deletion to be reflected
# 4. Assert the table is now inactive
# 5. Cleanup — restore the table for other tests
#
# Prerequisites: docker-setup.sh must have been run first.
# Usage: bash scripts/docker-test-inactive-tables.sh
set -euo pipefail
# ---------------------------------------------------------------------------
# Configuration
# ---------------------------------------------------------------------------
METABASE_URL="http://localhost:3000"
ADMIN_EMAIL="admin@test.com"
ADMIN_PASSWORD="Testpass1"
COMPOSE_FILE="docker-compose.yml"
MAX_WAIT_SYNC=60 # seconds to wait for sync (longer than docker-setup's 30s)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
info() { echo "==> $*"; }
ok() { echo " ✓ $*"; }
fail() { echo " ✗ $*" >&2; exit 1; }
# Parse JSON with Python (guaranteed available in this project)
json_get() {
python3 -c "import sys, json; data=json.load(sys.stdin); print($1)"
}
# ---------------------------------------------------------------------------
# Phase 1: Verify preconditions
# ---------------------------------------------------------------------------
info "Phase 1: Verifying preconditions..."
# Login
TOKEN=$(curl -sf "${METABASE_URL}/api/session" \
-H "Content-Type: application/json" \
-d "{
\"username\": \"${ADMIN_EMAIL}\",
\"password\": \"${ADMIN_PASSWORD}\"
}" | json_get "data['id']")
if [ -z "$TOKEN" ] || [ "$TOKEN" = "None" ]; then
fail "Could not obtain session token — is Metabase running?"
fi
AUTH_HEADER="X-Metabase-Session: ${TOKEN}"
# Get Sample DB ID
DB_ID=$(curl -sf "${METABASE_URL}/api/database" \
-H "$AUTH_HEADER" \
| json_get "[d['id'] for d in data.get('data', data) if d['name'] == 'Sample DB'][0]")
if [ -z "$DB_ID" ] || [ "$DB_ID" = "None" ]; then
fail "Could not find Sample DB — has docker-setup.sh been run?"
fi
# Get table ID from /api/table (use next() to safely handle empty list)
TABLE_ID=$(curl -sf "${METABASE_URL}/api/table" \
-H "$AUTH_HEADER" \
| json_get "next(iter([t['id'] for t in data if t.get('name') == 'old_user_stats' and t.get('db_id') == ${DB_ID}]), '')")
if [ -z "$TABLE_ID" ] || [ "$TABLE_ID" = "None" ]; then
fail "Table old_user_stats not found in Metabase — run 'just docker-setup' first"
fi
# SQL check: table is active in Metabase app DB
ACTIVE_FLAG=$(docker compose -f "$COMPOSE_FILE" exec -T postgres \
psql -U metabase -d metabase -t -A \
-c "SELECT active FROM metabase_table WHERE name = 'old_user_stats' AND db_id = ${DB_ID};")
if [ "$ACTIVE_FLAG" != "t" ]; then
fail "Table old_user_stats is not active in Metabase DB (expected 't', got '${ACTIVE_FLAG}')"
fi
ok "Table active in Metabase DB (SQL: active=$ACTIVE_FLAG)"
# API check: table is active via metadata endpoint
API_ACTIVE=$(curl -sf "${METABASE_URL}/api/database/${DB_ID}/metadata?skip_fields=true&remove_inactive=false" \
-H "$AUTH_HEADER" \
| json_get "[t['active'] for t in data.get('tables', []) if t['name'] == 'old_user_stats'][0]")
if [ "$API_ACTIVE" != "True" ]; then
fail "Table old_user_stats is not active via API (expected 'True', got '${API_ACTIVE}')"
fi
ok "Table active via API (remove_inactive=false: active=$API_ACTIVE)"
# ---------------------------------------------------------------------------
# Phase 2: Drop the table
# ---------------------------------------------------------------------------
info "Phase 2: Dropping table from sample database..."
docker compose -f "$COMPOSE_FILE" exec -T sample-db \
psql -U sample -d sample -c "DROP TABLE IF EXISTS analytics.old_user_stats;"
ok "Table analytics.old_user_stats dropped"
# ---------------------------------------------------------------------------
# Phase 3: Sync and wait for deletion to be reflected
# ---------------------------------------------------------------------------
info "Phase 3: Syncing Metabase and waiting for table to become inactive..."
curl -sf -X POST "${METABASE_URL}/api/database/${DB_ID}/sync_schema" \
-H "$AUTH_HEADER" > /dev/null
elapsed=0
while [ $elapsed -lt $MAX_WAIT_SYNC ]; do
FOUND=$(curl -sf "${METABASE_URL}/api/table" -H "$AUTH_HEADER" \
| json_get "[t['id'] for t in data if t.get('name') == 'old_user_stats' and t.get('db_id') == ${DB_ID}]")
if [ "$FOUND" = "[]" ]; then
ok "Table disappeared from active table list (${elapsed}s)"
break
fi
sleep 2
elapsed=$((elapsed + 2))
done
if [ $elapsed -ge $MAX_WAIT_SYNC ]; then
fail "Table old_user_stats did not become inactive within ${MAX_WAIT_SYNC}s"
fi
# ---------------------------------------------------------------------------
# Phase 4: Assert table is inactive
# ---------------------------------------------------------------------------
info "Phase 4: Asserting table is inactive..."
TEST_PASSED=true
# SQL check: table exists in Metabase app DB with active=false
ACTIVE_FLAG=$(docker compose -f "$COMPOSE_FILE" exec -T postgres \
psql -U metabase -d metabase -t -A \
-c "SELECT active FROM metabase_table WHERE name = 'old_user_stats' AND db_id = ${DB_ID};")
if [ "$ACTIVE_FLAG" != "f" ]; then
echo " ✗ Table old_user_stats is not inactive in Metabase DB (expected 'f', got '${ACTIVE_FLAG}')" >&2
TEST_PASSED=false
else
ok "Table found in Metabase app DB with active=false"
fi
# API check: table should also be returned by the metadata API with remove_inactive=false.
# The test FAILS if the table is in the app DB but missing from the API — that means
# remove_inactive=false does not actually include inactive tables.
API_TABLES=$(curl -sf "${METABASE_URL}/api/database/${DB_ID}/metadata?skip_fields=true&remove_inactive=false" \
-H "$AUTH_HEADER" \
| json_get "[t['name'] for t in data.get('tables', []) if t['name'] == 'old_user_stats']")
if [ "$API_TABLES" = "[]" ]; then
echo " ✗ Table found in Metabase app DB but NOT returned by metadata API with remove_inactive=false" >&2
TEST_PASSED=false
else
ok "Table returned by metadata API with remove_inactive=false"
fi
# ---------------------------------------------------------------------------
# Phase 5: Cleanup — restore the table
# ---------------------------------------------------------------------------
info "Phase 5: Restoring table for other tests..."
docker compose -f "$COMPOSE_FILE" exec -T sample-db \
psql -U sample -d sample -q <<'SQL'
CREATE SCHEMA IF NOT EXISTS analytics;
CREATE TABLE IF NOT EXISTS analytics.old_user_stats (
user_id INTEGER PRIMARY KEY,
booking_cnt INTEGER,
total_amount DECIMAL(10,2),
last_booking_date DATE,
user_type VARCHAR(50)
);
INSERT INTO analytics.old_user_stats (user_id, booking_cnt, total_amount, last_booking_date, user_type)
VALUES
(1, 5, 150.00, '2026-01-15', 'regular'),
(2, 12, 480.50, '2026-02-20', 'power'),
(3, 1, 25.00, '2026-03-01', 'new')
ON CONFLICT (user_id) DO NOTHING;
SQL
# Trigger sync
curl -sf -X POST "${METABASE_URL}/api/database/${DB_ID}/sync_schema" \
-H "$AUTH_HEADER" > /dev/null
# Wait for table to reappear
elapsed=0
while [ $elapsed -lt $MAX_WAIT_SYNC ]; do
FOUND=$(curl -sf "${METABASE_URL}/api/table" -H "$AUTH_HEADER" \
| json_get "[t['id'] for t in data if t.get('name') == 'old_user_stats' and t.get('db_id') == ${DB_ID}]")
if [ "$FOUND" != "[]" ]; then
ok "Table reappeared in active table list (${elapsed}s)"
break
fi
sleep 2
elapsed=$((elapsed + 2))
done
if [ $elapsed -ge $MAX_WAIT_SYNC ]; then
fail "Table old_user_stats did not reappear within ${MAX_WAIT_SYNC}s"
fi
# Verify restored: table is active again
API_ACTIVE=$(curl -sf "${METABASE_URL}/api/database/${DB_ID}/metadata?skip_fields=true&remove_inactive=false" \
-H "$AUTH_HEADER" \
| json_get "[t['active'] for t in data.get('tables', []) if t['name'] == 'old_user_stats'][0]")
if [ "$API_ACTIVE" != "True" ]; then
fail "Table old_user_stats should be active again (expected 'True', got '${API_ACTIVE}')"
fi
ok "Table restored and active again"
# ---------------------------------------------------------------------------
# Summary
# ---------------------------------------------------------------------------
echo ""
if [ "$TEST_PASSED" = true ]; then
echo "============================================================"
echo " Inactive Tables Test: PASSED"
echo "============================================================"
echo " All assertions verified:"
echo " ✓ Table initially active in Metabase DB and API"
echo " ✓ After deletion + sync: table found in Metabase app DB with active=false"
echo " ✓ After deletion + sync: table returned by metadata API with remove_inactive=false"
echo " ✓ Table restored and active again"
echo "============================================================"
else
echo "============================================================"
echo " Inactive Tables Test: FAILED"
echo "============================================================"
echo " Assertions:"
echo " ✓ Table initially active in Metabase DB and API"
echo " ✓ After deletion + sync: table found in Metabase app DB with active=false"
echo " ✗ After deletion + sync: table NOT returned by metadata API with remove_inactive=false"
echo " ✓ Table restored and active again"
echo "============================================================"
exit 1
fi
Raw
justfile
# Inactive tables integration test
# Requires: docker, just, python3
# Stop containers and destroy volumes (clean slate)
docker-reset:
docker compose -f docker-compose.yml down -v
# Start containers in background
docker-up:
docker compose -f docker-compose.yml up -d
# Bootstrap local Metabase: start containers + seed data + create cards
docker-setup: docker-up
bash docker-setup.sh
# Test that deleted tables appear as inactive via the metadata API
docker-test-inactive-tables:
bash docker-test-inactive-tables.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment