Databricks Connection

Connect your Databricks workspace to enable AI-powered analysis across your lakehouse data with full catalog, schema, and table introspection.

What you’ll need: Workspace URL, authentication (access token or service principal), SQL warehouse HTTP path, and catalog/schema names.

Quick Setup

Get connection details from SQL Warehouse

In Databricks:

Go to SQL Warehouses → Select your warehouse
Click Connection Details tab
Copy Server hostname and HTTP path

Generate authentication

Access Token: Settings → Developer → Access tokens → Generate new tokenOr use Service Principal (see Authentication section)

Connect in SignalPilot

Database icon → Add Connection → DatabricksEnter URL, auth, HTTP path, catalog → Test Connection

SignalPilot discovers all accessible catalogs, schemas, and tables.

Connection Fields

Field	Required	Description	Example
Connection Name	✅	Friendly label	`Databricks Production`
Connection URL	✅	Workspace URL	`https://adb-123456789012.3.azuredatabricks.net`
Authentication	✅	Access Token or Service Principal	See below
HTTP Path	✅	SQL warehouse path	`/sql/1.0/warehouses/abc123def456`
Catalog	✅	Unity Catalog name	`main`, `prod`
Schema	Optional	Default schema	`analytics`

Credentials are encrypted locally using AES-256.

Authentication

Access Token
Service Principal

For: Individual use, developmentHow to create:

Settings → Developer → Access tokens
Generate new token → Set description and lifetime
Copy token immediately

Paste in: Databricks Access Token field

Key Capabilities

Unity Catalog

Three-level namespace: catalog.schema.table

Delta Lake

Query Delta tables with time travel support

Schema Discovery

Auto-introspect catalogs, schemas, tables, columns

Optimized SQL

Generate Databricks SQL with proper syntax

Security Best Practices

Read-only service principal

-- Grant catalog access
GRANT USE CATALOG ON CATALOG main TO `sp-id`;
GRANT USE SCHEMA ON SCHEMA main.analytics TO `sp-id`;
GRANT SELECT ON SCHEMA main.analytics TO `sp-id`;

Rotate tokens regularly

Set expiration when generating tokens
Rotate OAuth secrets every 6-12 months
Revoke old credentials after rotation

Troubleshooting

Connection timeout

Solutions:

Verify workspace URL format: https://adb-<id>.<region>.azuredatabricks.net
Ensure SQL Warehouse is running
Check network/firewall settings

Authentication failed

Solutions:

Generate new access token
Verify service principal has catalog permissions
Check token hasn’t expired

'Catalog does not exist'

Solutions:

List catalogs: SHOW CATALOGS
Use hive_metastore if Unity Catalog isn’t enabled
Grant access: GRANT USE CATALOG ON CATALOG main TO user

Next Steps

Query Your Lakehouse

Start analyzing Databricks data

dbt + Databricks

Connect dbt for Unity Catalog lineage

Snowflake

Add data warehouse connection

MCP Overview

External context connections

Introduction

Getting Started

Integrations

Core Concepts

Databricks

Databricks Connection

Quick Setup

Connection Fields

Authentication

Key Capabilities

Unity Catalog

Delta Lake

Schema Discovery

Optimized SQL

Security Best Practices

Troubleshooting

Next Steps

Query Your Lakehouse

dbt + Databricks

Snowflake

MCP Overview

​Databricks Connection

​Quick Setup

​Connection Fields

​Authentication

​Key Capabilities

Unity Catalog

Delta Lake

Schema Discovery

Optimized SQL

​Security Best Practices

​Troubleshooting

​Next Steps

Query Your Lakehouse

dbt + Databricks

Snowflake

MCP Overview

Databricks Connection

Quick Setup

Connection Fields

Authentication

Key Capabilities

Security Best Practices

Troubleshooting

Next Steps