Permissions and Data Handling
Review every permission the SeeMore Data service principal holds in Databricks, what SeeMore Data reads with it, and how that data is handled.
This page is the reference for a security review of the Databricks integration. It lists every permission the service principal holds, what SeeMore Data reads with it, and how that data is stored. Create Databricks Service Principal and Privileges and Automated Setup explain how to set these permissions up.
At a glance
Rows in your tables, volumes or files
Doesn't read. It asks for no SELECT on your catalogs. See inherited access
Model prompts and responses
Doesn't read. It isn't granted inference tables or payload logs
Notebook and file contents, secret values
Doesn't read
Catalog, schema and table names, owners, comments, tags and view SQL
Reads, on the catalogs you grant BROWSE on
Column names and types
Doesn't read. They're visible to BROWSE
Cost, usage and prices
Reads
Warehouse, cluster, job and pipeline configuration and runs
Reads
Who ran each query, when, for how long, and its error message
Reads
Full SQL text
Reads, through the databricks_pii_access group that the setup adds the service principal to. You can skip that step
Audit events, with their request parameters
Reads
AI request metadata: requester, client IP, user agent, tokens, latency
Reads
MLflow runs, with their parameters, tags and metrics
Reads
Workspace user names and email addresses
Reads
Changes to your account
Only on SQL warehouses you grant Can manage on. There, SeeMore Data changes size, cluster limits or auto stop when one of your users saves a change. It doesn't start, stop or delete warehouses, or change their permissions. Without Can manage, it changes nothing. Can monitor would allow starting a warehouse; SeeMore Data doesn't
Permissions
Service principal with an OAuth secret
Account
Signing in. It holds no account role
Everything
Workspace membership, as a user, with the Workspace access and Databricks SQL access entitlements
Each workspace you add it to
Signing in to that workspace and using its SQL warehouses. It doesn't need cluster or pool creation
Query history, live queries and user lists for that workspace
USE CATALOG
system catalog
Reaching the system schemas
Everything
USE SCHEMA and SELECT
The nine system schemas
Reading the system tables listed below
Everything; tags, serving, ai_gateway and mlflow are optional
BROWSE
Each catalog you choose
Metadata only: no row of data
The asset catalog
Can use
seemore_warehouse in each region
Running SeeMore Data's reads
Everything
Can monitor
Each warehouse you want watched live
Seeing the warehouse, its query history and query profiles. Databricks also lets it start the warehouse and run queries; SeeMore Data does neither
Real Time Monitoring
Can view
Each model serving endpoint
Seeing the endpoint's configuration and state. It can't query or change the endpoint
AI Services endpoint details
Can manage (optional)
SQL warehouses you choose
Starting, stopping, editing and deleting the warehouse, and changing its permissions. SeeMore Data uses it only to read the warehouse's permission list, to confirm it holds the grant, and to change the warehouse's size, cluster limits and auto stop
Editing a warehouse's settings from SeeMore Data. See Can manage
databricks_pii_access membership (default; you can skip it)
Account group
Reading the full text of every query in the account
Similar-query grouping, query tagging and lineage, and the SQL shown for each query, including live queries
OAuth scopes
Scope the OAuth secret to the seven Databricks APIs SeeMore Data calls:
sql
Running its reads on the SQL warehouse, and reading and editing warehouse settings
query-history
Live queries, for Real Time Monitoring
access-management
Reading a warehouse's permission list, to confirm it holds Can manage
scim
The workspace's user list
model-serving
Model serving endpoint details, for AI Services
vector-search
The list of vector search endpoints, for AI Services
unity-catalog
The workspace's metastore, cloud and region
A token issued from a scoped secret can't call any other Databricks API, such as jobs, clusters, notebooks or workspace files. If the secret is missing one of the seven, the connection test names it. A secret with All APIs also works, but reaches every API the service principal is allowed to use.
Scopes limit which APIs the secret can call, not what it can read. That is still bounded by the permissions above and the inherited access below.
SeeMore Data never asks for SELECT, MODIFY or ALL PRIVILEGES on your catalogs, schemas or tables, or for workspace admin, metastore admin or any account role.
Inherited access
Like every identity in your account, the service principal is a member of the account users group, and of the users group of each workspace you add it to. It holds anything you've granted to those groups:
Data granted to
account users. To see it, run this for each catalog:Workspace objects open to all users, such as the
/Workspace/Sharedfolder, where Databricks gives every user full permissions by default, and clusters, warehouses or notebooks shared with all users.
SeeMore Data never queries your own tables or reads your notebooks and files. With a scoped secret, notebooks, files and clusters are out of the secret's reach, but data granted to account users and warehouses shared with all users are still reachable through SQL. With an All APIs secret, all of this access is reachable. Either way, store the secret as carefully as any other credential.
What SeeMore Data reads
system.access
Workspace inventory, account identification, table lineage, Databricks Assistant usage, and the audit log — each event's action, who made it and its full request parameters
system.billing
Consumption per SKU and workspace, and the list and negotiated rates behind it
system.compute
Warehouse and cluster configuration, start/stop/scale events, per-node CPU and memory utilization, and how each cluster node launched and ended
system.lakeflow
Jobs, tasks and pipelines, and the timeline of every run, so scheduled work can be costed
system.query
Per-query cost attribution and queue pressure, including who ran each query and the error message of a failed one
system.tags
The governed tag keys defined on your account, used to group spend by tag
system.serving
Which model served each inference request, for whom, at what token count, and the usage context your callers attach
system.ai_gateway
Every request through a Unity Gateway model service — destination, requester, client IP address, URL, user agent, your request tags, tokens and latency — and Databricks' estimate of what external model providers bill for them
system.mlflow
MLflow experiments and runs, with each run's parameters, tags and metrics
Some of these fields are free text that your teams write: query error messages, audit request parameters (which can include job and notebook parameters, file paths and cluster settings), MLflow parameters and tags, and request tags. SeeMore Data copies them as they are, so they can carry any value your teams put in them.
SeeMore Data never reads model prompts or responses. Nothing in system.serving or system.ai_gateway holds request or response bodies.
Beyond the system schemas, SeeMore Data reads system.information_schema for the asset catalog, on the catalogs you granted BROWSE on: catalog, schema and table names, table types, owners, comments and timestamps, their tags (keys and values), and the SQL definition of each view, which it uses for view lineage. Through the Databricks API, it also lists your workspace users (names and email addresses), your SQL warehouses and the queries running on them, your model serving endpoints, and your Vector Search endpoints.
Account-wide tables are read once, through the connected region's SQL warehouse. Everything else is read region by region — see what SeeMore Data reads through each SQL warehouse.
What BROWSE exposes
Through information_schema, a principal holding BROWSE can see catalog, schema and table names, comments, owners and timestamps, column names and data types, tags, constraints, and the SQL definition of views. It can't read a row of table data, and DESCRIBE TABLE is refused. SeeMore Data reads the subset above and not the columns. If your view definitions or tag values hold sensitive literals, grant BROWSE only on the catalogs you're comfortable sharing them from.
Query text and error messages
Databricks returns statement_text in system.query.history as <REDACTED> to everyone outside the account-level group databricks_pii_access. This is platform behavior that applies to every tool — see Databricks' query history reference. The setup adds the service principal to the group by default, so SeeMore Data stores the full text of your SQL as it is. It doesn't remove literals from Databricks queries.
Error messages aren't masked. Databricks returns error_message in full whether or not the service principal is in the group, and a message can quote values from the failed statement.
If your workspaces use customer-managed keys, Databricks encrypts statement_text and error_message, and no group membership changes that. An account admin can make them readable by adding a key configuration to the system catalog. Doing so resets the grants on system.query across the metastore, so run the grants, or the setup script, again afterwards.
Tables read
Five tables must be readable: system.access.workspaces_latest, system.billing.usage, system.billing.list_prices, system.query.history and system.compute.warehouse_events. Permission checks also reads the optional tables below and reports any it can't read as Ready with optional gaps. A missing one leaves only its feature's views empty. If your policy requires table-level grants rather than schema-level ones, grant all of them, and revisit the list when SeeMore Data starts reading a new table.
An empty result isn't a missing grant — a workspace with no job runs returns no rows either. A permission problem fails with TABLE_OR_VIEW_NOT_FOUND or INSUFFICIENT_PERMISSIONS.
What some gaps cost you:
node_timelinegives per-node CPU and memory for classic job and all-purpose clusters. Jobs on serverless compute or SQL warehouses show statement metrics fromsystem.queryinstead, with or without it.instance_eventsis in Public Preview at Databricks. Without it, cluster pages show no spot evictions or start-up time.auditis where cluster starts, stops and notebook attaches come from. Databricks records no reason when a cluster stops, so SeeMore Data infers it from these events. Without them, notebook time on a cluster can't be attributed to anyone.
How SeeMore Data handles your Databricks data
SeeMore Data copies the system-table rows and API results listed above into its own storage on AWS. The first import goes back between 14 and 180 days, depending on the table: for example, 30 days for query history and lineage, 90 days for audit events, Databricks Assistant events and job runs, and 180 days for billing. Price lists are imported in full.
Encryption. Everything is encrypted in transit (TLS) and at rest, including your connection credentials.
Deletion. To have imported data deleted, contact the SeeMore Data team.
Optional cloud billing access
The optional AWS integration adds AWS billing. It uses a separate read-only role to read billing reports from your S3 bucket; your Databricks service principal does not grant this access. The guide explains the report scope, AWS permissions, and how to revoke access and request deletion of imported cloud-cost data.
See Security Model & Architecture for certifications.
Last updated
