Skip to content

catalog: convert mz_views into a view over the catalog#35807

Draft
teskje wants to merge 3 commits intoMaterializeInc:mainfrom
teskje:mv_views-view
Draft

catalog: convert mz_views into a view over the catalog#35807
teskje wants to merge 3 commits intoMaterializeInc:mainfrom
teskje:mv_views-view

Conversation

@teskje
Copy link
Copy Markdown
Contributor

@teskje teskje commented Mar 31, 2026

Blocked by temporary views missing from the catalog. See Slack thread.


This PR converts the mz_views builtin table into a materialized view over the catalog.

It includes some pre-work to make this possible:

  • Introduces a constant builtin view mz_builtin_views that exposes... builtin views. The mz_views query joins against that view to augment the catalog contents (which only include user views) with all the builtin views.
  • Introduces two new internal SQL functions:
    • parse_catalog_create_sql, to extract information from the view create_sql
    • redact_sql, to derived redacted_create_sql from the view create_sql

Note that mz_builtin_views doesn't contain itself, so it will be hidden from the catalog. I think this is fine since no user should care about this view, but I don't think we have precedent for hidden objects. The reason why mz_builtin_views doesn't contain itself is that its create_sql would be self-recursive and we can't have that. A solution would be to use a placeholder create_sql string, but we also don't have precedent for that.

Alternatives to having mz_builtin_views as a constant builtin view:

  • Make it a builtin table that gets updated during bootstrap. We want to move away from builtin tables as they complicate having concurrent envds, so this seems like a step in the wrong direction. Builtin views pose no issues for concurrent envds.
  • Make it a builtin table function. This doesn't work because SQL funcs are defined in mz-expr, while builtins are defined in mz-catalog, and the dependency relationship is the wrong way round. We could introduce a way to add table functions that get evaluated during planning time, but afaict that infrastructure doesn't exist yet.
  • Insert builtins into the catalog so we can simply treat them like user views. This might be nice but is also a lot of work, and might also complicate 0dt upgrades by adding more state to the catalog that will need to be migrated.

Motivation

Part of SQL-118

@github-actions
Copy link
Copy Markdown
Contributor

Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone.

PR title guidelines

  • Use imperative mood: "Fix X" not "Fixed X" or "Fixes X"
  • Be specific: "Fix panic in catalog sync when controller restarts" not "Fix bug" or "Update catalog code"
  • Prefix with area if helpful: compute: , storage: , adapter: , sql:

Pre-merge checklist

  • The PR title is descriptive and will make sense in the git log.
  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).

@teskje teskje force-pushed the mv_views-view branch 4 times, most recently from 9e4da37 to b056134 Compare March 31, 2026 16:08
teskje added 3 commits March 31, 2026 18:37
This commit adds the first constant builtin view exposing builtin
objects to the catalog. Specifically, `mz_builtin_views` reports what
builtin views exist. It is required to define `mz_views` as a view over
the catalog, since the catalog does not contain builtin objects.

Further builtin views exposing other builtin object types will be added
as needed to support converting more builtin tables.
This commit adds two new internal sqlfuncs to support converting
`mz_views` into a view over the catalog.

* `parse_catalog_create_sql` parses a create_sql string and returns
  information extracted from it as JSON
* `redact_sql` redacts a given SQL statement
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant