catalog: convert mz_views into a view over the catalog#35807
Draft
teskje wants to merge 3 commits intoMaterializeInc:mainfrom
Draft
catalog: convert mz_views into a view over the catalog#35807teskje wants to merge 3 commits intoMaterializeInc:mainfrom
teskje wants to merge 3 commits intoMaterializeInc:mainfrom
Conversation
Contributor
|
Thanks for opening this PR! Here are a few tips to help make the review process smooth for everyone. PR title guidelines
Pre-merge checklist
|
9e4da37 to
b056134
Compare
This commit adds the first constant builtin view exposing builtin objects to the catalog. Specifically, `mz_builtin_views` reports what builtin views exist. It is required to define `mz_views` as a view over the catalog, since the catalog does not contain builtin objects. Further builtin views exposing other builtin object types will be added as needed to support converting more builtin tables.
This commit adds two new internal sqlfuncs to support converting `mz_views` into a view over the catalog. * `parse_catalog_create_sql` parses a create_sql string and returns information extracted from it as JSON * `redact_sql` redacts a given SQL statement
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Blocked by temporary views missing from the catalog. See Slack thread.
This PR converts the
mz_viewsbuiltin table into a materialized view over the catalog.It includes some pre-work to make this possible:
mz_builtin_viewsthat exposes... builtin views. Themz_viewsquery joins against that view to augment the catalog contents (which only include user views) with all the builtin views.parse_catalog_create_sql, to extract information from the viewcreate_sqlredact_sql, to derivedredacted_create_sqlfrom the viewcreate_sqlNote that
mz_builtin_viewsdoesn't contain itself, so it will be hidden from the catalog. I think this is fine since no user should care about this view, but I don't think we have precedent for hidden objects. The reason whymz_builtin_viewsdoesn't contain itself is that itscreate_sqlwould be self-recursive and we can't have that. A solution would be to use a placeholdercreate_sqlstring, but we also don't have precedent for that.Alternatives to having
mz_builtin_viewsas a constant builtin view:mz-expr, while builtins are defined inmz-catalog, and the dependency relationship is the wrong way round. We could introduce a way to add table functions that get evaluated during planning time, but afaict that infrastructure doesn't exist yet.Motivation
Part of SQL-118