Skip to content

Add resource: github_repository_files#3456

Open
novucs wants to merge 1 commit into
integrations:mainfrom
novucs:add-resource-github-repository-files
Open

Add resource: github_repository_files#3456
novucs wants to merge 1 commit into
integrations:mainfrom
novucs:add-resource-github-repository-files

Conversation

@novucs
Copy link
Copy Markdown

@novucs novucs commented May 28, 2026

Resolves #2909


Before the change?

When parallel_requests = true is set on the provider, managing many files in a single repository via github_repository_file with for_each fails frequently with HTTP 409 errors:

Error: PUT https://api.github.com/repos/xxx/yyy/contents/tenants/test01/myfile.yaml: 409 is at 30e15e8485d2808d7f9cdf27cecbb145252e1427 but expected 46affa9e9f78ca99714ed0e12a9c44e94e1e9055 []
│
│   with github_repository_file.tenant["myfile"],
│   on fluxcd.tf line 204, in resource "github_repository_file" "tenant":
│  204: resource "github_repository_file" "tenant" {

Cause: each github_repository_file instance issues an independent PUT /contents/{path} (Contents API), which creates one commit per file. Under Terraform's parallelism, the second concurrent write sees a branch SHA that has already advanced past the SHA it captured, and GitHub rejects it via
optimistic-concurrency check. Setting parallel_requests = false serializes HTTP at the transport layer but is documented as experimental, slows down all provider work globally, and still produces N commits when the user really wanted one.

The underlying mismatch is semantic: users want one commit containing N files, but the resource model is one resource per file.

After the change?

Adds a new resource, github_repository_files, that manages a set of files within a branch and writes them all in a single commit per apply via the Git Data API:

  1. Git.GetRefGit.GetCommit to capture the base commit + tree.
  2. Git.CreateBlob for each file content (bounded parallelism).
  3. Git.CreateTree(baseTree, entries) — a partial tree merged onto the current base, so unmanaged files in the repo are preserved.
  4. Git.CreateCommit + Git.UpdateRef.
  5. On 409/422 from UpdateRef (branch advanced under us), re-read HEAD, rebase the change onto the new base, and retry. Bounded to 3 attempts with exponential backoff. Blobs are reused across retries because they're content-addressed.

Highlights:

  • One commit per apply, regardless of how many file {} blocks the resource owns. Removes the 409-conflict failure mode entirely for users converting for_each patterns over a single repo.
  • Adopt-and-overwrite by default — a batch resource is a desired-state declaration. No overwrite_on_create flag.
  • Files outside the declared file {} blocks are never touched — the merged-tree commit only adds/updates listed paths and deletes paths that were removed from the configuration.
  • Self-healing against concurrent writers via the rebase retry loop, so the issue reporter's parallel_requests = true use case works.
  • Existing github_repository_file is unchanged — users opt in by switching.

Example, applied directly to the reporter's config:

resource "github_repository_files" "tenant" {
  repository     = data.github_repository.cluster.name
  branch         = local.cluster_branch
  commit_message = "chore: sync tenants"

  dynamic "file" {
    for_each = { for ns in local.tenant_namespaces : ns.name => ns if ns.flux.enabled }
    content {
      path    = "tenants/${local.cluster_id}/${file.key}.yaml"
      content = yamlencode(file.value.flux)
    }
  }
}

Mirrors the conventions of the existing resource: repository_id + CustomizeDiff: diffRepository for repo-rename detection, computed ref / commit_sha / tree_sha / per-file sha, commit_author/commit_email pairing, deleteResourceOn404AndSwallow304OtherwiseReturnError for ETag-cached reads,
and handleArchivedRepoDelete for archived-repo destroys.

Docs include a "Differences from github_repository_file" section (intentionally-omitted attributes) and a "Migrating from github_repository_file" guide with the terraform state rm + terraform import workflow.

Pull request checklist

  • Schema migrations have been created if needed (example)
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been reviewed and added / updated if needed (for bug fixes / features)

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

  • Yes
  • No

@github-actions
Copy link
Copy Markdown

👋 Hi! Thank you for this contribution! Just to let you know, our GitHub SDK team does a round of issue and PR reviews twice a week, every Monday and Friday! We have a process in place for prioritizing and responding to your input. Because you are a part of this community please feel free to comment, add to, or pick up any issues/PRs that are labeled with Status: Up for grabs. You & others like you are the reason all of this works! So thank you & happy coding! 🚀

@deiga deiga added the Type: Feature New feature or request label Jun 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Type: Feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MAINT]: Improve UX for github_repository_file when parallel_requests = true

2 participants