feat: add tarball overlays for source archive modification

docs/user/reference/config/overlays.md

@@ -47,21 +47,34 @@ successfully makes a replacement to at least one matching file.
 | `file-remove` | Removes a file | `file` | Glob pattern for files to remove |
 | `file-rename` | Renames a file within the same directory | `file`, `replacement` | Name of file to rename |
 
+### Tarball Overlays
+
+These overlays modify files **inside** source tarballs. The tarball is extracted into a temporary directory, modifications are applied, and the tarball is repacked with the same compression format. Extraction and repacking are handled natively; patch application requires the `patch` command on the host.
+
+> **Note:** Tarball overlays are applied before spec and file overlays, so subsequent overlays see the modified tarball. The `tarball-patch` overlay type requires the `patch` command to be installed on the host.
+
+| Type | Description | Required Fields |
+|------|-------------|-----------------|
+| `tarball-file-remove` | Removes file(s) matching a glob pattern from inside a tarball | `tarball`, `file` |
+| `tarball-search-replace` | Regex-based search and replace on file(s) inside a tarball | `tarball`, `file`, `regex` |
+| `tarball-patch` | Applies a unified diff patch to the extracted tarball contents | `tarball`, `source` |
+
 ## Field Reference
 
 | Field | TOML Key | Description | Used By |
 |-------|----------|-------------|---------|
 | Type | `type` | **Required.** The overlay type to apply | All overlays |
 | Description | `description` | Human-readable explanation documenting the need for the change; helps identify overlays in error messages | All (optional) |
+| Tarball | `tarball` | The source tarball filename to modify (must be a basename, not a path) | `tarball-file-remove`, `tarball-search-replace`, `tarball-patch` |
 | Tag | `tag` | The spec tag name (e.g., `BuildRequires`, `Requires`, `Version`) | `spec-add-tag`, `spec-insert-tag`, `spec-set-tag`, `spec-update-tag`, `spec-remove-tag` |
-| Value | `value` | The tag value to set, or value to match for removal | `spec-add-tag`, `spec-insert-tag`, `spec-set-tag`, `spec-update-tag`, `spec-remove-tag` (optional for matching) |
+| Value | `value` | The tag value to set, or value to match for removal. For `tarball-patch`, sets the patch strip level (default: `1`, equivalent to `patch -p1`). | `spec-add-tag`, `spec-insert-tag`, `spec-set-tag`, `spec-update-tag`, `spec-remove-tag` (optional for matching), `tarball-patch` (optional) |
 | Section | `section` | The spec section to target (e.g., `%build`, `%install`, `%files`, `%description`) | `spec-prepend-lines`, `spec-append-lines`, `spec-search-replace` (optional), `spec-remove-section` |
 | Package | `package` | The sub-package name for multi-package specs; omit to target the main package | All spec overlays (optional, except `spec-remove-subpackage` which **requires** it) |
-| Regex | `regex` | Regular expression pattern to match | `spec-search-replace`, `file-search-replace` |
-| Replacement | `replacement` | Literal replacement text; capture group references like `$1` are **not** expanded. Omit or leave empty to delete matched text. | `spec-search-replace`, `file-search-replace`, `file-rename` |
+| Regex | `regex` | Regular expression pattern to match | `spec-search-replace`, `file-search-replace`, `tarball-search-replace` |
+| Replacement | `replacement` | Literal replacement text; capture group references like `$1` are **not** expanded. Omit or leave empty to delete matched text. | `spec-search-replace`, `file-search-replace`, `file-rename`, `tarball-search-replace` |
 | Lines | `lines` | Array of text lines to insert | `spec-prepend-lines`, `spec-append-lines`, `file-prepend-lines` |
-| File | `file` | The name of the non-spec file to modify or add | `file-prepend-lines`, `file-search-replace`, `file-add`, `file-remove`, `file-rename`, `patch-add` (optional), `patch-remove` |
-| Source | `source` | Path to source file for `file-add` and `patch-add`; relative paths are relative to the config file | `file-add`, `patch-add` |
+| File | `file` | The name of the non-spec file to modify or add | `file-prepend-lines`, `file-search-replace`, `file-add`, `file-remove`, `file-rename`, `patch-add` (optional), `patch-remove`, `tarball-file-remove`, `tarball-search-replace` |
+| Source | `source` | Path to source file for `file-add` and `patch-add`; relative paths are relative to the config file | `file-add`, `patch-add`, `tarball-patch` |
 
 > **Note:** For `file-rename`, the `replacement` field is a **filename only** (not a path). The file is renamed within its current directory.
 
@@ -274,6 +287,55 @@ description = "Remove CVE patches that are now upstream"
 > `PatchN` tags. Macro-based tag numbering (e.g., `Patch%{n}`) is not expanded and may
 > conflict with auto-assigned numbers.
 
+### Removing a File from a Tarball
+
+The `tarball-file-remove` overlay deletes files matching a glob pattern from inside a source
+tarball. The tarball is extracted, matching files are removed, and the tarball is repacked.
+
+```toml
+[[components.mypackage.overlays]]
+type = "tarball-file-remove"
+tarball = "mypackage-1.0.tar.gz"
+file = "vendor/**"
+description = "Remove bundled vendor directory"
+```
+
+### Search and Replace Inside a Tarball
+
+```toml
+[[components.mypackage.overlays]]
+type = "tarball-search-replace"
+tarball = "mypackage-1.0.tar.xz"
+file = "configure.ac"
+regex = "AC_CHECK_LIB\\(old_lib"
+replacement = "AC_CHECK_LIB(new_lib"
+description = "Update library reference in configure script"
+```
+
+### Applying a Patch to Tarball Contents
+
+The `tarball-patch` overlay applies a unified diff patch to the extracted tarball contents.
+By default, it uses `patch -p1`. Use the `value` field to change the strip level.
+
+```toml
+[[components.mypackage.overlays]]
+type = "tarball-patch"
+tarball = "mypackage-1.0.tar.gz"
+source = "patches/fix-build.patch"
+description = "Fix build issue in upstream source"
+```
+
+With a custom strip level:
+
+```toml
+[[components.mypackage.overlays]]
+type = "tarball-patch"
+tarball = "mypackage-1.0.tar.gz"
+source = "patches/fix-build.patch"
+value = "0"
+description = "Apply patch with -p0 strip level"
+```
+
 ### Removing a Section
 
 The `spec-remove-section` overlay removes an entire section from the spec, including its

internal/app/azldev/cmds/component/preparesources.go

@@ -138,13 +138,7 @@ func PrepareComponentSources(env *azldev.Env, options *PrepareSourcesOptions) er
 		)
 	}
 
-	if options.AllowNoHashes {
-		preparerOpts = append(preparerOpts, sources.WithAllowNoHashes())
-	}
-
-	if options.SkipSources {
-		preparerOpts = append(preparerOpts, sources.WithSkipLookaside())
-	}
+	preparerOpts = appendPrepareSourcesOptions(env, preparerOpts, options, distro)
 
 	preparer, err := sources.NewPreparer(sourceManager, env.FS(), env, env, preparerOpts...)
 	if err != nil {
@@ -194,3 +188,23 @@ func CheckOutputDir(env *azldev.Env, options *PrepareSourcesOptions) error {
 			"use --force to delete and recreate it",
 		options.OutputDir)
 }
+
+// appendPrepareSourcesOptions appends conditional preparer options that control
+// hashing and lookaside behavior. Extracted from
+// [PrepareComponentSources] to keep cyclomatic complexity within limits.
+func appendPrepareSourcesOptions(
+	_ *azldev.Env,
+	opts []sources.PreparerOption,
+	options *PrepareSourcesOptions,
+	_ sourceproviders.ResolvedDistro,
+) []sources.PreparerOption {
+	if options.AllowNoHashes {
+		opts = append(opts, sources.WithAllowNoHashes())
+	}
+
+	if options.SkipSources {
+		opts = append(opts, sources.WithSkipLookaside())
+	}
+
+	return opts
+}

internal/app/azldev/core/sources/sourceprep.go

@@ -246,8 +246,7 @@ func (p *sourcePreparerImpl) PrepareSources(
 	}
 
 	if applyOverlays {
-		err := p.applyOverlaysToSources(ctx, component, outputDir)
-		if err != nil {
+		if err := p.applyOverlaysToSources(ctx, component, outputDir); err != nil {
 			return err
 		}
 
@@ -276,9 +275,6 @@ func (p *sourcePreparerImpl) PrepareSources(
 func (p *sourcePreparerImpl) applyOverlaysToSources(
 	ctx context.Context, component components.Component, outputDir string,
 ) error {
-	// Emit computed macros to a macros file in the output directory.
-	// If the build configuration produces no macros, no file is written and
-	// macrosFileName will be empty.
 	var macrosFileName string
 
 	macrosFilePath, err := p.writeMacrosFile(component, outputDir)
@@ -291,32 +287,27 @@ func (p *sourcePreparerImpl) applyOverlaysToSources(
 		macrosFileName = filepath.Base(macrosFilePath)
 	}
 
-	// Apply all overlays to prepared sources.
 	if err := p.applyOverlays(ctx, component, outputDir, macrosFileName); err != nil {
-		return fmt.Errorf("failed to apply overlays for component %#q:\n%w", component.GetName(), err)
+		return fmt.Errorf("failed to apply overlays for component %#q:\n%w",
+			component.GetName(), err)
 	}
 
 	return nil
 }
 
 // applyOverlays applies all overlays (user-defined and system-generated) to the
-// component sources. Overlay application is decoupled from git history generation:
-// overlays modify the working tree; synthetic history is recorded separately by
-// [trySyntheticHistory].
+// component sources.
 func (p *sourcePreparerImpl) applyOverlays(
-	_ context.Context, component components.Component, sourcesDirPath, macrosFileName string,
+	ctx context.Context, component components.Component, sourcesDirPath, macrosFileName string,
 ) error {
 	event := p.eventListener.StartEvent("Applying overlays", "component", component.GetName())
 	defer event.End()
 
-	// Resolve the spec path once for all overlay operations in this call.
 	absSpecPath, err := p.resolveSpecPath(component, sourcesDirPath)
 	if err != nil {
 		return err
 	}
 
-	// Collect all overlays in application order. This ensures every change is
-	// captured in the synthetic history, including build configuration changes.
 	allOverlays, err := p.collectOverlays(component, macrosFileName)
 	if err != nil {
 		return fmt.Errorf("failed to collect overlays for component %#q:\n%w", component.GetName(), err)
@@ -326,14 +317,54 @@ func (p *sourcePreparerImpl) applyOverlays(
 		return nil
 	}
 
-	// Apply all overlays to the working tree.
+	// Tarball overlays are applied first (they modify archived source files
+	// in-place), followed by spec and loose-file overlays. Each function
+	// self-filters to the overlay types it handles.
+	if err := p.applyTarballOverlayGroup(ctx, component, sourcesDirPath, allOverlays); err != nil {
+		return err
+	}
+
 	if err := p.applyOverlayList(allOverlays, sourcesDirPath, absSpecPath); err != nil {
 		return fmt.Errorf("failed to apply overlays for component %#q:\n%w", component.GetName(), err)
 	}
 
 	return nil
 }
 
+// applyTarballOverlayGroup applies tarball overlays. Skipped when source
+// downloads were not performed.
+func (p *sourcePreparerImpl) applyTarballOverlayGroup(
+	ctx context.Context, component components.Component,
+	sourcesDirPath string, tarballOverlays []projectconfig.ComponentOverlay,
+) error {
+	if len(tarballOverlays) == 0 {
+		return nil
+	}
+
+	if p.skipLookaside {
+		slog.Warn("Skipping tarball overlays because source downloads were skipped (--skip-sources)",
+			"component", component.GetName(),
+			"count", len(tarballOverlays))
+
+		return nil
+	}
+
+	cmdFactory, ok := p.dryRunnable.(opctx.CmdFactory)
+	if !ok {
+		return errors.New(
+			"tarball overlays require a CmdFactory; the provided DryRunnable does not implement it")
+	}
+
+	if err := applyTarballOverlays(
+		ctx, cmdFactory, p.fs, p.eventListener, sourcesDirPath, tarballOverlays,
+	); err != nil {
+		return fmt.Errorf("failed to apply tarball overlays for component %#q:\n%w",
+			component.GetName(), err)
+	}
+
+	return nil
+}
+
 // collectOverlays gathers all overlays for a component into a single ordered slice:
 // macros-load first, then user overlays, followed by check-skip and file-header overlays.
 func (p *sourcePreparerImpl) collectOverlays(
@@ -634,9 +665,17 @@ func (p *sourcePreparerImpl) DiffSources(
 // enforced by [projectconfig.ConfigFile.Validate]). Setting `ReplaceUpstream` = true without
 // a matching upstream entry is also an error: the user expressed intent to replace something
 // that isn't there, which almost certainly indicates a stale config or filename typo.
-func (p *sourcePreparerImpl) updateSourcesFile(component components.Component, outputDir string) error {
-	sourceFiles := component.GetConfig().SourceFiles
-	if len(sourceFiles) == 0 {
+func (p *sourcePreparerImpl) updateSourcesFile(
+	component components.Component, outputDir string,
+) error {
+	config := component.GetConfig()
+	sourceFiles := config.SourceFiles
+
+	// Derive tarball names from the component's overlays — no need to thread
+	// them through the overlay application chain.
+	modifiedTarballs := tarballNamesFromOverlays(config.Overlays)
+
+	if len(sourceFiles) == 0 && len(modifiedTarballs) == 0 {
 		return nil
 	}
 
@@ -647,7 +686,19 @@ func (p *sourcePreparerImpl) updateSourcesFile(component components.Component, o
 		return err
 	}
 
-	mergedLines, err := p.buildSourceEntries(sourceFiles, existingContent, component.GetName(), outputDir)
+	// Parse once, then rehash modified tarballs and merge source-files entries
+	// on the parsed representation — single parse, single write.
+	existingLines, err := fedorasource.ReadSourcesFile(existingContent)
+	if err != nil {
+		return fmt.Errorf("failed to parse 'sources' file %#q:\n%w", sourcesFilePath, err)
+	}
+
+	// Rehash tarballs that were modified by tarball overlays in-place.
+	if err := p.rehashModifiedEntries(existingLines, outputDir, modifiedTarballs); err != nil {
+		return err
+	}
+
+	mergedLines, err := p.buildSourceEntries(sourceFiles, existingLines, component.GetName(), outputDir)
 	if err != nil {
 		return err
 	}
@@ -666,6 +717,47 @@ func (p *sourcePreparerImpl) updateSourcesFile(component components.Component, o
 	return nil
 }
 
+// rehashModifiedEntries updates the Raw and Entry fields of parsed 'sources' lines
+// for tarballs that were modified by tarball overlays. The hash is recomputed using
+// the same hash type as the original entry.
+func (p *sourcePreparerImpl) rehashModifiedEntries(
+	lines []fedorasource.SourcesFileLine, outputDir string, modifiedTarballs []string,
+) error {
+	if len(modifiedTarballs) == 0 {
+		return nil
+	}
+
+	modified := make(map[string]bool, len(modifiedTarballs))
+	for _, name := range modifiedTarballs {
+		modified[name] = true
+	}
+
+	for idx, line := range lines {
+		if line.Entry == nil || !modified[line.Entry.Filename] {
+			continue
+		}
+
+		tarballPath := filepath.Join(outputDir, line.Entry.Filename)
+
+		newHash, err := fileutils.ComputeFileHash(p.fs, line.Entry.HashType, tarballPath)
+		if err != nil {
+			return fmt.Errorf("rehashing modified tarball %#q:\n%w", line.Entry.Filename, err)
+		}
+
+		slog.Debug("Rehashed modified tarball in 'sources' file",
+			"tarball", line.Entry.Filename,
+			"hashType", line.Entry.HashType,
+			"oldHash", line.Entry.Hash,
+			"newHash", newHash,
+		)
+
+		lines[idx].Raw = fedorasource.FormatSourcesEntry(line.Entry.Filename, line.Entry.HashType, newHash)
+		lines[idx].Entry.Hash = newHash
+	}
+
+	return nil
+}
+
 // readSourcesFileIfExists reads the 'sources' file content if it exists, returning empty string if not.
 func (p *sourcePreparerImpl) readSourcesFileIfExists(sourcesFilePath string) (string, error) {
 	exists, err := fileutils.Exists(p.fs, sourcesFilePath)
@@ -685,32 +777,24 @@ func (p *sourcePreparerImpl) readSourcesFileIfExists(sourcesFilePath string) (st
 	return string(data), nil
 }
 
-// buildSourceEntries validates [projectconfig.SourceFileReference] entries and returns
-// the merged set of lines ready to be written to the 'sources' file. Before returning,
-// it logs an INFO-level event indicating that the 'sources' file will be updated,
-// including the counts of newly added and replaced entries.
+// buildSourceEntries merges user-declared [projectconfig.SourceFileReference] entries
+// into the parsed 'sources' lines. Returns the final set of raw lines ready to be
+// written to the 'sources' file.
 //
 // Output ordering and preservation:
-//   - Each line of [existingContent] is emitted verbatim, except for entry lines whose
+//   - Each existing line is emitted verbatim, except for entry lines whose
 //     filename matches a replacement, which are swapped for the new formatted entry.
-//     Comments and blank lines from the original file are kept in their original positions.
-//   - Brand-new entries (no upstream filename collision) are appended after the upstream
-//     content in the order they appear in [sourceFiles].
+//     Comments and blank lines are kept in their original positions.
+//   - Brand-new entries (no upstream filename collision) are appended after the
+//     existing content in the order they appear in [sourceFiles].
 //
 // Collision rules and hash resolution are documented on [sourcePreparerImpl.processSourceRef].
 func (p *sourcePreparerImpl) buildSourceEntries(
 	sourceFiles []projectconfig.SourceFileReference,
-	existingContent string,
+	existingLines []fedorasource.SourcesFileLine,
 	componentName string,
 	outputDir string,
 ) (mergedLines []string, err error) {
-	existingLines, err := fedorasource.ReadSourcesFile(existingContent)
-	if err != nil {
-		return nil, fmt.Errorf(
-			"failed to parse existing 'sources' file at %#q:\n%w",
-			filepath.Join(outputDir, fedorasource.SourcesFileName), err)
-	}
-
 	// Index upstream entries by filename for O(1) collision lookup. The parser
 	// (fedorasource.ReadSourcesFile) errors on duplicate filenames, so the
 	// entries are guaranteed unique by the time we get here.

internal/app/azldev/core/sources/sourceprep_test.go

@@ -854,7 +854,7 @@ func TestPrepareSources_UpdatesSourcesFile(t *testing.T) {
 			existingSourcesContent: "SHA512 (dup.tar.gz) = aaaa1111\nSHA512 (dup.tar.gz) = bbbb2222\n",
 			expectError:            true,
 			errorContains: []string{
-				"failed to parse existing 'sources' file",
+				"failed to parse 'sources' file",
 				"duplicate filename",
 				"dup.tar.gz",
 			},