Skip to content

test: add integration tests for CLI client#8689

Merged
reedham-aws merged 6 commits intoaws:feat-buildkitfrom
reedham-aws:buildkit-tests-integ
Mar 5, 2026
Merged

test: add integration tests for CLI client#8689
reedham-aws merged 6 commits intoaws:feat-buildkitfrom
reedham-aws:buildkit-tests-integ

Conversation

@reedham-aws
Copy link
Contributor

Integration test PR for buildkit/container CLI support.

Added a few parameterized tests for some image based tests, as well as just general tests for using the --use-buildkit flag. Worth noting is that when using the CLI, there were a lot of resource warnings that the sockets were being left open. I couldn't figure out a fix to this, but I think they're harmless warnings so I ignored them.

Implementation PR: #8661.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added area/local/start-api sam local start-api command area/build sam build command area/local/invoke sam local invoke command area/local/start-invoke area/schema JSON schema file pr/internal labels Feb 24, 2026
@reedham-aws reedham-aws marked this pull request as ready for review February 24, 2026 01:31
@reedham-aws reedham-aws requested a review from a team as a code owner February 24, 2026 01:31
@roger-zhangg
Copy link
Member

prob need to merge from develop for the chardet thing

Copy link
Member

@roger-zhangg roger-zhangg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would suggest to add a ARM64 test, and then mark one test -> x86/arm64 as https://github.com/aws/aws-sam-cli/blob/develop/CONTRIBUTING.md#tier-1-cross-platform-smoke-tests for running that test on finch / windows (to be added)

@reedham-aws
Copy link
Contributor Author

would suggest to add a ARM64 test, and then mark one test -> x86/arm64 as https://github.com/aws/aws-sam-cli/blob/develop/CONTRIBUTING.md#tier-1-cross-platform-smoke-tests for running that test on finch / windows (to be added)

Done, I added as tier1, although not sure if this would have fallen more into the tier1_extra category. I think the testing of the CLI is important enough, not sure about the parameters.

@reedham-aws
Copy link
Contributor Author

reedham-aws commented Feb 27, 2026

command_list += ["--debug"]

if use_buildkit:
command_list += ["--use-buildkit"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will there be change in VS Code extension for this feature? If so, should we have a negate version of this as well: --no-use-buildkit. This should be similar to

"--use-container/--no-use-container",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--no-use-buildkit exists in the implementation, is it required to be in the test options as well?

@reedham-aws
Copy link
Contributor Author

reedham-aws commented Mar 2, 2026

Latest integration test run: https://github.com/aws/aws-sam-cli/actions/runs/22595806682. I removed the ARM64 test because it wasn't working well in x86 environment. While BuildKit should technically support this, that's more testing a BuildKit feature than our implementation. I think QEMU needs to be set up in some way in the environment for it to fully work.

Edit: All related and non-flaky tests are passing

@reedham-aws reedham-aws force-pushed the buildkit-tests-integ branch from e4c0cab to bacfe0c Compare March 4, 2026 00:35
@reedham-aws
Copy link
Contributor Author

Latest test run: https://github.com/aws/aws-sam-cli/actions/runs/22649580848

Before, the tier1-finch tests were failing. This is because the test was incorrectly finding that the finch CLI was unavailable. The problem was that in the CI runner, finch required sudo to run, but the code did not supply it. The solution was to make it so that sudo was not required for finch or nerdctl.

sudo chmod +s "$NERDCTL_PATH"
sudo chmod +s /usr/bin/finch

@reedham-aws reedham-aws merged commit b9aee06 into aws:feat-buildkit Mar 5, 2026
58 of 60 checks passed
@reedham-aws reedham-aws deleted the buildkit-tests-integ branch March 5, 2026 00:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/build sam build command area/local/invoke sam local invoke command area/local/start-api sam local start-api command area/local/start-invoke area/schema JSON schema file pr/internal

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants