[Task]: Update GitHub Actions Self-Hosted Runners to Ubuntu 24.04

[jrmccluskey]

What needs to happen?

The self-hosted runners are currently running on Ubuntu 20.04, which is not compatible with Python 3.14 and is blocking support. The base docker image should be updated to Ubuntu 24.04 and a new runner pool should be defined to use it, then we can slowly migrate jobs over to that image.

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[damccorm]

I think steps here are:

• Update https://github.com/apache/beam/blob/master/.github/gh-actions-self-hosted-runners/arc/images/Dockerfile to use 24.04
• After https://github.com/apache/beam/blob/master/.github/workflows/build_runner_image.yml runs, update

  beam/.github/gh-actions-self-hosted-runners/arc/environments/beam.env

  Line 38 in e2a2b29

  runner_image = "us-central1-docker.pkg.dev/apache-beam-testing/beam-github-actions/beam-arc-runner:d7cd81a1649bc665581951d2330c4b8acd19ed72"

  to add a new pool with the 24.04 label
• Migrate over a few workflows to test this, then migrate the rest
• Update

  beam/.github/gh-actions-self-hosted-runners/arc/environments/beam.env

  Line 38 in e2a2b29

  runner_image = "us-central1-docker.pkg.dev/apache-beam-testing/beam-github-actions/beam-arc-runner:d7cd81a1649bc665581951d2330c4b8acd19ed72"

  to make 24.04 the default runner/label