refactor: explicit Config.load() + lazy catalog config initialization

[jx2lee]

Closes #3028

Rationale for this change

This PR refactors config loading to avoid implicit IO and import-time config initialization.

Changes

• Make Config() a pure container (no implicit file/env reads)
• Add explicit Config.load() to load from config files + environment variables
• Replace import-time _ENV_CONFIG with a lazy cached getter in pyiceberg.catalog
• Update affected call sites to use Config.load() where runtime config reads are intended

Are these changes tested?

Yes!

Are there any user-facing changes?

No! (Maybe.. ?)

[jx2lee]

@kevinjqliu
I took a pass at a refactoring (a draft 🤗 ) based on your two concerns:

1. Config() implicit IO (env + local config files)
2. import-time _ENV_CONFIG initialization causing config reads during import

---

Direction I used is:

• make Config() a pure container (no implicit)
• add explicit Config.load() for file/env loading
• replace import-time _ENV_CONFIG with a lazy cached getter used by load_catalog() / list_catalogs()

---

Would you mind taking a quick look and sharing any guidance on:

• whether changing Config() semantics so config loading is explicit (e.g., via Config.load()) is acceptable in scope for this issue
• any compatibility concerns I should watch for before opening a PR

Thanks — happy to adjust the approach based on your feedback. 👍🏽

[jx2lee]

@kevinjqliu gentle ping.

[kevinjqliu]

Thanks for the PR! I think we should mark Config() as deprecated so folks can migrate to .load() instead

[kevinjqliu]

awesome!

[kevinjqliu]

nit move this to test_config?

[kevinjqliu]

nit i think we can clean up this test to use Config.load() instead

[kevinjqliu]

whether changing Config() semantics so config loading is explicit (e.g., via Config.load()) is acceptable in scope for this issue

Thinking about this again. Changing the signature from Config() to Config.load() might be unnecessarily complex for users to migrate. I think we can use the original signature (Config()) and change the __init__ be to lazy; i.e. only read config when a get_* function is called.
Feels like this is the right balance.

any compatibility concerns I should watch for before opening a PR

could you split up the _ENV_CONFIG change into another PR?
EDIT: btw what do you think about #3028 (comment)?

[jx2lee]

Thanks for the review, @kevinjqliu !

whether changing Config() semantics so config loading is explicit (e.g., via Config.load()) is acceptable in scope for this issue

Thinking about this again. Changing the signature from Config() to Config.load() might be unnecessarily complex for users to migrate. I think we can use the original signature (Config()) and change the __init__ be to lazy; i.e. only read config when a get_* function is called.

Feels like this is the right balance.

it makes sense. I agreed that, It's so complext for users.

could you split up the _ENV_CONFIG change into another PR?

EDIT: btw what do you think about #3028 (comment)?

I agree with that direction overall !

_ENV_CONFIG seems a bit misplaced and if we want a cached config object, I agree that config.py is probably the right place for it. It also makes sense to keep it lazy and avoid loading it automatically on import.

I can split that into a follow-up issue !