Support multiple EESSI versions by using ReFrame environments#326
Open
casparvl wants to merge 10 commits intoEESSI:mainfrom
Open
Support multiple EESSI versions by using ReFrame environments#326casparvl wants to merge 10 commits intoEESSI:mainfrom
casparvl wants to merge 10 commits intoEESSI:mainfrom
Conversation
… in a single run, by configuring ReFrame environments and using ReFrame's own find_modules function
Collaborator
Author
|
Ok, I now simply wrap the |
Collaborator
Author
|
Oh, this broke that's something we might want to fix if we do the wrapped thing (if it's easy...) |
Collaborator
Author
|
Metallwalls should also be adapted, since I think it doesn't use EESSI_mixin (?): |
Collaborator
Author
|
For me: At first look, that looks quite good: all CUDA-enabled modules are only scheduled on our GPU partitions, all non-CUDA modules only on the CPU partitions. Modules are discovered accross both EESSI 2023.06 and EESSI 2025.06. The execution takes quite long - possibly also because we don't cache (yet). Anyway, I'm not disappointed with the current state: it needs some work to fix the last tests, but not a bad starting point. |
casparvl
commented
Mar 26, 2026
casparvl
commented
Mar 26, 2026
…ssi_environments' into support_multiple_eessi_environments
…n of sys:part +feat notation directly
smoors
reviewed
Apr 4, 2026
Collaborator
|
i must be doing something wrong here (with my suggested fix above). |
Collaborator
|
ok, solved it by removing |
smoors
reviewed
Apr 5, 2026
Comment on lines
+45
to
+49
| environments = [ | ||
| {'name': 'EESSI-2023.06', 'modules': ['EESSI/2023.06']}, | ||
| {'name': 'EESSI-2025.06', 'modules': ['EESSI/2025.06']}, | ||
| ] | ||
| environs = ['EESSI-2023.06', 'EESSI-2025.06'] |
Collaborator
There was a problem hiding this comment.
as we are overwriting the environments, this change means that we can no longer use the default environment, which is needed to run local modules.
a simple solution would be to let it depend on an environment variable, e.g. USE_EESSI_MODULES:
if os.getenv('USE_EESSI_MODULES', True):
environments = [
{'name': 'EESSI-2023.06', 'modules': ['EESSI/2023.06']},
{'name': 'EESSI-2025.06', 'modules': ['EESSI/2025.06']},
]
environs = ['EESSI-2023.06', 'EESSI-2025.06']
else:
environments = [{'name': 'default'}]
environs = ['default']
Collaborator
Author
There was a problem hiding this comment.
Or should we append this to local config instead of overwrite?
Collaborator
There was a problem hiding this comment.
i prefer not to append to the local config, and only allow the environments that we support.
but thinking more about it, we can actually just do this:
environments = [
{'name': 'EESSI-2023.06', 'modules': ['EESSI/2023.06']},
{'name': 'EESSI-2025.06', 'modules': ['EESSI/2025.06']},
{'name': 'default'},
]
environs = ['EESSI-2023.06', 'EESSI-2025.06', 'default']- if the EESSI modules are in MODULEPATH, it will use them, and in that case you should not have any local modules in the MODULEPATH, so there will be no local modules found.
- if the EESSI modules are not in MODULEPATH, the user should add
-p defaultto avoid failure when trying to load an EESSI module, which i think isn't too bad.
Collaborator
Author
There was a problem hiding this comment.
Hmmm, I was thinking if you do have local environments that need to be loaded (e.g. we have local 2024, 2025 modules that load our local software stacks), it's annoying if everything is overwritten. And you don't really want to keep a separate config for local testing, because that means duplicating a lot of information. But what I then realized: you can just update the environments after calling set_common_required_config. That way, you can still have a single config file AND have local environs appended as well.
Collaborator
There was a problem hiding this comment.
ok, i didn't think of that use case. then i agree to just append them.
smoors
reviewed
Apr 5, 2026
Co-authored-by: Sam Moors <smoors@users.noreply.github.com>
Collaborator
Author
I have no clue why
|
Collaborator
|
yes, i get the expected behavior now: checking the ReFrame source code, the way it works as i understand: when it tries to load a module (here |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add initial functionality to schedule test for all EESSI environments in a single run, by configuring ReFrame environments and using ReFrame's own find_modules function.
For now, these work:
Still need to work on the other tests.