-
Notifications
You must be signed in to change notification settings - Fork 1.6k
test(framework): add dual DB engine coverage with flaky test fixes #6586
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
vividctrlalt
wants to merge
3
commits into
tronprotocol:develop
Choose a base branch
from
vividctrlalt:test/dual-db-engine-coverage
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
98 changes: 98 additions & 0 deletions
98
framework/src/test/java/org/tron/common/BaseMethodTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| package org.tron.common; | ||
|
|
||
| import java.io.IOException; | ||
| import java.util.Arrays; | ||
| import lombok.extern.slf4j.Slf4j; | ||
| import org.junit.After; | ||
| import org.junit.Before; | ||
| import org.junit.Rule; | ||
| import org.junit.rules.TemporaryFolder; | ||
| import org.tron.common.application.Application; | ||
| import org.tron.common.application.ApplicationFactory; | ||
| import org.tron.common.application.TronApplicationContext; | ||
| import org.tron.core.ChainBaseManager; | ||
| import org.tron.core.config.DefaultConfig; | ||
| import org.tron.core.config.args.Args; | ||
| import org.tron.core.db.Manager; | ||
|
|
||
| /** | ||
| * Base class for tests that need a fresh Spring context per test method. | ||
| * | ||
| * Each @Test method gets its own TronApplicationContext, created in @Before | ||
| * and destroyed in @After, ensuring full isolation between tests. | ||
| * | ||
| * Subclasses can customize behavior by overriding hook methods: | ||
| * extraArgs() — additional CLI args (e.g. "--debug") | ||
| * configFile() — config file (default: config-test.conf) | ||
| * beforeContext() — runs after Args.setParam, before Spring context creation | ||
| * afterInit() — runs after Spring context is ready (e.g. get extra beans) | ||
| * beforeDestroy() — runs before context shutdown (e.g. close resources) | ||
| * | ||
| * Use this when: | ||
| * - methods modify database state that would affect other methods | ||
| * - methods need different CommonParameter settings (e.g. setP2pDisable) | ||
| * - methods need beforeContext() to configure state before Spring starts | ||
| * | ||
| * If methods are read-only and don't interfere with each other, use BaseTest instead. | ||
| * Tests that don't need Spring (e.g. pure unit tests) should NOT extend either base class. | ||
| */ | ||
| @Slf4j | ||
| public abstract class BaseMethodTest { | ||
|
|
||
| @Rule | ||
| public final TemporaryFolder temporaryFolder = new TemporaryFolder(); | ||
|
|
||
| protected TronApplicationContext context; | ||
| protected Application appT; | ||
| protected Manager dbManager; | ||
| protected ChainBaseManager chainBaseManager; | ||
|
|
||
| protected String[] extraArgs() { | ||
| return new String[0]; | ||
| } | ||
|
|
||
| protected String configFile() { | ||
| return TestConstants.TEST_CONF; | ||
| } | ||
|
|
||
| @Before | ||
| public final void initContext() throws IOException { | ||
| String[] baseArgs = new String[]{ | ||
| "--output-directory", temporaryFolder.newFolder().toString()}; | ||
| String[] allArgs = mergeArgs(baseArgs, extraArgs()); | ||
| Args.setParam(allArgs, configFile()); | ||
| beforeContext(); | ||
| context = new TronApplicationContext(DefaultConfig.class); | ||
| appT = ApplicationFactory.create(context); | ||
| dbManager = context.getBean(Manager.class); | ||
| chainBaseManager = context.getBean(ChainBaseManager.class); | ||
| afterInit(); | ||
| } | ||
|
|
||
| protected void beforeContext() { | ||
| } | ||
|
|
||
| protected void afterInit() { | ||
| } | ||
|
|
||
| @After | ||
| public final void destroyContext() { | ||
| beforeDestroy(); | ||
| if (appT != null) { | ||
| appT.shutdown(); | ||
| } | ||
| if (context != null) { | ||
| context.close(); | ||
| } | ||
| Args.clearParam(); | ||
| } | ||
|
|
||
| protected void beforeDestroy() { | ||
| } | ||
|
|
||
| private static String[] mergeArgs(String[] base, String[] extra) { | ||
| String[] result = Arrays.copyOf(base, base.length + extra.length); | ||
| System.arraycopy(extra, 0, result, base.length, extra.length); | ||
| return result; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
32 changes: 32 additions & 0 deletions
32
framework/src/test/java/org/tron/common/TestConstants.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,39 @@ | ||
| package org.tron.common; | ||
|
|
||
| import static org.junit.Assume.assumeFalse; | ||
|
|
||
| import org.tron.common.arch.Arch; | ||
|
|
||
| /** | ||
| * Centralized test environment constants and utilities. | ||
| * | ||
| * <h3>DB engine override for dual-engine testing</h3> | ||
| * Gradle tasks ({@code test} on ARM64, {@code testWithRocksDb}) set the JVM system property | ||
| * {@code -Dstorage.db.engine=ROCKSDB}. Because test config files are loaded from the classpath | ||
| * via {@code ConfigFactory.load(fileName)}, Typesafe Config automatically merges system | ||
| * properties with higher priority than the config file values. This means the config's | ||
| * {@code storage.db.engine = "LEVELDB"} is overridden transparently, without any code changes | ||
| * in individual tests. | ||
| * | ||
| * <p><b>IMPORTANT:</b> Config files MUST be classpath resources (in {@code src/test/resources/}), | ||
| * NOT standalone files in the working directory. If a config file exists on disk, | ||
| * {@code Configuration.getByFileName} falls back to {@code ConfigFactory.parseFile()}, | ||
| * which does NOT merge system properties, and the engine override will silently fail. | ||
| */ | ||
| public class TestConstants { | ||
|
|
||
| public static final String TEST_CONF = "config-test.conf"; | ||
| public static final String NET_CONF = "config.conf"; | ||
| public static final String MAINNET_CONF = "config-test-mainnet.conf"; | ||
| public static final String DBBACKUP_CONF = "config-test-dbbackup.conf"; | ||
| public static final String LOCAL_CONF = "config-localtest.conf"; | ||
| public static final String STORAGE_CONF = "config-test-storagetest.conf"; | ||
| public static final String INDEX_CONF = "config-test-index.conf"; | ||
|
|
||
| /** | ||
| * Skips the current test on ARM64 where LevelDB JNI is unavailable. | ||
| */ | ||
| public static void assumeLevelDbAvailable() { | ||
| assumeFalse("LevelDB JNI unavailable on ARM64", Arch.isArm64()); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change and
validateConfigintroduce a regression: on ARM64, starting the node with the defaultconfig.conf(wheredb.engineis set toLEVELDB) now results in a startup failure.The issue can be reproduced as follows:
$ ./gradlew :framework:buildFullNodeJar ….. Building for architecture: aarch64, Java version: 17 ….. $ java -jar framework/build/libs/FullNode.jar $ tail -f logs/tron.log ….. 15:17:21.876 ERROR [main] [Exit](ExitManager.java:49) Shutting down with code: PARAMETER_INIT(1), reason: ARM64 architecture only supports RocksDB. Current engine: LEVELDBThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. I've opened an issue to continue the discussion: #6587
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe fail-fast is the correct behavior here. If a user runs on ARM64 with
db.engine=LEVELDB, the node should refuse to start with a clear error message, rather than silently falling back to RocksDB or deferring the failure to runtime.The principle is: no hidden behavior. If you're running on ARM64, you must explicitly configure a supported engine. A silent fallback would mask a misconfiguration and could lead to confusion later (e.g., user expects LevelDB but data is actually stored in RocksDB).
That said, I understand the concern about the default
config.confshipping withLEVELDB. This is really a config-level issue — the default config should either be platform-aware or the documentation should clearly state that ARM64 users need to setdb.engine=ROCKSDB. Happy to discuss further in #6587.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To add more context: the whole point of this series of changes is to eliminate silent behaviors in FullNode. Silently swapping the DB engine in production is exactly the kind of hidden behavior we're trying to remove — it should be strictly prohibited.
If an ARM64 node has
db.engine=LEVELDBinconfig.conf, that's a configuration management problem, not something the application should paper over at runtime. Mixing config-management concerns into production code leads to harder-to-debug issues and undermines the principle of explicit configuration.The correct fix belongs in the deployment/config layer — ARM64 environments should ship with the right
config.conffrom the start.