Skip to content

Load all storage data into cache during Onyx.init #752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
fabioh8010 wants to merge 10 commits into
base: main
Choose a base branch
from
+262 −1,487
Open

Load all storage data into cache during Onyx.init #752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
8a4acc0
Remove cache eviction system
fabioh8010 Mar 12, 2026
dce6b40
Load all storage data into cache during init
fabioh8010 Mar 12, 2026
ba345a2
Revert change
fabioh8010 Mar 12, 2026
8bfb429
Add error handling for getAll() failure during init
TMisiukiewicz Mar 13, 2026
3164734
Merge remote-tracking branch 'exfy/main' into sync-cache-2
TMisiukiewicz Mar 17, 2026
fa95648
remove storage mock usage from useOnyx perf tests
TMisiukiewicz Mar 17, 2026
8e1a33d
Minor fixes
fabioh8010 Mar 17, 2026
b61292e
Add tests for cache load during initialisation
fabioh8010 Mar 17, 2026
150e4b5
Remove duplicate test
fabioh8010 Mar 17, 2026
2c251c8
Fix Reassure tests
fabioh8010 Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 26 additions & 61 deletions API-INTERNAL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@
<dt><a href="#getDeferredInitTask">getDeferredInitTask()</a></dt>
<dd><p>Getter - returns the deffered init task.</p>
</dd>
<dt><a href="#afterInit">afterInit(action)</a> ⇒</dt>
<dd><p>Executes an action after Onyx has been initialized.
If Onyx is already initialized, the action is executed immediately.
Otherwise, it waits for initialization to complete before executing.</p>
</dd>
<dt><a href="#getSkippableCollectionMemberIDs">getSkippableCollectionMemberIDs()</a></dt>
<dd><p>Getter - returns the skippable collection member IDs.</p>
</dd>
Expand All @@ -29,7 +34,7 @@
<dt><a href="#setSnapshotMergeKeys">setSnapshotMergeKeys()</a></dt>
<dd><p>Setter - sets the snapshot merge keys allowlist.</p>
</dd>
<dt><a href="#initStoreValues">initStoreValues(keys, initialKeyStates, evictableKeys)</a></dt>
<dt><a href="#initStoreValues">initStoreValues(keys, initialKeyStates)</a></dt>
<dd><p>Sets the initial values for the Onyx store</p>
</dd>
<dt><a href="#reduceCollectionWithSelector">reduceCollectionWithSelector()</a></dt>
Expand Down Expand Up @@ -106,33 +111,17 @@ If the requested key is a collection, it will return an object with all the coll
<dt><a href="#sendDataToConnection">sendDataToConnection()</a></dt>
<dd><p>Sends the data obtained from the keys to the connection.</p>
</dd>
<dt><a href="#addKeyToRecentlyAccessedIfNeeded">addKeyToRecentlyAccessedIfNeeded()</a></dt>
<dd><p>We check to see if this key is flagged as safe for eviction and add it to the recentlyAccessedKeys list so that when we
run out of storage the least recently accessed key can be removed.</p>
</dd>
<dt><a href="#getCollectionDataAndSendAsObject">getCollectionDataAndSendAsObject()</a></dt>
<dd><p>Gets the data for a given an array of matching keys, combines them into an object, and sends the result back to the subscriber.</p>
</dd>
<dt><a href="#prepareSubscriberUpdate">prepareSubscriberUpdate(callback)</a></dt>
<dd><p>Delays promise resolution until the next macrotask to prevent race condition if the key subscription is in progress.</p>
</dd>
<dt><a href="#scheduleSubscriberUpdate">scheduleSubscriberUpdate()</a></dt>
<dd><p>Schedules an update that will be appended to the macro task queue (so it doesn&#39;t update the subscribers immediately).</p>
</dd>
<dt><a href="#scheduleNotifyCollectionSubscribers">scheduleNotifyCollectionSubscribers()</a></dt>
<dd><p>This method is similar to scheduleSubscriberUpdate but it is built for working specifically with collections
so that keysChanged() is triggered for the collection and not keyChanged(). If this was not done, then the
subscriber callbacks receive the data in a different format than they normally expect and it breaks code.</p>
</dd>
<dt><a href="#remove">remove()</a></dt>
<dd><p>Remove a key from Onyx and update the subscribers</p>
</dd>
<dt><a href="#retryOperation">retryOperation()</a></dt>
<dd><p>Handles storage operation failures based on the error type:</p>
<dd><p>Handles storage operation failures by retrying the operation.</p>
<ul>
<li>Storage capacity errors: evicts data and retries the operation</li>
<li>Invalid data errors: logs an alert and throws an error</li>
<li>Other errors: retries the operation</li>
<li>Other errors: retries the operation up to MAX_STORAGE_OPERATION_RETRY_ATTEMPTS times</li>
</ul>
</dd>
<dt><a href="#broadcastUpdate">broadcastUpdate()</a></dt>
Expand Down Expand Up @@ -223,6 +212,20 @@ Getter - returns the default key states.
Getter - returns the deffered init task.

**Kind**: global function
<a name="afterInit"></a>

## afterInit(action) ⇒
Executes an action after Onyx has been initialized.
If Onyx is already initialized, the action is executed immediately.
Otherwise, it waits for initialization to complete before executing.

**Kind**: global function
**Returns**: The result of the action

| Param | Description |
| --- | --- |
| action | The action to execute after initialization |

<a name="getSkippableCollectionMemberIDs"></a>

## getSkippableCollectionMemberIDs()
Expand All @@ -249,7 +252,7 @@ Setter - sets the snapshot merge keys allowlist.
**Kind**: global function
<a name="initStoreValues"></a>

## initStoreValues(keys, initialKeyStates, evictableKeys)
## initStoreValues(keys, initialKeyStates)
Sets the initial values for the Onyx store

**Kind**: global function
Expand All @@ -258,7 +261,6 @@ Sets the initial values for the Onyx store
| --- | --- |
| keys | `ONYXKEYS` constants object from Onyx.init() |
| initialKeyStates | initial data to set when `init()` and `clear()` are called |
| evictableKeys | This is an array of keys (individual or collection patterns) that when provided to Onyx are flagged as "safe" for removal. |

<a name="reduceCollectionWithSelector"></a>

Expand Down Expand Up @@ -393,7 +395,7 @@ For example:
- `getCollectionKey("sharedNVP_user_-1_something")` would return "sharedNVP_user_"

**Kind**: global function
**Returns**: The plain collection key or throws an Error if the key is not a collection one.
**Returns**: The plain collection key or undefined if the key is not a collection one.

| Param | Description |
| --- | --- |
Expand Down Expand Up @@ -444,48 +446,12 @@ keyChanged(key, value, subscriber => subscriber.initWithStoredValues === false)
## sendDataToConnection()
Sends the data obtained from the keys to the connection.

**Kind**: global function
<a name="addKeyToRecentlyAccessedIfNeeded"></a>

## addKeyToRecentlyAccessedIfNeeded()
We check to see if this key is flagged as safe for eviction and add it to the recentlyAccessedKeys list so that when we
run out of storage the least recently accessed key can be removed.

**Kind**: global function
<a name="getCollectionDataAndSendAsObject"></a>

## getCollectionDataAndSendAsObject()
Gets the data for a given an array of matching keys, combines them into an object, and sends the result back to the subscriber.

**Kind**: global function
<a name="prepareSubscriberUpdate"></a>

## prepareSubscriberUpdate(callback)
Delays promise resolution until the next macrotask to prevent race condition if the key subscription is in progress.

**Kind**: global function

| Param | Description |
| --- | --- |
| callback | The keyChanged/keysChanged callback |

<a name="scheduleSubscriberUpdate"></a>

## scheduleSubscriberUpdate()
Schedules an update that will be appended to the macro task queue (so it doesn't update the subscribers immediately).

**Kind**: global function
**Example**
```js
scheduleSubscriberUpdate(key, value, subscriber => subscriber.initWithStoredValues === false)
```
<a name="scheduleNotifyCollectionSubscribers"></a>

## scheduleNotifyCollectionSubscribers()
This method is similar to scheduleSubscriberUpdate but it is built for working specifically with collections
so that keysChanged() is triggered for the collection and not keyChanged(). If this was not done, then the
subscriber callbacks receive the data in a different format than they normally expect and it breaks code.

**Kind**: global function
<a name="remove"></a>

Expand All @@ -496,10 +462,9 @@ Remove a key from Onyx and update the subscribers
<a name="retryOperation"></a>

## retryOperation()
Handles storage operation failures based on the error type:
- Storage capacity errors: evicts data and retries the operation
Handles storage operation failures by retrying the operation.
- Invalid data errors: logs an alert and throws an error
- Other errors: retries the operation
- Other errors: retries the operation up to MAX_STORAGE_OPERATION_RETRY_ATTEMPTS times

**Kind**: global function
<a name="broadcastUpdate"></a>
Expand Down
38 changes: 0 additions & 38 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -314,51 +314,13 @@ If a platform needs to use a separate library (like using MMVK for react-native)

[Docs](./API.md)

# Cache Eviction

Different platforms come with varying storage capacities and Onyx has a way to gracefully fail when those storage limits are encountered. When Onyx fails to set or modify a key the following steps are taken:
1. Onyx looks at a list of recently accessed keys (access is defined as subscribed to or modified) and locates the key that was least recently accessed
2. It then deletes this key and retries the original operation

By default, Onyx will not evict anything from storage and will presume all keys are "unsafe" to remove unless explicitly told otherwise.

**To flag a key as safe for removal:**
- Add the key to the `evictableKeys` option in `Onyx.init(options)`
- Implement `canEvict` in the Onyx config for each component subscribing to a key
- The key will only be deleted when all subscribers return `true` for `canEvict`

e.g.
```js
Onyx.init({
evictableKeys: [ONYXKEYS.COLLECTION.REPORT_ACTIONS],
});
```

```js
const ReportActionsView = ({reportID, isActiveReport}) => {
const [reportActions] = useOnyx(
`${ONYXKEYS.COLLECTION.REPORT_ACTIONS}${reportID}_`,
{canEvict: () => !isActiveReport}
);

return (
<View>
{/* Render with reportActions data */}
</View>
);
};

export default ReportActionsView;
```

# Benchmarks

Provide the `captureMetrics` boolean flag to `Onyx.init` to capture call statistics

```js
Onyx.init({
keys: ONYXKEYS,
evictableKeys: [ONYXKEYS.COLLECTION.REPORT_ACTIONS],
captureMetrics: Config.BENCHMARK_ONYX,
});
```
Expand Down
12 changes: 2 additions & 10 deletions lib/Onyx.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,6 @@ import OnyxMerge from './OnyxMerge';
function init({
keys = {},
initialKeyStates = {},
evictableKeys = [],
maxCachedKeysCount = 1000,
shouldSyncMultipleInstances = !!global.localStorage,
enablePerformanceMetrics = false,
enableDevTools = true,
Expand Down Expand Up @@ -76,16 +74,10 @@ function init({
});
}

if (maxCachedKeysCount > 0) {
cache.setRecentKeysLimit(maxCachedKeysCount);
}

OnyxUtils.initStoreValues(keys, initialKeyStates, evictableKeys);
OnyxUtils.initStoreValues(keys, initialKeyStates);

// Initialize all of our keys with data provided then give green light to any pending connections
Promise.all([cache.addEvictableKeysToRecentlyAccessedList(OnyxUtils.isCollectionKey, OnyxUtils.getAllKeys), OnyxUtils.initializeWithDefaultKeyStates()]).then(
OnyxUtils.getDeferredInitTask().resolve,
);
OnyxUtils.initializeWithDefaultKeyStates().then(OnyxUtils.getDeferredInitTask().resolve);
}

/**
Expand Down
Loading
Loading