Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
CLAUDE.md
.claude/
# next.js
.next
node_modules
Expand Down
13 changes: 13 additions & 0 deletions pages/advanced-algorithms/available-algorithms/migrate.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,19 @@ filter, and convert relational data into a graph format.
| **Implementation** | Python |
| **Parallelism** | sequential |

<Callout type="info">
When running multiple migrations against the same source, avoid repeating the `config` map in every call.
Use [server-side parameters](/database-management/server-side-parameters) to store the connection config once
and reference it as `$config` across all your queries:

```cypher
SET GLOBAL PARAMETER pg_config = {user: 'memgraph', password: 'password', host: 'localhost', database: 'demo_db'};

CALL migrate.postgresql('users', $pg_config) YIELD row CREATE (u:User {id: row.id});
CALL migrate.postgresql('orders', $pg_config) YIELD row CREATE (o:Order {id: row.id});
```
</Callout>

---

## Procedures
Expand Down
8 changes: 4 additions & 4 deletions pages/ai-ecosystem/graph-rag/atomic-pipelines.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,9 +57,9 @@ Offer the ability to compute embeddings during the preprocessing or retrieval
stages.
11. [`llm.complete`](/advanced-algorithms/available-algorithms/llm) function:
Allows you to call any LLM under any given Cypher query.
12. Server-side parameters: Could be used for many different things, but in this
context, the parameters help you with managing configuration under any given
query or pipeline.
12. [Server-side parameters](/database-management/server-side-parameters): Could
be used for many different things, but in this context, the parameters help you
with managing configuration under any given query or pipeline.

## Question and Pipeline Types

Expand Down Expand Up @@ -107,4 +107,4 @@ exists, it synthesizes what is missing or not well-covered. Example use cases:
- **Publishing**: *"I want to write a book, which important topics in this
domain are not yet well covered in the existing literature?"*

![atomic_graphrag_pipelines](/pages/ai-ecosystem/graph-rag/atomic-pipelines/atomic-graphrag-pipelines.png)
![atomic_graphrag_pipelines](/pages/ai-ecosystem/graph-rag/atomic-pipelines/atomic-graphrag-pipelines.png)
12 changes: 12 additions & 0 deletions pages/clustering/high-availability.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -60,5 +60,17 @@ production-ready clusters, scaling patterns, cross–data center deployments, an
advanced architectures for low-latency or geographically distributed
environments.

## Upgrades & migration

### [Upgrades](/database-management/upgrades)

Plan Memgraph version upgrades, including standalone and HA clusters. Use the
[version-specific guides](/database-management/upgrades/specific-versions) when
moving between major releases.

### [In-service software upgrade (ISSU) on Kubernetes](/clustering/high-availability/setup-ha-cluster-k8s#in-service-software-upgrade-issu)

For HA on Kubernetes, Memgraph supports rolling upgrades with minimal downtime
when you follow the ISSU procedure on the Helm chart setup page.

<CommunityLinks/>
1 change: 1 addition & 0 deletions pages/clustering/high-availability/_meta.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ export default {
"best-practices": "Best practices",
"ha-commands-reference": "Reference commands",
"ha-reference-architectures": "Reference architectures",
"migrating-to-v3-9-ha": "Migrating to v3.9 HA",
}
13 changes: 13 additions & 0 deletions pages/clustering/high-availability/best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,19 @@ The configuration value can be controlled using the query:
SET COORDINATOR SETTING 'max_replica_read_lag' TO '10' ;
```

### `deltas_batch_progress_size`

Users can control how often REPLICAs report back to the MAIN that they're still processing the data (transactions, WALs, snapshots)
the MAIN has sent to them. The default value is 100'000, which should be enough for most of your transactions.
However, if processing 100'000 deltas takes more than 30s (because you're dealing with large deltas or you have older CPUs),
you can set the configuration value `deltas_batch_progress_size` to a smaller value. This will avoid timeouts on REPLICAs so
you won't see the query exception "At least one SYNC replica has not committed", but at the cost of lower throughput since REPLICAs
will be sending in-progress messages to the MAIN more often.

```
SET COORDINATOR SETTING 'deltas_batch_progress_size' TO '50000';
```

## Observability

Monitoring cluster health is essential. Key metrics include:
Expand Down
21 changes: 12 additions & 9 deletions pages/clustering/high-availability/ha-commands-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,18 @@ Memgraph High Availability (HA) cluster.
## Cluster registration commands

<Callout type="warning">
**Important:** **All registration commands (adding coordinators and registering
data instances) must be executed on the *same* coordinator.** You may choose any
**Important:** You may choose any
coordinator for the initial setup; it automatically becomes the leader. After
setup, the choice no longer matters.
</Callout>

<Callout type="info">
All queries can be run on any coordinator. If currently the coordinator is not a
leader, the query will be automatically forwarded to the current leader and
executed there. This is because the Raft protocol specifies that only the
leader should accept changes in the cluster.
</Callout>

### `ADD COORDINATOR`

Adds a coordinator to the cluster.
Expand Down Expand Up @@ -78,7 +84,6 @@ REMOVE COORDINATOR coordinatorId;

{<h4 className="custom-header"> Behavior & implications </h4>}

- Must be executed on the **leader** coordinator.
- Leader coordinator **cannot** remove itself. To remove the leader, first
trigger a leadership change.

Expand All @@ -90,7 +95,9 @@ REMOVE COORDINATOR 2;

### `UPDATE CONFIG`

Updates the configuration of the data instance/coordinator in the cluster.
Updates the configuration of the data instance/coordinator in the cluster. Use
this command to update the DNS of your bolt servers, for example, when
migrating to new load balancers.

```cypher
UPDATE CONFIG FOR ( INSTANCE instanceName | COORDINATOR coordinatorServerId ) configsMap=configMap ;
Expand All @@ -106,7 +113,6 @@ of each instance.

{<h4 className="custom-header"> Behavior & implications </h4>}

- Must be executed on the **leader** coordinator.
- Only bolt server can be updated.

{<h4 className="custom-header"> Example </h4>}
Expand Down Expand Up @@ -294,8 +300,7 @@ SHOW INSTANCE;

### `SHOW REPLICATION LAG`

Shows replication lag (in committed transactions) for all instances. Must be run
on the **leader**.
Shows replication lag (in committed transactions) for all instances.

```cypher
SHOW REPLICATION LAG;
Expand Down Expand Up @@ -326,7 +331,5 @@ FORCE RESET CLUSTER STATE;

{<h4 className="custom-header"> Implications </h4>}

- Must be executed on the **leader** coordinator.


<CommunityLinks/>
Original file line number Diff line number Diff line change
Expand Up @@ -133,9 +133,19 @@ Below is a cleaned-up categorization.

#### Coordinator → Coordinator RPCs

| RPC | Purpose | Description |
| -------------------- | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `ShowInstancesRpc` | Follower requests cluster state from leader | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW INSTANCES` through the follower. |
| RPC | Purpose | Description |
| ------------------------ | -------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
| `ShowInstancesRpc` | Follower requests cluster state from leader. | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW INSTANCES` through the follower. |
| `AddCoordinatorRpc` | Follower requests adding coordinator. | Sent by a follower coordinator to the leader coordinator when a user executes `ADD COORDINATOR` through the follower. |
| `RemoveCoordinatorRpc` | Follower requests removing coordinator. | Sent by a follower coordinator to the leader coordinator when a user executes `REMOVE COORDINATOR` through the follower. |
| `RegisterInstanceRpc` | Follower requests registering an instance. | Sent by a follower coordinator to the leader coordinator when a user executes `REGISTER INSTANCE` through the follower. |
| `UnregisterInstanceRpc` | Follower requests unregistering an instance. | Sent by a follower coordinator to the leader coordinator when a user executes `UNREGISTER INSTANCE` through the follower. |
| `SetInstanceToMainRpc` | Follower requests updating MAIN instance. | Sent by a follower coordinator to the leader coordinator when a user executes `SET INSTANCE TO MAIN` through the follower. |
| `DemoteInstanceRpc` | Follower requests demoting an instance. | Sent by a follower coordinator to the leader coordinator when a user executes `DEMOTE INSTANCE` through the follower. |
| `UpdateConfigRpc` | Follower requests updating config. | Sent by a follower coordinator to the leader coordinator when a user executes `UPDATE CONFIG` through the follower. |
| `ForceResetRpc` | Follower requests resetting cluster state. | Sent by a follower coordinator to the leader coordinator when a user executes `FORCE RESET` through the follower. |
| `GetRoutingTableRpc` | Follower requests a routing table. | Sent by a follower coordinator to the leader coordinator when a user connects using `bolt+routing` through the follower. |
| `CoordReplicationLagRpc` | Follower requests replication lag info. | Sent by a follower coordinator to the leader coordinator when a user executes `SHOW REPLICATION LAG` through the follower. |


#### Coordinator → Data Instance RPCs
Expand Down
42 changes: 42 additions & 0 deletions pages/clustering/high-availability/migrating-to-v3-9-ha.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
title: Migrating to v3.9 HA
description: Learn about the required migration steps when upgrading a Memgraph high-availability cluster to version 3.9.
---

import { Callout } from 'nextra/components'
import { Steps } from 'nextra/components'

# Migrating to v3.9 HA

Memgraph 3.9 includes a fix for a bug in how durability is handled on
coordinators in a high-availability (HA) setup. Because of this fix, **all
coordinator data directories must be cleaned and coordinators must be started
fresh**.

Data instances are not affected and can stay unchanged.

<Callout type="warning">
Upgrading to 3.9 **cannot be performed without downtime** if you are using a
routing connection to connect to the currently writable instance. You can,
however, keep using direct Bolt connections to data instances during this
maintenance period.
</Callout>

<Steps>

{<h3 className="custom-header">Clean coordinator data directories</h3>}

Remove the data directories of all coordinators. If you are running Memgraph HA
in Kubernetes, delete the coordinators' PVCs and PVs before the restart.

{<h3 className="custom-header">Restart coordinators</h3>}

Start all coordinators with the new Memgraph 3.9 version.

{<h3 className="custom-header">Reconnect the cluster</h3>}

After the coordinators are running, reconnect your cluster by executing
`REGISTER INSTANCE` and `ADD COORDINATOR` queries for each data instance and
coordinator respectively.

</Steps>
Loading