Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 11 additions & 6 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,21 @@
# run automated tests
name: CI Tests

on: [push]
on:
push:
pull_request:
workflow_dispatch:

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [20.x, 22.x, 24.x, 25.x]
steps:
- uses: actions/checkout@v1
- uses: actions/setup-node@v1
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
# VMs have 23.x installed
node-version: '23.x'
- run: npm install
node-version: ${{ matrix.node-version }}
- run: npm ci || npm install
- run: npm test
23 changes: 23 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# AGENTS

This document explains the repository structure and context for AI agents working with the `datastore-backup` tool.

## Purpose

The primary purpose of this tool is to back up a **subset of entities** from Google Cloud Datastore (or Firestore in Datastore mode) rather than an entire database. It allows specifying specific entities and groups them into "backup schedules" or schedules representing logical groupings of data to export together.

If the goal is to back up an entire database en masse, this simple Node.js script is generally not the recommended approach. Instead, administrators should prefer [Firestore scheduled backups](https://cloud.google.com/firestore/docs/manage-data/scheduled-backups) or native Datastore export jobs.

## Repository Structure

- `index.js`: The main entry point and CLI wrapper. Uses `commander` to parse arguments and define commands for backing up, restoring, auditing (`list`), and testing restores.
- `lib/cmd.js`: Helper functions for parsing command line arguments and extracting things like Google Cloud project IDs and bucket names.
- `lib/datastore-backup.js`: Core programmatic logic that interfaces with `@google-cloud/datastore` to execute export operations and execute validation commands via `gcloud`.
- `test/`: Contains the test suite files (currently using Mocha + Should). Tests validate configuration parsing and command generation.
- `examples/`: Example configuration files like `backup-schedule.json`.

## Technical Details

- **Dependencies**: Uses `@google-cloud/datastore` for interacting with the Google Cloud Datastore API and executes some `gcloud` CLI commands via `child_process.execSync` for testing and restoration outputs.
- **Testing**: Test suite verifies the core configuration logic but does not automatically hit live GCP endpoints unless explicitly configured.
- **CI**: There is a GitHub Actions workflow that runs standard `npm test` checks.
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# datastore-backup
npm script to backup/restore Google Cloud Datastore, across multiple projects
npm script to backup/restore Google Cloud Datastore, across multiple projects.

> **Note**: This tool is designed primarily to backup a **subset of entities** by defining a schedule. If you are backing up an entire database, using [Firestore scheduled backups](https://cloud.google.com/firestore/docs/manage-data/scheduled-backups) or native export features is probably a better solution.


## Installation
Expand Down
4 changes: 3 additions & 1 deletion lib/cmd.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ const child_process = require('child_process');
*/
const validateFrequency = (backupSchedule, frequency) => {
if (_.isUndefined(backupSchedule[frequency])) {
console.log(('Frequency value (' + frequency + ') unknown!').red);
const msg = 'Frequency value (' + frequency + ') unknown!';
console.log(msg.red);
throw new Error(msg);
}
}

Expand Down
23 changes: 18 additions & 5 deletions test/cmd.js
Original file line number Diff line number Diff line change
@@ -1,19 +1,32 @@

require('should');

const {getProjectId, getBackupBucket, validateFrequency} = require('../lib/cmd');
const { getProjectId, getBackupBucket, validateFrequency } = require('../lib/cmd');

describe('getProjectId', () => {
it ('parses from program', () => {
getProjectId({projectId : 'some-project-id'}).should.equal('some-project-id');
it('parses from program', () => {
getProjectId({ projectId: 'some-project-id' }).should.equal('some-project-id');
});
});

describe('validateFrequency', () => {
it('passes when frequency exists', () => {
(function () {
validateFrequency({ 'daily': ['Entity'] }, 'daily');
}).should.not.throw();
});

it('throws when frequency is missing', () => {
(function () {
validateFrequency({ 'daily': ['Entity'] }, 'weekly');
}).should.throw(/unknown/);
});
});

describe('getBackupBucket', () => {
it ('parses from program', () => {
it('parses from program', () => {

getBackupBucket({bucketPrefix: 'project_backups'}, 'some-project-id', 'daily')
getBackupBucket({ bucketPrefix: 'project_backups' }, 'some-project-id', 'daily')
.should.equal('project_backups_daily');
});

Expand Down
Loading