Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"ext-curl": "*",
"ext-openssl": "*",
"appwrite/appwrite": "19.*",
"utopia-php/database": "5.*",
"utopia-php/database": "dev-csv-import-upsert-v2 as 5.0.0",
"utopia-php/storage": "1.0.*",
"utopia-php/dsn": "0.2.*",
"halaxa/json-machine": "^1.2"
Expand Down
117 changes: 40 additions & 77 deletions composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

41 changes: 36 additions & 5 deletions src/Migration/Destinations/Appwrite.php
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,27 @@ class Appwrite extends Destination
*/
private array $rowBuffer = [];

private bool $overwrite = false;
private bool $skip = false;

public function setOverwrite(bool $overwrite): self
{
if ($overwrite && $this->skip) {
throw new \InvalidArgumentException('Cannot set both overwrite and skip to true.');
}
$this->overwrite = $overwrite;
return $this;
}

public function setSkip(bool $skip): self
{
if ($skip && $this->overwrite) {
throw new \InvalidArgumentException('Cannot set both skip and overwrite to true.');
}
$this->skip = $skip;
return $this;
}

/**
* @param string $project
* @param string $endpoint
Expand Down Expand Up @@ -986,7 +1007,7 @@ protected function createRecord(Row $resource, bool $isLast): bool
$this->cache->get($resource->getName())
);

if ($exists) {
if ($exists && !$this->overwrite) {
$resource->setStatus(
Resource::STATUS_SKIPPED,
'Row has already been created'
Expand Down Expand Up @@ -1067,10 +1088,20 @@ protected function createRecord(Row $resource, bool $isLast): bool
}
}
}
$dbForDatabases->skipRelationshipsExistCheck(fn () => $dbForDatabases->createDocuments(
'database_' . $databaseInternalId . '_collection_' . $tableInternalId,
$this->rowBuffer
));
$collectionName = 'database_' . $databaseInternalId . '_collection_' . $tableInternalId;

if ($this->overwrite) {
$dbForDatabases->skipRelationshipsExistCheck(fn () => $dbForDatabases->upsertDocuments(
$collectionName,
$this->rowBuffer
));
} else {
$dbForDatabases->skipRelationshipsExistCheck(fn () => $dbForDatabases->createDocuments(
$collectionName,
$this->rowBuffer,
ignore: $this->skip
));
Comment on lines +1093 to +1103
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Successful upserts not reflected in resource status

When overwrite is true, upsertDocuments is called but the return value is silently discarded and the method unconditionally returns true (line 1106). If the upsert fails for any reason (other than throwing an exception), the caller has no way to know — and even in the success case, there is no cache update after a successful upsert. A successfully upserted row won't be added to the cache, so a re-run would attempt to upsert it again instead of treating it as already migrated.

The same gap exists for the skip path: rows silently skipped at the database level (because ignore: true suppresses the duplicate error) are returned as true — indistinguishable from a genuine insert — meaning the status report will show them as successfully migrated when they were actually dropped.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache is already updated at the import() level (line 368: $this->cache->update($responseResource)) for every resource regardless of the code path in createRecord(), so re-runs will see the row as cached.

For the skip path: returning true is the intended behavior — skip means "do not fail on duplicates, keep going." The migration completed successfully from the caller's perspective.

}

} finally {
$this->rowBuffer = [];
Expand Down
Loading