Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ You can still iterate over it as if it was a normal array.

One typical use case would be to load a lot of datasets from a database at once. (There are reasons to prefer this over running multiple queries.) See *Usage* below for an example for this use case using this library.

In general, you might want to try the following PHP-builtin solutions first:

- [SplFixedArray](https://www.php.net/manual/class.splfixedarray.php): By being of fixed length, this saves some memory compared to traditional `array`s. It behaves pretty much like `array`s do. Although the savings are not that huge compared to other solutions, it is probably the easiest to adopt.
- [Generator](https://www.php.net/manual/language.generators.php): Instead of returning an `array` all at once, this allows to return one item after the other. Thereby a "consumer" is able to process one item after the other accordingly. One can also terminate the "generation" of further items. One of the downsides is, that a Generator can only be iterated over once. The memory consumption can be as good as constant, but this approach is quite different and therefore rather hard to adopt and not appropriate in every scenario.

It should be noted, that a `LargeArrayBuffer` can be converted to both of these using `toFixedArray()` and `toGenerator()` respectively.

## Install

Note: This library requires PHP 8.0+!
Expand Down
32 changes: 32 additions & 0 deletions src/ArrayBuffer.php
Original file line number Diff line number Diff line change
Expand Up @@ -112,4 +112,36 @@ public function toArray(): array {
return $this->array;
}
}

/**
* @psalm-return \SplFixedArray<E>
*/
public function toFixedArray(): \SplFixedArray {
if($this->buffer->count() > 0){
return $this->buffer->toFixedArray();
} else {
$res = new \SplFixedArray(count($this->array));
foreach($this->array as $idx => $item){
$res[$idx] = $item;
}
return $res;
}
}

/**
* @return \Generator send something other than null to terminate
* @psalm-return \Generator<int, E, mixed, void>
*/
public function toGenerator(): \Generator {
if($this->buffer->count() > 0){
yield from $this->buffer->toGenerator();
} else {
foreach($this->array as $item){
$cmd = yield $item;
if($cmd !== null){
break;
}
}
}
}
}
11 changes: 11 additions & 0 deletions src/ArrayBufferInterface.php
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,15 @@ public function push(mixed $item): void;
* @psalm-return list<E>
*/
public function toArray(): array;

/**
* @psalm-return \SplFixedArray<E>
*/
public function toFixedArray(): \SplFixedArray;

/**
* @return \Generator send something other than null to terminate
* @psalm-return \Generator<int, E, mixed, void>
*/
public function toGenerator(): \Generator;
}
24 changes: 24 additions & 0 deletions src/LargeArrayBuffer.php
Original file line number Diff line number Diff line change
Expand Up @@ -252,6 +252,30 @@ public function toArray(): array {
}
return $res;
}

/**
* @psalm-return \SplFixedArray<E>
*/
public function toFixedArray(): \SplFixedArray {
$res = new \SplFixedArray($this->count);
foreach($this as $idx => $item){
$res[$idx] = $item;
}
return $res;
}

/**
* @return \Generator send something other than null to terminate
* @psalm-return \Generator<int, E, mixed, void>
*/
public function toGenerator(): \Generator {
foreach($this as $item){
$cmd = yield $item;
if($cmd !== null){
break;
}
}
}

public function __destruct() {
/**
Expand Down
42 changes: 40 additions & 2 deletions test/LargeArrayBufferTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

use LargeArrayBuffer\LargeArrayBuffer;
use PHPUnit\Framework\TestCase;
use PHPUnit\Framework\Attributes\DataProvider;

/**
* @author Andreas Wahlen
Expand Down Expand Up @@ -77,7 +76,7 @@ public function testLoop(int $serializer, int $compression): void {
$count = 1500;
$buf = new LargeArrayBuffer(serializer: $serializer, compression: $compression);
$objs = [];
for($i=0;$i<$count;$i++){
for($i = 0; $i < $count; $i++){
$o = new \stdClass();
$o->idx = $i;
$objs[] = $o;
Expand Down Expand Up @@ -107,4 +106,43 @@ public function testToJSON(): void {
fclose($stream);
$this->assertEquals(json_encode([$o, $o], JSON_THROW_ON_ERROR), $json);
}

public function provideItems(): array {
return [
[['hello world!', 'just another string']]
];
}

/**
* @dataProvider provideItems
*/
public function testToArray(array $items): void {
$buf = new LargeArrayBuffer();
foreach($items as $item){
$buf->push($item);
}
$this->assertSame($items, $buf->toArray());
}

/**
* @dataProvider provideItems
*/
public function testToFixedArray(array $items): void {
$buf = new LargeArrayBuffer();
foreach($items as $item){
$buf->push($item);
}
$this->assertSame($items, $buf->toFixedArray()->toArray());
}

/**
* @dataProvider provideItems
*/
public function testToGenerator(array $items): void {
$buf = new LargeArrayBuffer();
foreach($items as $item){
$buf->push($item);
}
$this->assertSame($items, iterator_to_array($buf->toGenerator()));
}
}