# S3

## S3 Stream Wrapper

The S3 stream wrapper lets you use `s3://bucket/key` paths with PHP's file functions
(`file_get_contents()`, `file_put_contents()`, `fopen()`, `is_file()`, `scandir()`, ...). It is a
port of the stream wrapper of the AWS SDK for PHP (`Aws\S3\StreamWrapper`), which
pieceofcake2/aws registers, to AsyncAws, with the same scope and limits. It needs
`async-aws/simple-s3`.

Load the plugin with `'bootstrap' => true` to register it with the `AsyncAws` configuration (or
the `Aws` configuration, as the other features), or register it yourself:

```php
App::uses('S3StreamWrapper', 'AsyncAws.Lib');

// With the AsyncAws configuration
S3StreamWrapper::bootstrap();

// Or with a client
S3StreamWrapper::register($this->AsyncAws->createSimpleS3(['region' => 'us-east-1']));
```

When `async-aws/simple-s3` is not installed, the bootstrap logs a warning and does not register
the wrapper. Then:

```php
file_put_contents('s3://my-bucket/reports/today.csv', $csv);
echo file_get_contents('s3://my-bucket/reports/today.csv');

if (is_file('s3://my-bucket/reports/today.csv')) {
    echo filesize('s3://my-bucket/reports/today.csv');
}

// "today.csv", ...
foreach (scandir('s3://my-bucket/reports') as $name) {
    echo $name;
}
```

Options of the S3 operations can be passed with a stream context:

```php
$context = stream_context_create([
    's3' => [
        'ACL' => 'private',
        'ContentType' => 'text/csv',
        'CacheControl' => 'max-age=3600',
        'Metadata' => ['source' => 'reports'],
        'ServerSideEncryption' => 'AES256',
    ],
]);
file_put_contents('s3://my-bucket/reports/today.csv', $csv, 0, $context);
```

The options are passed to the AsyncAws operation (PutObject, GetObject, CopyObject, ...), which
uses those it knows: for example `VersionId` or `Range` for reading, and `VersionId` for
`unlink()`. These options are used by the wrapper itself:

| Option | Description |
| --- | --- |
| `seekable` | `true` makes a read stream seekable (see below). |
| `delimiter` | Delimiter of `opendir()`/`scandir()`. Default: `/`. An empty string lists all the keys under the prefix. |
| `listFilter` | A callable for `opendir()` that receives each entry (`Key`, `Size`, `LastModified`, ... or `Prefix`) and returns whether to list it. |
| `PartSize` | Part size of multipart uploads, in MB (default: 64). With `async-aws/simple-s3` 3.x, bodies larger than one part are uploaded in parts; 2.x (installed on PHP 8.1) uploads bodies up to 64 MB with PutObject whatever the part size. |

These functions are supported:

| Function | How |
| --- | --- |
| `fopen()` with `r` | GetObject. The body is streamed. |
| `fopen()` with `w`, `x` | The data is written to a `php://temp` buffer and uploaded when the stream is flushed (`fflush()`) or closed. Bodies larger than 64 MB are uploaded in parts. `x` fails when the object exists (HeadObject). |
| `fopen()` with `a` | The object is downloaded into the buffer (or the buffer starts empty when the object does not exist), and the result is uploaded. |
| `fread()`, `fwrite()`, `feof()`, `ftell()`, `fseek()`, `fflush()`, `fclose()`, `fstat()` | On the stream or its buffer. |
| `file_exists()`, `is_file()`, `is_dir()`, `filesize()`, `filemtime()`, `stat()`, ... | HeadObject. When the key is not an object, a prefix with objects (ListObjectsV2) is a directory. Buckets and `s3://` are directories. Files and directories have the mode `0777`. |
| `unlink()` | DeleteObject. |
| `rename()` | CopyObject (with the metadata) and DeleteObject, in a bucket or between buckets. |
| `copy()` | Reads and writes the object. |
| `mkdir()` | `s3://bucket` creates the bucket in the region of the client. `s3://bucket/dir` creates an empty `dir/` object. |
| `rmdir()` | `s3://bucket` deletes the bucket. `s3://bucket/dir` deletes the `dir/` object when nothing else has the prefix. |
| `opendir()`, `readdir()`, `rewinddir()`, `closedir()`, `scandir()` | ListObjectsV2 with the delimiter, page by page. `dir/` objects are skipped. |

### Limitations

These are the limitations of the stream wrapper of the AWS SDK for PHP, which this wrapper
keeps:

- Only the modes `r`, `w`, `a` and `x` are supported (`r+`, `w+`, `c`, ... are not), so a
  stream either reads or writes.
- A read stream (`r`) cannot seek, unless the `seekable` context option is `true`. Then the
  bytes read are kept in a `php://temp` buffer, so that the stream can seek backward and forward.
- Writes are uploaded when the stream is flushed or closed, not while you write. Each
  `fflush()` uploads the whole buffer again. An upload error triggers a warning, but
  `fclose()` and `file_put_contents()` cannot report it, so call `fflush()` and check its
  result when it matters.
- `a` downloads the whole object, and uploads it again when the stream is closed. Two processes
  that append at the same time overwrite each other's changes.
- `touch()`, `chmod()`, `chown()`, `flock()` (and `file_put_contents()` with `LOCK_EX`) and
  `ftruncate()` are not supported.
- Directories are emulated with key prefixes. `mkdir()` ignores `recursive`. `rmdir()` of a
  prefix deletes only its `dir/` object, and fails when the prefix has other objects; it
  returns `true` when the prefix has no objects at all.
- `mkdir()` without an `ACL` context option sends the ACL of the mode, as the AWS SDK does:
  `public-read` for `07xx` (including the default `0777`), `authenticated-read` for `06xx`, and
  `private` otherwise. Buckets with the "Bucket owner enforced" object ownership setting (the
  default for new buckets) accept only the `bucket-owner-full-control` ACL, so pass it as the
  `ACL` context option there.
- `rename()` copies the object with one CopyObject request, so it is limited to objects up to
  5 GB (the AWS SDK switches to a multipart copy above 5 GB). It is not atomic: the source is
  deleted after the copy.
- The results of `url_stat()` and the entries listed by `readdir()` are cached in the process
  (up to 1000 paths), and the wrapper clears them when it changes a path. Call
  `S3StreamWrapper::clearCache()` when other processes change the objects. PHP's `clearstatcache()`
  does not clear this cache.
- A read stream receives the body through AsyncAws, which keeps a copy of the bytes received in a
  `php://temp` stream (in memory up to 2 MB, then in a temporary file).
- Errors trigger an `E_USER_WARNING` and the function returns `false`. `file_exists()`,
  `is_file()` and the like report missing paths without a warning.
- The wrapper is registered with `STREAM_IS_URL`, so it needs `allow_url_fopen`, and paths
  cannot be included (`include 's3://...'`).

These differ from the AWS SDK:

- `opendir()` sends the first ListObjectsV2 request when the directory is opened, so a missing
  bucket makes `opendir()` and `scandir()` fail with a warning. An error on a later page makes
  `readdir()` return `false` with a warning instead of throwing an exception.
- `rmdir()` of a prefix also fails when the prefix has a `dir/` object and other objects (the
  AWS SDK deletes the `dir/` object then).
- `rename()` sends an ACL only when the `ACL` (or `acl`) context option is set. The AWS SDK
  always sends one (`private` by default), and buckets with ACLs disabled accept only
  `bucket-owner-full-control`.
- `a` treats only a missing object (404) as a new object; other errors make `fopen()` fail.
- `fstat()` returns the file mode `0100777`.
- Uploads with an `async-aws/s3` client (`S3StreamWrapper::register(new S3Client(...))`) use
  PutObject, up to 5 GB, as the AWS SDK. With an `async-aws/simple-s3` client (the default),
  bodies larger than 64 MB (`PartSize`) are uploaded in parts. The content type of a key is
  guessed from its extension with CakePHP's list of MIME types, when `ContentType` is not set.

### CakePHP's File and Folder Classes

CakePHP 2's `File` and `Folder` classes do not work with `s3://` paths, with this wrapper or with
the AWS SDK's: `Folder::isAbsolute()` recognizes the protocol of a stream wrapper only when it
has letters only, so `s3://bucket/dir` is handled as a relative path. With a protocol of letters
only, for example `S3StreamWrapper::register($client, 'awss')`, `Folder::read()`, `find()` and
`new Folder($path, true)` work, and `File` reads, writes, appends to and deletes existing objects.
`File::write()` and `File::create()` cannot create an object, because `File::create()` calls
`touch()`. Use `file_put_contents()` to create objects.

## S3 Storage

`S3Storage` takes the same `s3://bucket/key` paths and provides the common file operations
with [`async-aws/simple-s3`](https://async-aws.com/integration/simple-s3.html):

```php
App::uses('S3Storage', 'AsyncAws.Lib');

$storage = new S3Storage();

$storage->write('s3://my-bucket/reports/today.csv', $csv, ['ContentType' => 'text/csv']);
file_put_contents(TMP . 'today.csv', $storage->read('s3://my-bucket/reports/today.csv'));
```

| Method | Description |
| --- | --- |
| `write($path, $contents, $options = [])` | Uploads a string or a stream. Large objects are uploaded in parts. `$options` are those of `SimpleS3Client::upload()`, e.g. `ContentType`, `CacheControl`, `ACL`, `Metadata`. |
| `read($path)` | Returns the contents. Throws `AsyncAws\S3\Exception\NoSuchKeyException` when the object does not exist. |
| `readStream($path)` | Returns a stream (`resource`) of the contents. |
| `exists($path)` | Returns whether the object exists. |
| `delete($path)` | Deletes the object. |
| `url($path)` | Returns the URL of the object. |
| `presignedUrl($path, $expires = null)` | Returns a presigned download URL. `$expires` is a `DateTimeImmutable`. |
| `client()` | Returns the `SimpleS3Client`, for anything else. |

`new S3Storage()` creates the client with the `AsyncAws` configuration. Pass a
`SimpleS3Client` to use another one: `new S3Storage($this->AsyncAws->createSimpleS3($args))`.