HDDS-14505. [Website v2] [Docs] [User Guide] s3a#300
HDDS-14505. [Website v2] [Docs] [User Guide] s3a#300jojochuang merged 5 commits intoapache:HDDS-9225-website-v2from
Conversation
| - `core-site.xml` is configured as above | ||
| - An S3 bucket (for example `bucket1`) already exists (you can create it with `aws s3api --endpoint http://localhost:9878 create-bucket --bucket bucket1`) | ||
|
|
||
| S3A URLs use the form `s3a://<bucket>/<path>`. The bucket corresponds to an Ozone bucket under the `/s3v` volume or a bucket link. |
There was a problem hiding this comment.
The path is broken with the quoting
|
|
||
| ### Recommended settings for Ozone | ||
|
|
||
| Ozone S3 Gateway does not support ETags and object versioning in the same way as AWS S3. To avoid compatibility issues, set these options when using S3A with Ozone: |
There was a problem hiding this comment.
Etag support is added in S3 MPU. Do we still recommend below configs?
ashishkumar50
left a comment
ashishkumar50
left a comment
There was a problem hiding this comment.
@rich7420, Thanks for writing the doc.
|
|
||
| ```bash | ||
| # To local filesystem | ||
| hadoop fs -copyToLocal s3a://bucket1/path/local-file.txt /tmp/from-ozone.txt |
There was a problem hiding this comment.
nit:
hadoop fs -copyToLocal s3a://bucket1/path/file.txt /tmp/from-ozone.txt
| hadoop fs -copyToLocal s3a://bucket1/path/local-file.txt /tmp/from-ozone.txt | ||
|
|
||
| # Copy to HDFS | ||
| hadoop fs -cp s3a://bucket1/path/local-file.txt hdfs:///user/test/from-ozone.txt |
There was a problem hiding this comment.
nit:
hadoop fs -cp s3a://bucket1/path/file.txt hdfs:///user/test/from-ozone.txt
|
@ashishkumar50 thanks for the review! |
|
It looks good to me. Anything else you'd like to add? @ashishkumar50 @ssulav |
|
LGTM |
| <value>us-east-1</value> | ||
| <description> | ||
| Logical region name required by the S3A client. Ozone does not enforce regions, | ||
| but this must be a valid-looking value. |
There was a problem hiding this comment.
it would be nice to have a link to the spec for region name format.
There was a problem hiding this comment.
It doesn't look like Ozone s3 gateway does any checks for region string.
| <name>fs.s3a.path.style.access</name> | ||
| <value>true</value> | ||
| <description> | ||
| Ozone S3 Gateway defaults to path-style URLs (http://host:9878/bucket), |
There was a problem hiding this comment.
can ozone handle the style.access=false? if so what should be configured on ozone side?
There was a problem hiding this comment.
|
|
||
| ### Recommended settings for Ozone | ||
|
|
||
| Ozone S3 Gateway adds ETag support for S3 Multipart Upload (MPU). Object versioning and some other S3 behaviors may still differ from AWS S3. To avoid compatibility issues with older clients or when not using MPU, you can set these options when using S3A with Ozone: |
There was a problem hiding this comment.
it would be nice to describe the difference in Ozone's behavior comparing to AWS S3.
There was a problem hiding this comment.
|
@yandrey321 I addressed your comments so I think this is good to merge. |
|
thanks for the review! |
5 participants
What changes were proposed in this pull request?
Write about "s3a and Ozone"
What is the link to the Apache Jira?
HDDS-14505
How was this patch tested?
https://github.com/rich7420/ozone-site/actions/runs/21430617221