S3

Summary

• object store optimized for arbitrary sized files

Quickstart

common options

• acl

  • private

  • public-read

  • public-read-write

  • authenticated-read

  • aws-exec-read

  • bucket-owner-read

  • bucket-owner-full-control

upload a file

aws s3api put-object --bucket linkevi.curator --key datasets/$DATA_SET --body $PATH_TO_DATASET

download a file

aws s3api get-object --bucket linkevi.curator --key datasets/$DATA_SET $OUTFILE

Reference

Limitation

• can't nest buckets

• bucket names must be unique across all of s3

• 100 bucket limit, can be raised

Gotchas

Issues

403 denied, permission denied

• check if bucket is set to prevent public access by default, will prevent even admins from changing bucket permissions

Topics

Budget

Availability

Performance

Security