Skip to content
This repository was archived by the owner on May 29, 2025. It is now read-only.

Update sample to use boto3 #6

Merged
petermoon merged 2 commits into from
Dec 31, 2015
Merged

Update sample to use boto3 #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b4326b7
Update sample to use boto3
petermoon Dec 30, 2015
c4bca70
Add Python 3.5 in the README
petermoon Dec 31, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 18 additions & 12 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,28 @@
# AWS SDK for Python Sample Project

A simple Python application illustrating usage of the AWS SDK for Python (also
referred to as Boto).
referred to as `boto3`).

## Requirements

This sample project depends on Boto, the AWS SDK for Python, and requires
Python 2.6 or 2.7. You can install Boto using pip:
This sample project depends on `boto3`, the AWS SDK for Python, and requires
Python 2.6.5+, 2.7, 3.3, 3.4, or 3.5. You can install `boto3` using pip:

pip install boto
pip install boto3

## Basic Configuration

You need to set up your AWS security credentials before the sample code is able
to connect to AWS. You can do this by creating a file named "credentials" at
`~/.aws/` (`C:\Users\USER_NAME\.aws\` for Windows users)
and saving the following lines in the file:
to connect to AWS. You can do this by creating a file named "credentials" at ~/.aws/
(`C:\Users\USER_NAME\.aws\` for Windows users) and saving the following lines in the file:

[default]
aws_access_key_id = <your access key id>
aws_secret_access_key = <your secret key>

See the [Security Credentials](http://aws.amazon.com/security-credentials) page
for more information on getting your keys. It's also possible to configure your
credentials via other configuration files. See the [Boto Config documentation](http://boto.readthedocs.org/en/latest/boto_config_tut.html)
for more information.
for more information on getting your keys. For more information on configuring `boto3`,
check out the Quickstart section in the [developer guide](https://boto3.readthedocs.org/en/latest/guide/quickstart.html).

## Running the S3 sample

Expand All @@ -34,8 +32,16 @@ bucket name and file for you. All you need to do is run the code:

python s3_sample.py

The S3 documentation has a good overview of the [restrictions for bucket names](http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html)
for when you start making your own buckets.
You need to make sure the credentials you're using have the correct permissions to access the Amazon S3
service. If you run into 'Access Denied' errors while running this sample, please follow the steps below.

1. Log into the [AWS IAM Console](https://console.aws.amazon.com/iam/home)
2. Navigate to the Users page.
3. Find the AWS IAM user whose credentials you're using.
4. Under the 'Permissions' section, attach the policy called 'AmazonS3FullAccess'
5. Re-run the sample. Now your user should have the right permissions to run the sample.

The sample creates randomly generated bucket names for you, but please be aware of the [restrictions for bucket names](http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html) when you start creating your own buckets.

## License

Expand Down
150 changes: 102 additions & 48 deletions s3_sample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,77 +1,131 @@
# Copyright 2013. Amazon Web Services, Inc. All Rights Reserved.
#
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#
# http://www.apache.org/licenses/LICENSE-2.0
#
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Import the SDK
import boto
import boto3
import uuid

# Instantiate a new client for Amazon Simple Storage Service (S3). With no
# parameters or configuration, the AWS SDK for Python (Boto) will look for
# access keys in these environment variables:
# boto3 offers two different styles of API - Resource API (high-level) and
# Client API (low-level). Client API maps directly to the underlying RPC-style
# service operations (put_object, delete_object, etc.). Resource API provides
# an object-oriented abstraction on top (object.delete(), object.put()).
#
# While Resource APIs may help simplify your code and feel more intuitive to
# some, others may prefer the explicitness and control over network calls
# offered by Client APIs. For new AWS customers, we recommend getting started
# with Resource APIs, if available for the service being used. At the time of
# writing they're available for Amazon EC2, Amazon S3, Amazon DynamoDB, Amazon
# SQS, Amazon SNS, AWS IAM, Amazon Glacier, AWS OpsWorks, AWS CloudFormation,
# and Amazon CloudWatch. This sample will show both styles.
#
# AWS_ACCESS_KEY_ID='...'
# AWS_SECRET_ACCESS_KEY='...'
# First, we'll start with Client API for Amazon S3. Let's instantiate a new
# client object. With no parameters or configuration, boto3 will look for
# access keys in these places:
#
# For more information about this interface to Amazon S3, see:
# http://boto.readthedocs.org/en/latest/s3_tut.html
s3 = boto.connect_s3()
# 1. Environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY)
# 2. Credentials file (~/.aws/credentials or
# C:\Users\USER_NAME\.aws\credentials)
# 3. AWS IAM role for Amazon EC2 instance
# (http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)

s3client = boto3.client('s3')

# Everything uploaded to Amazon S3 must belong to a bucket. These buckets are
# in the global namespace, and must have a unique name.
#
# For more information about bucket name restrictions, see:
# http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html
bucket_name = "python-sdk-sample-%s" % uuid.uuid4()
print("Creating new bucket with name: " + bucket_name)
bucket = s3.create_bucket(bucket_name)
bucket_name = 'python-sdk-sample-{}'.format(uuid.uuid4())
print('Creating new bucket with name: {}'.format(bucket_name))
s3client.create_bucket(Bucket=bucket_name)

# Files in Amazon S3 are called "objects" and are stored in buckets. A specific
# object is referred to by its key (i.e., name) and holds data. Here, we create
# a new object with the key "python_sample_key.txt" and content "Hello World!".
#
# For more information on keys and set_contents_from_string, see:
# http://boto.readthedocs.org/en/latest/s3_tut.html#storing-data
from boto.s3.key import Key
k = Key(bucket)
k.key = 'python_sample_key.txt'

print ("Uploading some data to " + bucket_name + " with key: " + k.key)
k.set_contents_from_string('Hello World!')

# Fetch the key to show that we stored something. Key.generate_url will
# construct a URL that can be used to access the object for a limited time.
# Here, we set it to expire in 30 minutes.
# Now the bucket is created, and you'll find it in your list of buckets.

list_buckets_resp = s3client.list_buckets()
for bucket in list_buckets_resp['Buckets']:
if bucket['Name'] == bucket_name:
print('(Just created) --> {} - there since {}'.format(
bucket['Name'], bucket['CreationDate']))

# Files in Amazon S3 are called "objects" and are stored in buckets. A
# specific object is referred to by its key (i.e., name) and holds data. Here,
# we create (put) a new object with the key "python_sample_key.txt" and
# content "Hello World!".

object_key = 'python_sample_key.txt'

print('Uploading some data to {} with key: {}'.format(
bucket_name, object_key))
s3client.put_object(Bucket=bucket_name, Key=object_key, Body=b'Hello World!')

# Using the client, you can generate a pre-signed URL that you can give
# others to securely share the object without making it publicly accessible.
# By default, the generated URL will expire and no longer function after one
# hour. You can change the expiration to be from 1 second to 604800 seconds
# (1 week).

url = s3client.generate_presigned_url(
'get_object', {'Bucket': bucket_name, 'Key': object_key})
print('\nTry this URL in your browser to download the object:')
print(url)

try:
input = raw_input
except NameError:
pass
input("\nPress enter to continue...")

# As we've seen in the create_bucket, list_buckets, and put_object methods,
# Client API requires you to explicitly specify all the input parameters for
# each operation. Most methods in the client class map to a single underlying
# API call to the AWS service - Amazon S3 in our case.
#
# For a more detailed overview of generate_url's options, see:
# http://boto.readthedocs.org/en/latest/ref/s3.html#boto.s3.key.Key.generate_url
expires_in_seconds = 1800
# Now that you got the hang of the Client API, let's take a look at Resouce
# API, which provides resource objects that further abstract out the over-the-
# network API calls.
# Here, we'll instantiate and use 'bucket' or 'object' objects.

print('\nNow using Resource API')
# First, create the service resource object
s3resource = boto3.resource('s3')
# Now, the bucket object
bucket = s3resource.Bucket(bucket_name)
# Then, the object object
obj = bucket.Object(object_key)
print('Bucket name: {}'.format(bucket.name))
print('Object key: {}'.format(obj.key))
print('Object content length: {}'.format(obj.content_length))
print('Object body: {}'.format(obj.get()['Body'].read()))
print('Object last modified: {}'.format(obj.last_modified))

print("Generating a public URL for the object we just uploaded. This URL will be active for %d seconds" % expires_in_seconds)
print('')
print(k.generate_url(expires_in_seconds))
print('')
# Buckets cannot be deleted unless they're empty. Let's keep using the
# Resource API to delete everything. Here, we'll utilize the collection
# 'objects' and its batch action 'delete'. Batch actions return a list
# of responses, because boto3 may have to take multiple actions iteratively to
# complete the action.

print('\nDeleting all objects in bucket {}.'.format(bucket_name))
delete_responses = bucket.objects.delete()
for delete_response in delete_responses:
for deleted in delete_response['Deleted']:
print('\t Deleted: {}'.format(deleted['Key']))

try: input = raw_input
except NameError: pass
input("Press enter to delete both the object and the bucket...")
# Now that the bucket is empty, let's delete the bucket.

# Buckets cannot be deleted unless they're empty. Since we still have a
# reference to the key (object), we can just delete it.
print("Deleting the object.")
k.delete()
print('\nDeleting the bucket.')
bucket.delete()

# Now that the bucket is empty, we can delete it.
print("Deleting the bucket.")
s3.delete_bucket(bucket_name)
# For more details on what you can do with boto3 and Amazon S3, see the API
# reference page:
# https://boto3.readthedocs.org/en/latest/reference/services/s3.html