Python1.1

最近更新时间:2019-01-14 10:11:00

KS3 SDK for python使用指南


目录


1 前言

1.1 简介

1.2 下载

1.3 兼容性说明

适用于2.6、2.7的Python版本,目前不支持Python 3版本。

1.4 历史版本说明

2 安装

安装环境

KS3 Python SDK适用于Python 2.6、2.7,根据Python官网的引导安装合适的Python版本。

安装依赖模块

pip install six

安装SDK

在线安装
pip install ks3sdk
本地安装

1、通过git下载SDK到本地

git clone https://github.com/ks3sdk/ks3-python-sdk.git

2、进入ks3-python-sdk目录

cd  ks3-python-sdk

3、安装SDK

python setup.py install

3 初始化

from ks3.connection import Connection
ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
c = Connection(ak, sk, host='<YOUR_REGION_ENDPOINT>', is_secure=False, domain_mode=False)

参数说明

  • ak:金山云提供的ACCESS KEY ID

  • sk:金山云提供的SECRET KEY ID

  • host:金山云提供的各个Region的域名(例 ks3-cn-beijing.ksyun.com),具体定义可参考 API接口文档-Region(区域)。也可以是用户自定义域名,如果是用户自定义域名,需要将domain_mode设置为True。

  • is_secure:是否通过HTTPS协议访问Ks3,True:启用 False:关闭。

  • domain_mode:是否使用自定义域名访问Ks3(host填写自定义域名),True:是 False:否。

4 快速入门

创建一个存储空间(Bucket)

在建立了连接后,可以创建一个bucket。bucket在s3中是一个用于储存key/value的容器。用户可以将所有的数据存储在一个bucket里,也可以为不同种类数据创建相应的bucket。

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.create_bucket(bucket_name)
print bucket
print 'Bucket:%s Create success' % bucket_name

注:这里如果出现409 conflict错误,说明请求的bucket name有冲突,因为bucket name是全局唯一的

上传文件

将指定目录下某一个文件上传,同时可以指定文件ACL

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'
key_name = '<YOUR_Key_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
object = bucket.new_key(key_name)
#object policy : 'private' or 'public-read'
ret=object.set_contents_from_filename('<<YOUR_LOCAL_FILE>>', policy='private')
if ret and ret.status == 200:
    print '上传成功'

下载文件

下载object,并且保存到文件中

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'
key_name = '<YOUR_Key_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
object = bucket.get_key(key_name)
#保存到文件
object.get_contents_to_filename('<YOUT_LOCAL_FILE>')

列举文件

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
objects = bucket.list()
for object in objects:
    print object.name

删除文件

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'
key_name = '<YOUR_Key_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket=conn.get_bucket(bucket_name)
bucket.delete_key(key_name)

5 存储空间管理

5.1 列举Bucket列表

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

buckets = conn.get_all_buckets()
for bucket in buckets:
    print bucket.name

5.2 创建Bucket

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.create_bucket(bucket_name)
print bucket
print 'Bucket:%s Create success' % bucket_name

5.3 删除Bucket

删除一个bucket可以通过delete_bucket方法实现。

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

conn.delete_bucket(bucket_name)

如果bucket下面存在key,那么需要首先删除所有key

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
for object in bucket.list():
    object.delete()
conn.delete_bucket(bucket_name)

5.4 获取Bucket的region信息

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

region_name = conn.get_bucket_location(bucket_name)

print region_name

5.4 判断Bucket是否存在

5.5 设置Bucket ACL

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)

#设置bucket的权限, private or public-read or public-read-write
bucket.set_acl("public-read")

5.6 获取Bucket ACL

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
acp = bucket.get_acl()
for grant in acp.acl.grants:
    print grant.permission, grant.id, grant.uri

5.8 设置Bucket空间策略

5.9 获取Bucket空间策略

6 上传文件

6.1 简单上传

上传方法支持多种类型的输入源,输入源有如下几种类型:字符串、本地文件

上传字符串
from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
key = bucket.new_key(key_name)
#object policy : 'private' or 'public-read'
ret=key.set_contents_from_string("this is a string", policy="private")
if ret and ret.status == 200:
    print "上传成功"
上传本地文件
from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
key = bucket.new_key(key_name)
#object policy : 'private' or 'public-read'
ret=k.set_contents_from_filename("<yourLocalFile>", policy="private")
if ret and ret.status == 200:
    print "上传成功"

6.2 分片上传

如果上传一个大文件,可以将它分成几个小份,逐个上传,ks3会按照顺序把它们合成一个最终的object。整个过程需要几步来完成,

  1. 初始化(initiate_multipart_upload):获取Upload ID。
  2. 上传分片(upload_part_from_file):上传分片数据。这一步可以并发进行。
  3. 完成上传(complete_upload):所有分片上传完成后,合并分片,生成OSS文件

下面的demo程序是通过python的FileChunkIO模块来实现的。所以可能需要首先运行pip install FileChunkIO来安装。

import math, os
from ks3.connection import Connection
from filechunkio import FileChunkIO
from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)
bucket = conn.get_bucket(bucket_name)

source_path = '<YOUR_LOCAL_FILE>'
source_size = os.stat(source_path).st_size

# Create a multipart upload request
mp =bucket.initiate_multipart_upload(os.path.basename(source_path))

# Use a chunk size of 50 MiB (feel free to change this)
chunk_size = 52428800
chunk_count = int(math.ceil(source_size / float(chunk_size)))

# Send the file parts, using FileChunkIO to create a file-like object
# that points to a certain byte range within the original file. We
# set bytes to never exceed the original file size.
for i in range(chunk_count):
    offset = chunk_size * i
    bytes = min(chunk_size, source_size - offset)
    with FileChunkIO(source_path, 'r', offset=offset,bytes=bytes) as fp:
        mp.upload_part_from_file(fp, part_num=i + 1)

# Finish the upload
mp.complete_upload()

获取已上传分块列表

bucket = conn.get_bucket(bucket_name)

for part in bucket.list_multipart_uploads():
    print 'uploadId:%s,key:%s' % (part.id, part.key_name)
    for info in part:
        print info.part_number, info.size, info.etag, info.last_modified

7 下载文件

7.1 下载为字符串

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
key = bucket.get_key(key_name)
s = key.get_contents_as_string()
print s

7.2 下载为本地文件

from ks3.connection import Connection

ak = '<YOUR_ACCESS_KEY>'
sk = '<YOUR_SECRET_KEY>'
host='<YOUR_REGION_ENDPOINT>'
bucket_name = '<YOUR_BUCKET_NAME>'

conn = Connection(ak, sk, host,is_secure=False, domain_mode=False)

bucket = conn.get_bucket(bucket_name)
key = bucket.get_key(key_name)
k.get_contents_to_filename("<YOUR_LOCAL_FILE>")

7.3 流式下载

金山云,开启您的云计算之旅

立即注册