Client compatibility list

This section needs your help


Fully tested with the current API coverage. Here is a minimal configuration you can put in ~/.s3cfg:

host_base =
host_bucket = %(bucket)
access_key = YOUR_ACCESS_KEY
secret_key = YOUR_SECRET_KEY
use_https = True
signature_v2 = True

Adapt with your credentials and replace with the value you specified for service-uri. use_https is needed only if Pithos is served over TLS. Currently pithos doesn’t support v4 signatures so the signature_v2 flag is necessary.

When testing locally, the following configuration can be used:

host_base =
host_bucket = %(bucket)
access_key = YOUR_ACCESS_KEY
secret_key = YOUR_SECRET_KEY
use_https = False
signature_v2 = True
proxy_host = localhost
proxy_port = 8080


Working support with the S3 provider:

from import Provider
from import get_driver
cls = get_driver(Provider.S3)
driver = cls('api key', 'api secret key', host='')


On-going integration


Working support

s3fs - s3 fuse support

Working support. If you specified as service-uri, you can mount the bucket bucket with the following command:

s3fs bucket /mnt/bucket -o url=

The credentials have to be specified in ~/.passwd-s3fs:


WAL-E - continuous archiving for Postgres

Support for S3-compatible object stores was added in version 0.8 of WAL-E. Configure WAL-E with the following environment variables:

WALE_S3_ENDPOINT https+path://
WALE_S3_PREFIX s3://your-bucket/your-prefix

Archiving WAL files

Postgresql needs the following settings in postresql.conf:

wal_level = archive
archive_mode = on
archive_command = 'envdir /etc/wal-e.d/env /path/to/wal-e wal-push %p'
archive_timeout = 60

Once postgres is setup to send WAL files, make a base backup with envdir /etc/wal-e.d/env /path/to/wal-e backup-push /path/to/postgres/data

Restoring from archived WAL files

Pull a base backup:

envdir /etc/wal-e.d/env /path/to/wal-e backup-fetch /path/to/postgres/data LATEST

Create a recovery.conf file in the postgres data dir with the following content:

restore_command = 'envdir /etc/wal-e.d/env /path/to/wal-e wal-fetch "%f" "%p"'

Start postgresql and check the logs to see its restore status.

elasticsearch - index backup and restore

Snapshotting and restoring indices to Pithos is supported thanks to the AWS Cloud Plugin. To configure a snapshot repository that points to your pithos installation, simply add to your /etc/elasticsearch/elasticsearch.yml:

    access_key: <your key>
    secret_key: <your secret>
      protocol: https

Then create your repository:

$ curl -XPUT 'http://localhost:9200/_snapshot/pithos' -d '{
    "type": "s3",
    "settings": {
        "bucket": "es-snapshots"

Starting with version 2.4.2 of the plugin, all settings can be provided per-repository:

$ curl -XPUT 'http://localhost:9200/_snapshot/pithos' -d '{
    "type": "s3",
    "settings": {
        "bucket": "es-snapshots",
        "access_key": "your key",
        "secret_key": "your secret",
        "protocol": "http",
        "endpoint": "",

AWS Languages SDKs

In general, AWS Language SDKs can work with Pithos with the following configuration:

  • In ~/.aws/config:

    s3 =
        signature_version = s3
  • In ~/.aws/credentials:

    aws_access_key_id = <your key>
    aws_secret_access_key = <your secret>

You can have multiple profiles instead of altering the [default] configuration. Simply repeat configuration sections and name them [profile <profile name>]

Shell (awscli)

Install awscli, then:

aws s3 ls --endpoint-url=https://your-endpoint

To use a non-default profile:

aws s3 ls --endpoint-url=https://your-endpoint --profile=<profile-name>

Python (boto3)

Install boto3 and create a Pithos client like this:

import boto3.session

session = boto3.session.Session()
client = session.client('s3', endpoint_url='https://pithos-endpoint')

To use a non-default profile:

import boto3.session
session = boto3.session.Session(profile_name='profile-name')
client = session.client('s3', endpoint_url='https://pithos-endpoint')

Python (boto)

Boto version 2 is boto3’s ancestor but is still widely used. It doesn’t take ~/.aws/* configuration files into account.

from boto.s3.connection import S3Connection, OrdinaryCallingFormat

connection = S3Connection(key, secret, host='pithos-endpoint',
                          port=443, is_secure=True,
bucket = connection.get_bucket('your-bucket')


Install AWSSDK.S3, then:

Amazon.AWSConfigsS3.UseSignatureVersion4 = false;
var config = new Amazon.S3.AmazonS3Config()
    ServiceURL = host,
    SignatureVersion = "s3",
var client = new Amazon.S3.AmazonS3Client(apikey, secretKey, config);


Install AWS SDK for Java, then:

import com.amazonaws.ClientConfiguration;

ClientConfiguration config = new ClientConfiguration();
AmazonS3Client s3 = new AmazonS3Client(config);


Install AWS SDK for Java, then:

(ns sos.core
(:import com.amazonaws.ClientConfiguration

(defn test-s3
  (let [opts   (doto (ClientConfiguration.)
                 (.setSignerOverride "S3SignerType"))
        client (doto (AmazonS3Client. opts)
                 (.setEndpoint "https://your-endpoint"))]
    (.generatePresignedUrl client "batman" "foo.txt" (Date.))))