Skip to main content
Version: 0.6.0

Object Store

Seaplane comes with a built-in object store to build powerful data science and machine learning applications. The Seaplane SDK hides all the complexity of setting up and connecting to the object store and hands you a single object_store instance to interact with. Authentication is handled automatically by your API key. You can read more about authentication here.

To add an object store to any task simply import the object store from the Seaplane package.

from seaplane.object import object_store

def my_object_store_task(msg):
    # do something with your object store here
    object_store.list_buckets()
info

The maximum size of an object is currently set to 8MB. We are working on supporting larger objects in a future update, if you want access now let us know by emailing support@seaplane.io.

The object life cycle is decoupled from tasks and applications, in other words deleting a task or application does not automatically delete the objects.

In addition to the Python SDK, you can also access the object store through the plane command line interface (CLI) tool. Make sure to configure plane before use. You can learn more about configuring plane here.

Buckets And Bucket Operations​

The seaplane object store supports the use of buckets. Buckets are used to group objects together. In a future version, we are introducing permissions such as public and read-only on buckets.

Creating A Bucket​

To create a bucket call the create_bucket() method on the object_store and supply it with the required arguments.

  • The bucket name type:string
from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.create_bucket("my-bucket")

Expected Output:

  • True - If successful type:bool
  • False - If unsuccessful type:bool

Deleting A Bucket​

To delete a bucket call the delete_bucket() method on the object_store and supply it with the required arguments. You can only delete empty buckets, to prevent accidental data loss.

  • The bucket name type:string
from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.delete_bucket("my-bucket")

Expected Output:

  • True - If successful type:bool
  • False - If unsuccessful type:bool

Listing All Buckets​

You can list all the buckets in your tenant by calling the list_buckets() method on the object_store. It requires no additional arguments.

from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.list_buckets()

Expected Output:

A list of strings containing all your available buckets.

['bucket-1', 'bucket-2', 'bucket-3']

Objects And Object Operations​

Uploading An Object​

To upload an object call the upload() or upload_file() method on the object_store and supply it with the three required arguments.

  • The bucket name in which you want to store the object type:string
  • The object name type:string
  • The file type:file if using upload()
  • The path to the file type:string if using upload_file()

Upload:

from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.upload("my-bucket", "my-remote-obj-name", file)

Upload File:

from seaplane import task, object_store

def my_object_store_task(msg):
    object_store.upload_file('my-bucket', 'my-remote-obj-name', 'path/to/file.txt')

Expected Output:

  • True - If successful type:bool
  • False - If unsuccessful type:bool

Downloading an object​

To download an object call the download() method through a context manager expression on the object_store and supply it with the two required arguments.

  • The bucket name in which the object resides type:string
  • The object name type:string
from seaplane.object import object_store

def my_object_store_task(msg):
    with object_store.download("my-bucket", "my-remote-obj-name") as file:
        # do something with your object

Expected Output:

A byte string of your object.

b'bytes string'

Deleting An Object​

To delete an object call the delete() method on the object_store and supply it with the two required arguments.

  • The bucket name in which the object resides type:string
  • The object name type:string
from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.delete('my-bucket', "my-remote-obj-name")

Expected Output:

To update an object call the update() method on the obj_store and supply it with the three required arguments.

  • True - If successful type:bool
  • False - If unsuccessful type:bool

Listing All Objects In A Bucket​

You can list all objects in a bucket with the list_objects() method on the object_store. List objects has two required arguments.

  • The bucket name type:string
  • Path prefix type:string usually the default / is required
from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.list_objects('my-bucket', '/')

Expected Output:

A list of dicts one for each object in the bucket.

[{'name': 'remote-object-name',
  'digest': 'SHA-256=9frluGfQAwakKSY8t5q993tRVGdP5lWSY4li_203L_8=',
  'created_at': datetime.datetime(2023, 10, 5, 13, 11, 52),
  'size': '15.0B'}]

Check If An Object Exists​

To check if an object exists call the exists() method on the object_store with the two required arguments. The exists() method returns True if the object exists and False if it does not.

  • The bucket name in which the object resides type:string
  • The proposed object name type:string
from seaplane.object import object_store

def my_object_store_task(msg):
    object_store.exists("my-bucket", "my-remote-obj-name")

Expected Output:

  • True - If successful type:bool
  • False - If unsuccessful type:bool

Object Store Access From Outside The Seaplane Ecosystem​

You can access all the same object store functionality outside of the Seaplane ecosystem by importing the object_store in your Python project and authenticating with your API key.

example.py
from seaplane.object import object_store
from seaplane.config import config
import os

config.set_api_key(os.get_env('SEAPLANE_KEY'))

# any object store call here
object_store.list_buckets()