Skip to main content
Version: 0.6.0

Project Creation And Default Structure

This document describes the default project structure of Seaplane applications. It assumes you have the Seaplane SDK and CLI tool installed. You can learn more about installation and setup here.

Creating A New Project​

To create a new Seaplane application run the following command inside your terminal. We recommend you set up a virtual environment to avoid dependency conflicts.

seaplane init my-project-name

Seaplane automatically generates the default project structure and populates the main.py file with a "hello world" application.

Creating Virtual Environments​

Run the following commands to create a virtual environment. You can learn more about virtual environments here.

python3 -m venv my-seaplane-project
source my-seaplane-project/bin/activate

Default Project Structure and Files​

A default seaplane project consists of the following elements.

my-project-name/
├── my-project-name/
│    └── main.py
├── .env
└── pyproject.toml

The main my-project-name directory contains all the files associated with this project. Including a directory for the project's Python scripts, a pyproject.toml file for build configuration and a .env file for declaring project secrets.

Python File Directory​

Each Seaplane project requires another directory by the same name containing at least one Python script indicated as the main Python script by the pyproject.toml file. You can learn more about the Pyproject file in the section below.

The only requirement for the main Python script is that it includes a new app instance by using App(), a response to return via app.respond(), an app.run() command to tell Seaplane to run the app, a DAG (Directed Acyclic Graph), and a task. The init function generates a sample project with one file main.py containing all the components of an application, a DAG, and a task.

For larger projects, we recommend you define your App() and your main DAG inside your main.py. Create a new file for all other DAGs and tasks to enable easier collaboration and to keep your project structured and easy to understand.

info

The application, DAG, and task names are currently limited to 28 characters and can only include the following characters:

  • a-z
  • 0-9
  • -

A sample project structure might look something like this.

my-project-name/
├── my-project-name/
│    ├── task1.py
│    ├── task2.py
│    ├── task3.py
│    ├── dag1.py
│    └── main.py
├── .env
└── pyproject.toml

Pyproject​

The pyproject.toml file is the main configuration file. It contains basic information about your app and its dependencies, and it indicates which Python script in your project contains the app.run() command. More information on each section below.

  • [tool.poetry] - Includes the name, version and optionally a description and authors of the project.
  • [tool.seaplane] - Indicates which Python script includes the app.run() command. By default, it is set to main.py but you can use any Python script you like as long as it contains the start command.
  • [tool.poetry.dependencies] - This section should include all dependencies of your project. Add each dependency as a new line. To avoid dependency conflicts we recommend using loosely defined dependencies i.e 'package>=1.0.0' instead of 'package==1.0.0'
  • [build-system] - This section defines the build backend and relevant configuration options for building your project.

Environment Variables​

Use the .env file to define application secrets. During deployment, Seaplane automatically creates environment variables for all secrets defined in .env.

Make sure you include the following line inside your .gitignore file to avoid pushing application secrets to your GitHub repository.

.gitignore
.env

You can load environment secrets inside your application using os.getenv('SECRET_NAME'). For example, assume you have the following .env file.

.env
SEAPLANE_API_KEY=<YOUR-SEAPLANE-KEY>
OPEN_AI_API_KEY=<API-KEY-1>
MY_EXTERNAL_SERVICE=<API-KEY-2>

During deployment, Seaplane automatically creates the required secrets as environment variables. You can load them inside a sql_access object as follows.

task.py
import os

def my_sql_task(msg):
    # load API keys from environment variables
    open_ai_key = os.getenv('OPEN_AI_API_KEY')
    external_key = os.getenv('MY_EXTERNAL_SERVICE')