Use Case

Use Coiled. Save the planet.

Dask provides the foundation for scaling spatial analysis in Python.

Popular geospatial libraries integrate tightly with Dask, allowing you to scale workloads to Earth-sized datasets. Coiled makes it easy to launch clusters next to that data, providing fast access to both public and your own private datasets in the cloud.

Fundamental packages like Xarray and GeoPandas integrate natively with Dask to scale. You can use the same interface for working with a 1GB file locally as for a 1TB dataset distributed across a cluster.

A rich ecosystem of geospatial tools are available, thanks to a large community of academic, industry, and government users.

Dask’s flexibility enables advanced spatial workflows that would be impractical to run at scale with traditional GISs or databases.

# code used to produce the image above ^
import xarray as xr
import geogif
import coiled

ds = xr.open_dataset(
    "s3://era5-pds/zarr/2020/01/data/eastward_wind_at_100_metres.zarr",
    engine="zarr",
    chunks={},
)
hourly_wind = (
    ds['eastward_wind_at_100_metres']
    sel(lon=slice(180, None))
    .coarsen(lat=2, lon=2, boundary='trim')
    .mean()
)

cluster = coiled.Cluster(backend_options={'region': 'us-east-1'}) 
# ^ same region as `era5-pds` bucket
client = cluster.get_client()

geogif.dgif(hourly_wind).compute()

Dask + Coiled

Docs

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat.

Submit arbitrary functions for computation in a parallelized, eager, and non-blocking way.

Explore
Learn More arrow

Dask Bag implements operations like map, filter, groupby and aggregations on collections of Python objects.

Explore
Learn More arrow

What if you don’t have an array or dataframe? Instead of having blocks where the function is applied to each block, you can decorate functions with @delayed and have the functions themselves be lazy.

Explore
Learn More arrow
Heading

What if you don’t have an array or dataframe? Instead of having blocks where the function is applied to each block, you can decorate functions with @delayed and have the functions themselves be lazy.

Explore
Learn More arrow
Heading

What if you don’t have an array or dataframe? Instead of having blocks where the function is applied to each block, you can decorate functions with @delayed and have the functions themselves be lazy.

Explore
Learn More arrow
Heading

What if you don’t have an array or dataframe? Instead of having blocks where the function is applied to each block, you can decorate functions with @delayed and have the functions themselves be lazy.

Explore
Learn More arrow

With GitHub, Google or email.

Use your AWS or GCP account.

Start scaling.

$ pip install coiled
$ coiled setup
$ ipython
>>> import coiled
>>> cluster = coiled.Cluster(n_workers=500)