Dask Performance
Tech Blog

Shuffling large data at constant memory in Dask

February 23, 2023
With release 2023.2.1, dask.dataframe introduces a new shuffling method called P2P, making sorts, merges, and joins faster...

Prioritizing Pragmatic Performance for Dask

November 15, 2022
Dask developers care about performance, we’ve always taken a pragmatic rather than exciting approach to the problem...

Reducing memory usage in Dask workloads by 80%

November 15, 2022
The latest version of Dask (2022.11.0) can significantly reduce your memory usage. Here's how we did it.

Introducing the Dask Active Memory Manager

December 6, 2021
Dask release 2021.10.0 introduces the first piece of a new modular system called Active Memory Manager, which aims to alleviate memory issues.
Dask Usage

High-Performance Data Visualization with Datashader and Dask

Address performance issues for large-scale data visualizations by making smart choices about cluster memory, data types and data partitioning.

How Popular is Matplotlib?

This analysis tracks the growth of Matplotlib on the preprint server arXiv beginning in 2002 with 1% up to 2022 with 17% of all papers using Matplotlib...

Automate your ETL Jobs in the Cloud with Github Actions, S3 and Coiled

This post will demonstrate how running Github Actions on Coiled can be a useful way to schedule automated data-processing jobs...
Cost Savings
Scalable Computing

Save Money with Spot

January 5, 2023
The cloud is wonderful but expensive. Spot/preemptible instances offer dramatic cost savings, but using them well requires considerable nuance... 

Cost Savings with Dask and Coiled

November 3, 2022
Coiled can often save money for an organization running Dask. This article goes through the most common ways in which we see that happen. 
Tech Blog

AWS Cost Explorer Tips and Tricks

January 6, 2023
In this post, I’m going to share some views that I’ve found especially useful when digging in to our costs for running data-science workloads on EC2 instances...
Coiled Product
Scalable Computing

Why we passed on Kubernetes

October 20, 2022
Coiled deploys Dask clusters in the cloud. We do this using raw cloud APIs rather than with Kubernetes...

Coiled Cloud Architecture

August 17, 2022
Over the last couple of years, Coiled has made a cloud-SaaS application that runs Dask for folks smoothly and securely in the cloud...
Tech Blog

How many PEPs does it take to install a package?

March 15, 2023
A few months ago we releaseda feature that takes your Python environment and replicates it in the cloud with zero effort...
Tech Blog

Just in time Python environments

February 23, 2023
In the latest Coiled release, version 0.4.0, you can use a new system for creating software environments on the fly using only mamba instead...
Coiled Social

Social Feeds

Dask Tutorials

Hands-on training, totally free. Tutorials offered every month.

With GitHub, Google or email.

Use your AWS or GCP account.

Start scaling.

$ pip install coiled
$ coiled setup
$ ipython
>>> import coiled
>>> cluster = coiled.Cluster(n_workers=500)