Coiled Blog

Accelerating Data Science, Python and Open Source

RSS Feed

Save Money with Spot

Nat Tabris
January 5, 2023
The cloud is wonderful but expensive. Spot/preemptible instances offer dramatic cost savings, but using them well requires considerable nuance which we explore in this post. 
Category:
No items found.

High-Performance Data Visualization with Datashader and Dask

Richard Pelgrim
November 17, 2022
Address performance issues for large-scale data visualizations by making smart choices about cluster memory, data types and data partitioning.
Category:
No items found.

Reducing memory usage in Dask workloads by 80%

Gabe Joseph
November 15, 2022
The latest version of Dask (2022.11.0) can significantly reduce your memory usage. Here's how we did it.
Category:
Dask Scheduler

Prioritizing Pragmatic Performance for Dask

Matthew Rocklin
November 15, 2022
This article will go into a few things that I think are great ideas, but are not yet major bottlenecks and why. Then it will go into a few things that I think are less-than-exciting-but-really-important improvements for common workloads.
Category:
Dask
Python

How Popular is Matplotlib?

Matthew Rocklin
November 4, 2022
Anecdotally the Matplotlib maintainers were told "About 15% of arXiv papers use Matplotlib" arXiv is the preeminent repository for scholarly preprint articles. It stores millions of journal articles used across science. It's also public access, and so we can just scrape the entire thing given enough compute power.
Category:
Data Science
Distributed Computing
Scalable Computing

Cost Savings with Dask and Coiled

Coiled Team
November 3, 2022
Coiled can often save money for an organization running Dask. This article goes through the most common ways in which we see that happen. 
Category:
Coiled

Why we passed on Kubernetes

Matthew Rocklin
October 20, 2022
Kubernetes is great if you need to organize many always-on services and have in-house expertise, but can add an extra burden and abstraction when deploying a single bursty service like Dask, especially in a user environment with quickly changing needs.‍
Category:
Distributed Computing

How Coiled sets memory limit for Dask workers

Nat Tabris
August 17, 2022
Having Dask workers die from memory overuse is common, so we thought that we’d investigate this further.
Category:
Coiled Computing
Dask

Level up your Dask using Coiled

Coiled makes it easy to scale Dask maturely in the cloud