Coiled Product Detail

Rock-solid
Infrastructure

Coiled is a 24x7 managed service designed with best practices from the ground up.

Centralized Cluster Manager

What is it?
A 24x7 managed watchdog over every Dask cluster

Why it matters.
It’s Dask that just works. You get a cluster actively managed by a team of professionals that’s been robustly tested, monitored and battle-hardened.

Our Design.
This is a REST API that runs in Coiled’s cloud account, backed by a set of services that work in unison to idempotently  create and track clusters. This system has automated failover, automated monitoring, and a dedicated team of DevOps professionals keeping it in optimal health.

When you send requests to the Coiled control plane, it uses your credentials to set up resources on your behalf in your cloud account, connects you to those resources, and then gets out of the way. This system is built with raw cloud APIs, using optimal deployment choices based on years of experience (including countless mistakes and battle scars).

Coiled Manages Dask for you

Zero Trust Network Architecture

What is it?
Coiled's zero trust network architecture protects your cluster and your data.

Why it matters.
Using encryption in your cluster and between the client and cluster prevents attackers from reading your data, without you needing to configure security groups and SSL keys (but you can if you want!).

Our Design.
Coiled generates a distinct TLS key pair for each cluster for communication between the client and scheduler. This uniquely authenticates the client with the scheduler and ensures all traffic is encrypted.

Your Data Stays in Your Control

Flexible VM Architecture

What is it?
Full access to most VM instance types on the fly.

Why it matters.
Quickly burst from zero-cost idle to thousand-node clusters in about 2 minutes and back, while choosing the optimal instance types for your workflow. Access cheaper instance types like ARM, the newest GPUs, or fast SSD storage, no pre-configuration or permissioning needed.

Our Design.
We use raw cloud APIs to ask for instances on the fly. This differs from Kubernetes-based approaches which require pre-set node pools with autoscaling. Coiled uses EC2 instance fleets to provide a much richer interface to cloud resources with infinite configurability.Naive approaches can be tricky due to rate limits, quota limits for instance types, instance unavailability, and more cloud issues that arise when managing 10,000 instances. Our system tracks and resolves these issues. After providing your VMs of choice, we can download containers or dynamically install packages and set up native cloud networking.

Flexible VM Architecture

Networking

What is it?
Connect Dask workers to each other, to users, and to your data cheaply and securely.

Why it matters.
Things need to talk to each other, and Coiled makes sensible network decisions for you so you can avoid costly networking charges due to novice configuration. Plus, no one likes thinking about VPCs.

Our Design.
We ensure your Dask workers have cheap internet access while still using private routes for secure intra-cluster traffic. We put your cluster in a public subnet and secure it with security groups and route tables, rather than using costly NAT Gateways.Communication between the client and scheduler is secured by mTLS, which provides both authentication and encryption. This means by default, we don’t restrict network access to the scheduler, so you can connect from anywhere on the internet.If you have stricter security requirements, this is highly configurable. We make it easy for Coiled to use your existing infrastructure, assuming that infrastructure allows workers to speak to the public internet.

Level up your Dask using Coiled

Coiled makes it easy to scale Dask maturely in the cloud