Welcome to the very first edition of the Coiled Heartbeat, a monthly publication that will bring you an overview of all the latest features, updates, and bug fixes in Coiled.
September was a big month for Coiled with lots of great news to share!
[poll id="2"]
2. Coiled now includes an Analytics dashboard so you can track your (team’s) cluster usage statistics, historical costs, performance reports, and detailed task metrics and timing information for each cluster.
[poll id="3"]
3. Full support for Google Cloud Platform (GCP) is now available for all users, meaning everyone can now configure Coiled to run in their own GCP account and project.
[poll id="4"]
Read more about each of these (and all of the other updates) below!
You can now instruct your Coiled clusters to automatically scale depending on their workloads using the coiled.Cluster.adapt() method. The adapt method allows you to specify a range between the minimum and the maximum number of workers, as shown in the below code example. Coiled will handle scaling the number of workers in the cluster up or down for you based on a number of scaling heuristics. Learn more about adaptive scaling in Coiled documentation.
//import coiled
cluster = coiled.Cluster()
cluster.adapt(minimum=2, maximum=40)//]]>
Coiled now includes an “Analytics” page with an overview of your activity on Coiled, including total compute time, the number of tasks and workers run, and other usage statistics and visualizations. In addition to account-level statistics, you can view detailed activity on a given cluster in your account, including timing data, cost, and profiling information.
Coiled now provides full support for running managed Dask clusters on Google Cloud Platform. You can choose to run clusters within the Coiled multi-tenant GCP environment or in your own GCP account to make use of data access controls, compliance standards, and promotional credits that you already have in place. Read more about setting Coiled up with GCP in our documentation.
We have a bunch of product ideas in our pipeline that we’d love to get your feedback on. Below is a selection, you can see the complete list and leave feedback on our Feedback page:
- Ability to create heterogeneous CPU/GPU clusters and route tasks to appropriate workers
- Streaming integration and use cases with services like Kafka, streamz, etc.
- SQL examples and integration
[poll id="5"]
We’d love your input on any of these features in development...and we’re always looking for people who enjoy testing and breaking things :) If that’s you, please drop us a line in the Coiled Community Slack channel!
- Deprecated the ECS backend for Coiled clusters and migrated all users AWS EC2 VMs. The EC2 VMs provide a better balance of performance, consistency across cloud providers, explicit control over CPU/RAM/GPU resources, ease of use for working with larger instances and GPUs, and scalability. Read more here.
- Expanded the Azure documentation for users to configure Coiled to run in their own accounts (reach out to us if you are interested in trying it out!).
- Improved documentation for Teams to make for better distinction between Accounts and Teams.
- Added the account kwarg to coiled.create_notebook() to be consistent with similar actions in the Coiled API.
- Removed mention of deprecated region kwargs for coiled.Cluster()
- Improved log messaging.
- Updated CUDA drivers on Dask workers to version 11.2.
For a complete list of updates and bug fixes, see the Release Notes page in our docs.
If you have any questions or other feedback, please do reach out to us via our Coiled Community Slack, by writing to support@coiled.io, or by tweeting at us. We’d love to hear from you!
Thanks for reading! And if you’re interested in trying out Coiled Cloud, which provides hosted Dask clusters, docker-less managed software, and one-click deployments, you can do so for free today when you click below.
