The Dask community is highly distributed with different teams working independently. This is powerful but sometimes makes it hard for people within the community to see everything that is going on. The Dask Heartbeat by Coiled is a bi-weekly publication intended to centralize and broadcast Dask news over the previous two weeks.
If you want something added to this list either send an e-mail at info@coiled.io, or tweet and tag @dask_dev and we’ll try to include it.
We can celebrate Dask’s birthday! Dask was created a little over six years ago.
Last month Dask had a major release, 2020.12.0, which included some significant internal changes. Unsurprisingly, some of these changes had negative effects which Dask maintainers are busy trying to resolve now.
Under a heavy load, the `Worker` can sometimes skip a beat, and lose a key. This is currently being resolved in https://github.com/dask/distributed/pull/4360 by Florian Jetter (Blue Yonder) and Gil Forsyth (Capital One).
This comes from a large change restructuring the `Worker`s task state, which should result in better long term maintenance.
Data ingestion operations like reading Parquet or CSV files can sometimes result in serialization issues. This is being resolved by Rick Zamora (NVIDIA) at https://github.com/dask/dask/pull/7042
With the recent JupyterLab 3.0 release, the infrastructure to load extensions has been heavily modified. This has resulted in a needed refresh of the Dask-JupyterLab extension. Ian Rose (Coiled) is handling this here: https://github.com/dask/dask-labextension/pull/162
Update: this is done!
https://twitter.com/dask_dev/status/1349359797502701569
There has been a lot of activity over the last few months, which is great to see. However, this has also resulted in higher-than-typical churn and we thank you for your patience.
Xarray, a Dask-related project, is publishing its annual user survey. If you are an Xarray user then we encourage you to participate here: https://docs.google.com/forms/d/e/1FAIpQLSfhVUao634zgpWP3BdrMPwzCd3WUqRbZZ4Baq_l2shoMhcIlQ/viewform
Chan Zuckerberg is hiring a bio-imaging scientist to work with Dask on large scale light-sheet microscopy. Learn more here: https://apply.workable.com/czbiohub/j/F87328FDEA/
Markus Schmitt (Data Revenue) added a new story on discovering rare diseases using Dask’s lower-level APIs. You can read more here: https://stories.dask.org/en/latest/datarevenue.html
https://lh5.googleusercontent.com/AbIA3bdMCCKmfbQR-qG14B5c6MzcEeTcbHjvoWBr2Yuu_WfTbdBYm1kPwxC5DdxiIdm6-mvue-EASNucDQMVAF5WUun3BsJ2k_Yyvnc9PteMVH3G40z_3mUGICkIFSmGfoWlFuyt
That’s it. Thanks for reading all.
If you’re interested in taking Coiled Cloud for a spin, you can do so for free today when you click below.
