MLOps Needs a Better Way to Manage GPUs

2 years ago thenewstack.io

Summary: This is a summary of an article originally published by The New Stack. Read the full original article here →

GPUs are a necessity for deep learning and other large-scale forms of machine learning, yet we don’t yet have the tools to manage them effectively as we can with regular CPUs. Two https://www.run.ai/ software engineers — https://www.linkedin.com/in/natasha-romm-ba4707149/?originalSubdomain=il and http://www.razrotenberg.com/about/ (Software Team Lead) — have been investigating ways to improve GPU utilization.

Today, GPUs are allocated statically, and with not much nuance, usually by user or AI workload.

Genv was created as a way to introduce AI users to the idea of better managing GPUs.

The Run.AI platform also offers management features to manage AI workloads by creating projects and departments, as well as managing users and enforcing more sophisticated quotas.

DevOps Articles