How to improve scalability for Celery on Kubernetes?

4 years ago medium.com

Summary: This is a summary of an article originally published by the source. Read the full original article here →

This article has been written with Nicolas Bonnotte. We can configure Kubernetes to add more pods automatically according to the current load, a feature called Horizontal Pod Autoscaling (HPA).

The beta version, which includes support for scaling on memory and custom metrics, can be found in autoscaling/v2beta2 and is more interesting to have multiple metrics to scale the application.

As we use Datadog, we have set up their solution of custom metrics server paired with DatadogMetric CRD.

We have several job openings in tech, with offices in Paris, Berlin, Bordeaux, but also the possibility to work remotely from anywhere.

DevOps Articles