DevOps Articles

Curated articles, resources, tips and trends from the DevOps World.

Step Functions Distributed Map – A Serverless Solution for Large-Scale Para

2 years ago aws.amazon.com

Summary: This is a summary of an article originally published by AWS DevOps Blog. Read the full original article here →

https://aws.amazon.com/polly/ I am excited to announce the availability of a distributed map for https://aws.amazon.com/step-functions. This limit makes it challenging to scale data processing workloads to process thousands of items (or even more) in parallel.

The new distributed map state allows you to write Step Functions to coordinate large-scale parallel workloads within your serverless applications.

The new distributed map state can launch up to ten thousand parallel workflows to process data.

Original map state flow New distributed map flow Sub workflows Runs a sub-workflow for each item in an array.

Made with pure grit © 2024 Jetpack Labs Inc. All rights reserved. www.jetpacklabs.com