Nextdoor’s Plan for Catching New Release Troubles Early on

2 years ago thenewstack.io

Summary: This is a summary of an article originally published by The New Stack. Read the full original article here →

Hyperlocal social networking service https://about.nextdoor.com/ uses old-school statistics to isolate problems early on with new mobile app updates. To keep up with regular weekly releases, Nextdoor built an App Release Anomaly Detection tool that stops client-side regressions before they have a severe impact on the application.

Nextdoor started using “difference-in-differences” analysis to identify app session decline 10 days earlier than week-over-week figures.

Out-of-the-box methods fall short in early phases for two reasons — early data is very specific to the early users and very small.

Nextdoor can flag an app sessions decline 10 days earlier than the previous, diagnose the decline to a specific release, and isolate the regression to less than 1% of users.

DevOps Articles