Designing distributed applications with streams

1 . DRIVETRIBE ENGINEERING PANTA RHEI: DESIGNING DISTRIBUTED APPLICATIONS WITH STREAMS 

2 .DRIVETRIBE 2 DRIVETRIBE ▸ The world biggest motoring community. ▸ A social platform for petrolheads. ▸ By Clarkson, Hammond and May. 

3 .DRIVETRIBE 3 DRIVETRIBE ▸ A content destination at the core. ▸ Users consume feeds of content: images, videos, long-form articles. ▸ Content is organised in homogenous categories called “tribes”. ▸ Different users have different interests and the tribe model allows to mix and match at will. 

4 .DRIVETRIBE 4 DRIVETRIBE ARTICLE ▸ Single article by Richard Hammond. ▸ Contains a plethora of content and engagement information. ▸ What kind of data do we need in order to compute an aggregation like this? 

5 .DRIVETRIBE 5 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) 

6 .DRIVETRIBE 6 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) 

7 .DRIVETRIBE 7 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) 

8 .DRIVETRIBE 8 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) ▸ countViews(id: Id[Article]) 

9 .DRIVETRIBE 9 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) ▸ countViews(id: Id[Article]) ▸ countComments(id: Id[Article]) 

10 .DRIVETRIBE 10 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) ▸ countViews(id: Id[Article]) ▸ countComments(id: Id[Article]) ▸ countBumps(id: Id[Article]) 

11 .DRIVETRIBE 11 DRIVETRIBE FEED OF ARTICLES ▸ rankArticles(forUserId).flatMap { a => … } ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) ▸ countViews(id: Id[Article]) ▸… 

12 .DRIVETRIBE 12 QUINTESSENTIAL PREREQUISITES ▸ Scalable. Jeremy Clarkson has 7.2M Twitter followers. Cannot really hack it and worry about it later. ▸ Performant. Low latency is key and mobile networks add quite a bit of it. ▸ Flexible. Almost nobody gets it right the first time around. The ability to iterate is paramount. ▸ Maintainable. Spaghetti code works like interest on debt. 

13 .DRIVETRIBE 13 THREE TIER APPROACH ▸ Clients interact with a fleet of stateless servers (aka “API” servers or “Backend”) via HTTP (which is stateless). ▸ Global shared mutable state (aka the Database). ▸ Starting simple: Store data in a DB. ▸ Starting simple: Compute the aggregated views on the fly. 

14 .DRIVETRIBE 14 DRIVETRIBE ARTICLE ▸ getUser(id: Id[User]) ▸ getTribe(id: Id[Tribe]) ▸ getArticle(id: Id[Article]) ▸ countComments(id: Id[Article]) ▸ countBumps(id: Id[Article]) ▸ countViews(id: Id[Article]) 

15 .DRIVETRIBE 15 DRIVETRIBE FEED OF ARTICLES ▸ (6 queries per Item) x (Y items per page) ▸ Cost of ranking and personalisation. ▸ Quite some work at read time. ▸ Slow. Not really Performant. ▸ Also very expensive to scale. 

16 .DRIVETRIBE 16 WRITE TIME AGGREGATION ▸ Compute the aggregation at write time. ▸ Then a single query can fetch all the views at once. That scales. 

17 .DRIVETRIBE 17 WRITE TIME AGGREGATION ▸ Compute the aggregation at write time. ▸ Then a single query can fetch all the views at once. That scales. 

18 .DRIVETRIBE 18 WRITE TIME AGGREGATION ▸ Compute the aggregation at write time. ▸ Then a single query can fetch all the views at once. That scales. 

19 .DRIVETRIBE 19 WRITE TIME AGGREGATION ▸ Compute the aggregation at write time. ▸ Then a single query can fetch all the views at once. That scales. 

20 .DRIVETRIBE 20 WRITE TIME AGGREGATION ▸ Compute the aggregation at write time. ▸ Then a single query can fetch all the views at once. That scales. 

21 .DRIVETRIBE 21 WRITE TIME AGGREGATION - EVOLUTION ▸ sendNotification 

22 .DRIVETRIBE 22 WRITE TIME AGGREGATION - EVOLUTION ▸ sendNotification ▸ updateUserStats 

23 .DRIVETRIBE 23 WRITE TIME AGGREGATION - EVOLUTION ▸ sendNotification ▸ updateUserStats. ▸ What if we have a cache? ▸ Or a different database for search? 

24 .DRIVETRIBE 24 WRITE TIME AGGREGATION - FAN OUT ▸ Requests may need to fan out and update multiple views. ▸ Eg: Authors can update their names. ▸ This needs to be propagated to all materialised views. 

25 .DRIVETRIBE 25 WRITE TIME AGGREGATION ▸ A simple user action is triggering an unbound sequence of side effects. ▸ Most of which need network IO. ▸ Many of which can fail. 

26 .DRIVETRIBE 26 ATOMICITY Atomicity? ▸ What happens if one of them fails? What happens if the server fails in the middle? ▸ Inconsistent. 

27 .DRIVETRIBE 27 CONCURRENCY Concurrency ▸ Concurrent mutations on a global shared state entail race conditions. ▸ State mutations are destructive and can not be (easily) undone. ▸ A bug can corrupt the data permanently. 

28 .DRIVETRIBE 28 ITALIAN PASTA Extensibility ▸ Model evolution becomes difficult. Reads and writes are tightly coupled. ▸ Migrations are scary. ▸ This is neither Extensible nor Maintainable. 

29 .DRIVETRIBE 29 DIFFERENT APPROACH ▸ Let’s take a step back and try to decouple things. ▸ Clients send events to the API: “John liked Jeremy’s post”, “Maria updated her profile” ▸ Events are immutable. They capture a user action at some point in time. ▸ Every application state instance can be modelled as a projection of those events.