Finding Bad Acorns

1 . ANDREW GAO FINDING BAD & ACORNS JEFF SHARPE FLINK FORWARD 2018 

2 .ANDREW GAO JEFF SHARPE 

3 .Our journey to build a Fraud Decisioning Platform and use Flink to build out the use cases Developing a Fraud Fraud Defense at the Defense Platform Teller Using Flink 

4 .DEVELOPING A FRAUD DEFENSE PLATFORM 

5 . OUR USERS Fraud Customer Operator Data Data Scientist Analyst Product Engineer Owner 

6 .

7 . OUR USERS Fraud Customer Operator Data Data Scientist Analyst Product Engineer Owner 

8 . ARCHITECTURE DATA ACTIONS MAGIC! 

9 .RUNNING ON 

10 .

11 .RUNNING ON 

12 .Developing on Kubernetes has been challenging but very rewarding PROS CONS • Community support for • Maintaining your own Docker/Kube Kubernetes solution • Resilient • Containing blast radius • Easy to tear down and bring • Edge cases when combining # back of technology solutions • Maximizing resource efficiency 

13 .

14 .FRAUD DEFENSE AT THE TELLER 

15 .

16 . A FLINK MONOLITH • Problem: Develop a stream processing workflow for two legacy batch data sources • First Attempt: Do everything in Flink and take advantage of Flink Connected Streams 

17 .Using Flink operators to build our application workflow 2 3 4 1 

18 .AWS Lambda is a good fit for our use case and works well with our underlying technologies PROS CONS • Cheap • Not truly stateless • Start-up time • Not a lot of Code/Config • Scalability / Availability • Deployments are a breeze 

19 .Using Flink operators to build our application workflow 2 3 4 1 

20 .CUSTOM WINDOWS FOR OPTIMIZATION AND PORTABILITY 90 Day Storage Window 30 Day Virtual View 90 Day Filtered View 

21 . CUSTOM WINDOWS FOR OPTIMIZATION AND PORTABILITY Most-Recent-Beyond-24-Hours Window 24 Hour Offset Dynamic Window 

22 .Using Flink operators to build our application workflow 2 3 4 1 

23 . USING JYTHON TO BRIDGE THE GAP TO DATA SCIENTISTS Windows Data Flink Data Featur Featur Featur Featur Jython Adapter e e e e Featur Featur Featur Featur e e e e .py .py .py .py .py .py .py .py 

24 . GITFLOW AND JYTHON IMPROVE TRACEABILITY Featur Maven Develop Denied e JAR Import Build v1.0.42 Failed Junit Junit Tests Flink Tests Merge Job JAR Pull Build Request Commit 

25 .Using Flink operators to build our application workflow 2 3 4 1 

26 .FEATURES EXIST TO FEED MODELS Model Model Score Feature Feature H20 Tensor Flow Seldon (whatever) 

27 .

28 . BREAKING UP THE MONOLITH • Problem: Back Pressure leading to Delayed Transactions • Solution: Break up the monolith Flink App into small Queryable State Apps 

29 .CHIPMUNKS