Golang in GrabFood Discovery System

1 . GOLANG IN GRABFOOD DISCOVERY SYSTEM 温同鑫 1 

2 .AGENDA 1. System Preview 2. Grab-kit 3. Chimera ML pipeline 4. Personalize Discovery 5. Why golang 

3 .SYSTEM PREVIEW Mobile GateWay API 派单系统 ⽀付系统 { Keyword FoodDiscovery non- Keyword 个 热 优 性 免 专 ⻔ 惠 化 送 业 推 推 推 推 推 荐 荐 荐 荐 荐 

4 .SYSTEM PREVIEW 

5 .SYSTEM PREVIEW Mobile GateWay API 派单系统 ⽀付系统 { Keyword FoodDiscovery non- Keyword Grab-kit 个 热 优 性 免 专 ⻔ 惠 化 送 业 推 推 推 推 推 荐 荐 荐 荐 荐 

6 .GRAB-KIT (pain points) 1. 可维护性 2. 伸缩性 3. 容错性 4. 观察性 

7 .GRAB-KIT - 默认⽀持http/rpc传输协议 - ⾼度⾃动化的代码⾃动⽣成功能 - 丰富的微服务中间件 - 和第三⽅平台的⾼度集成 

8 .GRAB-KIT (scaffolding) // step1: 定义接⼜ // StringService provides operations on strings. import "context" type StringService interface { Uppercase(string) (string, error) } ref: https://gokit.io/examples/stringsvc.html 

9 .GRAB-KIT (scaffolding) // step4： 创建endpoint⼯⼚函数 // step3： 定义接⼜req/resp协议 import ( type uppercaseRequest struct { "context" S string `json:”s"` "github.com/go-kit/kit/endpoint" } ) type uppercaseResponse struct { func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint { V string `json:"v"` return func(_ context.Context, request interface{}) (interface{}, error) { Err string `json:"err,omitempty"` req := request.(uppercaseRequest) } v, err := svc.Uppercase(req.S) if err != nil { return uppercaseResponse{v, err.Error()}, nil } return uppercaseResponse{v, ""}, nil } } 

10 .GRAB-KIT (scaffolding) // step5： 实现http / rpc server func main() { svc := stringService{} uppercaseHandler := httptransport.NewServer( makeUppercaseEndpoint(svc), decodeUppercaseRequest, encodeResponse, ) http.Handle("/uppercase", uppercaseHandler) log.Fatal(http.ListenAndServe(":8080", nil)) } // step6： 实现请求的encode/decode func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) { var request uppercaseRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error { return json.NewEncoder(w).Encode(response) } 

11 .GRAB-KIT (scaffolding) grab-kit create gopher-china . ├── GETTING-STARTED.md ├── client ├── cmd ├── config-ci.json ├── conveyor.jsonnet ├── databases ├── doc.go ├── doc_test.go ├── dto ├── examples ├── gandalftests ├── grabkit.yaml ├── internal ├── mockers ├── out ├── pb ├── server └── stream 

12 .GRAB-KIT (scaffolding) // Step1: 定义接⼝ // Step2: 定义rpc协议 // gopher-china/pb/gopherchina.proto // gopher-china/pb/gopherchina.proto service GopherChina { message UpperCaseRequest { rpc UpperCase(UpperCaseRequest) returns (UpperCaseResponse) { string sentence = 1 [(grab.kit.validate) = "string,min=2,max=32,required"]; option idempotency_level = NO_SIDE_EFFECTS; } option (google.api.http) = { get: "/uppercase" message UpperCaseResponse { }; string sentence = 1; } } } step3: grab-kit gen 

13 .GRAB-KIT (scaffolding) 

14 .GRAB-KIT (middleware) Request Response Request Response 

15 .(middleware — defensive) Defensive: 检测typed nil func returnsError() error {error { func returnsError() var p *MyError if err :== bad(); nil err != nil { if bad() {return err p =} ErrBad } return nil return } p // Will always return a non-nil error. } 

16 .(middleware — Chaos) Chaos: 混沌测试, refer ChaosMonkey / GrabChaoEngineering 

17 .(middleware — Chaos) ⽬前我们⽀持的失败类型： 1. ERROR - FAILS THE REQUEST WITH AN ERROR 2. CPU LOAD - CREATES A LOAD ON THE CPU 3. MEMORY LEAK - CREATES SOME MEMORY WHICH IS NEVER FREED 4. LATENCY - PAUSES THE REQUEST’S EXECUTION FOR A RANDOM AMOUNT OF TIME 5. DISK SPACE - CREATES SOME TEMPORARY FILES ON THE MACHINE 6. GOROUTINE LEAK - CREATES AND LEAKS GOROUTINES 7. PANIC - CREATES A PANIC IN THE REQUEST 8. THROTTLE - CREATES A RATE LIMITER INSIDE THE REQUEST THAT REJECTS LIMIT-EXCEEDING REQUESTS 

18 .(middleware — Chaos) 

19 .(middleware — Resilient) 众多的弹性控件 RateLimit（local/global） Throttling （cpu usage） Cache （redis/memcache） Timeout 

20 .

21 .(middleware — Stats) // config.yaml statsd: host: {{ string .statsdServer }} port: 8xxx appName: hello 

22 .(middleware — Stats) 

23 .(middleware — Stats) 

24 .GRAB-KIT https://engineering.grab.com/ 

25 .SYSTEM PREVIEW FoodDiscovery 候选集合(recall) 过滤(filter) 重排序(rerank) 后排序(post-rank) 

26 .ML PIPELINE Mobile App Event log DataLake Train Pipeline MetaData API Discovery server Model serving 

27 .ML PIPELINE 机器学习pipeline的⼏个痛点： ➤ Complex Models Erode Boundaries：Entanglement, Cascades ➤ Data Dependencies Cost More than Code Dependencies ➤ Feedback Loops ➤ ML system Anti-patterns ➤ Configuration ➤ Team Cooperation Hidden Technical Debt in Machine Learning Systems — NIPS 2015, Google 

28 .ML PIPELINE (chimera) MLOps: Continuous delivery and automation pipelines in machine learning Chimera ML pipeline 把模型从架构管理、代码开发、测试、上线、监控等 各个环节⾃动化起来。同时⽅便⼯程师进⾏持续集成 (CI)、持续训练(CT)、持续交付(CD)。 

29 .ML PIPELINE 1. 持续训练 2. ⾃动实验 3. ⾃动部署 4. 模型监控 Level 1 Level 0