申请试用
HOT
登录
注册
 
账号已存在
Deep dive#7 Milvus 2.0 质量保障系统详解
Milvus.io
/
发布于
/
22
人观看

Deep dive是由Milvus社区发起的代码解析系列直播,针对开源数据库 Milvus 整体架构开放式解读,与社区交流与分享 Milvus 最核心的设计理念。

对本期内容感兴趣的小伙伴,想要和讲师实时QA,欢迎大家添加小助手微信:Zilliz-tech 备注”直播“加入讨论群与大家共同交流!

本期分享大纲:

  1. Miluvs 2.0 版本测试流程介绍
  2. 测试模块详解
  3. Milvus 2.0 测试效率提升技术手段
  4. QA
展开查看详情

1 .Deep dive#7 MIlvus2.0 Quality Assurance System Details

2 .About me Wenxing Zhu Zilliz QA Engineer Education: Master, Shanghai University Interests: chaos engineering , deep learning, cloud native Github: https://github.com/zhuwenxing

3 .Agenda • An overview of the testing process for a Milvus release • Detailed Introduction to each test module • Methods to improve test efficiency • QA

4 .An overview of the testing process for a Milvus release

5 .Milvus overview https://milvus.io https://github.com/milvus-io/milvus

6 .What we care Function: Interfaces work as design Deployment: Milvus is able to deploy, reinstall and upgrade with cluster/standalone mode by different ways Performance: The performance of insert, index, search and query Stability: Milvus runs stably under a normal level workload in 5-10 days Reliability: Milvus behaves if some specific error occurs Configuration: Milvus configuration items work as expected Compatibility: Milvus is able to run on different hardware/ software configurations

7 .How dev and QA cooperate

8 .Issues? How we manage them

9 .When to release?

10 .Detailed Introduction to each test module

11 .Issues origin, find issues and fix issues

12 .Efforts of Dev and QA

13 .Efforts of Dev and QA

14 .UT and Codecov

15 .Code review

16 .Test Framework

17 .Function Test Test goal: verify the interfaces work as design • Return expect result when param or operation is normal • Handle errors when param or operation is abnormal and returns a reasonable error message

18 .Testcases TestObjectParams: • Indicates the parameter test of corresponding interface • Tests the target category/method under different parameter inputs TestObjectOperations: • Indicates the function/operation test of corresponding interface. • Tests the target category/method with legit parameter inputs and interaction with other interfaces.

19 .Deployment Test

20 .Why we need chaos test

21 .Why chaos mesh

22 .What chaos we have implemented Chaos type Test scope Test goal Pod kill All pods Pod failure Pods with multiple replicas Verify that Milvus is capable to keep working or recover from an unexpected error with no user data Pods belongs to the worker Memory stress lost node Network partition Pods in different layers

23 .How we implement

24 .Benchmark • max numbers of Milvus connections • max qps of insert with/out index built • response time of search with a max qps in different matrix of nq, nlist, nprobe, topK • avg response time of load • avg response time of index build with different index types • algorithm performance test for different index type

25 .Methods to improve test efficiency

26 .SDK Test Framework

27 .Github Action Why GitHub action •Deep integration with Github •Uniformly configured machine environment (and pre-installed common software development tools: docker, docker-compose) •Support multiple operating systems and versions (ubuntu, mac, window-server) •Rich Actions marketplace •Matrix supports concurrent jobs, reusing the same test flow, but concurrency can improve efficiency Where are they used? • The process is the same, only some configurations need to be changed • Deploy test and chaos test

28 .Github Action Main plug-ins used •actions/setup-python: Configure Python version and virtual environment dependencies •helm/kind-action: Quickly create a local K8S cluster •actions/upload-artifact: Upload artifacts, mainly used to export log files during testing •dawidd6/action-send-mail: Send email alert after job fails •nick-invision/retry: To avoid accidental network anomalies

29 .Benchmark test

0 点赞
0 收藏
0下载
确认
3秒后跳转登录页面
去登陆