Effective Testing for Live Applications

当应用程序在实时生产环境中遇到问题时,通常很难:
重复一个有问题的场景,而不冒使事情变得更糟的风险
查找导致问题的查询或操作序列
准备一个数据集与支持团队共享,该数据集可用于在他们的网站上调查您的问题。
同时,在不了解故障原因的情况下,解决故障是不可能的。这就是为什么有必要清楚地了解导致问题的步骤。您需要有一个可重复的测试用例,或者至少清楚地了解应用程序何时遇到错误。
在这次网络研讨会上,我将:
指导您完成有助于确定问题的一般步骤
涵盖在每个步骤中最有效的测试方法
讨论如何最小化测试用例,以便更好地准备向您的支持团队提供信息(或者只是在您的测试服务器上方便使用它)。

展开查看详情

1.Effective Testing for Live Applications March, 29, 2018 Sveta Smirnova

2.Table of Contents •Sometimes You Have to Test on Production •Wrong Data SELECT Returns Nonsense Wrong Data in the Database •Performance Issues •Crashes 2

3.MySQL Troubleshooting Webinars • I taught to • Repeat problematic queries • Crash servers 3

4.MySQL Troubleshooting Webinars • I taught to • Repeat problematic queries • Crash servers • Not what you want to do in production 3

5.Not Acceptable • Updating wrong data • Intended performance slowdowns • Crashes 4

6.Sandboxes • Slower machines • Smaller disks • Blocking backups 5

7.Concurrency • Hard to imitate • Load testing tools are not perfect 6

8.Sometimes You Have to Test on Production

9.Safety First • Data must stay consistent • No intended crashes • Tests should not make performance worse 8

10.We will Discuss • Wrong data • SELECT • UPDATE/INSERT/DELETE • DDL 9

11.We will Discuss • Wrong data • Performance • Slow queries • Locking issues • Hardware-related 9

12.We will Discuss • Wrong data • Performance • Crashes 9

13.General Workflow • Measure: record what is wrong • Actual and expected results • Query execution time • Overall application performance 10

14.General Workflow • Measure: record what is wrong • Actual and expected results • Query execution time • Overall application performance • Plan changes 10

15.General Workflow • Measure: record what is wrong • Actual and expected results • Query execution time • Overall application performance • Plan changes • Implement them • May require maintenance window 10

16.General Workflow • Measure: record what is wrong • Actual and expected results • Query execution time • Overall application performance • Plan changes • Implement them • May require maintenance window • Measure again 10

17.Challenges • We need to know what causes bad behavior 11

18.Challenges • We need to know what causes bad behavior • It is hard to find out 11

19.Challenges • We need to know what causes bad behavior • It is hard to find out • Issue can have many potential solutions • You can find one only after many iterations 11

20.Challenges • We need to know what causes bad behavior • It is hard to find out • Issue can have many potential solutions • You can find one only after many iterations • You don’t want to cause worse damage 11

21.Wrong Data

22.Wrong Data SELECT Returns Nonsense

23.There is No Harm in Experimenting • This is just a query 14

24.There is No Harm in Experimenting • This is just a query • Does not change anything 14

25.There is No Harm in Experimenting • This is just a query • Does not change anything • Works fast 14

26.There is No Harm in Experimenting • This is just a query • Does not change anything • Works fast • No effect on others 14

27.Just Run It • Use all troubleshooting techniques you know 15

28.Just Run It • Use all troubleshooting techniques you know • Which we discussed • Troubleshooting Slow Queries 15

29.Just Run It • Use all troubleshooting techniques you know • Which we discussed • Troubleshooting Slow Queries • And not 15