MySQL Test Framework for Troubleshooting

1 .MySQL Test Framework for Troubleshooting April, 12, 2018 Sveta Smirnova 

2 .Code Examples https://github.com/svetasmirnova/mtr_test 2 

3 .The MySQL Test Framework (MTR) These tools constitute the MySQL test framework that provides a means for verifying that MySQL Server and its client programs operate according to expectations. The test cases consist mostly of SQL statements, but can also use test language constructs that control how to run tests and verify their results. Distributions also provide facilities for running unit tests and creating new unit tests. dev.mysql.com/doc/dev/mysql-server/latest/PAGE_INTRODUCTION.html 3 

4 .Unit Test Framework • Same scenario 4 

5 .Unit Test Framework • Same scenario • Custom options 4 

6 .Unit Test Framework • Same scenario • Custom options • Multiple servers 4 

7 .Support Engineer Investigates • Why customer’s SQL works wrongly 5 

8 .Support Engineer Investigates • Why customer’s SQL works wrongly • Is this true with all versions? 5 

9 .Support Engineer Investigates • Why customer’s SQL works wrongly • Is this true with all versions? • With all branches? 5 

10 .I Often Need to Test • Customer’s server version 6 

11 .I Often Need to Test • Customer’s server version • Same major branch, but latest version 6 

12 .I Often Need to Test • Customer’s server version • Same major branch, but latest version • Latest version 6 

13 .I Often Need to Test • Customer’s server version • Same major branch, but latest version • Latest version • Upstream 6 

14 .I Often Need to Test • Customer’s server version • Same major branch, but latest version • Latest version • Upstream • Other forks (e.g. MariaDB) 6 

15 .I Often Need to Test • Customer’s server version • Same major branch, but latest version • Latest version • Upstream • Other forks (e.g. MariaDB) • Same scenario 6 

16 .Test Manually? Not an Option! 7 

17 .Solutions • MySQL Sandbox • Complicated installations with single command • Scripts to start/stop/restart mysqld • Does NOT enter code for you! 8 

18 .Solutions • MySQL Sandbox • Docker • Runs everything you wish with single command • Not much version variations pre-installed • You need to write Dockerfile if you want Custom setup Custom options Run SQL inside Docker • Does NOT check results! 8 

19 .Solutions • MySQL Sandbox • Docker • MySQL Test Framework • Write once • Execute as many times as you want • Checks result for you 8 

20 .Where to get MTR? • MySQL Server • Standard tarball package mysql-5.7.21-linux-glibc2.12-x86 64.tar.gz • Together with test suite tarball mysql-test-5.7.21-linux-glibc2.12-x86 64.tar.gz 9 

21 .Where to get MTR? • MySQL Server • Standard tarball package mysql-5.7.21-linux-glibc2.12-x86 64.tar.gz • Together with test suite tarball mysql-test-5.7.21-linux-glibc2.12-x86 64.tar.gz • Test suite package for your OS mysql-testsuite 5.7.21-1ubuntu17.10 amd64.deb 9 

22 .Where to get MTR? • MySQL Server • Percona Server • Standard tarball package Percona-Server-5.7.21-20-Linux.x86 64.ssl102.tar.gz 9 

23 .Where to get MTR? • MySQL Server • Percona Server • Standard tarball package Percona-Server-5.7.21-20-Linux.x86 64.ssl102.tar.gz • Test suite package for your OS percona-server-test-5.7 5.7.21-20-1.artful amd64.deb 9 

24 .How to Run Tests • All tests sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr Logging: ./mtr MySQL Version 5.7.19 Checking supported features... - SSL connections supported - binaries are debug compiled Using suites: main,sys_vars,binlog,... Collecting tests... - adding combinations for binlog - adding combinations for rpl - adding combinations for tokudb.rpl - adding combinations for rocksdb.rpl Checking leftover processes... Removing old var directory... Creating var directory ’/home/sveta/build/ps-5.7/mysql-test/var’... Installing system database... Using parallel: 1 10 

25 .How to Run Tests ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13029 gcol.gcol_ndb [ skipped ] ndbcluster disabled ... 10 

26 .How to Run Tests • Single test sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr innodb Logging: ./mtr innodb MySQL Version 5.7.19 Checking supported features... ... ============================================================================== TEST RESULT TIME (ms) or COMMENT -------------------------------------------------------------------------- worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009 innodb.innodb [ pass ] 45802 -------------------------------------------------------------------------- The servers were restarted 0 times Spent 45.802 of 61 seconds executing testcases 10 Completed: All 1 tests were successful. 

27 .How to Run Tests • In suite sveta@delly:~/build/ps-5.7/mysql-test$ ./mtr --suite=rpl rpl_alter 10 

28 .Setting Up sveta@delly:~$ cd ~/build/ps-5.7/mysql-test 11 

29 .Setting Up sveta@delly:~$ cd ~/build/ps-5.7/mysql-test sveta@delly:~/build/ps-5.7/mysql-test$ mkdir suite/mtr_test sveta@delly:~/build/ps-5.7/mysql-test$ mkdir suite/mtr_test/t sveta@delly:~/build/ps-5.7/mysql-test$ mkdir suite/mtr_test/r 11