W3cubDocs

/MariaDB

mysql-test-run.pl Options

Syntax

./mysql-test-run.pl [ OPTIONS ] [ TESTCASE ]

Where the test case can be specified as: testcase[.test] Runs the test case named 'testcase' from all suits

path-to-testcase
[suite.]testcase[,combination]

Examples

alias main.alias 'main' is the name of the suite for the 't' directory. 

rpl.rpl_invoked_features,mix,xtradb_plugin
suite/rpl/t/rpl.rpl_invoked_features

Options

Options to Control What Engine/Variation to Run

Option Description
--embedded-server Use the embedded server, i.e. no mysqld daemons.
--ps-protocol Use the binary protocol between client and server.
--cursor-protocol Use the cursor protocol between client and server (implies --ps-protocol).
--view-protocol Create a view to execute all non updating queries.
--sp-protocol Create a stored procedure to execute all queries.
--compress Use the compressed protocol between client and server if both support it.
--ssl If mysql-test-run.pl is started with the --ssl option, it sets up a secure connection for all test cases. In this case, if mysqld does not support TLS, mysql-test-run.pl exits with an error message: Couldn´t find support for SSL.
--skip-ssl Dont start server with support for TLS connections.
--vs-config Visual Studio configuration used to create executables (default: MTR_VS_CONFIG environment variable).
--parallel=num How many parallel tests should be run. Default is 1, use --parallel=auto for auto-setting of num.
--defaults-file=<config template> Use fixed config template for all tests.
--defaults_extra_file=<config template> Extra config template to add to all generated configs.
--combination=<opt> Extra options to pass to mysqld. The value should consist of one or more comma-separated mysqld options. This option is similar to --mysqld but should be given two or more times. mysql-test-run.pl executes multiple test runs, using the options for each instance of --combination in successive runs. If --combination is given only once, it has no effect. For test runs specific to a given test suite, an alternative to the use of --combination is to create a combinations file in the suite directory. The file should contain a section of options for each test run.
--dry-run Don't run any tests, print the list of tests that were selected for execution.

Options to Control Directories to Use

Option Description
--tmpdir=DIR The directory where temporary files are stored (default: ./var/tmp). The environment variable MYSQL_TMP_DIR will be set to the path for this directory, whether it has the default value or has been set explicitly. This may be referred to in tests.
--vardir=DIR The directory where files generated from the test run is stored (default: ./var). Specifying a ramdisk or tmpfs will speed up tests. The environment variable MYSQLTEST_VARDIR will be set to the path for this directory, whether it has the default value or has been set explicitly. This may be referred to in tests.
--mem Run testsuite in "memory" using tmpfs or ramdisk. This can decrease test times significantly, in particular if you would otherwise be running over a remote file system. Attempts to find a suitable location using a builtin list of standard locations for tmpfs (/dev/shm). The option can also be set using environment variable MTR_MEM=[DIR]. If DIR is given, it is added to the beginning of the list of locations to search, so it takes precedence over any built-in locations. Once you have run tests with --mem within a mysql-testdirectory, a soflink var will have been set up to the temporary directory, and this will be re-used the next time, until the soflink is deleted. Thus, you do not have to repeat the --mem option next time.
--client-bindir=PATH Path to the directory where client binaries are located.
--client-libdir=PATH Path to the directory where client libraries are located.

Options to Control What Test Suites or Cases to Run

Option Description
--force Normally, mysql-test-run.pl exits if a test case fails. --force causes execution to continue regardless of test case failure.
--with-ndbcluster-only Run only tests that include "ndb" in the filename.
--skip-ndb[cluster] Skip all tests that need cluster. Default.
--do-test=PREFIX or REGEX Run test cases with names prefixed with PREFIX or which fulfil the REGEX. For example, --do-test=testa matches tests that begin with testa, --do-test=main.testa matches tests in the main test suite that begin with testa, and --do-test=main.*testa matches test names that contain main followed by testa with anything in between. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as xmainytestz.
--skip-test=PREFIX or REGEX Skip test cases with names prefixed with PREFIX or which fulfil the REGEX. See -do-test for examples.
--start-from=PREFIX Sorts the list of names of the test cases to be run, and then starts with the test prefixed with PREFIX, where the prefix may be suite.testname or just testname.
--suite[s]=NAME1,..,NAMEN Comma separated list of suite names to run. The default, as of MariaDB 10.4.5, is:
"main-, archive-, binlog-, binlog_encryption-, csv-, compat/oracle-, encryption-, federated-, funcs_1-, funcs_2-, gcol-, handler-, heap-, innodb-, innodb_fts-, innodb_gis-, innodb_zip-, json-, maria-, mariabackup-, multi_source-, optimizer_unfixed_bugs-, parts-, perfschema-, plugins-, roles-, rpl-, sys_vars-, sql_sequence-, unit-, vcol-, versioning-,period-".
--skip-rpl Skip the replication test cases.
--big-test Allow tests marked as "big" to run. Tests can be thus marked by including the line --source include/big_test.inc, and they will only be run if this option is given, or if the environment variable BIG_TEST is set to 1. Repeat this option twice to run only "big" tests. This is typically used for tests that take a very long to run, or that use many resources, so that they are not suitable for running as part of a normal test suite run
--staging-run Run a limited number of tests (no slow tests). Used for running staging trees with valgrind.
--enable-disabled Ignore any disabled.def file, and also run tests marked as disabled. Success or failure of those tests will be reported the same way as other tests.
--print-testcases Don't run the tests but print details about all the selected tests, in the order they would be run.
--skip-test-list=FILE Skip the tests listed in FILE. Each line in the file is an entry and should be formatted as: <TESTNAME> : <COMMENT>

Options That Specify Ports

Option Description
--[mtr-]port-base=num Base for port numbers. Ports from this number to number+9 are reserved. Should be divisible by 10; if not it will be rounded down. May be set with environment variable MTR_PORT_BASE. If this value is set and is not "auto", it overrides build-thread.
--[mtr-]build-thread=num Specify unique number to calculate port number(s) from. Can be set in environment variable MTR_BUILD_THREAD. Set MTR_BUILD_THREAD="auto" to automatically acquire a build thread id that is unique to current host. The more logical --port-base is supported as an alternative.

Options For Test Case Authoring

Option Description
--record TESTNAME (Re)generate the result file for TESTNAME.
--check-testcases Check testcases for side-effects. This is done by checking system state before and after each test case; if there is any difference, a warning to that effect will be written, but the test case will not be marked as failed because of it. This check is enabled by default. Use --nocheck-testcases to disable.
mark-progress Log line number and elapsed time to <testname>.progress

Options That Pass On Options

Option Description
--mysqld=ARGS Specify additional arguments to "mysqld"
--mysqltest=ARGS Specify additional arguments to "mysqltest". Use additional --mysqld-env options to set more than one variable.

Options to Run Test On Running Server

Option Description
extern option=value Use an already running server. The option/value pair is what is needed by the mysql client to connect to the server. Each --extern option can only take one option/value pair as an argument, so you need to repeat --extern for each pair needed. Example: ./mysql-test-run.pl --extern socket=var/tmp/mysqld.1.sock alias. Note: If a test case has an .opt file that requires the server to be restarted with specific options, the file will not be used. The test case likely will fail as a result.

Options For Debugging the Product

Option Description
--boot-dbx Run the mysqld server used for bootstrapping the database through the dbx debugger.
--boot-ddd Run the mysqld server used for bootstrapping the database through the ddd debugger.
--boot-gdb Run the mysqld server used for bootstrapping the database through the gdb debugger.
--client-dbx Start mysqltest client in the dbx debugger.
--client-ddd Start mysqltest client in the ddd debugger.
--client-debugger=NAME Start mysqltest in the selected debugger.
--client-gdb Start mysqltest client in the gdb debugger.
--dbx Start the mysqld(s) in the dbx debugger.
--ddd Start the mysqld(s) in the ddd debugger.
--debug Dump trace output for all servers and client programs.
--debug-common Same as --debug, but sets the 'd' debug flags to "query,info,error,enter,exit"
--debug-server Use debug version of server, but without turning on tracing.
--debugger=NAME Start mysqld in the selected debugger.
--gdb[=COMMANDS] Start the mysqld(s) in the gdb debugger. From MariaDB 10.3.6, optionally execute specified commands (semicolon-separated) in gdb.
--manual-debug Let user manually start mysqld in debugger, before running test(s).
--manual-dbx Let user manually start mysqld in the dbx debugger, before running test(s).
--manual-ddd Let user manually start mysqld in the ddd debugger, before running test(s).
--manual-gdb Let user manually start mysqld in the gdb debugger, before running test(s).
--manual-lldb Let user manually start mysqld in the lldb debugger, before running test(s).
--max-save-core Limit the number of core files saved (to avoid filling up disks for heavily crashing server). Defaults to 5, set to 0 for no limit. Set its default with MTR_MAX_SAVE_CORE.
--max-save-datadir Limit the number of datadir saved (to avoid filling up disks for heavily crashing server). Defaults to 20, set to 0 for no limit. Set its default with MTR_MAX_SAVE_DATDIR.
--max-test-fail Limit the number of test failurs before aborting the current test run. Defaults to 10, set to 0 for no limit. Set its default with MTR_MAX_TEST_FAIL.

Options For Valgrind

To get less warnings when running the test system with Valgrind, you should consider compiling MariaDB with the BUILD/compile-pentium64-valgrind-max script. This will initialize some structures, that normally don't have to be initialized, to remove some false positive warnings from Valgrind.

Option Description
--valgrind Run the "mysqltest" and "mysqld" executables using valgrind with default options. This and the following valgrind options require that the executables have been built with valgrind support.
--valgrind-all Synonym for --valgrind.
--valgrind-mysqld Run the "mysqld" executable with valgrind.
--valgrind-mysqltest Run the "mysqltest" and "mysql_client_test" executables with valgrind.
--valgrind-options=ARGS Deprecated, use --valgrind-option.
--valgrind-option=ARGS Option to give valgrind. Replaces default option(s). Can be specified more then once.
--valgrind-path=<EXE> Path to the valgrind executable.
--callgrind Instruct valgrind to use callgrind.

Options For Strace

Option Description
--strace Run the "mysqld" executables using strace. Default options are -f -o var/log/'mysqld-name'.strace.
--strace-option=ARGS Option to give strace, replaces default option(s).
--strace-client=[path] Create strace output for mysqltest client, optionally specifying name and path to the trace program to use. Example: ./mysql-test-run.pl --strace-client=ktrace.

Misc Options

Option Description
--user=USER User for connecting to mysqld (default: root)
--comment=STR Write STR to the output within lines filled with #, as a form of banner.
--timer Show test case execution time. Use no-timer to disable.
--verbose More verbose output(use multiple times for even more)
--verbose-restart Write when and why servers are restarted between test cases.
--start Only initialize and start the servers, using the startup settings for the first specified test case Example: ./mysql-test-run.pl --start alias & start-dirty Only start the servers (without initialization) for the first specified test case
--start-and-exit Same as --start, but mysql-test-run terminates and leaves just the server running.
--start-dirty This is similar to --start, but will skip the database initialization phase and assume that database files are already available. Usually this means you must have run another test first.
--user-args In combination with start* and no test name, drops arguments to mysqld except those specified with --mysqld (if any).
--wait-all If --start or --start-dirty option is used, wait for all servers to exit before finishing the process. Otherwise, it will terminate if one (of several) servers is restarted.
--fast Do not perform controlled shutdown when servers need to be restarted or at the end of the test run. This is equivalent to using --shutdown-timeout=0.
--force-restart Always restart servers between tests.
--parallel=N Run tests in N parallel threads (default 1) Use parallel=auto for auto-setting of N.
--repeat=N Run each test N number of times.
--retry=N If a test fails, it is retried up to a maximum of N runs (default 1). Retries are also limited by the maximum number of failures before stopping, set with the --retry-failure option. This option has no effect unless --force is also used; without it, test execution will terminate after the first failure. The --retry and --retry-failure options do not affect how many times a test repeated with --repeat may fail in total, as each repetition is considered a new test case, which may in turn be retried if it fails.
--retry-failure=N When using the --retry option to retry failed tests, stop when N failures have occured (default 2). Setting it to 0 or 1 effectively turns off retries.
--reorder Reorder tests to get fewer server restarts. This is the default behavior. There is no guarantee that a particular set of tests will always end up in the same order. Use -no-reorder to disable.
--help Display help text.
--testcase-timeout=MINUTES Max test case run time in minutes (default 15).
--suite-timeout=MINUTES Max test suite run time in minutes (default 360).
--shutdown-timeout=SECONDS Max number of seconds to wait for server shutdown before killing servers (default 10).
--warnings Scan the log files for warnings and report any suspicious ones; if any are found, the test will be marked as failed. Use --nowarnings to turn off.
--stop-file=file If this file is detected, mysqltest will not start new tests until the file is removed (also MTR_STOP_FILE environment variable).
--stop-keep-alive=sec Works with --stop-file, print messages every sec seconds when mysqltest is waiting to remove the file (for buildbot) (also MTR_STOP_KEEP_ALIVE environment variable).
--sleep=SECONDS Passed to mysqltest; will be used as fixed sleep time.
--debug-sync-timeout=NUM Set default timeout for WAIT_FOR debug sync actions. Disable facility with NUM=0.
--gcov Collect coverage information after the test. The result is a dgcov file per source and header file and a last_changes.dgcov file in the vardir with the coverage for the uncommitted changes if any (or the last commit).
--gprof Collect profiling information using the gprof profiling tool.
--experimental=<file> Specify a file that contains a list of test cases that should be displayed with the [ exp-fail ] code rather than [ fail ] if they fail. For an example of a file that might be specified via this option, see mysql-test/collections/default.experimental.
--report-features First run a "test" that reports MariaDB features, displaying the output of SHOW ENGINES and SHOW VARIABLES. This can be used to verify that binaries are built with all required features.
--timestamp Print timestamp before each test report line, showing when the test ended.
--timediff Used with --timestamp, also print time passed since the previous test started.
--max-connections=N Maximum number of simultaneous server connections that may be used per test. Default is 128. Minimum is 8, maximum is 5120. Corresponds to the same option for mysqltest.
--default-myisam Set default storage engine to MyISAM for non-innodb tests. This is needed after switching default storage engine to InnoDB.
--report-times Report how much time has been spent on different phases of test execution.
--stress=ARGS Run stress test, providing options to mysql-stress-test.pl. Options are separated by comma.
Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.

© 2019 MariaDB
Licensed under the Creative Commons Attribution 3.0 Unported License and the GNU Free Documentation License.
https://mariadb.com/kb/en/mysql-test-runpl-options/