TestCase QML Type

Represents a unit test case. More...

Import Statement:	import QtTest 1.15
Since:	Qt 4.8
Inherits:	Item

• List of all members, including inherited members
• Obsolete members

Properties

• completed : bool
• name : string
• optional : bool
• running : bool
• when : bool
• windowShown : bool

Methods

• cleanup()
• cleanupTestCase()
• compare(actual, expected, message)
• object createTemporaryObject(Component component, object parent, object properties)
• object createTemporaryQmlObject(string qml, object parent, string filePath)
• expectFail(tag, message)
• expectFailContinue(tag, message)
• fail(message)
• QtObject findChild(parent, objectName)
• fuzzyCompare(actual, expected, delta, message)
• object grabImage(item)
• ignoreWarning(message)
• init()
• initTestCase()
• bool isPolishScheduled(object item)
• keyClick(key, modifiers, delay)
• keyPress(key, modifiers, delay)
• keyRelease(key, modifiers, delay)
• keySequence(keySequence)
• mouseClick(item, x, y, button, modifiers, delay)
• mouseDoubleClickSequence(item, x, y, button, modifiers, delay)
• mouseDrag(item, x, y, dx, dy, button, modifiers, delay)
• mouseMove(item, x, y, delay)
• mousePress(item, x, y, button, modifiers, delay)
• mouseRelease(item, x, y, button, modifiers, delay)
• mouseWheel(item, x, y, xDelta, yDelta, button, modifiers, delay)
• skip(message)
• sleep(ms)
• TouchEventSequence touchEvent(object item)
• tryCompare(obj, property, expected, timeout, message)
• tryVerify(function, timeout, message)
• verify(condition, message)
• wait(ms)
• bool waitForItemPolished(object item, int timeout)
• waitForRendering(item, timeout)
• warn(message)

Detailed Description

Introduction to QML Test Cases

Test cases are written as JavaScript functions within a TestCase type:

import QtQuick 2.0
import QtTest 1.2

TestCase {
    name: "MathTests"

    function test_math() {
        compare(2 + 2, 4, "2 + 2 = 4")
    }

    function test_fail() {
        compare(2 + 2, 5, "2 + 2 = 5")
    }
}

Functions whose names start with "test_" are treated as test cases to be executed. The name property is used to prefix the functions in the output:

********* Start testing of MathTests *********
Config: Using QTest library 4.7.2, Qt 4.7.2
PASS   : MathTests::initTestCase()
FAIL!  : MathTests::test_fail() 2 + 2 = 5
   Actual (): 4
   Expected (): 5
   Loc: [/home/.../tst_math.qml(12)]
PASS   : MathTests::test_math()
PASS   : MathTests::cleanupTestCase()
Totals: 3 passed, 1 failed, 0 skipped
********* Finished testing of MathTests *********

Because of the way JavaScript properties work, the order in which the test functions are found is unpredictable. To assist with predictability, the test framework will sort the functions on ascending order of name. This can help when there are two tests that must be run in order.

Multiple TestCase types can be supplied. The test program will exit once they have all completed. If a test case doesn't need to run (because a precondition has failed), then optional can be set to true.

Data-driven Tests

Table data can be provided to a test using a function name that ends with "_data". Alternatively, the init_data() function can be used to provide default test data for all test functions in a TestCase type:

import QtQuick 2.0
import QtTest 1.2

TestCase {
    name: "DataTests"

    function init_data() {
      return [
           {tag:"init_data_1", a:1, b:2, answer: 3},
           {tag:"init_data_2", a:2, b:4, answer: 6}
      ];
    }

    function test_table_data() {
        return [
            {tag: "2 + 2 = 4", a: 2, b: 2, answer: 4 },
            {tag: "2 + 6 = 8", a: 2, b: 6, answer: 8 },
        ]
    }

    function test_table(data) {
        //data comes from test_table_data
        compare(data.a + data.b, data.answer)
    }

    function test__default_table(data) {
        //data comes from init_data
        compare(data.a + data.b, data.answer)
    }
}

The test framework will iterate over all of the rows in the table and pass each row to the test function. As shown, the columns can be extracted for use in the test. The tag column is special - it is printed by the test framework when a row fails, to help the reader identify which case failed amongst a set of otherwise passing tests.

Benchmarks

Functions whose names start with "benchmark_" will be run multiple times with the Qt benchmark framework, with an average timing value reported for the runs. This is equivalent to using the QBENCHMARK macro in the C++ version of QTestLib.

TestCase {
    id: top
    name: "CreateBenchmark"

    function benchmark_create_component() {
        var component = Qt.createComponent("item.qml")
        var obj = component.createObject(top)
        obj.destroy()
        component.destroy()
    }
}

RESULT : CreateBenchmark::benchmark_create_component:
     0.23 msecs per iteration (total: 60, iterations: 256)
PASS   : CreateBenchmark::benchmark_create_component()

To get the effect of the QBENCHMARK_ONCE macro, prefix the test function name with "benchmark_once_".

Simulating Keyboard and Mouse Events

The keyPress(), keyRelease(), and keyClick() methods can be used to simulate keyboard events within unit tests. The events are delivered to the currently focused QML item. You can pass either a Qt.Key enum value or a latin1 char (string of length one)

Rectangle {
    width: 50; height: 50
    focus: true

    TestCase {
        name: "KeyClick"
        when: windowShown

        function test_key_click() {
            keyClick(Qt.Key_Left)
            keyClick("a")
            ...
        }
    }
}

The mousePress(), mouseRelease(), mouseClick(), mouseDoubleClickSequence() and mouseMove() methods can be used to simulate mouse events in a similar fashion.

Note: keyboard and mouse events can only be delivered once the main window has been shown. Attempts to deliver events before then will fail. Use the when and windowShown properties to track when the main window has been shown.

Managing Dynamically Created Test Objects

A typical pattern with QML tests is to dynamically create an item and then destroy it at the end of the test function:

TestCase {
    id: testCase
    name: "MyTest"
    when: windowShown

    function test_click() {
        var item = Qt.createQmlObject("import QtQuick 2.0; Item {}", testCase);
        verify(item);

        // Test item...

        item.destroy();
    }
}

The problem with this pattern is that any failures in the test function will cause the call to item.destroy() to be skipped, leaving the item hanging around in the scene until the test case has finished. This can result in interference with future tests; for example, by blocking input events or producing unrelated debug output that makes it difficult to follow the code's execution.

By calling createTemporaryQmlObject() instead, the object is guaranteed to be destroyed at the end of the test function:

TestCase {
    id: testCase
    name: "MyTest"
    when: windowShown

    function test_click() {
        var item = createTemporaryQmlObject("import QtQuick 2.0; Item {}", testCase);
        verify(item);

        // Test item...

        // Don't need to worry about destroying "item" here.
    }
}

For objects that are created via the createObject() function of Component, the createTemporaryObject() function can be used.

See also SignalSpy and Qt Quick Test.

Property Documentation

completed : bool

This property will be set to true once the test case has completed execution. Test cases are only executed once. The initial value is false.

See also running and when.

name : string

This property defines the name of the test case for result reporting. The default value is an empty string.

TestCase {
    name: "ButtonTests"
    ...
}

optional : bool

Multiple TestCase types can be supplied in a test application. The application will exit once they have all completed. If a test case does not need to run (because a precondition has failed), then this property can be set to true. The default value is false.

TestCase {
    when: false
    optional: true
    function test_not_run() {
        verify(false)
    }
}

See also when and completed.

running : bool

This property will be set to true while the test case is running. The initial value is false, and the value will become false again once the test case completes.

See also completed and when.

when : bool

This property should be set to true when the application wants the test cases to run. The default value is true. In the following example, a test is run when the user presses the mouse button:

Rectangle {
    id: foo
    width: 640; height: 480
    color: "cyan"

    MouseArea {
        id: area
        anchors.fill: parent
    }

    property bool bar: true

    TestCase {
        name: "ItemTests"
        when: area.pressed
        id: test1

        function test_bar() {
            verify(bar)
        }
    }
}

The test application will exit once all TestCase types have been triggered and have run. The optional property can be used to exclude a TestCase type.

See also optional and completed.

windowShown : bool

This property will be set to true after the QML viewing window has been displayed. Normally test cases run as soon as the test application is loaded and before a window is displayed. If the test case involves visual types and behaviors, then it may need to be delayed until after the window is shown.

Button {
    id: button
    onClicked: text = "Clicked"
    TestCase {
        name: "ClickTest"
        when: windowShown
        function test_click() {
            button.clicked();
            compare(button.text, "Clicked");
        }
    }
}

Method Documentation

cleanup()

This function is called after each test function that is executed in the TestCase type. The default implementation does nothing. The application can provide its own implementation to perform cleanup after each test function.

See also init() and cleanupTestCase().

cleanupTestCase()

This function is called after all other test functions in the TestCase type have completed. The default implementation does nothing. The application can provide its own implementation to perform test case cleanup.

See also initTestCase() and cleanup().

compare(actual, expected, message = "")

Fails the current test case if actual is not the same as expected, and displays the optional message. Similar to QCOMPARE(actual, expected) in C++.

See also tryCompare() and fuzzyCompare.

object createTemporaryObject(Component component, object parent, object properties)

This function dynamically creates a QML object from the given component with the specified optional parent and properties. The returned object will be destroyed (if it was not already) after cleanup() has finished executing, meaning that objects created with this function are guaranteed to be destroyed after each test, regardless of whether or not the tests fail.

If there was an error while creating the object, null will be returned.

This function calls component.createObject() internally.

This method was introduced in Qt 5.9.

See also Managing Dynamically Created Test Objects.

object createTemporaryQmlObject(string qml, object parent, string filePath)

This function dynamically creates a QML object from the given qml string with the specified parent. The returned object will be destroyed (if it was not already) after cleanup() has finished executing, meaning that objects created with this function are guaranteed to be destroyed after each test, regardless of whether or not the tests fail.

If there was an error while creating the object, null will be returned.

If filePath is specified, it will be used for error reporting for the created object.

This function calls Qt.createQmlObject() internally.

This method was introduced in Qt 5.9.

See also Managing Dynamically Created Test Objects.

expectFail(tag, message)

In a data-driven test, marks the row associated with tag as expected to fail. When the fail occurs, display the message, abort the test, and mark the test as passing. Similar to QEXPECT_FAIL(tag, message, Abort) in C++.

If the test is not data-driven, then tag must be set to an empty string.

See also expectFailContinue().

expectFailContinue(tag, message)

In a data-driven test, marks the row associated with tag as expected to fail. When the fail occurs, display the message, and then continue the test. Similar to QEXPECT_FAIL(tag, message, Continue) in C++.

If the test is not data-driven, then tag must be set to an empty string.

See also expectFail().

fail(message = "")

Fails the current test case, with the optional message. Similar to QFAIL(message) in C++.

QtObject findChild(parent, objectName)

Returns the first child of parent with objectName, or null if no such item exists. Both visual and non-visual children are searched recursively, with visual children being searched first.

compare(findChild(item, "childObject"), expectedChildObject);

This method was introduced in Qt 5.4.

fuzzyCompare(actual, expected, delta, message = "")

Fails the current test case if the difference betwen actual and expected is greater than delta, and displays the optional message. Similar to qFuzzyCompare(actual, expected) in C++ but with a required delta value.

This function can also be used for color comparisons if both the actual and expected values can be converted into color values. If any of the differences for RGBA channel values are greater than delta, the test fails.

See also tryCompare() and compare().

object grabImage(item)

Returns a snapshot image object of the given item.

The returned image object has the following properties:

• width Returns the width of the underlying image (since 5.10)
• height Returns the height of the underlying image (since 5.10)
• size Returns the size of the underlying image (since 5.10)

Additionally, the returned image object has the following methods:

• red(x, y) Returns the red channel value of the pixel at x, y position
• green(x, y) Returns the green channel value of the pixel at x, y position
• blue(x, y) Returns the blue channel value of the pixel at x, y position
• alpha(x, y) Returns the alpha channel value of the pixel at x, y position
• pixel(x, y) Returns the color value of the pixel at x, y position
• equals(image) Returns true if this image is identical to image - see QImage::operator== (since 5.6)

  For example:

  var image = grabImage(rect);
  compare(image.red(10, 10), 255);
  compare(image.pixel(20, 20), Qt.rgba(255, 0, 0, 255));
  
  rect.width += 10;
  var newImage = grabImage(rect);
  verify(!newImage.equals(image));

• save(path) Saves the image to the given path. If the image cannot be saved, an exception will be thrown. (since 5.10)

  This can be useful to perform postmortem analysis on failing tests, for example:

  var image = grabImage(rect);
  try {
      compare(image.width, 100);
  } catch (ex) {
      image.save("debug.png");
      throw ex;
  }

ignoreWarning(message)

Marks message as an ignored warning message. When it occurs, the warning will not be printed and the test passes. If the message does not occur, then the test will fail. Similar to QTest::ignoreMessage(QtWarningMsg, message) in C++.

Since Qt 5.12, message can be either a string, or a regular expression providing a pattern of messages to ignore.

For example, the following snippet will ignore a string warning message:

ignoreWarning("Something sort of bad happened")

And the following snippet will ignore a regular expression matching a number of possible warning messages:

ignoreWarning(new RegExp("[0-9]+ bad things happened"))

Note: Despite being a JavaScript RegExp object, it will not be interpreted as such; instead, the pattern will be passed to QRegularExpression.

See also warn().

init()

This function is called before each test function that is executed in the TestCase type. The default implementation does nothing. The application can provide its own implementation to perform initialization before each test function.

See also cleanup() and initTestCase().

initTestCase()

This function is called before any other test functions in the TestCase type. The default implementation does nothing. The application can provide its own implementation to perform test case initialization.

See also cleanupTestCase() and init().

bool isPolishScheduled(object item)

Returns true if updatePolish() has not been called on item since the last call to polish(), otherwise returns false.

When assigning values to properties in QML, any layouting the item must do as a result of the assignment might not take effect immediately, but can instead be postponed until the item is polished. For these cases, you can use this function to ensure that the item has been polished before the execution of the test continues. For example:

verify(isPolishScheduled(item))
verify(waitForItemPolished(item))

Without the call to isPolishScheduled() above, the call to waitForItemPolished() might see that no polish was scheduled and therefore pass instantly, assuming that the item had already been polished. This function makes it obvious why an item wasn't polished and allows tests to fail early under such circumstances.

This method was introduced in Qt 5.13.

See also waitForItemPolished(), QQuickItem::polish(), and QQuickItem::updatePolish().

keyClick(key, modifiers = Qt.NoModifier, delay = -1)

Simulates clicking of key with optional modifiers on the currently focused item. If delay is larger than 0, the test will wait for delay milliseconds.

The event will be sent to the TestCase window or, in case of multiple windows, to the current active window. See QGuiApplication::focusWindow() for more details.

See also keyPress() and keyRelease().

keyPress(key, modifiers = Qt.NoModifier, delay = -1)

Simulates pressing a key with optional modifiers on the currently focused item. If delay is larger than 0, the test will wait for delay milliseconds.

The event will be sent to the TestCase window or, in case of multiple windows, to the current active window. See QGuiApplication::focusWindow() for more details.

Note: At some point you should release the key using keyRelease().

See also keyRelease() and keyClick().

keyRelease(key, modifiers = Qt.NoModifier, delay = -1)

Simulates releasing a key with optional modifiers on the currently focused item. If delay is larger than 0, the test will wait for delay milliseconds.

The event will be sent to the TestCase window or, in case of multiple windows, to the current active window. See QGuiApplication::focusWindow() for more details.

See also keyPress() and keyClick().

keySequence(keySequence)

Simulates typing of keySequence. The key sequence can be set to one of the standard keyboard shortcuts, or it can be described with a string containing a sequence of up to four key presses.

Each event shall be sent to the TestCase window or, in case of multiple windows, to the current active window. See QGuiApplication::focusWindow() for more details.

This method was introduced in Qt 5.10.

See also keyPress(), keyRelease(), GNU Emacs Style Key Sequences, and Shortcut.sequence.

mouseClick(item, x = item.width / 2, y = item.height / 2, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates clicking a mouse button with optional modifiers on an item. The position of the click is defined by x and y. If x and y are not defined the position will be the center of item. If delay is specified, the test will wait for the specified amount of milliseconds before pressing and before releasing the button.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

See also mousePress(), mouseRelease(), mouseDoubleClickSequence(), mouseMove(), mouseDrag(), and mouseWheel().

mouseDoubleClickSequence(item, x = item.width / 2, y = item.height / 2, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates the full sequence of events generated by double-clicking a mouse button with optional modifiers on an item.

This method reproduces the sequence of mouse events generated when a user makes a double click: Press-Release-Press-DoubleClick-Release.

The position of the click is defined by x and y. If x and y are not defined the position will be the center of item. If delay is specified, the test will wait for the specified amount of milliseconds before pressing and before releasing the button.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

This QML method was introduced in Qt 5.5.

See also mousePress(), mouseRelease(), mouseClick(), mouseMove(), mouseDrag(), and mouseWheel().

mouseDrag(item, x, y, dx, dy, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates dragging the mouse on an item with button pressed and optional modifiers The initial drag position is defined by x and y, and drag distance is defined by dx and dy. If delay is specified, the test will wait for the specified amount of milliseconds before releasing the button.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

See also mousePress(), mouseClick(), mouseDoubleClickSequence(), mouseMove(), mouseRelease(), and mouseWheel().

mouseMove(item, x, y, delay = -1)

Moves the mouse pointer to the position given by x and y within item. If a delay (in milliseconds) is given, the test will wait before moving the mouse pointer.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

See also mousePress(), mouseRelease(), mouseClick(), mouseDoubleClickSequence(), mouseDrag(), and mouseWheel().

mousePress(item, x = item.width / 2, y = item.height / 2, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates pressing a mouse button with optional modifiers on an item. The position is defined by x and y. If x or y are not defined the position will be the center of item. If delay is specified, the test will wait for the specified amount of milliseconds before the press.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

See also mouseRelease(), mouseClick(), mouseDoubleClickSequence(), mouseMove(), mouseDrag(), and mouseWheel().

mouseRelease(item, x = item.width / 2, y = item.height / 2, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates releasing a mouse button with optional modifiers on an item. The position of the release is defined by x and y. If x or y are not defined the position will be the center of item. If delay is specified, the test will wait for the specified amount of milliseconds before releasing the button.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

See also mousePress(), mouseClick(), mouseDoubleClickSequence(), mouseMove(), mouseDrag(), and mouseWheel().

mouseWheel(item, x, y, xDelta, yDelta, button = Qt.LeftButton, modifiers = Qt.NoModifier, delay = -1)

Simulates rotating the mouse wheel on an item with button pressed and optional modifiers. The position of the wheel event is defined by x and y. If delay is specified, the test will wait for the specified amount of milliseconds before releasing the button.

The position given by x and y is transformed from the co-ordinate system of item into window co-ordinates and then delivered. If item is obscured by another item, or a child of item occupies that position, then the event will be delivered to the other item instead.

The xDelta and yDelta contain the wheel rotation distance in eighths of a degree. see QWheelEvent::angleDelta() for more details.

See also mousePress(), mouseClick(), mouseDoubleClickSequence(), mouseMove(), mouseRelease(), mouseDrag(), and QWheelEvent::angleDelta().

skip(message = "")

Skips the current test case and prints the optional message. If this is a data-driven test, then only the current row is skipped. Similar to QSKIP(message) in C++.

sleep(ms)

Sleeps for ms milliseconds without processing Qt events.

See also wait() and waitForRendering().

TouchEventSequence touchEvent(object item)

Begins a sequence of touch events through a simulated QTouchDevice::TouchScreen. Events are delivered to the window containing item.

The returned object is used to enumerate events to be delivered through a single QTouchEvent. Touches are delivered to the window containing the TestCase unless otherwise specified.

Rectangle {
    width: 640; height: 480

    MultiPointTouchArea {
        id: area
        anchors.fill: parent

        property bool touched: false

        onPressed: touched = true
    }

    TestCase {
        name: "ItemTests"
        when: windowShown
        id: test1

        function test_touch() {
            var touch = touchEvent(area);
            touch.press(0, area, 10, 10);
            touch.commit();
            verify(area.touched);
        }
    }
}

This method was introduced in Qt 5.9.

See also TouchEventSequence::press(), TouchEventSequence::move(), TouchEventSequence::release(), TouchEventSequence::stationary(), TouchEventSequence::commit(), and QTouchDevice::TouchScreen.

tryCompare(obj, property, expected, timeout = 5000, message = "")

Fails the current test case if the specified property on obj is not the same as expected, and displays the optional message. The test will be retried multiple times until the timeout (in milliseconds) is reached.

This function is intended for testing applications where a property changes value based on asynchronous events. Use compare() for testing synchronous property changes.

tryCompare(img, "status", BorderImage.Ready)
compare(img.width, 120)
compare(img.height, 120)
compare(img.horizontalTileMode, BorderImage.Stretch)
compare(img.verticalTileMode, BorderImage.Stretch)

SignalSpy::wait() provides an alternative method to wait for a signal to be emitted.

See also compare() and SignalSpy::wait().

tryVerify(function, timeout = 5000, message = "")

Fails the current test case if function does not evaluate to true before the specified timeout (in milliseconds) has elapsed. The function is evaluated multiple times until the timeout is reached. An optional message is displayed upon failure.

This function is intended for testing applications where a condition changes based on asynchronous events. Use verify() for testing synchronous condition changes, and tryCompare() for testing asynchronous property changes.

For example, in the code below, it's not possible to use tryCompare(), because the currentItem property might be null for a short period of time:

tryCompare(listView.currentItem, "text", "Hello");

Instead, we can use tryVerify() to first check that currentItem isn't null, and then use a regular compare afterwards:

tryVerify(function(){ return listView.currentItem })
compare(listView.currentItem.text, "Hello")

This method was introduced in Qt 5.8.

See also verify(), compare(), tryCompare(), and SignalSpy::wait().

verify(condition, message = "")

Fails the current test case if condition is false, and displays the optional message. Similar to QVERIFY(condition) or QVERIFY2(condition, message) in C++.

wait(ms)

Waits for ms milliseconds while processing Qt events.

See also sleep() and waitForRendering().

bool waitForItemPolished(object item, int timeout = 5000)

Waits for timeout milliseconds or until updatePolish() has been called on item.

Returns true if updatePolish() was called on item within timeout milliseconds, otherwise returns false.

This method was introduced in Qt 5.13.

See also isPolishScheduled(), QQuickItem::polish(), and QQuickItem::updatePolish().

waitForRendering(item, timeout = 5000)

Waits for timeout milliseconds or until the item is rendered by the renderer. Returns true if item is rendered in timeout milliseconds, otherwise returns false. The default timeout value is 5000.

See also sleep() and wait().

warn(message)

Prints message as a warning message. Similar to QWARN(message) in C++.

See also ignoreWarning().