Table of Contents
Learn how to test applications which are based on the Qt framework.
Table of Contents
This tutorial will show you how to create, run, and modify tests for an example Qt application. In the process you will learn about Squish's most frequently used features so that by the end of the tutorial you will be able to start writing your own tests for your own applications.
This chapter presents most of the major concepts behind Squish and provides the information you need to get started using Squish for testing your own applications. This tutorial does not discuss all of Squish's features, and those that it does cover are not covered in full detail. After reading this tutorial we recommend reading the User Guide (Chapter 5), and at least skimming the API Reference Manual (Chapter 6) and the Tools Reference Manual (Chapter 7), so that you are familiar with all the features that Squish has to offer, even if you don't need to use them all straight away.
This tutorial is divided into several sections. If you are new to Squish, it is best to read all of them. If you are already using Squish you might want to just skim the tutorial, stopping only to read those sections that cover any new features that you haven't used before—or you could just skip straight to the User Guide (Chapter 5).
Squish comes with an IDE and command line tools. Using the IDE is the easiest and best way to start, but once you build up lots of tests you will want to automate them, (e.g., doing nightly runs of your regression test suite), so it is worth knowing how to use the command line tools since they can be run from batch files or shell scripts.
For this Chapter we will use a simple Address Book application as our
AUT. The application is shipped with Squish in
SQUISHDIR/examples/qt/addressbook. This is a very
basic application that allows users to load an existing address book or
create a new one, add, edit, and remove entries, and save (or save as),
the new or modified addressbook. Despite the application's simplicity,
it has all the key features that most standard applications have: a menu
bar with pull down menus, a toolbar, and a central area—in this
case showing a table. It supports in-place editing and also has a pop-up
modal dialog for adding items. All the ideas and practices that you
learn to test this application can easily be adapted to your own
applications. And naturally, the User Guide (Chapter 5) has many
more examples and shows how to test lots of Qt-specific features
including model/view models and views, as well as all the standard
editing widgets.
The screenshot shows the application in action with a user adding a new name and address.
The Qt addressbook example.
![[Note]](images/note.png) | Using the Examples |
|---|---|
The first time you try running a test for one of the example AUTs you might get a fatal error that begins “Squish couldn't find the AUT to start...”. If this occurs, click the toolbar button, and in the Application Under Test (AUT) section choose the AUT from the combobox if it is available, or click the button and navigate to the AUT's executable via the file open dialog that pops up. (Some versions of Squish will automatically pop up this dialog if no AUT is specified.) This only needs to be done once per example AUT. (This doesn't arise when testing your own AUTs.) |
In the following sections we will create a test suite and then create some tests, but first we will very briefly review some key Squish concepts.
To perform testing, two things are required:
an application to test—known as the Application Under Test (AUT), and
a test script that exercises the AUT.
One fundamental aspect of Squish's approach is that the AUT and the test script that exercises it are always executed in two separate processes. This ensures that even if the AUT crashes, it should not crash Squish. (In such cases the test script will fail gracefully and log an error message.) In addition to insulating Squish and test scripts from AUT crashes, running the AUT and the test script in separate processes brings other benefits. For example, it makes it easier to store the test scripts in a central location, and it also makes it possible to perform remote testing on different machines and platforms. The ability to do remote testing is particularly useful for testing AUTs that run on multiple platforms, and also when testing AUTs that run on embedded devices.
Squish runs a small server (squishserver) that handles the communication between the AUT and the test script. The test script is executed by the squishrunner tool, which in turn connects to the squishserver. The squishserver starts the AUT and injects the Squish hook into it. The hook is a small library that makes the AUT's live running objects accessible and that can communicate with the squishserver. With the hook in place, the squishserver can query AUT objects regarding their state and can execute commands—all on behalf of the squishrunner. And the squishrunner itself requests that the AUT performs whatever actions the test script specifies. All the communication takes place using network sockets which means that everything can be done on a single machine, or the test script can be executed on one machine and the AUT can be tested over the network on another machine.
The following diagram illustrates how the individual Squish tools work together.
From the test engineer's perspective this separation is not noticeable, since all the communication is handled transparently behind the scenes.
Tests can be written and executed using the Squish IDE, in which case the squishserver is started and stopped automatically, and the test results are displayed in the Squish IDE's Test Results view (Section 8.2.18). The following diagram illustrates what happens behind the scenes when the Squish IDE is used.
The Squish tools can also be used from the command line without the Squish IDE—this is useful for those testers who prefer to use their own tools (for example, their favorite editor), and also for performing automatic batch testing (for example, when running regression tests overnight). In these cases, the squishserver must be started manually, and stopped when all the testing is complete (or, if preferred, started and stopped for each test).
For Squish to make it possible for test scripts to be able to query and control an AUT, Squish must be able to access the AUT's internals, and this is made possible by the use of bindings. Bindings are in effect libraries that provide access to the objects—and in turn to the objects' properties and methods—that are available from a GUI toolkit, or from the AUT itself.
There are two sets of bindings that are of interest when developing tests using Squish.
bindings—Squish are provided for most of the built-in classes of Qt (sometimes additional support must be provided when a new class is added to the library). This means that all the standard objects (including the GUI widgets) provided by Qt can be queried and controlled by Squish test scripts.
AUT-specific bindings—it is possible to create bindings that provide access to the AUT's own API for those cases where the toolkit's bindings don't provide sufficient functionality for proper testing. (Note that for Qt-based AUTs Squish automatically creates bindings to the AUTs QObjects—including custom classes; see How to Create and Access Application Bindings (Section 5.27).)
The need to make AUT-specific bindings is rarely needed in practice, but if it is necessary, Squish provides a tool to make the process as simple as possible. The tool, squishidl (Section 7.4.5), is used to instrument the AUT (and any additional components) to generate AUT-specific bindings. The generated bindings library is seamlessly integrated with the standard GUI toolkit bindings and in the same way will automatically be loaded on demand by the Squish test tools.
When Squish creates bindings to AUT classes, for Qt
applications this means that the properties and slots of the AUT's
custom QObjects and Q_GADGETs
are automatically found and made available to test scripts.
In most cases, nothing special needs to be done to make an application testable, since the toolkit's API (e.g., Qt) provides enough functionality to implement and record test scripts. The connection to the squishserver is also established automatically, when the Squish IDE starts the AUT.
| The Squish Directory |
|---|---|
Throughout the manual, we often refer to the |
A test suite is a collection of one or more test cases (tests). Using a test suite is convenient since it makes it easy to share tests scripts and test data between tests.
Here, and throughout the tutorial, we will start by describing how to do things using the IDE, with the information for command line users following.
To begin with start up the Squish IDE, either by clicking or double-clicking the squishide icon, or by launching squishide from the taskbar menu or by executing squishide on the command line—whichever you prefer and that is suitable for the platform you are using. Once Squish starts up you might be greeted with a Welcome Page in case you're starting the squishide for the first time. Click the button in the upper right to dismiss it. Then, the squishide will look similar to the screenshot—but probably slightly different depending on the windowing system, colors, fonts, and theme that you use, and so on.
Once Squish has started click | to pop-up the New Squish Test Suite wizard (Section 8.3.13) shown below.
Enter a name for your test suite and choose the folder where you want
the test suite to be stored. In the screenshot we have called the test
suite suite_py and will put it inside the
addressbook folder. (For your own tests you might
use a more meaningful name such as "suite_addressbook"; we chose
"suite_py" because for the sake of the tutorial we will create several
suites, one for each scripting language that Squish supports.)
Naturally, you can choose whatever name and folder you prefer. Once the
details are complete, click to go on to the
Toolkit (or Scripting Language) page.
If you get this wizard page, click the toolkit your AUT uses. For this example, we must click Qt since we are testing a Qt application. Then click to go to the Scripting Language page.
Choose whichever scripting language you want—the only constraint is that you can only use one scripting language per test suite. (So if you want to use multiple scripting languages, just create multiple test suites, one for each scripting language you want to use.) The functionality offered by Squish is the same for all languages. Having chosen a scripting language, click once more to get to the wizard's last page.
If you are creating a new test suite for an AUT that Squish already
knows about, simply click the combobox to drop-down the list of AUTs and
choose the one you want. If the combobox is empty or your AUT isn't
listed, click the button to the right
of the combobox—this will pop-up a file open dialog from which you
can navigate to SQUISHDIR/examples/qt/addressbook/
[1]
to select your AUT. In the case of Qt programs, the AUT is the
application's executable (e.g., addressbook.exe on
Windows). Once you have chosen the AUT, click
and Squish will create a sub-folder with
the same name as the test suite, and will create a file inside that
folder called suite.conf that contains the test
suite's configuration details. Squish will also register the AUT with
the squishserver. The wizard will then close and Squish's IDE will
look similar to the screenshot below.
We are now ready to start creating tests.
Squish records tests using the scripting language that was specified for the test suite, rather than using a proprietary language. Once a test has been recorded we can run the test and Squish will faithfully repeat all the actions that we performed when recording the test, but without all the pauses that humans are prone to but which computers don't need. It is also possible—and very common—to edit recorded tests, or to copy parts of recorded tests into manually created tests, as we will see later on in the tutorial.
Recordings are made into existing test cases. We begin by
creating a New Script Test Case.
There are two ways we can do this. One way
is to click |. This will pop up the New Squish Test Case wizard (Section 8.3.10)—simply enter the name
for the test case and then click . Another
way is to click the (
) button (to
the right of the Test Cases label in the
Test Suites view); this will create a new test
case with a default name (which you can easily change). Use one of these
methods and give the new test case the name “tst_general”.
Squish automatically creates a sub-folder inside the test suite's
folder with this name and also a test file, for example
test.py. (If we had chosen JavaScript as our
scripting language the file would be called
test.js, and correspondingly for Perl, Ruby, or Tcl.)
| Note |
|---|---|
If you get a sample |
To make the test script file (e.g., test.py) appear
in an Editor view (Section 8.2.6), click—or double-click
depending on the ||
setting—the test case. This selects the Script as the active one
and makes visible its corresponding (
) and (
) buttons.
The checkboxes are used to control which test cases are
run when the (
) toolbar button is clicked; we can also run
a single test case by clicking its () button. If the test case is not
currently active, the button may be invisible until the mouse is hovered over
it.
Initially, the
script's main() logs "Hello World" to the test results. If
we were to create a test manually (as we will do later on in the tutorial), we
must create a main function, and we
should import the same imports at the top.
The name "main" is special to Squish. Tests may contain as many functions
and other code as we like (providing it is legal for the scripting language), but when the
test is executed (i.e., run), Squish always executes the
main function. It is also possible to share commonly used code
between test scripts—this is covered in the User Guide (Chapter 5). (In fact, two other function names are special
to Squish, cleanup and init; see Tester-Created Special Functions (Section 6.1) for details.)
Once the new test case has been created, we are free to write
test code manually, or to record a test. Clicking on the test case's
() replaces the test's code with a new recording.
It is also possible to record snippets and insert them into existing test
cases, covered in Users Guide and not in this tutorial.
| For command-line users |
|---|---|
Creating a new test case from the command line is an easy two-step process: first, create a test case directory; and second, create a test case script with the same elements (imports, main() function) that the IDE does when it creates a hello-world script in that language. |
Before we dive into recording let's briefly review our very simple test scenario:
Open the
MyAddresses.adraddress file.Navigate to the second address and then add a new name and address.
Navigate to the fourth address (that was the third address) and change the surname field.
Navigate to the first address and remove it.
Verify that the first address is now the new one that was added.
We are now ready to record our first test. Click the toolbar button () that's to the right of the
tst_general test case shown in the Test Suites view (Section 8.2.19)'s Test Cases list. This will cause
Squish to run the AUT so that we can interact with it. Once the AUT is
running perform the following actions—and don't worry about how
long it takes since Squish doesn't record idle time:
Click |, and once the file dialog appears, click the
MyAddresses.adrfilename, then click the button.Click the second row, then click |, then in the Add dialog's first line edit type in "Jane". Now click (or tab to) the second line edit and type in "Doe". Continue similarly, to set an email address of "jane.doe@nowhere.com" and a phone number of "555 123 4567". Don't worry about typing mistakes—just backspace delete as normal and fix them. Finally, click the button. There should now be a new second address with the details you typed in.
Click the fourth row's second (surname) column, delete its text and replace it with "Doe". (You can do this simply by overtyping.) Then press Enter to confirm your edit.
Click in the first row, then click |, and then click the button in the message box. At the end of this, your "Jane Doe" entry should be the first row.
Click the toolbar button in the Squish Control Bar Window (Section 8.1.3) (the second button from the left) and select .
This will make the Squish IDE appear. In the Application Objects view expand the
MainWindowobject, then theQTableWidgetobject. Click the “item_0/0”QModelIndexobject to make its properties appear in the Properties view (Section 8.2.12), and then check the text property's checkbox. Now click the “item_0/1”QModelIndexand check its text property. Finally, click the button (at the bottom of the Verification Point Creator view (Section 8.2.22)) to have the forename and surname verifications for the first row inserted into the recorded test script. (See the screenshot below.) Once the verification points are inserted the Squish IDE's window will be hidden again and the Control Bar window and the AUT will be back in view.We've now completed the test, so in the AUT click |, then click in the message box, since we don't want to save any changes.
Once we quit the AUT, the recorded test will appear in Squish's IDE as the screenshot illustrates. (Note that the exact code that is recorded will vary depending on how you interact. For example, you might invoke menu options by clicking them or by using key sequences—it doesn't matter which you use, but since they are different, Squish will record them differently.)
If the recorded test doesn't appear, click (or double-click depending on
your platform and settings) the tst_general test
case; this will make Squish show the test's
test.py file in an editor window as shown in the
screenshot.
Now that we've recorded the test we are able to play it back, i.e., run it. This in itself is useful in that if the play back failed it might mean that the application has been broken. Furthermore, the two verifications we put in will be checked on play back as the screenshot shows.
Inserting verification points during test recording is very convenient. Here we inserted two in one go, but we can insert as many as we like as often as we like during the test recording process. However, sometimes we might forget to insert a verification, or later on we might want to insert a new verification. We can easily insert additional verifications into a recorded test script as we will see in the next section, Inserting Additional Verification Points (Section 4.1.1.4).
Before going further we will see how to run a test, and we will also look at some of the code that Squish generated to record the test and discuss some of its features.
Running the Test
To run a test case in the IDE just click the () that appears
when the test case is hovered or selected in the Test Suites view (Section 8.2.19).
When we have two or more test cases, we can run them all, one after another,
(or only those that are checked) by clicking ().
| For command-line users |
|---|---|
As noted earlier, the squishserver must always be running when
recording or running a test, or the To play back a recorded test from the command line we execute the squishrunner program and specify the test suite our recorded script is in and the test case we want to play. For example (assuming we are in the directory that contains the test suite's directory): squishrunner --testsuite suite_py --testcase tst_general --local |
If you look at the code in the screenshot (or the code snippet shown
below) you will see that it consists of lots of waitForObject calls as parameters to various
other calls such as activateItem,
clickButton,
clickItem, and type. The waitForObject function waits until a GUI object
is ready to be interacted with (i.e., becomes visible and enabled), and
is then followed by some function that interacts with the object. The
typical interactions are activate (pop-up) a menu, click a menu option
or a button, or type in some text. (For a complete overview of
Squish's script commands see the User Guide (Chapter 5), the
API Reference Manual (Chapter 6), and the Tools Reference Manual (Chapter 7). Objects are
identified by names that Squish generates. (See How to Identify and Access Objects (Section 5.1) for full details.)
The generated code is about 35 lines of code. Here's an extract that just shows how Squish records clicking the Edit menu's Add option, typing in Jane Doe's details into the Add dialog, and clicking OK at the end to close the dialog and update the table.
| Scripting Language Support |
|---|---|
Although the screenshots only show the Python test suite in action, for the code snippets quoted here and throughout the tutorial, we show the code for all the scripting languages that Squish supports. In practice you would normally only use one of them of course, so feel free to just look at the snippets in the language you are interested in and skip the others. (In the HTML version of this manual you can use the combobox at the top of the page to select the language you use—this will hide the code snippets in other languages.) |
clickButton(waitForObject(names.address_Book_MyAddresses_adr_Add_QToolButton))
snooze(.5) # qt4/linux workaround
type(waitForObject(names.forename_LineEdit), "Jane")
type(waitForObject(names.forename_LineEdit), "<Tab>")
type(waitForObject(names.surname_LineEdit), "Doe")
type(waitForObject(names.surname_LineEdit), "<Tab>")
type(waitForObject(names.email_LineEdit), "jane.doe@nowhere.com")
type(waitForObject(names.email_LineEdit), "<Tab>")
type(waitForObject(names.phone_LineEdit), "123 555 1212")
clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
clickButton(waitForObject(names.addressBookMyAddressesAdrAddQToolButton));
type(waitForObject(names.forenameLineEdit), "Jane");
type(waitForObject(names.forenameLineEdit), "<Tab>");
type(waitForObject(names.surnameLineEdit), "Doe");
type(waitForObject(names.surnameLineEdit), "<Tab>");
type(waitForObject(names.emailLineEdit), "Jane.doe@nowhere.com");
type(waitForObject(names.emailLineEdit), "<Tab>");
type(waitForObject(names.phoneLineEdit), "123 555 1212");
clickButton(waitForObject(names.addressBookAddOKQPushButton));
clickButton(waitForObject($Names::address_book_myaddresses_adr_add_qtoolbutton));
type(waitForObject($Names::forename_lineedit), "Jane");
type(waitForObject($Names::forename_lineedit), "<Tab>");
type(waitForObject($Names::surname_lineedit), "Doe");
type(waitForObject($Names::surname_lineedit), "<Tab>");
type(waitForObject($Names::email_lineedit), "Jane.Doe\@nowhere.com");
type(waitForObject($Names::email_lineedit), "<Tab>");
type(waitForObject($Names::phone_lineedit), "123 555 1212");
clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
clickButton(waitForObject(Names::Address_Book_MyAddresses_adr_Add_QToolButton))
type(waitForObject(Names::Forename_LineEdit), "Jane")
type(waitForObject(Names::Forename_LineEdit), "<Tab>")
type(waitForObject(Names::Surname_LineEdit), "Doe")
type(waitForObject(Names::Surname_LineEdit), "<Tab>")
type(waitForObject(Names::Email_LineEdit), "jane.doe@nowhere.com")
type(waitForObject(Names::Email_LineEdit), "<Tab>")
type(waitForObject(Names::Phone_LineEdit), "123 555 1212")
clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
invoke clickButton [waitForObject $names::Address_Book_MyAddresses_adr_Add_QToolButton]
invoke type [waitForObject $names::Forename_LineEdit] "Jane"
invoke type [waitForObject $names::Forename_LineEdit] "<Tab>"
invoke type [waitForObject $names::Surname_LineEdit] "Doe"
invoke type [waitForObject $names::Surname_LineEdit] "<Tab>"
invoke type [waitForObject $names::Email_LineEdit] "jane.doe@nowhere.com"
invoke type [waitForObject $names::Email_LineEdit] "<Tab>"
invoke type [waitForObject $names::Phone_LineEdit] "123 555 1212"
invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
As you can see the tester used the keyboard to tab from one text field to another and clicked the OK button using the mouse rather than with a key press. If the tester had clicked the button any other way (for example, by pressing Alt+O, or by tabbing to the OK button and pressing the spacebar), the outcome would be the same, but of course Squish will have recorded the actual actions that were taken.
| Object Names |
|---|---|
Squish recordings refer to objects using variables that begin with a
|
![[Tip]](images/tip.png) | Editor Context Menu |
|---|---|
When a Symbolic Name is under the cursor, the editor's context menu allows you to , showing its entry in the Object Map, or , which places an inline mapping in your desired script language at the cursor, allowing you to hand-edit the properties in the script itself. |
Now that we have seen how to record and play back a test and have seen the code that Squish generates, let's go a step further and make sure that at particular points in the test's execution certain conditions hold.
In the previous section we saw how easy it is to insert verification
points during the recording of test scripts. Verification points can
also be inserted into existing test scripts, either by setting a
breakpoint and using the Squish IDE, or simply by editing a test script and
putting in calls to Squish's test functions such as test.compare and test.verify.
Squish supports many kinds of verification points: those that verify that object properties have particular values—known as "Object Property Verifications"; those that verify that an entire table has the contents we expect—known as "Table Verifications"; those that verify that two images match—known as "Screenshot Verifications"; and a hybrid verification type that includes properties and screenshots from multiple objects, known as "Visual Verifications". In addition, it is possible to verify that a search image exists somewhere on the screen, or that certain text is found by OCR. The most commonly used kind is object property verifications, and it is these that we will cover in the tutorial. For further reading, see How to Create and Use Verification Points (Section 5.22)).
Regular (non-scriptified) property
verification points are stored as XML files in the test case or test suite
resources, and contain the value(s) that need to be passed to
test.compare. These verification points can be reused across test
cases, and can verify many values in a single line of script code.
Scriptified property verification points are direct
calls to the test.compare function, with two
arguments—the value of a particular property for a particular
object, and an expected value. We can manually insert calls to the test.compare function in a recorded or hand
written script, or we can get Squish to insert them for us using scriptified
verification points. In the previous section we showed how to use the Squish IDE to insert
verifications during recording. Here we will first show how to use the
Squish IDE to insert verifications into an existing test script, and then we
will show how to insert a verification by hand.
Before asking Squish to insert verification points, it is best to make
sure that we have a list of what we want to verify and when. There are many
potential verifications we could add to the tst_general
test case, but since our concern here is simply to show how to do it, we
will only do two—we will verify that the "Jane Doe"
entry's email address and phone number match the ones entered, and put
the verifications immediately after the ones we inserted during
recording.
To insert a verification point using the IDE we start by putting a break point in the script (whether recorded or manually written—it does not matter to Squish), at the point where we want to verify.
The Squish IDE showing the tst_general test case with a breakpoint
As the above screenshot shows, we have set a breakpoint at line 35. This is done simply by double-clicking, or right-clicking in the gutter (next to the line number in the editor) and selecting the context menu item. We chose this line because it follows the script lines where the first address is removed, so at this point (just before invoking the File menu to close the application), the first address should be that of "Jane Doe". The screenshot shows the verifications that were entered using the Squish IDE during recording. Our additional verifications will follow them. (Note that your line number may be different if you recorded the test in a different way, for example, using keyboard shortcuts rather than clicking menu items.)
Having set the breakpoint, we now run the test as usual by clicking the
() button, or by clicking the
| menu option. Unlike a normal test run the
test will stop when the breakpoint is reached (i.e., at line 33, or at
whatever line you set), and Squish's main window will reappear (which
will probably obscure the AUT). At this point the Squish IDE will
automatically switch to the Squish Test Debugging Perspective (Section 8.1.2.3).
| Perspectives and Views |
|---|---|
The Squish IDE works just like the Eclipse IDE. If you aren't used to Eclipse it is crucial to understand one key concept: Views and Perspectives. In Eclipse (and therefore in the Squish IDE), a View is essentially a child window (perhaps a dock window, or a tab in an existing window). And a Perspective is a collection of Views arranged together. Both are accessible through the menu. The Squish IDE is supplied with three Perspectives—the Squish Test Management Perspective (Section 8.1.2.2) (which is the Perspective that the Squish IDE starts with, and the one we have seen in all previous screenshots), Squish Test Debugging Perspective (Section 8.1.2.3), and Squish Spy Perspective (Section 8.1.2.1). You can change these Perspectives to include additional Views (or to get rid of any Views that you don't want), and you can create your own Perspectives with exactly the Views you want. So if your windows change dramatically it just means that the Perspective changed; you can always use the menu to change back to the Perspective you want. In practice, Squish will automatically change perspective to reflect the current situation, so it isn't really necessary to change perspective manually. |
As the screenshot below shows, when Squish stops at a breakpoint the Squish IDE automatically changes to the Squish Test Debugging Perspective (Section 8.1.2.3). The perspective shows the Variables view (Section 8.2.21), the Editor view (Section 8.2.6), the Debug view (Section 8.2.5), the Application Objects view (Section 8.2.1), and the Properties view (Section 8.2.12), Methods view (Section 8.2.9), and Test Results view (Section 8.2.18).
To insert a verification point we can expand items in the Application
Objects view until we find the object we want to verify. In this example
we want to verify the QTableWidget's first row's texts, so
we expand the “Address Book - MyAddresses_adr” item, and
its child items until we find the QTableWidget, and within
that the item we are interested in. Once we click the item object its
properties are shown in the Properties view (Section 8.2.12) as
the screenshot shows.
The normal Squish Test Management Perspective (Section 8.1.2.2) can be returned to at any time by choosing it from the menu (or by clicking its toolbar button), although the Squish IDE will automatically return to it if you stop the script or run it to completion.
Here, we can see that the text property of the item in row 0 and column 0 has the value “Jane”; we already have a verification for this that we inserted during recording. Scroll down so that you can see the item in row 0 column 2: this is the email address. To make sure that this is verified every time the test is run, click the “item_0/2” item in the Application Objects view (Section 8.2.1)to make its properties appear, and then click the text property to check its check box. When we check it, the Verification Point Creator view (Section 8.2.22) appears as shown in the screenshot.
At this point the verification point has not been added to the test script. We could easily add it by clicking the button. But before doing that we'll add one more thing to be verified.
Scroll down and click the “item_0/3” item in the Application Objects view (Section 8.2.1); then click its text property. Now both verifications will appear in the Verification Point Creator view (Section 8.2.22) as the screenshot shows.
We have now said that we expect these properties to have the values shown, that is, an email address of “jane.doe@nowhere.com” and phone number of “555 123 4567”. We must click the button to actually insert the verification point, so do that now.
We don't need to continue running the test now, so we can either stop running the test at this point (by clicking the toolbar button), or we can continue (by clicking the button).
Once we have finished inserting verifications and stopped or finished
running the test we should now disable the break point. Just right click
the break point and click the menu option in the context menu. We are now
ready to run the test without any breakpoints but with the verification
points in place. Click the () button. This
time we will get some additional test results—as the screenshot
shows—one of which we have expanded to show its details. (We have
also selected the lines of code that Squish inserted to perform the
verifications—notice that the code is structurally identical to
the code inserted during recording.)
These particular verification points generate four tests comparing the forename, surname, email, and phone number of the newly inserted entry.
Another way to insert verification points is to insert them in code
form. In theory we can just add our own calls to Squish's test
functions such as test.compare and test.verify anywhere we like in an existing
script. In practice it is best to make sure that Squish knows about
the objects we want to verify first so that it can find them when the
test is run. This involves a very similar procedure to inserting them using the Squish IDE.
First we set a breakpoint where we intend adding our verifications. Then
we run the test script until it stops. Next we navigate in the
Application Objects view (Section 8.2.1) until we find the
object we want to verify. At this point it is wise to right-click the
object we are interested in and click the context menu option. This will ensure that Squish
can access the object. Then right click again and click the
context
menu option—this gives us the name of the object that Squish
will use to identify it. Now we can edit the test script to add in our
own verification and finish or stop the execution. (Don't forget to
disable the break point once it isn't needed any more.)
Although we can write our test script code to be exactly the same style as the automatically generated code, it is usually clearer and easier to do things in a slightly different style, as we will explain in a moment.
For our manual verifications we want to check the number of addresses
present in the QTableWidget after reading in the
MyAddresses.adr file, then after the new address is
added, and finally after the first address is removed. The screenshot
shows two of the lines of code we entered to get one of these three
verifications, plus the results of running the test script.
When writing scripts by hand, we use Squish's test
module's functions to verify conditions at certain points during our
test script's execution. As the screenshot (and the code snippets below)
show, we begin by retrieving a reference to the object we are interested
in. Using the waitForObject function is
standard practice for manually written test scripts. This function waits
for the object to be available (i.e., visible and enabled), and then
returns a reference to it. (Otherwise it times out and raises a
catchable exception.) We then use this reference to access the item's
properties—in this case the QTableWidget's
rowCount property—and verify that the value is what
we expect it to be using the test.compare
function. (Incidentally, we got the name for the object from the
previous line so we didn't need to set a breakpoint and manually add the
table's name to the Object Map to ensure that Squish would remember it
in this particular case because Squish had already added it during the
test recording.)
Here is the code we entered manually for the first verification for all
the scripting languages that Squish supports. Naturally, you only need
to look at the code for the language that you will be using for your own
tests. (For the other verifications we just did calls to the test.compare function reusing the table
object reference we obtained in the code shown below.)
table = waitForObject(names.address_Book_MyAddresses_adr_File_QTableWidget)
test.compare(table.rowCount, 125)
var table = waitForObject(names.addressBookMyAddressesAdrFileQTableWidget)
test.compare(table.rowCount, 125);
my $table = waitForObject($Names::address_book_myaddresses_adr_file_qtablewidget);
test::compare($table->rowCount, 125);
table = waitForObject(Names::Address_Book_MyAddresses_adr_File_QTableWidget)
Test.compare(table.rowCount, 125)
set table [waitForObject $names::Address_Book_MyAddresses_adr_File_QTableWidget]
test compare [property get $table rowCount] 125
The coding pattern is very simple: we retrieve a reference to the object we are interested in and then verify its properties using one of Squish's verification functions. And we can, of course, call methods on the object to interact with it if we wish.
We will see more examples of manually written code shortly, in the Creating Tests by Hand (Section 4.1.1.5) section, and further examples are in the User Guide (Chapter 5).
For complete coverage of verification points, see How to Create and Use Verification Points (Section 5.22) in the User Guide (Chapter 5).
After each test run finishes, the test results—including those for the verification points—are shown in the Test Results view at the bottom of the Squish IDE.
This is a detailed report of the test run and would also contain details of any failures or errors, etc. If you click on a Test Results item, the Squish IDE highlights the script line which generated the test result. And if you expand a Test Results item, you can see additional details of the test.
Now that we have seen how to record a test and modify it by inserting verification points, we are ready to see how to create tests manually. The easiest way to do this is to modify and refactor recorded tests, although it is also perfectly possible to create manual tests from scratch.
Potentially the most challenging part of writing manual tests is to use the right object names, but in practice, this is rarely a problem. We can either copy the symbolic names that Squish has already added to the Object Map when recording previous tests, or we can copy object names directly from recorded tests. And if we haven't recorded any tests and are starting from scratch we can use the Spy. We do this by clicking the toolbar button. This starts the AUT and switches to the Squish Spy Perspective (Section 8.1.2.1). We can then interact with the AUT until the object we are interested in is visible. Then, inside the Squish IDE we can navigate to, or pick the object so it is selected in the Application Objects view and use the context menu to both and (so that we can paste it into our test script). And at the end we can click the toolbar button to terminate the AUT and return Squish to the Squish Test Management Perspective (Section 8.1.2.2). (See How to Use the Spy (Section 5.21.3) in the User Guide (Chapter 5) for more details on using the Spy.)
We can view the Object Map by clicking the toolbar button, or from the Script Editor context menu, when right-clicking on an object name in script (see also, the Object Map view (Section 8.2.10)). Every application object that Squish interacts with is listed here, either as a top-level object, or as a child object (the view is a tree view). We can retrieve the symbolic name used by Squish in recorded scripts by right-clicking the object we are interested in and then clicking the context menu's (to get the symbolic name variable) or (to get the actual key-value pairs stored in the variable). This is useful for when we want to modify existing test scripts or when we want to create test scripts from scratch, as we will see later on in the tutorial.
Squish's Object Map
Suppose we want to test the AUT's Add functionality by adding three new names and addresses. We could of course record such a test but it is just as easy to do everything in code. The steps we need the test script to do are: first click | to create a new address book, then for each new name and address, click |, then fill in the details, and click . And finally, click | without saving. We also want to verify at the start that there are no rows of data and at the end that there are three rows. We will also refactor as we go, to make our code as neat and modular as possible.
First, we must create a new test case. Click
| and set the test case's name to be
tst_adding. Squish will automatically create a
test.py (or test.js, and
so on) file.
Command line users can simply create a tst_adding
directory inside the test suite's directory and create and edit the
test.py file (or test.js and
so on) within that directory.
The first thing we need is a way to start the AUT and then invoke a menu
option. Here are the first few lines from the
recorded tst_general script:
import names
import os
def main():
startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
activateItem(waitForObjectItem(names.address_Book_QMenuBar, "File"))
activateItem(waitForObjectItem(names.address_Book_File_QMenu, "Open..."))
import * as names from 'names.js';
function main() {
startApplication('"' + OS.getenv("SQUISH_PREFIX") + '/examples/qt/addressbook/addressbook"');
activateItem(waitForObjectItem(names.addressBookQMenuBar, "File"));
activateItem(waitForObjectItem(names.addressBookFileQMenu, "Open..."));
require 'names.pl';
sub main
{
startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
activateItem(waitForObjectItem($Names::address_book_qmenubar, "File"));
activateItem(waitForObjectItem($Names::address_book_file_qmenu, "Open..."));
require 'names'
include Squish
def main
startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
activateItem(waitForObjectItem(Names::Address_Book_QMenuBar, "File"))
activateItem(waitForObjectItem(Names::Address_Book_File_QMenu, "Open..."))
source [findFile "scripts" "names.tcl"]
proc main {} {
startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
invoke activateItem [waitForObjectItem $names::Address_Book_QMenuBar "File"]
invoke activateItem [waitForObjectItem $names::Address_Book_File_QMenu "Open..."]
Notice that the pattern in the code is simple: start the AUT, then wait
for the menu bar, then activate the menu bar; wait for the menu item,
then activate the menu item. In both cases we have used the
waitForObjectItem function. This function
is used for a multi-valued objects (such as lists, tables,
trees—or in this case, a menubar and a menu), and allows us to
access the object's items (which are themselves objects of course), by
passing the name of the object containing the item and the item's text
as arguments.
| Note |
|---|---|
It may seem a waste to put our functions in
|
If you look at the recorded test (tst_general) or
in the Object Map you will see that Squish sometimes uses
different names for the same things.
For example, the menubar is identified in three different ways,
initially as AddressBook_QMenuBar
then if the user clicks |,
it is identified as AddressBook_Unnamed_QMenuBar,
and if the user clicks |
and opens the MyAddresses.adr
file, then the menubar is identified as
AddressBook_MyAddressesadr_QMenuBar.
The reason for this is that Squish needs to uniquely identify every
object in a given context, and it uses whatever information it has to
hand. So in the case of identifying menubars (and many other objects),
Squish uses the window title text to give it some context. (For
example, an application's File or Edit menus may have different options
depending on whether a file is loaded and what state the application is
in.)
Naturally, when we write test scripts we don't want to have to know or care which particular variation of a name to use, and Squish supports this need by providing alternative naming schemes, as we will see shortly.
Sometimes the AUT will appear to freeze during test execution. When this happens, just wait for Squish to time out the AUT (about 20 seconds), and then it will pop up an Object Not Found dialog (Section 8.3.14), indicating an error like this:
don't worry! It just means that Squish doesn't have an object with the
given name in the Object Map. From here, we can
,
, or after picking a new object,
. Picking a new object will update the object
map entry for the symbolic name.
In addition to the Object Picker (
), we can also use the Spy's Application Objects view (Section 8.2.1) to locate the objects we
are interested in and use the context menu to access real or symbolic names
of them.
We've spent a bit of time on the issue of naming since it is probably the part of writing scripts that leads to the most error messages (usually of the object ... not found kind shown above.) Once we have identified the objects we are going to access in our tests, writing test scripts using Squish is very straightforward. And of course you can almost certainly use the scripting language you are most familiar with since Squish supports the most popular ones available.
We are now almost ready to write our own test script. It is probably
easiest to begin by recording a dummy test. So click
| and set the test case's name to be
tst_dummy. Then click the dummy test case's
().
Once the AUT starts, click
|, then click the (empty)
table, then click | and
add an item, then press Return or click . Finally,
click | to finish, and say
to saving changes. Then replay
this test just to confirm that everything works okay. The sole purpose
of this is to make sure that Squish adds the necessary names to the
Object Map since it is probably quicker to do it this way than to use
the Spy for every object of interest. After replaying the dummy test
you can delete it if you want to.
With all the object names we need in the Object Map, we can now write our
own test script completely from scratch. We will start with the
main function, and then we will look at the supporting
functions that the main function uses.
import names
import os
def main():
startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
invokeMenuItem("File", "New")
table = waitForObject({"type": "QTableWidget"})
test.verify(table.rowCount == 0)
data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
for oneNameAndAddress in data:
addNameAndAddress(oneNameAndAddress)
waitForObject(table);
test.compare(table.rowCount, len(data), "table contains as many rows as added data")
closeWithoutSaving()
import * as names from 'names.js';
function invokeMenuItem(menu, item)
{
activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}
require 'names.pl';
sub main
{
startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
invokeMenuItem("File", "New");
my $table = waitForObject({"type" => "QTableWidget"});
test::verify($table->rowCount == 0);
my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
foreach my $oneNameAndAddress (@data) {
addNameAndAddress(@{$oneNameAndAddress});
}
test::compare($table->rowCount, scalar(@data), "table contains as many rows as added data");
closeWithoutSaving();
}
require 'names'
include Squish
def main
startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
invokeMenuItem("File", "New")
table = waitForObject({:type => "QTableWidget"})
Test.verify(table.rowCount == 0)
data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
data.each do |oneNameAndAddress|
addNameAndAddress(oneNameAndAddress)
end
Test.compare(table.rowCount, data.length, "table contains as many rows as added data")
closeWithoutSaving
end
source [findFile "scripts" "names.tcl"]
proc main {} {
startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
set table [waitForObject [::Squish::ObjectName type QTableWidget]]
test compare [property get $table rowCount] 0
invokeMenuItem "File" "New"
set data [list \
[list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
[list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
[list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
for {set i 0} {$i < [llength $data]} {incr i} {
addNameAndAddress [lindex $data $i]
}
waitForObject $table
test compare [property get $table rowCount] [llength $data] "table contains as many rows as added data"
closeWithoutSaving
}
We begin by starting the application with a call to the startApplication function. The name we pass as a
string is the name registered with Squish (normally the name of the
executable). Then we obtain a reference to the QTableWidget.
The object name we used is a Real Name, containing
key-value pairs. From the Object Map Editor, we found another
QTableWidget object map entry, and obtained its real name from
. We pasted it into our
tst_adding testcase, and removed the properties
that were too specific for this situation.
The waitForObject
function waits until an object is ready (visible and enabled) and
returns a reference to it—or it times out and raises a catchable
exception. Once we have the table reference we can use
it to access any of the QTableWidget's
public methods and properties.
The invokeMenuItem function is one we have created
specially for this test. It takes a menu name and a menu option name and
invokes the menu option. It also uses real names to describe objects,
and demonstrates how to parametrize values from variables in each script
language.
After using the invokeMenuItem
function to do
|, we verify that the
table's row count is 0.
Next, we create some sample data and call a custom
addNameAndAddress function to populate the table with the
data using the AUT's Add dialog. Then we again compare the table's row
count, this time to the number of rows in our sample data. And finally
we call a custom closeWithoutSaving function to terminate
the application.
We will now review each of the three supporting functions, so as to
cover all the code in the tst_adding test case,
starting with the invokeMenuItem function.
def invokeMenuItem(menu, item):
activateItem(waitForObjectItem({"type": "QMenuBar"}, menu))
activateItem(waitForObjectItem({'type': 'QMenu', 'title': menu}, item))
function invokeMenuItem(menu, item)
{
activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}
sub invokeMenuItem
{
my ($menu, $item) = @_;
activateItem(waitForObjectItem({"type" => "QMenuBar"}, $menu));
activateItem(waitForObjectItem({"title" => $menu, "type" => "QMenu"}, $item));
}
def invokeMenuItem(menu, item)
activateItem(waitForObjectItem({:type => "QMenuBar"}, menu))
activateItem(waitForObjectItem({:title => menu, :type => "QMenu"}, item))
end
proc invokeMenuItem {menu item} {
invoke activateItem [waitForObjectItem [::Squish::ObjectName type QMenuBar] $menu]
invoke activateItem [waitForObjectItem [::Squish::ObjectName title $menu type QMenu] $item]
}
As we mentioned earlier, the object names Squish uses for menus and menu items (and other objects) can vary depending on the context, and the context is partially derived from the window's title. For our reusable functions, we would like object names to match the desired objects regardless of the context or window title. So, instead of reusing an existing symbolic name, we will copy its real name and remove the properties we don't want to check for.
Every real name must specify the type property, and usually at least one other property. Here we've used the type to uniquely identify the menubar, and type and title properties to uniquely identify the menu. By using real names, we can create an object name that is general to enough match our desired objects regardless of the window title.
Once we have identified the object we want to interact with, we use the
waitForObjectItem function to retrieve a
reference to it and in this case we then apply the
activateItem function to it. The waitForObjectItem function pauses Squish until
the specified object and its item are visible and enabled. So, here, we
waited for the menu bar and one of its menu bar items, and then we
waited for a menu bar item and one of its menu items. And as soon as the
waiting is over each time we activate the object and its item using the
activateItem function.
def addNameAndAddress(oneNameAndAddress):
invokeMenuItem("Edit", "Add...")
type(waitForObject(names.forename_LineEdit), oneNameAndAddress[0])
type(waitForObject(names.surname_LineEdit), oneNameAndAddress[1])
type(waitForObject(names.email_LineEdit), oneNameAndAddress[2])
type(waitForObject(names.phone_LineEdit), oneNameAndAddress[3])
clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
function addNameAndAddress(oneNameAndAddress)
{
invokeMenuItem("Edit", "Add...");
type(waitForObject(names.forenameLineEdit), oneNameAndAddress[0]);
type(waitForObject(names.surnameLineEdit), oneNameAndAddress[1]);
type(waitForObject(names.emailLineEdit), oneNameAndAddress[2]);
type(waitForObject(names.phoneLineEdit), oneNameAndAddress[3]);
clickButton(waitForObject(names.addressBookAddOKQPushButton));
}
sub addNameAndAddress
{
my(@oneNameAndAddress) = @_;
invokeMenuItem("Edit", "Add...");
type(waitForObject($Names::forename_lineedit), $_[0]);
type(waitForObject($Names::surname_lineedit), $_[1]);
type(waitForObject($Names::email_lineedit), $_[2]);
type(waitForObject($Names::phone_lineedit), $_[3]);
clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
}
def addNameAndAddress(oneNameAndAddress)
invokeMenuItem("Edit", "Add...")
type(waitForObject(Names::Forename_LineEdit), oneNameAndAddress[0])
type(waitForObject(Names::Surname_LineEdit), oneNameAndAddress[1])
type(waitForObject(Names::Email_LineEdit), oneNameAndAddress[2])
type(waitForObject(Names::Phone_LineEdit), oneNameAndAddress[3])
clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
end
proc addNameAndAddress {oneNameAndAddress} {
invokeMenuItem "Edit" "Add..."
invoke type [waitForObject $names::Forename_LineEdit] [lindex $oneNameAndAddress 0]
invoke type [waitForObject $names::Surname_LineEdit] [lindex $oneNameAndAddress 1]
invoke type [waitForObject $names::Email_LineEdit] [lindex $oneNameAndAddress 2]
invoke type [waitForObject $names::Phone_LineEdit] [lindex $oneNameAndAddress 3]
invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
}
For each set of name and address data we invoke the
| menu option
to pop up the Add dialog. Then for each value received we
populate the appropriate field by waiting for the relevant QLineEdit to be
ready and then typing in the text using the type function.
And at the end we click the dialog's button.
The code from this function was mostly copied and modified from
the tst_general test case.
def closeWithoutSaving():
sendEvent("QCloseEvent", waitForObject(names.mainWindow));
clickButton(waitForObject(names.address_Book_No_QPushButton))
function closeWithoutSaving()
{
sendEvent("QCloseEvent", waitForObject(names.mainWindow));
clickButton(waitForObject(names.addressBookNoQPushButton));
}
sub closeWithoutSaving
{
sendEvent( "QCloseEvent", waitForObject($Names::mainwindow) );
clickButton(waitForObject($Names::address_book_no_qpushbutton));
}
def closeWithoutSaving
sendEvent("QCloseEvent", waitForObject(Names::MainWindow))
clickButton(waitForObject(Names::Address_Book_No_QPushButton))
end
proc closeWithoutSaving {} {
sendEvent QCloseEvent [waitForObject $names::MainWindow]
invoke clickButton [waitForObject $names::Address_Book_No_QPushButton]
}
Here we use the invokeMenuItem function to do
|,
and then click the Save unsaved changes? dialog's
button. The last line was copied from the recorded test.
The entire test is under 30 lines of code—and would be even less
if we put some of the common functions (such as
invokeMenuItem and closeWithoutSaving) in
a shared script. And much of the code was copied directly from the
recorded test, and in some cases parametrized.
This should be sufficient to give a flavor of writing test scripts for an AUT. Keep in mind that Squish provides far more functionality than we used here, (all of which is covered in the API Reference Manual (Chapter 6) and the Tools Reference Manual (Chapter 7)). And Squish also provides access to the entire public APIs of the AUT's objects.
However, one aspect of the test case is not very satisfactory. Although embedding test data as we did here is sensible for small amounts, it is rather limiting, especially when we want to use a lot of test data. Also, we didn't test any of the data that was added to see if it correctly ended up in the QTableWidget. In the next section we will create a new version of this test, only this time we will pull in the data from an external data source, and check that the data we add to the QTableWidget is correct.
In the previous section we put three hard-coded names and addresses in
our test. But what if we want to test lots of data?
Or what if we want to change the data without having to change our test
script's source code. One approach is to import a dataset into Squish
and use the dataset as the source of the values we insert into our
tests. Squish can import data in .tsv
(tab-separated values format), .csv
(comma-separated values format), .xls, or
.xlsx (Microsoft® Excel™ spreadsheet formats).
[2]
Test data can either be imported using the Squish IDE, or manually using a file manager or console commands. We will describe both approaches, starting with using the Squish IDE.
For the addressbook application we want to
import the MyAddresses.tsv data file. To do this we
must start by clicking
| to pop-up the Import Squish Resource dialog (Section 8.3.7). Inside the dialog click
the button to choose the file to
import—in this case MyAddresses.tsv. Make
sure that the Import As combobox is set to
“TestData”. By default the Squish IDE will import the test data
just for the current test case, but we want the test data to be
available to all the test suite's test cases: to do this check the
radio button. Now
click the button. You can now see the file
listed in the Test Suite Resources view (in the Test Data tab), and if
you click the file's name it will be shown in an Editor view (Section 8.2.6). The screenshot shows Squish after the
test data has been added.
| For command-line users |
|---|---|
It is also possible to import test data outside the Squish IDE using a file
manager (such as File Explorer) or console commands. To do this, create
a directory inside the test suite's directory called
|
Squish with some imported test data
Although in real life we would modify our
tst_adding test case to use the test data, for the
purpose of the tutorial we will make a new test case called
tst_adding_data that is a copy of
tst_adding and which we will modify to make use of
the test data.
The only function we have to change is main, where
instead of iterating over hard-coded items of data, we iterate over all
the records in the dataset. We also need to update the expected row
count at the end since we are adding a lot more records now, and we will
also add a function to verify each record that's added.
import names
import os
def main():
startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
invokeMenuItem("File", "New")
table = waitForObject({"type": "QTableWidget"})
test.verify(table.rowCount == 0)
limit = 10 # To avoid testing 100s of rows since that would be boring
for row, record in enumerate(testData.dataset("MyAddresses.tsv")):
forename = testData.field(record, "Forename")
surname = testData.field(record, "Surname")
email = testData.field(record, "Email")
phone = testData.field(record, "Phone")
table.setCurrentCell(0, 0) # always insert at the start
addNameAndAddress((forename, surname, email, phone)) # pass as a single tuple
checkNameAndAddress(table, record)
if row > limit:
break
test.compare(table.rowCount, row + 1, "table contains as many rows as added data")
closeWithoutSaving()
import * as names from 'names.js';
function invokeMenuItem(menu, item)
{
activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}
require 'names.pl';
sub main
{
startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
invokeMenuItem("File", "New");
my $table = waitForObject({"type" => "QTableWidget"});
test::verify($table->rowCount == 0);
my $limit = 10; # To avoid testing 100s of rows since that would be boring
my @records = testData::dataset("MyAddresses.tsv");
my $row = 0;
for (; $row < scalar(@records); ++$row) {
my $record = $records[$row];
my $forename = testData::field($record, "Forename");
my $surname = testData::field($record, "Surname");
my $email = testData::field($record, "Email");
my $phone = testData::field($record, "Phone");
$table->setCurrentCell(0, 0); # always insert at the start
addNameAndAddress($forename, $surname, $email, $phone);
checkNameAndAddress($table, $record);
if ($row > $limit) {
last;
}
}
test::verify($table->rowCount == $row + 1);
closeWithoutSaving();
}
require 'names'
include Squish
def main
startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
invokeMenuItem("File", "New")
table = waitForObject({:type => "QTableWidget"})
Test.verify(table.rowCount == 0)
limit = 10 # To avoid testing 100s of rows since that would be boring
rows = 0
TestData.dataset("MyAddresses.tsv").each_with_index do
|record, row|
forename = TestData.field(record, "Forename")
surname = TestData.field(record, "Surname")
email = TestData.field(record, "Email")
phone = TestData.field(record, "Phone")
table.setCurrentCell(0, 0) # always insert at the start
addNameAndAddress([forename, surname, email, phone]) # pass as a single Array
checkNameAndAddress(table, record)
break if row > limit
rows += 1
end
Test.compare(table.rowCount, rows + 1, "table contains as many rows as added data")
closeWithoutSaving
end
source [findFile "scripts" "names.tcl"]
proc main {} {
startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
invokeMenuItem "File" "New"
set table [waitForObject [::Squish::ObjectName type QTableWidget]]
test compare [property get $table rowCount] 0
# To avoid testing 100s of rows since that would be boring
set limit 10
set data [testData dataset "MyAddresses.tsv"]
set columns [llength [testData fieldNames [lindex $data 0]]]
set row 0
for {} {$row < [llength $data]} {incr row} {
set record [lindex $data $row]
set forename [testData field $record "Forename"]
set surname [testData field $record "Surname"]
set email [testData field $record "Email"]
set phone [testData field $record "Phone"]
set details [list $forename $surname $email $phone]
invoke $table setCurrentCell 0 0
addNameAndAddress $details
checkNameAndAddress $table $record
if {$row > $limit} {
break
}
}
test compare [property get $table rowCount] [expr $row + 1]
closeWithoutSaving
}
Squish provides access to test data through its testData
module's functions—here we used the testData.dataset function to access the data file
and make its records available, and the testData.field function to retrieve each record's
individual fields.
Having used the test data to populate the QTableWidget we
want to be confident that the data in the table is the same as what we
have added, so that's why we added the
checkNameAndAddress function. We also added a limit to
how many records we would compare, just to make the test run faster.
def checkNameAndAddress(table, record):
for column in range(len(testData.fieldNames(record))):
test.compare(table.item(0, column).text(), # New addresses are inserted at the start
testData.field(record, column))
function checkNameAndAddress(table, record)
{
for (var column = 0; column < testData.fieldNames(record).length; ++column)
test.compare(table.item(0, column).text(), // New addresses are inserted at the start
testData.field(record, column));
}
sub checkNameAndAddress
{
my($table, $record) = @_;
my @columnNames = testData::fieldNames($record);
for (my $column = 0; $column < scalar(@columnNames); $column++) {
test::compare($table->item(0, $column)->text(), # New addresses are inserted at the start
testData::field($record, $column));
}
}
def checkNameAndAddress(table, record)
for column in 0...TestData.fieldNames(record).length
Test.compare(table.item(0, column).text(),
TestData.field(record, column))
# New addresses are inserted at the start
end
end
proc checkNameAndAddress {table record} {
set columns [llength [testData fieldNames $record]]
for {set column 0} {$column < $columns} {incr column} {
set value [invoke [invoke $table item 0 $column] text]
test compare $value [testData field $record $column]
}
}
This function accesses the QTableWidget's
first row and extracts each of its columns' values. We use Squish's
testData.fieldNames function to get a
column count and then use the test.compare
function to check that each value in the table is the same as the value in the test
data we used. Note that for this particular test we always insert new
rows at the start of the table. The effect of this is that every new
name and address is always added as the first row, so this is why we
hard-coded the row to be 0.
The screenshot show Squish's Test Summary log after the data-driven tests have been run.
Squish after a successful data-driven test run
Squish can also do keyword-driven testing. This is a bit more sophisticated than data-driven testing. See How to Do Keyword-Driven Testing (Section 5.16).
We have now completed the tutorial! Squish can of course do much more than we have shown here, but the aim has been to get you started with basic testing as quickly and easily as possible. The User Guide (Chapter 5) provides many more examples, including those that show how tests can interact with particular widgets such as spinboxes, comboboxes, date/time editors, and line editors, and of course with model/view models and views.
The API Reference Manual (Chapter 6) and Tools Reference Manual (Chapter 7) give full details of Squish's testing API and the numerous functions it offers to make testing as easy and efficient as possible. It is well worth reading the User Guide (Chapter 5) and at least skimming the API Reference Manual (Chapter 6) and Tools Reference Manual (Chapter 7)—especially since the time invested will be repaid because you'll know what functionality Squish provides out of the box and can avoid reinventing things that are already available.
The key Qt examples with links to the places they are used are given below.
The Payment Form example (How to Test Stateful and Single-Valued Widgets (Section 5.2.6.2)) shows how to test single-valued Qt widgets such as: QCheckBox, QComboBox, QDateEdit, QLineEdit, QPushButton, QRadioButton, and QSpinBox.
The Item Views example (How to Test Items in Item Views, Item Widgets, and Models (Section 5.2.6.3)) shows how to test Qt's multi-item widgets and views, including: QListWidget and QListView, QTableWidget and QTableView, QTreeWidget and QTreeView, as well as the underlying Qt models that hold the data for the views.
The CsvTable example (How to Test Table Widgets and Use External Data Files (Section 5.2.6.4)) shows how to do data-driven testing using external data files.
In addition to the documented examples listed above, further Qt example
applications and their corresponding tests are provided in
SQUISHDIR/examples/qt.
Table of Contents
This tutorial will show you how to create, run, and modify Behavior Driven Development (BDD) tests for an example application. You will learn about Squish's most frequently used features. By the end of the tutorial you will be able to write your own tests for your own applications.
For this chapter we will use a simple Address Book application as our Application Under Test (AUT). This is a very basic application that allows users to load an existing address book or create a new one, add, edit, and remove entries. The screenshot shows the application in action with a user adding a new name and address.
The Qt addressbook example.
Behavior-Driven Development (BDD) is an extension of the Test-Driven Development approach which puts the definition of acceptance criteria at the beginning of the development process as opposed to writing tests after the software has been developed. With possible cycles of code changes done after testing.
Behavior Driven Tests are built out of a set of Feature files, which
describe product features through the expected application behavior in one or many
Scenarios. Each Scenario is built out of a sequence
of steps which represent actions or verifications that need to be
tested for that Scenario.
BDD focuses on expected application behavior, not on implementation details. Therefore BDD tests are described in a human-readable Domain Specific Language (DSL). As this language is not technical, such tests can be created not only by programmers, but also by product owners, testers or business analysts. Additionally, during the product development, such tests serve as living product documentation. For Squish usage, BDD tests shall be created using Gherkin syntax. The previously written product specification (BDD tests) can be turned into executable tests. This step by step tutorial presents automating BDD tests with Squish IDE support.
Gherkin files describe product features through the expected application behavior in one or many Scenarios. An example showing the "Filling of addressbook" feature of the addressbook example application.
Feature: Filling of addressbook
As a user I want to fill the addressbook with entries
Scenario: Initial state of created address book
Given addressbook application is running
When I create a new addressbook
Then addressbook should have zero entries
Scenario: State after adding one entry
Given addressbook application is running
When I create a new addressbook
And I add a new person 'John','Doe','john@m.com','500600700' to address book
Then '1' entries should be present
Scenario: State after adding two entries
Given addressbook application is running
When I create a new addressbook
And I add new persons to address book
| forname | surname | email | phone |
| John | Smith | john@m.com | 123123 |
| Alice | Thomson | alice@m.com | 234234 |
Then '2' entries should be present
Scenario: Forename and surname is added to table
Given addressbook application is running
When I create a new addressbook
When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
Then previously entered forename and surname shall be at the top
Most of the above is free form text (does not have to be English). It's just the
Feature/Scenario structure and the leading keywords
like "Given", "And", "When" and "Then" that are fixed. Each of those keywords marks a
step defining preconditions, user actions and expected results. Above application
behavior description can be passed to software developers to implement these features and
at the same time the same description can be passed to software testers to implement
automated tests.
First, we need to create a Test Suite, which is a container for all Test Cases. Start the squishide and select |. Please follow the New Test Suite wizard, provide a Test Suite name, choose the Qt Toolkit and scripting language of your choice and finally register Address Book application as AUT. Please refer to Creating a Test Suite (Section 4.1.1.2) for more details about creating new Test Suites.
Squish offers two types of Test Cases: "Script Test Case" and "BDD Test Case". As "Script Test Case" is the default one, in order to create new "BDD Test Case" we need to use the context menu by clicking on the expander next to button and choosing the option . The Squish IDE will remember your choice and the "BDD Test Case" will become the default when clicking on the button in the future.
The newly created BDD Test Case consists of a test.feature file
(filled with a Gherkin template while creating a new BDD test case), a file named
test.(py|js|pl|rb|tcl) which will drive the execution (there is no need
to edit this file), and a Test Suite Resources file named
shared/steps/steps.(py|js|pl|rb|tcl) where step
implementation code will be placed.
We need to replace the Gherkin template with a Feature for the addressbook example application. To do this, copy the Feature
description below and paste it into the Feature file.
Feature: Filling of addressbook
As a user I want to fill the addressbook with entries
Scenario: Initial state of created address book
Given addressbook application is running
When I create a new addressbook
Then addressbook should have zero entries
When editing the test.feature file, a Feature file
warning No implementation found is displayed for each
undefined step. The implementations are in the
steps subdirectory, in Test Case Resources,
or in Test Suite Resources. Running our
Feature test now will currently fail at the first step with a
No Matching Step Definition and the following
steps will be skipped.
In order to record the Scenario, press the
button next to the respective Scenario that
is listed in the Scenarios tab in Test Case Resources view.
This will cause Squish to run the AUT so that we can interact with it. Additionally,
the Control Bar is displayed with a list of all steps that need to be
recorded. Now all interaction with the AUT or any verification points added to
the script will be recorded under the first step Given addressbook application is
running (which is bolded in the Step list on the Control Bar). In order to verify that
this precondition is met, we will add a Verification Point. To do this, click on
in the Control Bar
and select .
As a result, the Squish IDE is put into Spy mode which displays all application objects and
their properties. In the Application Objects view, select
(don't check) the addressbook MainWindow item.
Selecting it will update the Properties view to its right.
Next, click on the checkbox in front of the property
enabled in the Properties View.
Finally, click on the button .
The Squish IDE disappears and the Control Bar is shown again.
When we are done with each step, we can move to the next
undefined step (playing back the ones that were previously defined)
by clicking on the (
) arrow button in the Control Bar that is
located to the left of the current step.
Next, for the step When I create a new addressbook, click on the
button (
) in the toolbar of the
addressbook application and click on () to move on to the
next step.
Finally, for the step Then addressbook should have zero entries
verify that the table containing the address entries is empty. To record this
verification, click on while recording, and select
. In the Application Objects,
navigate or use the Object Picker () to select (not check) the table object containing
the address book entries (in our case this table is empty).
Then, check the rowCount property
from the Properties View, and click
.
Finally, click on the last () arrow button in the Control Bar.
As a result, Squish will generate the following step
definitions in the steps.* file (at
Test Suites+Test Suite
Resources):
@Given("addressbook application is running")
def step(context):
startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
test.compare(waitForObjectExists(names.address_Book_MainWindow).enabled, True)
@Step("I create a new addressbook")
def step(context):
clickButton(waitForObject(names.address_Book_New_QToolButton))
@Then("addressbook should have zero entries")
def step(context):
test.compare(waitForObjectExists(names.address_Book_Unnamed_File_QTableWidget).rowCount, 0)
Given("addressbook application is running") do |context|
startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
Test.compare(waitForObjectExists(Names::Address_Book_MainWindow).enabled, true)
end
When("I create a new addressbook") do |context|
clickButton(waitForObject(Names::Address_Book_New_QToolButton))
end
Then("addressbook should have zero entries") do |context|
Test.compare(waitForObjectExists(Names::Address_Book_Unnamed_File_QTableWidget).rowCount, 0)
end
Given("addressbook application is running", function(context) {
startApplication('"' + OS.getenv("SQUISH_PREFIX") + '/examples/qt/addressbook/addressbook"');
test.compare(waitForObjectExists(names.addressBookMainWindow).enabled, true);
});
When("I create a new addressbook", function(context) {
clickButton(waitForObject(names.addressBookNewQToolButton));
});
Then("addressbook should have zero entries", function(context) {
test.compare(waitForObjectExists(names.addressBookUnnamedFileQTableWidget).rowCount, 0);
});
Given("addressbook application is running", sub {
my $context = shift;
startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
test::compare(waitForObjectExists($Names::address_book_mainwindow)->enabled, 1 );
});
When("I create a new addressbook", sub {
my $context = shift;
clickButton(waitForObject($Names::address_book_new_qtoolbutton) );
});
Then("addressbook should have zero entries", sub {
my $context = shift;
test::compare(waitForObjectExists($Names::address_book_unnamed_file_qtablewidget)->rowCount,0);
});
Given "addressbook application is running" {context} {
startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
test compare [property get [waitForObjectExists $names::Address_Book_MainWindow] enabled] true
}
When "I create a new addressbook" {context} {
invoke clickButton [waitForObject $names::Address_Book_New_QToolButton]
}
Then "addressbook should have zero entries" {context} {
test compare [property get [waitForObjectExists $names::Address_Book_Unnamed_File_QTableWidget] rowCount] 0
}
The application is automatically started at the beginning of the first step due
to the recorded startApplication() call. At the end of each
Scenario, the onScenarioEnd hook is called, causing
detach() to be called on the application context.
Because the AUT was started with startApplication(),
this causes it to terminate.
This hook function is found in the file bdd_hooks.(py|js|pl|rb|tcl),
which is located in the Scripts tab of the Test
Suite Resources view. You can define additional hook functions here.
For a list of all available hooks, please refer to Performing Actions During Test Execution Via Hooks (Section 6.19.10).
@OnScenarioEnd
def OnScenarioEnd(context):
for ctx in applicationContextList():
ctx.detach()
OnScenarioEnd(function(context) {
applicationContextList().forEach(function(ctx) { ctx.detach(); });
});
OnScenarioEnd(sub {
foreach (applicationContextList()) {
$_->detach();
}
});
OnScenarioEnd do |context|
applicationContextList().each { |ctx| ctx.detach() }
end
OnScenarioEnd {context} {
foreach ctx [applicationContextList] {
applicationContext $ctx detach
}
}
So far, our steps did not use any parameters and all values
were hardcoded. Squish has different types of parameters like
any, integer or word, allowing our
step definitions to be more reusable. Let us add a new
Scenario to our Feature file which will
provide step parameters for both the Test Data and the
expected results. Copy the below section into your Feature file.
Scenario: State after adding one entry
Given addressbook application is running
When I create a new addressbook
And I add a new person 'John','Doe','john@m.com','500600700' to address book
Then '1' entries should be present
After auto-saving
the Feature file, the Squish IDE provides a hint that only 2
steps need to be implemented: When I add a new person 'John',
'Doe','john@m.com','500600700' to address book and Then '1' entries should
be present. The remaining steps already have a matching
step implementation.
To record the missing steps, hit the record button next to the
test case name in the Test Suites view. The script will play until it gets to the missing
step and then prompt you to implement it. If you select the Add
button, then you can type in the information for a new entry. Click on the
() to move to the next step. For the second missing step, we
could record an object property verification
like we did with the step Then addressbook should have zero entries.
Or we could copy that step's implementation in the steps.(py|js|pl|rb|tcl) file
and increment the number
at the end of the test.compare line. Instead of testing for zero items, we are
testing for one item.
Now we parametrize the generated When step implementation by
replacing the values with parameter types.
Since we want to be able to add different names, replace 'John' with '|word|'.
Note that each parameter will be passed to the step implementation function in
the order of appearance in the descriptive name of the step. Finish
parametrizing by editing the typed values into keywords, looking like this
example step:
When I add a new person 'John', 'Doe','john@m.com','500600700'
@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
clickButton(waitForObject(names.address_Book_Add_QToolButton))
type(waitForObject(names.forename_LineEdit), forename)
type(waitForObject(names.surname_LineEdit), surname)
type(waitForObject(names.email_LineEdit), email)
type(waitForObject(names.phone_LineEdit), phone)
clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
function (context, forename, surname, email, phone){
clickButton(waitForObject(names.addressBookUnnamedAddQToolButton));
type(waitForObject(names.forenameLineEdit), forename);
type(waitForObject(names.surnameLineEdit), surname);
type(waitForObject(names.emailLineEdit), email);
type(waitForObject(names.phoneLineEdit), phone);
clickButton(waitForObject(names.addressBookAddOKQPushButton));
context.userData["forename"] = forename;
context.userData["surname"] = surname;
});
When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
my $context = shift;
my ($forename, $surname, $email, $phone) = @_;
clickButton(waitForObject($Names::address_book_unnamed_add_qtoolbutton));
type(waitForObject($Names::forename_lineedit), $forename);
type(waitForObject($Names::surname_lineedit), $surname);
type(waitForObject($Names::email_lineedit), $email);
type(waitForObject($Names::phone_lineedit), $phone);
clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
$context->{userData}{'forename'} = $forename;
$context->{userData}{'surname'} = $surname;
});
When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
type(waitForObject(Names::Forename_LineEdit), forename)
type(waitForObject(Names::Surname_LineEdit), surname)
type(waitForObject(Names::Email_LineEdit), email)
type(waitForObject(Names::Phone_LineEdit), phone)
clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
context.userData = Hash.new
context.userData[:forename] = forename
context.userData[:surname] = surname
end
When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
invoke type [waitForObject $names::Forename_LineEdit] $forename
invoke type [waitForObject $names::Surname_LineEdit] $surname
invoke type [waitForObject $names::Email_LineEdit] $email
invoke type [waitForObject $names::Phone_LineEdit] $phone
invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
$context userData [dict create forename $forename surname $surname]
}
If we recorded the final Then as a missing step, and
verified the rowCount is 1 in the table, we can modify the
step so that it takes a parameter, so it can verify other integer values later.
@Then("'|integer|' entries should be present")
def step(context, num):
snooze(.25);
test.compare(waitForObjectExists(names.address_Book_Unnamed_File_QTableWidget).rowCount, num)
Then("'|integer|' entries should be present", function(context, rowCount) {
snooze(0.25)
test.compare(waitForObjectExists(names.addressBookUnnamedFileQTableWidget).rowCount, rowCount);
});
Then("'|integer|' entries should be present", sub {
my $context = shift;
my $num = shift;
test::compare(waitForObjectExists($Names::address_book_unnamed_file_qtablewidget)->rowCount, $num);
});
Then("'|integer|' entries should be present") do |context, num|
Test.compare(waitForObjectExists(Names::Address_Book_Unnamed_File_QTableWidget).rowCount, num)
end
Then "'|integer|' entries should be present" {context num} {
test compare [property get [waitForObjectExists $names::Address_Book_Unnamed_File_QTableWidget] rowCount] $num
}
The next Scenario will test adding multiple entries to the address
book. We could use step When I add a new person John','Doe','john@m.com','500600700'
to address book multiple times just with different data. But lets instead define a
new step called When I add a new person to address book
which will handle data from a table.
Scenario: State after adding two entries
Given addressbook application is running
When I create a new addressbook
And I add new persons to address book
| forename | surname | email | phone |
| John | Smith | john@m.com | 123123 |
| Alice | Thomson | alice@m.com | 234234 |
Then '2' entries should be present
The step implementation to handle such tables looks like this:
@Step("I add new persons to address book")
def step(context):
table = context.table
# Drop initial row with column headers
table.pop(0)
for (forname, surname, email, phone) in table:
clickButton(waitForObject(names.address_Book_Add_QToolButton))
snooze(.5) # workaround for qt4
type(waitForObject(names.forename_LineEdit), forname)
type(waitForObject(names.surname_LineEdit), surname)
type(waitForObject(names.email_LineEdit), email)
type(waitForObject(names.phone_LineEdit), phone)
clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
test.log("Added entry: "+forname+","+surname+","+email+","+phone);
When("I add new persons to address book", function(context) {
var table = context.table;
// Drop initial row with column headers
for (var i = 1; i < table.length; ++i) {
var forename = table[i][0];
var surname = table[i][1];
var email = table[i][2];
var phone = table[i][3];
clickButton(waitForObject(names.addressBookUnnamedAddQToolButton))
snooze(0.5) // workaround for qt4
type(waitForObject(names.forenameLineEdit), forename)
type(waitForObject(names.surnameLineEdit), surname)
type(waitForObject(names.emailLineEdit), email)
type(waitForObject(names.phoneLineEdit), phone)
clickButton(waitForObject(names.addressBookAddOKQPushButton))
}
});
When("I add new persons to address book", sub {
my %context = %{shift()};
my @table = @{$context{'table'}};
# Drop initial row with column headers
shift(@table);
for my $row (@table) {
my ($forename, $surname, $email, $phone) = @{$row};
clickButton( waitForObject($Names::address_book_unnamed_add_qtoolbutton) );
snooze(0.5); # workaround for qt4
type( waitForObject($Names::forename_lineedit), $forename );
type( waitForObject($Names::surname_lineedit), $surname );
type( waitForObject($Names::email_lineedit), $email );
type( waitForObject($Names::phone_lineedit), $phone );
clickButton( waitForObject($Names::address_book_add_ok_qpushbutton) );
}
});
When("I add new persons to address book") do |context|
table = context.table
# Drop initial row with column headers
table.shift
for forename, surname, email, phone in table do
clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
snooze(0.5) # qt4 workaround
type(waitForObject(Names::Forename_LineEdit), forename)
type(waitForObject(Names::Surname_LineEdit), surname)
type(waitForObject(Names::Email_LineEdit), email)
type(waitForObject(Names::Phone_LineEdit), phone)
clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
end
end
When "I add new persons to address book" {context} {
set table [$context table]
# Drop initial row with column headers
foreach row [lreplace $table 0 0] {
foreach {forename surname email phone} $row break
invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
# qt4 workaround:
snooze 0.5
invoke type [waitForObject $names::Forename_LineEdit] $forename
invoke type [waitForObject $names::Surname_LineEdit] $surname
invoke type [waitForObject $names::Email_LineEdit] $email
invoke type [waitForObject $names::Phone_LineEdit] $phone
invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
}
}
Lets add a new Scenario to the Feature file. This
time we would like to check not the number of entries in address book list, but if this
list contains proper data. Because we enter data into the address book in one
step and verify them in another, we must share information about
entered data among those steps in order to perform a verification.
Scenario: Forename and surname is added to table
Given addressbook application is running
When I create a new addressbook
When I add a new person 'Bob','Doe','Bob@m.com','123321231' to address book
Then previously entered forename and surname shall be at the top
To share this data, the context.userData property can be used.
@When("I add a new person '|word|','|word|','|any|','|integer|' to address book")
def step(context, forename, surname, email, phone):
clickButton(waitForObject(names.address_Book_Add_QToolButton))
type(waitForObject(names.forename_LineEdit), forename)
type(waitForObject(names.surname_LineEdit), surname)
type(waitForObject(names.email_LineEdit), email)
type(waitForObject(names.phone_LineEdit), phone)
clickButton(waitForObject(names.address_Book_Add_OK_QPushButton))
context.userData = {}
context.userData['forename'] = forename
context.userData['surname'] = surname
When("I add a new person '|word|','|word|','|any|','|integer|' to address book",
function (context, forename, surname, email, phone){
clickButton(waitForObject(names.addressBookUnnamedAddQToolButton));
type(waitForObject(names.forenameLineEdit), forename);
type(waitForObject(names.surnameLineEdit), surname);
type(waitForObject(names.emailLineEdit), email);
type(waitForObject(names.phoneLineEdit), phone);
clickButton(waitForObject(names.addressBookAddOKQPushButton));
context.userData["forename"] = forename;
context.userData["surname"] = surname;
});
When("I add a new person '|word|','|word|','|any|','|integer|' to address book", sub {
my $context = shift;
my ($forename, $surname, $email, $phone) = @_;
clickButton(waitForObject($Names::address_book_unnamed_add_qtoolbutton));
type(waitForObject($Names::forename_lineedit), $forename);
type(waitForObject($Names::surname_lineedit), $surname);
type(waitForObject($Names::email_lineedit), $email);
type(waitForObject($Names::phone_lineedit), $phone);
clickButton(waitForObject($Names::address_book_add_ok_qpushbutton));
$context->{userData}{'forename'} = $forename;
$context->{userData}{'surname'} = $surname;
});
When("I add a new person '|word|','|word|','|any|','|integer|' to address book") do |context, forename, surname, email, phone|
clickButton(waitForObject(Names::Address_Book_Unnamed_Add_QToolButton))
type(waitForObject(Names::Forename_LineEdit), forename)
type(waitForObject(Names::Surname_LineEdit), surname)
type(waitForObject(Names::Email_LineEdit), email)
type(waitForObject(Names::Phone_LineEdit), phone)
clickButton(waitForObject(Names::Address_Book_Add_OK_QPushButton))
context.userData = Hash.new
context.userData[:forename] = forename
context.userData[:surname] = surname
end
When "I add a new person '|word|','|word|','|any|','|integer|' to address book" {context forename surname email phone} {
invoke clickButton [waitForObject $names::Address_Book_Unnamed_Add_QToolButton]
invoke type [waitForObject $names::Forename_LineEdit] $forename
invoke type [waitForObject $names::Surname_LineEdit] $surname
invoke type [waitForObject $names::Email_LineEdit] $email
invoke type [waitForObject $names::Phone_LineEdit] $phone
invoke clickButton [waitForObject $names::Address_Book_Add_OK_QPushButton]
$context userData [dict create forename $forename surname $surname]
}
All data stored in context.userData can be accessed in all
steps and Hooks in all Scenarios
of the given Feature. Finally, we need to implement
the step Then previously entered forename and lastname shall be at the top.
@Then("previously entered forename and surname shall be at the top")
def step(context):
test.compare(waitForObjectExists(names.file_0_0_QModelIndex).text,context.userData['forename']);
test.compare(waitForObjectExists(names.file_0_1_QModelIndex).text,context.userData['surname']);
Then("previously entered forename and surname shall be at the top",function(context){
test.compare(waitForObjectExists(names.file00QModelIndex).text,context.userData["forename"], "forname?");
test.compare(waitForObjectExists(names.file01QModelIndex).text,context.userData["surname"], "surname?");
});
Then("previously entered forename and surname shall be at the top", sub {
my $context = shift;
test::compare(waitForObjectExists($Names::file_0_0_qmodelindex)->text, $context->{userData}{'forename'}, "forename?" );
test::compare(waitForObjectExists($Names::file_0_1_qmodelindex)->text, $context->{userData}{'surname'}, "surname?" );
});
Then("previously entered forename and surname shall be at the top") do |context|
Test.compare(waitForObjectExists(Names::File_0_0_QModelIndex).text, context.userData[:forename], "forename?")
Test.compare(waitForObjectExists(Names::File_0_1_QModelIndex).text, context.userData[:surname], "surname?")
end
Then "previously entered forename and surname shall be at the top" {context} {
test compare [property get [waitForObjectExists $names::File_0_0_QModelIndex] text] [dict get [$context userData] forename]
test compare [property get [waitForObjectExists $names::File_0_1_QModelIndex] text] [dict get [$context userData] surname]
}
By recording a snippet that verifies the text
property of the table cells in row 0, columns 0 and 1, we get scriptified
verification points on QModelIndex objects. We replaced
the actual values in the snippet with the stored
context.userData values from the previous step.
Assume our Feature contains the following two Scenarios:
Scenario: State after adding one entry
Given addressbook application is running
When I create a new addressbook
And I add a new person 'John','Doe','john@m.com','500600700' to address book
Then '1' entries should be present
Scenario: State after adding one entry
Given addressbook application is running
When I create a new addressbook
And I add a new person 'Bob','Koo','bob@m.com','500600800' to address book
Then '1' entries should be present
As we can see, those Scenarios perform the same actions using different
test data. The same can be achieved by using a Scenario Outline (a
Scenario template with placeholders) and Examples (a table with
parameters).
Scenario Outline: Adding single entries multiple time
Given addressbook application is running
When I create a new addressbook
And I add a new person '<forename>','<lastname>','<email>','<phone>' to address book
Then '1' entries should be present
Examples:
| forename | lastname | email | phone |
| John | Doe | john@m.com | 500600700 |
| Bob | Koo | bob@m.com | 500600800 |
Please note that the OnScenarioEnd hook will be executed at the end of
each loop iteration in a Scenario Outline.
In the Squish IDE, users can execute all Scenarios in a
Feature, or execute only one selected Scenario. In
order to execute all Scenarios, the proper Test Case has to be
executed by clicking on the button in the Test Suites view.
In order to execute only one Scenario, you need to open the
Feature file, right-click on the given Scenario and
choose . An alternative approach is to click on the
button next to the respective Scenario in
the Scenarios tab in Test Case Resources.
After a Scenario is executed, the Feature file is
colored according to the execution results. More detailed information (like logs) can
be found in the Test Results View.
Squish offers the possibility to pause an execution of a Test Case at any point in
order to check script variables, spy application objects or run custom
code in the Squish script console. To do this, a breakpoint has to be placed before
starting the execution, either in the Feature file at any line
containing a step or at any line of executed code (i.e. in middle of
step definition code).
After the breakpoint is reached, you can inspect all application objects and their properties. If a breakpoint is placed at a step definition or a hook is reached, then you can additionally add Verification Points or record code snippets.
BDD test maintainability can be increased by reusing step definitions in test cases located in another directory. For more information, see collectStepDefinitions().
This chapter is for users that have existing Squish tests and who would like to introduce Behavior Driven Testing. The first section describes how to keep the existing tests and just create new tests with the BDD approach. The second section describes how to convert Script Test Cases into BDD tests.
The first option is to keep any existing Squish tests and extend them by
adding new BDD tests. It's possible to have a Test Suite containing
both Script Test Cases and BDD Test Cases. Simply open existing
Test Suite with test cases and choose
option from drop down menu.
Assuming your existing Test Cases make use of a library and you are calling shared functions to interact with the AUT, those functions can be used in newly created BDD Test Cases also. In the example below, a function is used from multiple Script Test Cases:
def createNewAddressBook():
clickButton(waitForObject(":Address Book.New_QToolButton"))
function createNewAddressBook(){
clickButton(waitForObject(":Address Book.New_QToolButton"));
}
sub createNewAddressBook{
clickButton(waitForObject(":Address Book.New_QToolButton"));
}
def createNewAddressBook
clickButton(waitForObject(":Address Book.New_QToolButton"))
end
proc createNewAddressBook {} {
invoke clickButton [waitForObject ":Address Book.New_QToolButton"]
}
New BDD Test Cases can easily use the same function:
@When("I create a new addressbook")
def step(context):
createNewAddressBook()
When("I create a new addressbook",function(context){
createNewAddressBook()
});
When("I create a new addressbook", sub {
createNewAddressBook();
});
When("I create a new addressbook") do |context|
createNewAddressBook
end
When "I create a new addressbook" {context} {
createNewAddressBook
}
The second option is to convert an existing Test Suite that contains
Script Test Cases into behavior driven tests. Since a Test Suite
can contain Script Test Cases and also BDD Test Cases, migration can be done
gradually. A Test Suite containing a mix
of both Test Case types can be executed and results analyzed without
any extra effort required.
The first step is to review all Test Cases of the existing Test Suite
and group them by the Feature they test. Each Script Test Case will
be transformed into a Scenario, which is a part of a
Feature. For example, assume we have 5 Script Test Cases.
After review, we realize that those Script Test Cases examine two
Features. Therefore, when migration is completed, our Test
Suite will contain two BDD Test Cases, each of them containing one
Feature. Each Feature
will contain multiple Scenarios. In our example, the first
Feature contains three Scenarios and the second
Feature contains two Scenarios.
First, open a Test Suite in the Squish IDE that contains
Script Squish tests that are planned to be migrated to BDD tests. Next, create a
by choosing option from its
drop-down menu.
Each BDD Test Case contains a test.feature file
that can be filled with maximum one Feature. Next, open the
test.feature file to describe the Features
using the Gherkin language. Following the syntax from the template, edit the
Feature name and optionally provide a short description. Next,
analyze which actions and verifications are performed in the Script Test Case that
need to be migrated. This is how an example Test Case for the addressbook
application might look:
import names
import os
def main():
startApplication('"' + os.environ["SQUISH_PREFIX"] + '/examples/qt/addressbook/addressbook"')
invokeMenuItem("File", "New")
table = waitForObject({"type": "QTableWidget"})
test.verify(table.rowCount == 0)
data = [("Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"),
("Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"),
("Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654")]
for oneNameAndAddress in data:
addNameAndAddress(oneNameAndAddress)
waitForObject(table);
test.compare(table.rowCount, len(data), "table contains as many rows as added data")
closeWithoutSaving()
import * as names from 'names.js';
function invokeMenuItem(menu, item)
{
activateItem(waitForObjectItem({"type": "QMenuBar"}, menu));
activateItem(waitForObjectItem({"title": menu, "type": "QMenu"}, item));
}
require 'names.pl';
sub main
{
startApplication("\"$ENV{'SQUISH_PREFIX'}/examples/qt/addressbook/addressbook\"");
invokeMenuItem("File", "New");
my $table = waitForObject({"type" => "QTableWidget"});
test::verify($table->rowCount == 0);
my @data = (["Andy", "Beach", "andy.beach\@nowhere.com", "555 123 6786"],
["Candy", "Deane", "candy.deane\@nowhere.com", "555 234 8765"],
["Ed", "Fernleaf", "ed.fernleaf\@nowhere.com", "555 876 4654"]);
foreach my $oneNameAndAddress (@data) {
addNameAndAddress(@{$oneNameAndAddress});
}
test::compare($table->rowCount, scalar(@data), "table contains as many rows as added data");
closeWithoutSaving();
}
require 'names'
include Squish
def main
startApplication("\"#{ENV['SQUISH_PREFIX']}/examples/qt/addressbook/addressbook\"")
invokeMenuItem("File", "New")
table = waitForObject({:type => "QTableWidget"})
Test.verify(table.rowCount == 0)
data = [["Andy", "Beach", "andy.beach@nowhere.com", "555 123 6786"],
["Candy", "Deane", "candy.deane@nowhere.com", "555 234 8765"],
["Ed", "Fernleaf", "ed.fernleaf@nowhere.com", "555 876 4654"]]
data.each do |oneNameAndAddress|
addNameAndAddress(oneNameAndAddress)
end
Test.compare(table.rowCount, data.length, "table contains as many rows as added data")
closeWithoutSaving
end
source [findFile "scripts" "names.tcl"]
proc main {} {
startApplication "\"$::env(SQUISH_PREFIX)/examples/qt/addressbook/addressbook\""
set table [waitForObject [::Squish::ObjectName type QTableWidget]]
test compare [property get $table rowCount] 0
invokeMenuItem "File" "New"
set data [list \
[list "Andy" "Beach" "andy.beach@nowhere.com" "555 123 6786"] \
[list "Candy" "Deane" "candy.deane@nowhere.com" "555 234 8765"] \
[list "Ed" "Fernleaf" "ed.fernleaf@nowhere.com" "555 876 4654"] ]
for {set i 0} {$i < [llength $data]} {incr i} {
addNameAndAddress [lindex $data $i]
}
waitForObject $table
test compare [property get $table rowCount] [llength $data] "table contains as many rows as added data"
closeWithoutSaving
}
After analyzing the above Script Test Case we can create
the following Scenario, with the first
three steps:
Scenario: Initial state of created address book Given addressbook application is running When I create a new addressbook Then addressbook should have zero entries
Next, right-click on the Scenario and choose the option
from the context menu. This
will create a skeleton of steps definitions:
@Given("addressbook application is running")
def step(context):
test.warning("TODO implement addressbook application is running")
@When("I create a new addressbook")
def step(context):
test.warning("TODO implement I create a new addressbook")
@Then("addressbook should have zero entries")
def step(context):
test.warning("TODO implement addressbook should have zero entries")
Given("addressbook application is running", function(context) {
test.warning("TODO implement addressbook application is running");
});
When("I create a new addressbook", function(context) {
test.warning("TODO implement I create a new addressbook");
});
Then("addressbook should have zero entries", function(context) {
test.warning("TODO implement addressbook should have zero entries");
});
Given("addressbook application is running", sub {
my $context = shift;
test::warning("TODO implement addressbook application is running");
});
When("I create a new addressbook", sub {
my $context = shift;
test::warning("TODO implement I create a new addressbook");
});
Then("addressbook should have zero entries", sub {
my $context = shift;
test::warning("TODO implement addressbook should have zero entries");
});
Given("addressbook application is running") do |context|
Test.warning "TODO implement addressbook application is running"
end
When("I create a new addressbook") do |context|
Test.warning "TODO implement I create a new addressbook"
end
Then("addressbook should have zero entries") do |context|
Test.warning "TODO implement addressbook should have zero entries"
end
Given "addressbook application is running" {context} {
test warning "TODO implement addressbook application is running"
}
When "I create a new addressbook" {context} {
test warning "TODO implement I create a new addressbook"
}
Then "addressbook should have zero entries" {context} {
test warning "TODO implement addressbook should have zero entries"
}
Now we put code snippets from the Script Test Case into respective
step definitions and remove the lines containing
test.warning. If your Script Test Cases make use of shared scripts, you
can call those functions from the step definition as well. For
example, the final result might look like this:
@Given("addressbook application is running")
def step(context):
startApplication("addressbook")
@When("I create a new addressbook")
def step(context):
invokeMenuItem("File", "New")
@Then("addressbook should have zero entries")
def step(context):
table = waitForObject({"type": "QTableWidget"})
test.compare(table.rowCount, 0)
Given("addressbook application is running", function(context) {
startApplication("addressbook");
});
When("I create a new addressbook", function(context) {
invokeMenuItem("File", "New");
});
Then("addressbook should have zero entries", function(context) {
var table = waitForObject({"type": "QTableWidget"});
test.compare(table.rowCount, 0);
});
Given("addressbook application is running", sub {
my $context = shift;
startApplication("addressbook");
});
When("I create a new addressbook", sub {
my $context = shift;
invokeMenuItem("File", "New");
});
Then("addressbook should have zero entries", sub {
my $table = waitForObject({"type" => "QTableWidget"});
test::compare($table->rowCount, 0);
});
Given("addressbook application is running") do |context|
startApplication("addressbook")
end
When("I create a new addressbook") do |context|
invokeMenuItem("File", "New")
end
Then("addressbook should have zero entries") do |context|
table = waitForObject({:type => "QTableWidget"})
Test.compare(table.rowCount, 0)
end
Given "addressbook application is running" {context} {
startApplication "addressbook"
}
When "I create a new addressbook" {context} {
invokeMenuItem "File" "New"
}
Then "addressbook should have zero entries" {context} {
set table [waitForObject [::Squish::ObjectName type QTableWidget]]
test compare [property get $table rowCount] 0
}
Note that the test.log("Create new addressbook") got removed while migrating
this Script Test Case to BDD. When the step
I create a new addressbook is executed, the step name
will be logged into Test Results, so the test.log call would have been
redundant.
After creating the first BDD Test Case in your suite,
a shared/scripts/hooks file is created in the correct
script language. From the IDE, it is located in the
Scripts tab of Test Suite Resources.
Initially, these files contain a hook for OnScenarioEnd, called
when each Scenario is finished. It attempts to ensure that
the AUT is terminated.
@OnScenarioEnd
def hook(context):
for ctx in applicationContextList():
ctx.detach()
OnScenarioEnd(function(context) {
applicationContextList().forEach(function(ctx) { ctx.detach(); });
});
OnScenarioEnd(sub {
foreach (applicationContextList()) {
$_->detach();
}
});
OnScenarioEnd do |context|
applicationContextList().each { |ctx| ctx.detach() }
end
OnScenarioEnd {context} {
foreach ctx [applicationContextList] {
applicationContext $ctx detach
}
}
The above example was simplified for this tutorial. In order to take full advantage of Behavior Driven Testing in Squish, please familiarize yourself with the section Behavior Driven Testing (Section 6.19) in API Reference Manual (Chapter 6).
GUI Coverage is a form of program verification that tests whether all GUI elements of a program were activated, in a test suite. Squish now has support for GUI coverage of Qt Widgets and QtQuick2 elements. There is no support for QtQuick1 (also known as QtDeclarative). This section describes how Squish GUI coverage is activated and used.
By default, GUI coverage is switched off. To activate it, set
the environment variable SQUISH_GUI_COVERAGE to 1. With
any other value, GUI coverage will be switched off.
From the Squish IDE, GUI coverage can be activated in a test suite for
AUTs started with startApplication(). From , select the AUT tab. Then enter the
variable SQUISH_GUI_COVERAGE into the
Environment table and set its value to 1.
| For command-line users |
|---|---|
To run Squish from the command line with GUI coverage enabled,
set the $ export SQUISH_GUI_COVERAGE=1 $ squish/bin/squishserver |
The test cases can be run as usual, but now, there is an extra entry at the end of the result list in the Test Results window, with the title GUI coverage results. In the Message column, the location of the XML file with the test results is shown. This can be useful for debugging.
Double-click the entry, and the uibrowser is started and displays the coverage results.
The UI browser display has five windows:
Element Tree (top left) – A hierarchy of elements is shown, together with their coverage.
GUI coverage is expressed by terms like "3/4 75%". Such an entry means that 3 of 4 required activities were done with the GUI element, 75% of all. In a complex GUI element, like a window or a menu, the numbers refer to the element together with the elements that it contains. In the element tree, the subelements of an elements are displayed hierarchically below it.
Screen Shot (bottom left) – Display of the selected element.
Element Properties (top right) – This windows shows static properties of the selected GUI element.
class is the Qt class that is used to implement the element.
type is an abstract type that describes the element independent of the GUI toolkit that was used.
Usage (middle right) – List of the events registered for the selected element and its subelements.
Each entry in this list refers to one usage counter. Each counter refers to a certain activity of the element. E.g. the counter "shown" is incremented every time an element is displayed, and "clicked" is incremented when an element is clicked. Each type of GUI element has its own list of required events – "clicked" belongs e.g. to menu entries, but not to menus.
Style check (bottom right) – If the design of the element does not follow certain style rules, warnings are displayed in this window.
| For command-line users |
|---|---|
When squishrunner has ended, it prints the location of the results file in a form like
2015-07-07T15:30:55 ATTACHMENT GUI Coverage Result \
addressbook/suite_py/tst_development/coverage_200790021.xml
One can then call the uibrowser directly to see the content of this file: $ squish/bin/uibrowser addressbook/suite_py/tst_development/coverage_200790021.xml
For the full usage of this command, see uibrowser (Section 7.4.10). |
GUI coverage can be configured by changing the settings in the
XML file
SQUISHDIR/lib/qt/extensions/squishqtuicoverage.ext.
The file has three entries which can be set to 0 or 1 to disable or
enable the corresponding feature. By default all are enabled.
enable– enables or disables GUI coverage as a whole.qtwidgets– enables or disables coverage of Qt widgets.qtquick– enables or disables coverage of Qt Quick.takeScreenshotsOfElements– enables or disables taking screen shots of elements. Disable this if you experience memory problems or if you are not interested in them.
You can use qtwidgets and qtquick to disable GUI coverage
partially if there are problems. The use of enable is
however not recommended; use SQUISH_GUI_COVERAGE
instead.