By default, Squish supports the Tesseract OCR engine. In order to use the Tesseract engine, you need to install the Tesseract for Squish package for your operating system:

The packages are available in our download center. The Tesseract package installer can register the chosen path with Squish. If that step was omitted during installation, the path to the engine can be specified on the OCR Preferences pane, or in the ocr.ini (Section 7.6.1.2) file.

Squish supports multiple OCR engines, but each require a few steps before they can be used.

Both the OCR.Space and Amazon Rekognition engines require the authentication keys that indentify a particular user of the service. The instruction for obtaining the keys for these engines can be found here and here. The authentication data needs to be specified in the OCR Preferences pane, or in the ocr.ini (Section 7.6.1.2) file.

While we would normally automate the push of the Start new game button through a property-based object access (mouseClick(waitForObject(":Start_Button"))) we will now use Optical Character Recognition (OCR) to interact with that button.

Start New Game

The chess figures are not controls recognized by Squish, so we are forced to use their pixel representation. The first image cut out from the chessboard will be the white pawn on square C2:

white_pawn.png

At this point there are still eight such pawns on the board, with four of them placed on the same white background that we have captured. Upon accepting the selection Squish will search the desktop screenshot, find all positions matching the selection and compute the occurrence index. That index will be recorded as a part of the waitForImage call. Therefore, it is important to select the specific image of interest, even if there are other identical occurences of it on the screen.

White pawn image selection

A click on square C4 will perform the move. Once we activate the Insert > mouseClick(<Image>) action (Section 8.1.3.8) () we get to choose previously recorded images:

Image Reuse or Creation

The image of the pawn is not applicable for reuse at this point. We will use the New Image... button to record an image for the empty square at C4 using the same steps as above.

Recording search images that consist only of a flat background is not recommended. In order to avoid that, the selected image should include the margins of neighboring squares.

white_square.png

As before, other empty white squares will be considered, including the occurrence index recorded as the occurrence parameter to the waitForImage function.

The next recorded move is a pawn going from E2 to E4. We activate the Insert > mouseClick(<Image>) action (Section 8.1.3.8) () , and select the previously created white_pawn file. Because there are multiple pawns still visible on the board, Squish will open the Image Search Preview dialog (Section 8.3.6) with several matches marked with a red frame.

Selection of Image Occurrence

This time the occurrence index recorded in the test script will be based on the selected image.

The same steps can be repeated to use the chess_square.png file to click on the square E4.

Once the Image-based action insertions are done, we press the Stop Recording action () in the Control Bar. The generated script will look like this (or equivalent in your chosen scripting language):

function main() {
    startApplication("chess2012");
    mouseClick(waitForOcrText("Start new game"));
    mouseClick(waitForImage("white_pawn", { occurrence: 2 }));
    mouseClick(waitForImage("chess_square", { occurrence: 7 }));
    mouseClick(waitForImage("white_pawn", { occurrence: 3 }));
    mouseClick(waitForImage("chess_square", { occurrence: 4 }));
}

At this point we have automated a test without having access to internals of the chess application. The drawback of this low barrier is the reliance on a particular visual appearance of the application.

In situations where small changes in the appearance of the application occur, Squish can introduce a tolerance to the image search in order to find matches against previously captured images.

The Chess 2012 application that we used to record the test script scales its contents according to the amount of available space. By maximizing the application window we can resize the entire chessboard.

function main() {
    startApplication("chess2012");
    mouseClick(waitForImage("start_game"));
    mouseClick(waitForImage("white_pawn"));
}

If the above test case is executed with the Chess 2012 application's window maximized, the image search will fail, and the IDE will show the Image Not Found dialog (Section 8.3.4).

Image Not Found Dialog

We could try to select the Attempt to fix image search errors automatically. checkbox and allow Squish to enable the tolerant image search mode and adjust its parameters. The automatic fix procedure is not perfect and risks relaxing the image search to include incorrect matches. Instead, we will click the Adjust Search Parameters button that opens the Image Search Preview dialog (Section 8.3.6). The dialog will attempt to lower the threshold until any matches are found. It displays the image search parameters and allows manipulating it to observe the search preview according to currently set values.

Image Search Preview

For now, the automatically computed values should be correct, and we can confirm them by clicking the Set as Defaults button. This will set the default image search parameters to the values displayed above and continue the test script execution.

The default values of the image search parameters can be edited on the Image Search (Section 8.2.16.5) tab of the Test Suite Settings view (Section 8.2.16).

So far the sole purpose of image searches was a later interaction, like a Insert > mouseClick(<Image>) action (Section 8.1.3.8). The mere existence (or absence) of an image can also be be used for the purpose of testing the GUI state, to answer questions like: Does a warning sign appear? Is a status icon red or green? Or (in the case of a test for our chess application): which player's turn is it?

To verify that it is the white player's turn we will first cut out an image encompassing the text label It is white's turn. by recording, for example, a mouseClick on the label:

Selection of the whites_turn.png image

The recorded script statement will be:

mouseClick(waitForImage("whites_turn.png"));

To turn this into a check we'll drop the mouseClick call and reduce ourselves to verifying that the image search by itself succeeds:

try {
   waitForImage("whites_turn.png", {timeout: 1000})
   test.pass("It is white's turn");
} catch (e) {
   test.fail("It is not white's turn. Must be black's.")
}

Note how we reduced the maximum time we were willing to wait from 20 seconds to 1 second. This is to speed up the verification in case of a negative result. When we do not need to wait for an update of the UI before performing the test, we can simply use findImage. It will error out right away on failure to find the specified image.

Also note that a failure to find a specific image does not rule out with 100% safety that the information is not displayed with a differing visual appearance. The absence of one image does not always mean that another state of pixels is present for sure. It is therefore prudent to have the failing check for one image to be followed by the check for its expected alternative. In other words, a failed check for the display reading that it is white's turn is not sufficient for when we expect a visual representation of the text It is black's turn. We would need the expected alternative.