In many cases the application to test is not physically located on the tester’s desktop, which requires that the generated application data, such as code coverage information, gets transferred from the target to a host system.

This can be realized in different ways:

1. By a manual transfer through a physical storage medium (USB stick, HDD, …), which is not convenient,
2. By mounting a shared network drive from the target and the host system using it as location for all generated files,
3. By a dedicated transfer mechanism.

In general, the most convenient way is to use a shared network drive on which the target system generates the coverage information, which then can be analyzed directly on the host system. This is not always possible on embedded systems or due to security requirements. (Most of the remote file systems are not secure enough for using them over the internet. In this case, it is necessary to use a VPN or SSH.) For this reason, Squish Coco provides a simple way to customize CoverageBrowser in order to automate the download and analysis of the coverage traces.

What is necessary?

To use the automatic download feature, CoverageBrowser requires two console commands, one for fetching the content of the execution report (usually a file on the target machine with the extension .csexe) and one optional command for removing it on the target.

The fetch command has to print the content of the execution report to the standard output and return the exit code 0 on success. Any other exit code will be recognized as an error and stops the process. The standard error output is displayed on the terminal and can be used for debugging purposes. Also, if the command requires a user interaction (for example, a confirmation or a password), it is possible to enter it directly in the terminal window.

Here are some examples of possible fetch commands:

• ‘cat app.csexe’: The same as importing a the file app.csexe directly.
• ‘ssh target.com "cat app.csexe"’: Import the execution report app.csexe from the computer target.com with SSH.

The command for removing the execution report on the target is optional and is executed only upon a successful import. As for the fetch command, a null exit code is expected and the error output can be used for debugging.

Here are some examples of possible removal commands:

• ‘rm app.csexe’: The same as when using the file mode. The file app.csexe is removed after a successful import.
• ‘ssh target.com "rm app.csexe"’: Remove the execution report app.csexe from the computer target.com with SSH.

A working example

Let us suppose that we have an instrumented server application app.exe at the address x.x.x.x that can be compiled to generate a report with the signal SIGUSR1. We will write a script which generates an execution report, fetches it and removes it automatically after the import.

The implementation could be:

• ‘ssh target.com "killall -SIGUSR1 app.exe; sleep1; cat app.exe.csexe"’: This will generate the execution report through a signal and import it from the computer target.com by SSH.
• ‘ssh target.com "rm app.exe.csexe"’: This will remove the imported execution report on target.com.

First open the import dialog, by clicking on the menu entry File->”Load Execution Report”. The following dialog appears:

For the first execution, we activate script debugging by clicking on the “Options” button and selecting “Debug output for scripts”. This will have the following effect:

1. The complete output of the scripts will be printed in the terminal output. The standard output is printed in black and should contain only the content of the execution report, whereas the standard error output is printed in blue.
2. The output terminal will not be closed when the script finishes successfully.

Now we click on “Next”. The script is then executed:

The text in black is the content of the execution report. It is only displayed when the debug mode is activated, to avoid performance issues. The text in blue is the standard error output, and the exit code displayed in green at the end indicates that the command was successful. This page is only visible because the debug mode is activated: When the exit code is zero, CoverageBrowser jumps directly to the next page if the debug mode is deactivated.

A click on “Next” then shows the imported coverage data:

Here a summary of the imported executions is displayed. In our case, it shows that the coverage of one test called ‘Test Name’ is imported. It is now possible to cancel this import, then nothing will be imported. We can also press the “Back” button, which lets us return to the initial page, where we may redo the import with other settings. But if we accept the import by clicking on “Import & Delete”, the cleanup script will be executed. We see then the following screen, but only because the debug mode is active. (Otherwise, the dialog would be automatically closed since the script was successful.)

Conclusion

With this simple customization using scripts, it is possible to automate the collection of coverage information on remote systems or on embedded devices.

The post Coco Tip: Manual testing on a remote system with Coco appeared first on froglogic.