This section describes how to run Selenic from the command line so that you can automate Selenic as part of your continuous test execution process. In this section:
Introduction
Parasoft Selenic is implemented as two .jar files. Run the selenic_agent.jar file first as part of your continuous test execution process to capture information about the test execution. Next, run the selenic_analyzer.jar file to analyze the data captured by the agent.
The analyzer will generate recommendations based on what Selenic has learned about your tests and application. You can review the report and manually apply the recommendations, or enable the self-healing feature so that the Selenic agent automatically applies recommendations and/or heals wait conditions during successive test executions. This enables automated UI testing activities to continue without affecting downstream processes.
If you are using the Selenic IDE plug-in, both processes (capturing test execution behavior and analyzing the data) are performed under a single action. The following video provides an overview of how to use Selenic on the command line.
Capturing Test Execution Data
Specify and configure the selenic_agent.jar file using the -javaagent JVM argument when executing your tests to teach Selenic your test logic. The captureDom option must be enabled to capture test execution data. At least one successful test execution is required for Selenic to provide recommendations.
Java Executable
The following example demonstrates how the Selenic agent may be run using the Java executable:
java -javaagent:/path/to/selenic_agent.jar=captureDom=true -cp /path/to/junit.jar;/path/to/folder/containing/test/classes junit.textui.TestRunner com.mypackage.MyTestClass
You can also run Cucumber projects by calling the Cucumber CLI runner (io.cucumber.core.cli.Main) instead of the JUnit or TestNG runner. Additionally, you must specify the feature file to execute and the glue code, for example:
java -javaagent:/path/to/selenic_agent.jar=captureDom=true -cp /path/to/cucumber-core.jar;/path/to/folder/containing/glue/classes io.cucumber.core.cli.Main /path/to/feature.feature --glue com.mygluecode
Refer to the Cucumber CLI documentation for additional usage information: https://cucumber.io/docs/cucumber/api/#running-cucumber
Maven Commands
The command line examples in this documentation use Maven commands:
mvn test -DargLine=-javaagent:/path/to/selenic_agent.jar=captureDom=true
You can leverage additional Selenic functionality by configuring the Java agent argument using the following format:
mvn test -DargLine=-javaagent:/path/to/selenic_agent.jar=arg1=value1,arg2=value2
See Selenic Agent Options for details.
Selenic will save the test execution data to the <USER_HOME>/parasoft/recorded_data directory. Run the Selenic analyzer to analyze the data and create recommendations for updating any bad locators.
Test execution results will be saved to the <PROJECT>/target/surefire-report directory.
Example Commands
The following examples show how to use the available Selenic Agent Options.
Capture DOM and Take Screenshots During Execution
mvn test -DargLine=-javaagent:/Applications/parasoft_selenic/selenic_agent.jar=captureDom=true,screenshot=failures
Create API Tests and Self-heal Bad Locators and Wait Conditions
In this example, the main selfHealing option is enabled, which enables Selenic to apply default self-healing functionality to locators and wait conditions. The test must have been executed successfully at least once with the Selenic Java agent to automatically heal bad locators. There are no prerequisites for healing wait conditions.
The createApiTests option has the following prerequisites:
- A SOAtest server must be running
- The Parasoft Web Proxy must be running
mvn test -DargLine=-javaagent:/Applications/parasoft_selenic/selenic_agent.jar=createApiTests=true,proxyPort=40090,soatestHost=localhost,soatestPort=9080,username=admin,password=admin,selfHealing=true
The API Tests will be created in the SOAtest server's workspace.
Integrating with Applitools
Applitools is a third-party tool that captures screenshots or "checkpoints" during test execution and displays them in a web application so that you can visually verify UI changes. You can configure Selenic to call the Applitools API automatically when running tools with the Selenic agent. Checkpoints will be captured before each Selenium navigation action, before each Selenium click action, and at the end of each test.
Prerequisites
The Applitools JAR files must already be on the classpath of the project that contains the Selenium tests. An error message will print to the console if the JAR files are not present, but the tests will continue to run.
The APPLITOOLS_API_KEY environment variable must be configured within the CI system or passed as a Java system property to the process that executes the tests. Refer to the Applitools documentation for instructions on how to add the environment variable:
https://applitools.com/docs/topics/integrations/github-integration-ci-setup.html
Refer to the Applitools documentation for instructions on how to obtain an API key:
https://applitools.com/docs/topics/overview/obtain-api-key.html
Usage
Run the Selenic agent with the applitools and applitoolsApplicationName arguments to execute your tests with Applitools:
mvn test -DargLine=-javaagent:/Applications/parasoft_selenic/selenic_agent.jar=captureDom=true,applitools=true,applitoolsApplicationName=MyApplication
After test execution, the checkpoints will appear in the Applitools web application so you can use Applitools functionality.
You can simultaneously use Applitools and the Selenic screenshots option, but Applitools is intended to provide additional visual testing functionality.
Viewing Results
Run the Selenic analyzer to generate the HTML report (see Generating Recommendations). No additional arguments are necessary. Information about visual changes captured by Applitools appear in the Selenic HTML report. See Viewing Results from Applitools for details.
Executing Tests Affected by Change
You can leverage Selenic's test impact analysis functionality to determine which tests are affected by changes to the application under test. See Test Impact Analysis for details about using this functionality.
Generating Recommendations
Run the selenic_analyzer.jar file to generate recommendations after you have executed your tests with Selenic. The analyzer will analyze the data in the <USER_HOME>/parasoft/recorded_data directory and generate recommendations for failed locators. The report Selenic produces will also contain recommendations for wait conditions if self-healing for wait conditions was enabled. If the -allLocators option is used, then recommendations will be generated for all passed and failed locators.
java -jar /path/to/selenic_analyzer.jar
You can specify additional arguments to customize how Selenic outputs the results of its analysis. See Selenic Analyzer Arguments for information about all available arguments.
java -jar /path/to/selenic_analyzer.jar -<arg> <arg_value>
Selenic Agent Options
You can specify the following options during test execution with Selenic:
- applitools
- applitoolsApplicationName
- captureDom
- collectCoverage
- coverageAgentHost
- coverageAgentPort
- coverageAgentUserID
- coverageDir
- createApiTests
- data
- proxyPort
- soatestHost
- soatestPort
- secure
- username
- password
- maxSessionDaysToAnalyze
- maxSessionDaysToKeep
- screenshot
- selfHealing
- selfHealLocators
- selfHealWaitConditions
- additionalWaitTimePercent
- minAdditionalWaitTime
- sessionId
applitools
This option enables Selenic to integrate with Applitools. When set to true, the applitoolsApplicationName options are required.
Default is false.
applitoolsApplicationName
This option specifies the name of the application you are testing. The name specified is used in the App field in the Applitools web application UI so that you can filter by application.
This option is required if the applitools option is true .
captureDom
This option enables Selenic to capture the DOM when Selenium attempts to locate an element. This option is required for Selenic to automatically heal bad locators and/or generate recommendations for locators.
Default is false
collectCoverage
This option enables Selenic to collect coverage data for test impact analysis during test execution. Refer to Test Impact Analysis for details.
Default is false
coverageAgentHost
This option specifies the host where the AUT and coverage agent are running so that coverage data can be captured for test impact analysis during test execution. Refer to Test Impact Analysis for details.
Default is localhost
coverageAgentPort
This option specifies the port where the AUT and coverage agent are running so that coverage data can be captured for test impact analysis during test execution. Refer to Test Impact Analysis for details.
Default is 8050
coverageAgentUserID
This option specifies a user ID so that different users can be distinguished when collecting coverage in multi-session mode. Refer to Test Impact Analysis for details.
coverageDir
This option specifies a local directory for coverage data to be downloaded from the AUT server for test impact analysis during test execution. Refer to Test Impact Analysis for details.
createApiTests
This option enables Selenic to create API tests using Parasoft SOAtest during UI test execution.
Default is false
Only one .tst file can be created at a time if you enable this option. If you are running tests concurrently, API tests will only be created for one scenario at time. The following error will be shown in the log for other scenarios running at the same time:
[ERROR] ParasoftSelenicAgent - Failed to start web proxy session: A session is currently in process. Wait for the session to finish before starting another one.
The test framework should be configured to run in a single thread (one test scenario at a time) in order to capture API tests for all test scenarios.
data
This option specifies the location of the data recorded during test execution. The -data option for the Selenic analyzer should be configured to match the value configured for this option.
Default is <USER>/parasoft/recorded_data
proxyPort
If you are creating API tests during UI test execution, specify the port number for the SOAtest Web Proxy.
Refer to the Parasoft Recorder documentation for additional information.
Default is 40090
soatestHost
If you are creating API tests during UI test execution, specify the host where the SOAtest server is running.
Refer to the Parasoft Recorder documentation for additional information.
Default is localhost
soatestPort
If you are creating API tests during UI test execution, specify the port number for the SOAtest Server.
Refer to the Parasoft Recorder documentation for additional information.
Default is 9080
secure
If you are creating API tests during UI test execution, this option enables Selenic to communicate with the SOAtest server over HTTPS.
Default is false
username
If you are creating API tests during UI test execution and user authentication is enabled for SOAtest server, this option specifies the user name for logging into the server.
password
If you are creating API tests during UI test execution and user authentication is enabled for SOAtest server, this option specifies the password for logging into the server.
maxSessionDaysToAnalyze
This option indicates how many days worth of test sessions to include when applying the selfHealLocators option during execution. By default, there is no limit, so Selenic will analyze all available recorded data.
In the following example, the Selenic agent will use 12 days worth of recorded data to determine the best option for automatically healing a failed locator or wait condition.
maxSessionDaysToAnalyze=12
The number of days refers to the most recent days for which data was recorded, regardless of how many calendar days have passed.
maxSessionDaysToKeep
This option indicates how many days worth of test session data to keep on disk.
Default is 90.
In the following example, the Selenic agent will keep 12 days worth of recorded data on disk:
maxSessionDaysToKeep=12
The specified value refers to days for which data was recorded and not the number calendar days.
screenshot
This option enables the test machine to take a screenshot for any WebDriver commands Selenic logs.
You can set one of the following values:
all- A screenshot for eachfind elementcall in the Selenium test is capturedfailures- Only screenshots for failingfind elementcalls are capturednone- No screenshots are captured
Default is none.
selfHealing
This option enables self-healing functionality. When set to true, the selfHealLocators and selfHealWaitConditions options are enabled.
Default is false.
selfHealLocators
This option enables Selenic to automatically attempt to update a locator at runtime based on historical information collected during test execution.
Default is true when selfHealing is enabled.
selfHealWaitConditions
This option enables Selenic to extend Selenium wait conditions to prevent timeouts or other types of errors due to insufficient wait conditions during test execution.
Default is true when selfHealing is enabled.
additionalWaitTimePercent
This option specifies how much additional time a wait condition should be extended to prevent a timeout or other errors related to insufficient wait conditions during test execution. The value specified with this option is a percentage of the original wait condition. Selenic will calculate the additional time based on the percentage and extend the Selenium tests' wait conditions. In the following example, a wait condition originally configured for 4 seconds will be extended by 50% to 6 seconds.
additionalWaitTimePercent=50
Default is 50 when selfHealing and selfHealWaitConditions are enabled.
minAdditionalWaitTime
This option specifies a minimum number of seconds by which original wait conditions will be extended to prevent a timeout or other errors related to insufficient wait conditions during test execution. The value for this option overrides the value dynamically set with the additionalWaitTimePercent option if the resulting percentage is less than the minimum.
Default is true when selfHealing and selfHealWaitConditions are enabled.
sessionId
This option specifies an identifier for test execution data. This setting is optional and should only be used when running tests across multiple projects so that Selenium data from all projects can be grouped under a single session. Otherwise, Selenium data for each project will be recorded under separate session IDs. As a result, the analyzer will need to run for each test execution in a project before you can run tests in a different project.
Example:
mvn test -DargLine=-javaagent:/path/to/selenic_agent.jar=captureDom=true,sessionId=ID_1
When analyzing the data, use the -sessionId option when calling the selenic_analyzer.jar file to reference the ID set during test execution. For example:
java -jar /path/to/selenic_analyzer.jar -sessionId ID_1
By default, a date and time stamp is used.
Selenic stores execution data in the directory specified with the data property. Each subdirectory within the recorded data directory contains execution data for individual sessions. The subdirectories are named with the sessionId value. If the sessionId parameter is not configured, Selenic will store the data in a subdirectory identified with a timestamp by default.
If you do not specify a session ID, then you must run the analyzer each time you run tests in a project before running tests in a different project. A session ID should be different for each test run so that results from an earlier test run to do get combined with later test runs. Therefore, using a build identifier or the default timestamp is recommended.
Selenic Analyzer Arguments
The following options are supported:
- -allLocators
- -data <path_to_recorded_data>
- -encodepass <password>
- -help
- -machineId
- -maxSessionDaysToAnalyze <value>
- -performanceBenchmarking
- -performanceBenchmarkingMinRuns <number_of_runs>
- -performanceBenchmarkingPercent <percent_threshold>
- -publish
- -report <path_to_filename>
- -sessionId <value>
- -settings <location_of_properties_file>
- -source <directory_or_file>
- -surefire <path_to_file>
-allLocators
Use this option to get locator recommendations for all actions in Selenium tests. By default, Selenic only provides locator recommendations for failed locators. Recommendations for locators that succeeded will only be visible in the JSON report and not in the HTML report.
-data <path_to_recorded_data>
This option specifies the location of the recorded data. It should be configured to match the location specified with the data option when running the Selenic agent.
Default is <USER>/parasoft/recorded_data
-encodepass <password>
Use this option to print an encoded password that you can use in the selenic.properties file. See Licensing.
An encoded string will be printed to the console in the following example that can be used in place of myPassword:
java -jar selenic_analyzer.jar -encodepass myPassword
-help
This option prints the list of supported arguments to the console.
-machineId
This option prints the machine ID to the console. Parasoft requires the machine ID to issue a local license. See Licensing.
-maxSessionDaysToAnalyze <value>
This option indicates how many days worth of test sessions to include when analyzing recorded data to generate recommendations. By default, there is no limit, so Selenic will analyze all available recorded data.
In the following example, the Selenic analyzer will use 12 days worth of recorded data to generate locator or wait condition recommendations.
java -jar selenic_analyzer.jar -maxSessionDaysToAnalyze 12
The number of days refers to the most recent days for which data was recorded, regardless of how many calendar days have passed.
-performanceBenchmarking
This option enables Selenic to calculate and report test execution performance benchmarks. Selenic collects and reports test method execution times, and when this option is enabled Selenic calculates the average execution duration per test method based on a minimum set of test runs. Test methods that exceed the average duration by a specified percentage are flagged in the report. Only runs for passing tests are included in the calculation when determining the average.
The minimum number of test runs required to determine the average, as well as the threshold for exceeding the average duration are configurable. See -performanceBenchmarkingMinRuns and -performanceBenchmarkingPercent for information about configuring the performance benchmarking parameters.
This option is disabled by default.
In the following example, performance benchmarking is enabled, the -performanceBenchmarkingMinRuns option is set to 5, and the -performanceBenchmarkingPercent option is set to 20.
java -jar selenic_analyzer.jar -performanceBenchmarking -performanceBenchmarkingMinRuns 5 -performanceBenchmarkingPercent 20
After five test runs, Selenic determines that a test method averages 20 seconds to complete. On the sixth run, the test takes 24.2 seconds to complete, which is 21% over the average. The test will be flagged in the HTML report. See Viewing the Report for information about how performance benchmarking is reported.
-performanceBenchmarkingMinRuns <number_of_runs>
This option sets the minimum number of test runs required for test methods that exceed the percentage set with the -performanceBenchmarkingPercent option to be flagged in the report. The average test duration serves as the baseline for the performance benchmarking functionality. See -performanceBenchmarking for details about the performance benchmarking functionality.
Default is 5
-performanceBenchmarkingPercent <percent_threshold>
This option sets the test execution duration threshold for the current test run against the average execution time. If tests in the current run exceed the average test duration by more than the specified percentage, tests will be flagged in the HTML report. See -performanceBenchmarking for details about the performance benchmarking functionality.
Default is 20
-publish
Use this option to publish test results to your project in DTP. The DTP connection and project settings must be configured (see Reporting Results to DTP).
-report <path_to_filename>
This option specifies the location and name of the report file to be generated.
Absolute paths to a directory/file are supported.
Examples:
java -jar selenic_analyzer.jar -report /home/reports
java -jar selenic_analyzer.jar -report "C:\reports\my-report.html"
The following location is used by default:
"{WORKING_DIR}/report.html"
-sessionId <value>
This option enables you to specify the ID of the test session that contains the recorded data to process. If not specified, the data from the last recorded session is used.
-settings <location_of_properties_file>
Use this option to specify a .properties configuration file containing the execution options for the selenic_analyzer.jar. The .properties file contains licensing and reporting configurations that can be implemented during Selenic analysis. Refer to the following sections for additional information:
-source <directory_or_file>
This option enables you to specify the location of source files or a directory containing source files you want to publish to DTP. You can use multiple -source flags to specify a set of directories or files. This option has no effect if Selenic is not configured to publish sources to DTP (see Publishing Source Files to DTP).
Default is the current working directory.
Example:
java -jar selenic_analyzer.jar -source C:\my_project1 -source C:\my_project2
-surefire <path_to_file>
This option enables you to specify the location of the Surefire XML report. The Surefire XML report is created when running tests in Maven and is used to create the HTML report.
Absolute paths to a directory/file are supported.
The following wildcards are also supported:
** - Use two asterisks to match directories.
* - Use a single asterisk to match file names.
Examples:
-surefire "/home/folder"
-surefire "C:\folder\file.xml"
In the following example, Selenic will find XML files in /folder1/folder2/folder3, /folder2/folder3, and /folder3:
-surefire "**/folder3"
In the following example, Selenic will find all XML files prefixed with "ProjectA" in folder3:
-surefire "**/folder3/ProjectA*.xml"
The following location is used by default:
"**/target/surefire-reports/"
