While it is good practice to design automated tests to be independent, it is not always expeditious. There are situations where the data state of a test environment is difficult to manage or where the ‘stateful’ nature of a transaction requires components to be built one upon another. In these cases, a system of managing test dependencies may be required.
The components of the solution
The mechanism for building a system for managing test dependencies is based on an .ini file that records the test status of each test. The format of the file is simple. Each .ini file section is labeled by the ‘Test Identifier’ that is built into AscentialTest. The value of the key field ‘Status’ is set to either ‘Pass’ or ‘Fail’ as displayed in the example below.
Project Settings
In order to implement the test dependency system described in this paper, the user must enable the ‘Test Identifier’ feature of AscentialTest:
Project Data
Enabling the ‘Test Identifier’ adds a field to the Project Data dialog. The user must provide a value, typically the name of the AscentialTest project:
Plan Data
Enabling the ‘Test Identifier’ also adds a field to the Plan Data dialog. The user must provide a value, typically the name of the AscentialTest plan:
Example: DependentTestAppState
The ‘TestCaseStatus.ini’ file described above is generated in the ‘OnFinish’ action of the AscentialTest appstate:
Notice that the action makes use of the attribute ‘TestId’ which is automatically assigned to each test in every test plan when the ‘Test Identifier’ feature is enabled. If the error count of the current test is equal to 0 (zero), the ‘Status’ written to the ‘TestCaseStatus.ini’ file is ‘Pass’. Otherwise, the value of ‘Fail’ is assigned.
ReturnPrerequisiteTestStatus
The function ‘ReturnPrerequisiteTestStatus’ returns the status of a test by reading the ‘TestCaseStatus.ini’ file using the ‘TestID’ that is passed in as a parameter:
CheckPrerequisiteTestStatus
The step ‘CheckPrerequisiteTestStatus’ is implemented in each test that has a dependency on a prior test. It calls the function ‘ReturnPrerequisiteTestStatus’ with the TestID of the prerequisite test. If ‘true’ is passed in for the parameter ‘RaiseErrorForNotRunTest’, the step raises an error, halts the test and passes execution control back to the ‘OnFinish’ of the appstate. If the value of ‘RaiseErrorForNotRunTest’ is false, the step raises a warning and passes a switch back to the calling test so that the test can halt execution and return to ‘OnFinish’:
Example: TestWithDependency
The image below displays a test that implements the step ‘CheckPrerequisiteTestStatus’. In this example, the user has decided to raise a warning so a conditional statement is required to return to ‘OnFinish’ if the prerequisite test has failed. If the user had decided to raise an error instead, then the conditional statement would not be required:
RemoveTestStatusFile
Removing the ‘TestCaseStatus.ini’ file at the beginning of a test run is a good idea to ensure that the test status from a prior execution is cleared. This can be accomplished by either a test or a step added to the first test in the plan as displayed below. The first option, adding the test ‘RemoveTestStatusFile’ to the plan is easier but it will increase the test count by one. The step which will get dragged into the first test in the plan will not impact the test count, but it does require that the first test always be executed. The choice is up to the user.
Example: Plan with Test Dependencies
The following image displays a plan with a dependent test selected so that the parameters ‘PriorTest’ and ‘RaiseErrorForNotRunTest’ are visible in the ‘Test’ tab below the test plan.
Example: Results file for Plan with Test Dependencies
The image below displays the results file for the sample plan. Notice the difference in the status of the dependent tests, based on whether or not the value ‘RaiseErrorForNotRunTest’ is set to true or false. In the first case, the dependent test displays with the test status of ‘Fail’. In the latter, the test status displays as ‘Pass’. The choice is up to the user.
Conclusion
At some point in the future, it is likely that AscentialTest will have a built-in mechanism for managing dependent tests so that the status of a dependent test will display a status of ‘Not Run’ in the Test Set. In the meantime, the mechanism described above will save time by skipping the execution of tests that would not run successfully due to a failure in a prerequisite test.
