Life is a Non-Deterministic Finite State Automata
Automation ? (*pGeekiness)++ : Code /eneration;

June 1, 2014

Triksy v1.0 released on Github (PowerShell advanced functions for processing MsTest TRX results)

Filed under: Automation,BrekIT,Programming,Testing — admin @ 12:46 am

Triksy v1.0 has been released. It is open source and can be downloaded from the Github Repository.

What is it?
Triksy is intended to help you query, aggregate and reshape MsTest TRX files to simplify working with other test management tools or custom reporting solutions.

And it was a cool way to learn PowerShell Advanced Functions and Pester (used for testing) :-)

The following section shows some examples of using Triksy (the Github repository contains all of the test data to produce these results – just run these commands from the Functions\TestData folder).

To return the test result from every TRX file:

Get-ChildItem "*.trx" -Recurse | Get-Trx | Get-TrxResult | Format-Table -Property TestName,Outcome,Path -Autosize

Sample output is:

TestName                   Outcome      Path
--------                   -------      ----
WorkItemPassesAndFailsPass Passed       F:\Github\Triksy\Functions\TestData\Valid\MsTestAttributes.trx
AlwaysPass                 Passed       F:\Github\Triksy\Functions\TestData\Valid\MsTestSingleResult.trx
AlwaysPass                 Passed       F:\Github\Triksy\Functions\TestData\Valid\MsTestSummary1.trx
AlwaysPass                 Passed       F:\Github\Triksy\Functions\TestData\Valid\MsTestSummary2.trx
AlwaysPass                 Passed       F:\Github\Triksy\Functions\TestData\Valid\MsTestTwoResults.trx
... etc

To get summary of all tests run (passed, failed, inconslusive etc) in each Trx file, execute:

Get-ChildItem *.trx -Recurse | Get-TrxSummary | Format-Table -Property Valid,Outcome,Total,Executed,Passed,Error,Failed,Timeout,Aborted,Inconclusive,Path -Autosize

The sample output might look like this:

Valid Outcome   Total Executed Passed Error Failed Timeout Aborted Inconclusive Path
----- -------   ----- -------- ------ ----- ------ ------- ------- ------------ ----
False                                                                           F:\Github\Triksy\Functions\TestData\...
 True Failed    16    16       12     0     3      0       0       1            F:\Github\Triksy\Functions\TestData\...
 True Completed 0     0        0      0     0      0       0       0            F:\Github\Triksy\Functions\TestData\...
 True Completed 20    19       18     17    16     15      14      13           F:\Github\Triksy\Functions\TestData\...
 True Completed 20    19       18     17    16     15      14      13           F:\Github\Triksy\Functions\TestData\...
 True Completed 120   119      118    117   116    115     114     113          F:\Github\Triksy\Functions\TestData\...
 True Failed    2     2        1      0     1      0       0       0            F:\Github\Triksy\Functions\TestData\...

It is often useful to aggregate results across many files – for example, the use of the -Aggregate switch in the above command line will sum the totals in every TRX file:

Get-ChildItem *.trx -Recurse | Get-TrxSummary -Aggregate | Format-Table -Property Valid,Total,Executed,Passed,Error,Failed,Timeout,Aborted,Inconclusive -Autosize

Will produce a single line of output:

Valid Total Executed Passed Error Failed Timeout Aborted Inconclusive
----- ----- -------- ------ ----- ------ ------- ------- ------------
    6   178      175    167   151    152     145     142          140

The Workitem Attribute
A very useful feature of the MsTest framework is the [Workitem(id)] attribute you can add to your test methods – this is a way of correlating a particular test with a test case or user story in your ALM or test management tool of choice. The same Workitem id can be used on multiple test methods so you typically want to know if all of the tests associated with a particular workitem passed. The Workitem attribute is output in the TRX file by MsTest so it is just a scripting effort to correlate the result of that unit test with the associated work item in your ALM/Test Management tool of choice.

Triksy provides a CmdLet to do the aggregation Get-TrxWorkitemSummary. If every test associated with the same Workitem id passes, the result is Passed. If any of the results are NOT Passed (including: Error, Failed, Aborted, Inconclusive etc) then the overall status of that Workitem is ‘Failed’:

Get-ChildItem *.trx -Recurse | Get-Trx | Get-TrxResult | Get-TrxWorkitemSummary

Will produce a single Pass/Fail for each Workitem:

Workitem                                                    Outcome
--------                                                    -------
4000                                                        Failed
3000                                                        Failed
1000                                                        Passed
1001                                                        Passed
2000                                                        Passed
2001                                                        Failed
2002                                                        Failed

It is sometimes useful to associate all unit tests that do NOT have a [Workitem] attribute with the same Workitem as a catch all. To do this, provide the -DefaultWorkitems “ID” parameter to the Get-TrxWorkitemSummary.

Grey Ham

Powered by WordPress