Applause - TestRail Integration is an integration service between the Applause platform and TestRail test case management system to allow the transfer of test cases and results between the two platforms to keep the test cases and test results in sync for analysis and reporting.

The integration is based on TestRail APIs and comes with a few limitations. Please make sure to check the Limitations section below for more information.

Once the integration is successful, you should be able to do the following:

• Transfer test cases between the Applause platform and TestRail
• Transfer test results to TestRail from the Applause platform (with TS team assistance)

To set up the integration, you need the following things:

• Cloud-hosted instance of TestRail (can be referred by a URL) with existing projects
• A TestRail user account for syncing with Applause; we suggest using a separate account only for Applause
• A TestRail API key. To create an API key in TestRail, please refer to this link

 

Setting Up the Applause TestRail Integration

There are four main steps required to integrate TestRail with the Applause platform.

Step 1: Establish the Connection

1. Open Applause platform and select a product.
2. From the left navigation bar expand the Configuration dropdown and open Integrations.
3. Select Data Export and click the “New Connector” button.
4. Provide a connector name and select “TestRail” as the connector type.
5. Provide your TestRail URL, user account email and the API key.
6. Click “Test & Establish Connection”.
   
   • If the connection is successful, the next step with product selection and field mapping sections will become available.
   • If the connection is not successful, check your TestRail instance and API key details.

Step 2: Required Fields

Next, at the beginning of step 2, the TestRail project needs to be mapped to the Applause Products. 

1. Select the TestRail project from the list.
2. Next, select whether to include attachments as URLs only, or as files as well during the export.

Note: You need to create one connector for each TestRail project. Also, remember an Applause Product can be part of only one TestRail Project.

Moving on, map each Applause test case result field to a relevant TestRail field - review the fields and test results mapping carefully.

Next, map each Applause test case required field to a relevant TestRail field.

• Select “None” if there’s no relevant TestRail field for a particular Applause field and provide a default value for the specific field

Then, map Applause step fields to TestRail fields.

Next, map the TestRail required fields to Applause fields.

• You need to map the Priority field as well as the values between TestRail and Applause, so that the Priority value can be correctly interpreted and assigned to the test case. The Priority field can take one of the following values: Critical, High, Medium, Low.

In the next step (step 3: Optional Fields), map the optional Applause fields to TestRail fields if you want and vice versa - TestRail optional fields to Applause fields below that.

• In the Applause optional fields there’s the References field. The Reference value is a single text field with values separated by commas. Each value before the comma is read and added to the Reference integration value and added to the Reference value field. Reference Integration Applause specifies which system the value comes from.

Your TestRail instance is now successfully integrated with the Applause platform.

Limitations

• Applause TestRail connector uses TestRail APIs based on TestRail release 7.1 and above. We recommend connecting to the cloud based TestRail instance versus on-premise installation which might have an older TestRail version
• Applause TestRail connector can map a single TestRail project to one or more Applause products. However, you can use multiple connectors if you have multiple TestRail projects
• Once the Project to Products mapping of a TestRail connector is saved, it cannot be changed. If any changes are needed, then the connector should be deleted and a new connector needs to be set up
• The TestRail connector doesn’t provide any conflict detection or resolution when the same test case has different changes across the systems. The imported test case will override the existing one
• Transferring test results feature is not available in the Applause platform. Consult with your SDM if you want to transfer test results to TestRail

Best Practices

• We strongly recommend that you establish your primary system (either TestRail or Applause platform) as a system of truth and make all the changes in that system and update the changes to the other system through the connector. If there are different changes made in the two systems, then when a sync happens, the changes will be overwritten in the destination system
• There is not a clear one-to-one mapping of result steps between TestRail and Applause. In Applause, we have Step Instruction, Expected Result, Status, Actual Result and attachments for a step. To match this correctly, we recommend you to create custom Step Results fields, which will result in a separate field for Actual Result and Status for each step:

  "custom_step_results": [
  {
  "content": "Step 1",
  "expected": "Expected Result 1",
  "actual": "Actual Result 1",
  "status_id": 1
  },
  

Connector Logs

Connector synchronization logs show details for all test case synchronization jobs performed with a selected connector.

To view a list of logs for a certain connector:

1. Select the connector to view logs for.
2. Specify the date range to access the logs.
3. Select one or more statuses of the logs - Success, Failure and Partial Success.
   1. Partial Success can happen when a test case was successfully imported from TestRail and created in Applause but some of the test case attachments weren’t imported properly.

Each line in the log shows the log for a synchronization job. When clicking on a specific job, the tasks are shown in detail to make it easier to understand where the failure is.

To view more details on the specific synchronization task, click on the task log.