Currently we are able to run tests on following environment types: vm, k8s, helm

Please, refer to this article if you don't have VM environment -  /wiki/spaces/RedboxHome/pages/2562764310, (skip if you already have an environment)

Ways how to run tests

1. From Azure Devops
2. From Visual Studio using Test Explorer

Test framework structure

On the top we have solution which contains multiple projects, every project contains at least one test class and every test class contains at least one test method (test scenario)

Below is the example of how it looks in code

We can run following sets of tests:

1. All tests in the solution
2. Tests for specific project
3. Test class (feature)
4. Test method (scenario)
5. Smoke set of tests - all tests in the ApiAutomation solution marked with SmokeTest attribute

Running tests from Azure Devops

There 5 builds depending on suite of tests:

Single test case - https://redboxdev.visualstudio.com/Nubis/_build?definitionId=381
Single test class - https://redboxdev.visualstudio.com/Nubis/_build?definitionId=372
Project - https://redboxdev.visualstudio.com/Nubis/_build?definitionId=258
Smoke tests - https://redboxdev.visualstudio.com/Nubis/_build?definitionId=313
All tests - https://redboxdev.visualstudio.com/Nubis/_build?definitionId=187

To run all tests trigger build Api tests regression

1. Open build page 
2. Click button "Run pipeline" 
   
   
   
   
3. Select branch from the dropdown list:
   
   
   
   
   
4. Open "Variables" tab and provide "environmentTag" and "envKind" values:
   
   EnvironmentTag is a part of virtual machine environment URL:
   
   
   
   vm: http://dev-vm-TAG.westeurope.cloudapp.azure.com 
   helm: https://TAG.dev.redboxdev.com
   k8s: 

   List of existing kubernetes environments is described on this page - Environments

   Based on Environments page there's environments.json file in ApiAutomation solution

   environments.json file contains mapping between IP address and environmentTag
   
   Environment kind (envKind) is one of following environment types: vm, k8s, helm

5. 
   
   
   
6. Click "Run" button

To run tests for specific project trigger build Api test run for single project

1. Open build page
2. Click button "Run pipeline" 
3. Select branch, environmentTag, envKind and projectName
   
   projectName value should be equal to one of project names in the ApiAutomation solution, e.g. AuditApiTest, AnnotateCallTest.....
   
   
   
   
4. Click button "Run"

To run tests for specific test class (feature) trigger build Api tests single feature

1. Open build page
2. Click "Run pipeline" button
3. Select branch, environmentTag, envKind and test class name (feature)
   
   feature parameter should be equal to test class name
   
   
   
   
4. Click "Run" button

Running smoke tests - Api tests run smoke test suite

1. Open build page
2. Click "Run pipeline" button
3. Select branch branch, environmentTag, envKind
   
   
   
   
4. Click "Run" button

Running tests from Visual Studio Test Explorer

Steps to run tests:

1. Clone AutomationTests repository
2. Switch to the branch you're interested in (you might have to do a fetch through [Team Explorer > Synchronisation > Fetch])
3. Go to AutomationTests\ApiAutomation
4. Open ApiAutomation solution in Visual Studio
5. Configure RBR package source in the Visual Studio if it's not configured, please refer to the article "How to add package source"
6. Build the solution (ensure that there are no errors else reach out to automation engineers)
7. In the ApiTestCommon project find "appsettings.common.json" file
8. In the "appsettings.common.json" file replace placeholder "%envHost%" to the target environment tag, e.g: "fenrir3" or "fenrir4
9. In the "appsettings.common.json" file replace ip "172.16.250.14" to the ip of the target environment
10. In the "appsetting.k8s.json" file set "environmentTag" to the target environment tag, e.g: "fenrir3" or "fenrir4
11. In the "appsetting.environment.json" file add the ip of the target environment
12. Re-build the solution
13. Open Test Explorer
14. In Test Explorer right lick on project/class/method to run specific tests or click "Run All" link to run all tests

In order to run Smoke tests:

1. Open Test Explorer
   
   
   
2. Click on "Open playlist file"
   
   
   
3. Select "smoke_tests.playlist" file from ApiAutomation folder in the AutomationTests repository
4. There're only smoke tests will be displayed in the Test Explorer
   

Sign Off

• Vikash Mahabir (Unlicensed)
• Jayasri (Unlicensed)
• Baz Chougule (Unlicensed)
• Denis Shevchenko (Deactivated)
• Abdullah Arshad (Unlicensed)
• Kirill Martyshchenko (Unlicensed)
•