How to Configure postman / newman API tests in Azure DevOps or TFS and Publish HTML Results?

7 min readMay 16, 2019

Postman is one of the easy and fastest tool for API Testing in the previous article we have covered how to write api tests using postman?. Once we write the postman API tests we need to configure it to CI or Build. In this article I will answer your two questions i.e

1. How to configure postman API tests to Azure DevOps or TFS using newman?

2. How to publish HTML Results in Azure DevOps or TFS?

Pre-requiste: You have created a set of postman tests we call it as postman api tests, That is postman test collection.

Exporting your test collection and Push it into Repository

Step 1: Export postman Tests Collection

a. In your postman tool Test Collection (looks like a folder) on the left Side you see 3 dots (…) Click on that, from the menu choose to export

b. A window Export Collection appears

c. Choose a specific folder and export it
d. Once you click on save it will be saved as .json file

Step 2: Export Environment Variables (Optional)

Note: Use this step if you are using an environment variable in your tests else skip this step.

a. Top Right corner Postman Tool, you see the gear icon Click on that.

b. Once you click on that you see Manage Environment Window

c. Click on download to save the environment as the JSON file

After Completion to Step 1 and Step 2, you have exported your postman collection and postman environment to a specific folder.

Now, Check-in or Push this to your git Repository

Using newman tool to run the postman API test in the command line

What is Newman?

Newman is a command-line collection runner for Postman. It allows you to effortlessly run and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. Read more about newman here (https://www.npmjs.com/package/newman)

Configuring Postman API Tests to Newman tool in Azure DevOps / TFS Build (CI) Pipeline

Step 1: Create a build pipeline in Azure DevOps/TFS

Note: If you are using TFS directly navigate to builds

a) Navigate to your Azure DevOps website

b) From the Left side, menu choose Pipelines and click on the Builds

c. Create a New build pipeline

d. Next, Choose to Use the classic editor

e. Choose your Repository

f. From the template Choose an empty job

g. Name your pipeline and Choose Agent

i. Ensure your Get Sources pointing to correct repository

Click on the Get Sources modify the following

Repsitoty : Should be pointing to correct repository where your postman test collection is locatedDefault branch : Choose the correct branch (typically this will be master branch)Clean : Set the clean option to true (default is false, modify from the dropdown)Clean Options : Set clean option to Source

j. Modify Agent job 1 property

Display Name : Any name which you like (ex: Postman API Testing)Agent pool : Choose the correct Agent pool from the drop down

Step 5: Adding build tasks to Azure DevOps / TFS build pipeline

a. Add Command Line Steps to install newman in your build agent

i. Click on + button in your agent panel (Refer Image)
ii. Search for task Command Line (Refer Image)
iii. Click on Add

b. Configure Command Line task to install new man

Display name : Mention task name as Install newmanScript : npm install -g newmanClick on Advanced section and choose Working directoryWorking Directory : $(System.DefaultWorkingDirectory)

Note: $(System.DefaultWorkingDirectory) is in built Azure DevOps variable which will point to your source code (Know more about Azure DevOps Variable)

c. Add task to run Postman tests in newman

i. Add one more command line task as discussed above

ii. Modify newly added second command line task

Display name : Run API TestsScript :newman run mytest.postman_collection.json -e myenv.postman_environment.json --reporters cli,junit --reporter-junit-export Results\junitReport.xml Working Directory : Choose Correct Directory where your tests are located in git repository and folder (You can optionally use option "..." (3 dots right to this textbox manually search the folder)Control Options: Continue on Error (Check the Checkbox)

Note: Few lines about Script, the command we entered in the script is to run the postman tests in newman tool

If you are not using the enviroment variable simply use like below
newman run <your_postman_collection.json> --reporters cli,junit --reporter-junit-export Results\junitReport.xmlIf you are using envirinment variable you need to mention the environment variable json file which you exported at the begining use like below
newman run <your_postman_collection.json> -e <your_environmentfile.json> --reporters cli,junit --reporter-junit-export Results\junitReport.xmlIf you are using data driven testing with data file use the -d option like below
newman run <your_postman_collection.json> -e <your_environmentfile.json> -d <your_datafile.csv or .json> --reporters cli,junit --reporter-junit-export Results\junitReport.xml

What happens after the execution of this step?

1. Executes All the postman tests
2. Creates a directory called Results in the Working Folder which you mentioned in option
3. Creates junitReport.xml Results File inside the Results Folder

In the above command — reporters cli,junit — reporter-junit-export Results\junitReport.xml This line is responsible for generating result. Results is folder name and junitReport.xml is file name (Optionally you can modify it to any name).

— reporters cli,junit With this line we are specifying output as junit and command line interface both. because we want to see the result in both command line and html format.

d. Add a task to Publish the Result

Click on The + button on the Agent panel

Search for the Task Publish Test Results and Click on ADD

e. Modify Publish Result Task

Display name : Publish Test ResultsTest Result Format : JunitTest Result Files : $(System.DefaultWorkingDirectory)\QA\Results\*.xml (In your case it might be different this should point to the file where your junitReport.xml file is located in the previous step.)Upload test results files: Check the Checkbox

Please note that you will see that “unable to find the filename after execution if this path is wrong”, so please always do a double check on this.