How to Configure postman / newman API tests in Azure DevOps or TFS and Publish HTML Results?

Sample Postman/Newman Test Report

Postman is one of the easy and fastest tool for API Testing in the previous article we have covered how to write api tests using postman?. Once we write the postman API tests we need to configure it to CI or Build. In this article I will answer your two questions i.e

1. How to configure postman API tests to Azure DevOps or TFS using newman?

2. How to publish HTML Results in Azure DevOps or TFS?

Pre-requiste: You have created a set of postman tests we call it as postman api tests, That is postman test collection.

Exporting your test collection and Push it into Repository

Step 1: Export postman Tests Collection

a. In your postman tool Test Collection (looks like a folder) on the left Side you see 3 dots (…) Click on that, from the menu choose to export

b. A window Export Collection appears

c. Choose a specific folder and export it
d. Once you click on save it will be saved as .json file

Step 2: Export Environment Variables (Optional)

Note: Use this step if you are using an environment variable in your tests else skip this step.

a. Top Right corner Postman Tool, you see the gear icon Click on that.

b. Once you click on that you see Manage Environment Window

c. Click on download to save the environment as the JSON file

After Completion to Step 1 and Step 2, you have exported your postman collection and postman environment to a specific folder.

Now, Check-in or Push this to your git Repository

Using newman tool to run the postman API test in the command line

What is Newman?

Newman is a command-line collection runner for Postman. It allows you to effortlessly run and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems. Read more about newman here (https://www.npmjs.com/package/newman)

Configuring Postman API Tests to Newman tool in Azure DevOps / TFS Build (CI) Pipeline

Step 1: Create a build pipeline in Azure DevOps/TFS

Note: If you are using TFS directly navigate to builds

a) Navigate to your Azure DevOps website

b) From the Left side, menu choose Pipelines and click on the Builds

c. Create a New build pipeline

d. Next, Choose to Use the classic editor

e. Choose your Repository

f. From the template Choose an empty job

g. Name your pipeline and Choose Agent

i. Ensure your Get Sources pointing to correct repository

Click on the Get Sources modify the following

Repsitoty : Should be pointing to correct repository where your postman test collection is locatedDefault branch : Choose the correct branch (typically this will be master branch)Clean : Set the clean option to true (default is false, modify from the dropdown)Clean Options : Set clean option to Source 

j. Modify Agent job 1 property

Display Name : Any name which you like (ex: Postman API Testing)Agent pool : Choose the correct Agent pool from the drop down

Step 5: Adding build tasks to Azure DevOps / TFS build pipeline

a. Add Command Line Steps to install newman in your build agent

i. Click on + button in your agent panel (Refer Image)
ii. Search for task Command Line (Refer Image)
iii. Click on Add

b. Configure Command Line task to install new man

Display name : Mention task name as Install newmanScript : npm install -g newmanClick on Advanced section and choose Working directoryWorking Directory : $(System.DefaultWorkingDirectory)

Note: $(System.DefaultWorkingDirectory) is in built Azure DevOps variable which will point to your source code (Know more about Azure DevOps Variable)

c. Add task to run Postman tests in newman

i. Add one more command line task as discussed above

ii. Modify newly added second command line task

Display name : Run API TestsScript :newman run mytest.postman_collection.json -e myenv.postman_environment.json --reporters cli,junit --reporter-junit-export Results\junitReport.xml Working Directory : Choose Correct Directory where your tests are located in git repository and folder (You can optionally use option "..." (3 dots right to this textbox manually search the folder)Control Options: Continue on Error (Check the Checkbox)

Note: Few lines about Script, the command we entered in the script is to run the postman tests in newman tool

If you are not using the enviroment variable simply use like below
newman run <your_postman_collection.json> --reporters cli,junit --reporter-junit-export Results\junitReport.xml
If you are using envirinment variable you need to mention the environment variable json file which you exported at the begining use like below
newman run <your_postman_collection.json> -e <your_environmentfile.json> --reporters cli,junit --reporter-junit-export Results\junitReport.xml
If you are using data driven testing with data file use the -d option like below
newman run <your_postman_collection.json> -e <your_environmentfile.json> -d <your_datafile.csv or .json> --reporters cli,junit --reporter-junit-export Results\junitReport.xml

What happens after the execution of this step?

1. Executes All the postman tests
2. Creates a directory called Results in the Working Folder which you mentioned in option
3. Creates junitReport.xml Results File inside the Results Folder

In the above command — reporters cli,junit — reporter-junit-export Results\junitReport.xml This line is responsible for generating result. Results is folder name and junitReport.xml is file name (Optionally you can modify it to any name).

— reporters cli,junit With this line we are specifying output as junit and command line interface both. because we want to see the result in both command line and html format.

d. Add a task to Publish the Result

Click on The + button on the Agent panel

Search for the Task Publish Test Results and Click on ADD

e. Modify Publish Result Task

Display name : Publish Test ResultsTest Result Format : JunitTest Result Files : $(System.DefaultWorkingDirectory)\QA\Results\*.xml (In your case it might be different this should point to the file where your junitReport.xml file is located in the previous step.)Upload test results files: Check the Checkbox

Please note that you will see that “unable to find the filename after execution if this path is wrong”, so please always do a double check on this.

f. Queue the build now.

Step 4: Verifying the results after execution

If you have configured everything properly, as expected all the tasks should pass and will look like below

a. Viewing the Test Summary

Click on the Tests Tab to view the Test Summary

b. View Test Execution Details

i. Click on Logs Tab

ii. Click on the Publish Test Results Task

iii. Now Publish Test Results Window appears.

iv. Click on Published Test Run link as shown in image

The link will take you to the Result Details page

Click on the Test Results Tab to view each test result

That’s All!!!

--

--

--

https://www.linkedin.com/in/ganeshsirsi/

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

MacOS setup automation with Homebrew

Chatbot using django rest framework + api.ai + slack — Part 1/3

How to Build Ultra -low Latency Data Center Networks

Nvidia Web Driver Mac Os High Sierra

Nvidia web driver mac os high sierra 10 13 6

Connect Environments with Alibaba Cloud Express Connect — Part 2: Use Cases

Anatomy of an Event Streaming Platform — Part 2

Get started with programming the right way

Designing Data Intensive Applications: Consistency with Linearizability

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ganesh Hegde

Ganesh Hegde

https://www.linkedin.com/in/ganeshsirsi/

More from Medium

How to build a REST API with ASP.NET Web API

API Testing in Node.js — using Chai, Mocha and Github Actions

Connecting to Aws RDS database with IAM authentication in NHibernate with C# .net

Hosting ASP.NET