Configuring Jmeter Tests in VSTS/TFS (Azure DevOps) and publishing Results

4 min readSep 17, 2018

Running Performance Tests part of CI/CD is now a days essential. Whenever we make changes to the code its important ensure our code changes doesn’t affect performance. JMeter is widely used tool for performance testing. Running JMeter Tests as a part of CI/CD helps us to understand our application performance instantly.

Step by Step Guide to Run JMeter Tests in VSTS/TFS (AzureDevOps)

Pre-Requisites

i) Download and install Java JDK/SDK on the VM where your TFS agents are running

ii) Add Java Environment Variable JAVA_HOME

iii) Download and install the JMeter Set up on the VM where your TFS agents are running (Ex:D:\apache-jmeter-4.0)

iv) Download Saxon Parser (http://saxon.sourceforge.net, Choose Saxon-HE) Java Version and Unzip on VM (Ex: D:/saxon)

v) Copy the .xsl file from https://github.com/ganeshsirsi/Resources/blob/master/jm.xsl to your VM (Ex: D:\Jmeter\xslt\jm.xsl)

JMeter Project Settings.

i). Add Summery Result Listener to your JMeter Project.

ii). Choose Configure save as .xml

iii) Mention xml Result file Name in JMeter tool (ex:D:\jmeter\Results\Result.xml)

Awesome!!! Now you are done with your set up

Configuring TFS Build

Create CI Build by Navigating to Repo

Give the Name for your CI and Set the Agent Queue,

2. Now Add the Agent Phase as shown below

3. Add Task to Agent Phase, Search for Command Line Task and Add

4. Name First Task as Initial Set Up ,Enter Tool as setx and Arguments as JMETER_HOME “D:\apache-jmeter-4.0”. Please note that you should specify the JMeter directory here in my case its D:\apache-jmeter-4.0.

5. Repeat Step 3 to add one more command Line Task. Mention Name as Run Jmeter Test, Tool as D:\<path_to_Jmeter_bin_Folder>\jmeter and Arguments as -n -f -t MyJmeterTest.jmx (.jmx is your Jmeter Project file)

6. Repeat Step 3 add Once again Command Line Task. Name it as Create Report, Tool as java and Arguments as -jar <path_to_Saxon_Jar_File> -s:<Path_to_JmeterGeneratedResult.xml_File> -xsl:<path_to_copied.xsl_File> -o:<Output_File_path_to.xml>

ex: -jar D:/saxon/saxon9he.jar -s:D:\jmeter\Results\Result.xml -xsl:D:\jmeter\xslt\jm.xsl -o:D:\jmeter\Junit\JmeterJunitReport.xml

-s is the source file name, this file is generated by Jmeter (see JMeter Project Settings in this article)
-xsl is Junit xslt (jm.xsl) file, which you copied from the github (see Prerequisites point number v)
-o is output file and folder Name, this will be used to publish result in Next Step

7. Add Task “Publish Results in TFS”. Name it as Publish Results , Test result format as JUnit, Test results files as JmeterJunitReport.xml (The output file Name in Step 6) and finally add Search folder location as D:\jmeter\Junit\ (The output folder location which you mentioned in Step 6)