Testing WSO2 Identity Server Product Compatibility with Integration Tests.

Shanika Wickramasinghe
6 min readMar 7, 2022



Are you familiar with Integration Testing. Lets See How we can verify a product stability or compatibility against different infrastructures with the help of integration tests.

What is Integration Testing

Integration testing — also known as integration and testing (I&T) — is a type of software testing in which the different units, modules or components of a software application are tested as a combined entity.

Integration testing involves integrating the various modules of an application and then testing their behaviour as a combined, or integrated, unit. Verifying if the individual units are communicating with each other properly and working as intended post-integration is essential.

Lets Run WSO2 Identity Server Integration Test Suite

If you want to run the integration tests against a pack which is below IS 5.11.0 then you need to follow the wum guide and get a GA pack updated to latest wum version.

If you want to run the integration tests against a pack which is equal and above IS 5.11.0 then latest updates are not available in the WUM channel. You need to get the GA pack updated from U2

In this blog post I will take IS 5.11.0 to describe the required steps.

  1. Get the IS 5.11.0 GA pack downloaded from the website (If you want to get the pack updated with U2 then dont download the pack from Github. Because update manager tool is not zipped in the packs released for Github.)
  2. Install latest maven version in the host machine where you will be executing the integration tests.


If maven version is 3.6 and above

Then you dont need to build the dependency repositories

If maven version is below 3.6

You might face dependency issues when running the integration tests. To resolve them. Get the below repositories cloned from support branch and get them build in the host machine.

Navigate to https://github.com/wso2-support/product-is

Click on tags and check the latest tag for IS 5.11.0. In my case its v5.11.0.115

  1. Get the orbit repository cloned
git clone https://github.com/wso2-support/orbit.git

Take the master branch

Navigate to root pom file residing folder path

Execute the maven build skipping the tests

mvn clean install -Dmaven.test.skip=true

Note — You will get a build failure for this repo. Dont mind still you will get the required dependencies for your maven local repository.

2. Get the carbon framework

git clone https://github.com/wso2-support/carbon-identity-framework.git

git checkout support-5.18.187.x-full ( You can check which branch you need to checkout by going through the readme of this repository. It has mentioned which version is compatible with IS 5.11.0.x. )

3. Get the kernel

git clone https://github.com/wso2-support/carbon-kernel.gitgit checkout support-4.6.1.x-fullmvn clean install -Dmaven.test.skip=true

4. Get the Identity Inbound Oauth

git clone git clone https://github.com/wso2-support/identity-inbound-auth-oauth.gitgit checkout support-6.4.111.x-fullmvn clean install -Dmaven.test.skip=true


Other repositories such as kernel,framework,inbound-oauth should give a build success when you build the repository skipping the tests.

Now you are done with all the required dependencies

Prerequisites to run the Integration Tests

Java Installed and JAVA HOME set correctly

Maven installed

Required Dependency repositories built (if maven version is below 3.6)

How To Run Integration Tests

  1. Get the product IS repository cloned from support branch
git clone https://github.com/wso2-support/product-is.gitgit checkout v5.11.0.115cd product-ismvn clean install -Dmaven.test.skip=true

Note — First you need to build the repository from the root pom skipping the tests. Once you successfully build this there will be a folder generated as


Here you can see a wso2is-5.11.0.zip file generated

2. Now delete the generated wso2is-5.11.0.zip and copy the wum updated wso2is-5.11.0.zip pack which you got updated with U2 to here.

3. Now navigate to below folder location

cd home/shanika/product-is/modules/integration/tests-integration/tests-backend

4. Run the integration tests with nohup

nohup mvn clean install

5. If you want to see the integration test running log open another terminal and navigate to the same folder location and tail the log with

tail -1000f nohup.out

How To Debug a Integration Test to identify test failures.

  1. Navigate to below file path
cd modules/integration/tests-integration/tests-backend/src/test/resources/automation.xml

2. Open automation.xml from a code editor and comment out below line

<parameter name=”cmdArg” value=”debug 5005" />

Note :- If you want to quit out from debug mode you need to comment this line again after your debug is completed.

3. Then get the source code cloned for your host machine and open it using Intellij Idea

4. Click on Edit Configurations > + icon > Remote Debug option and add configure debug support > Apply > OK

4. Now open the java file related to the test you want to debug and add debug points in the required places by right clicking on the start location of each lines.

5. Now Navigate to below location and run the tests with debug mode using below command

cd product-is/modules/integration/tests-integration/tests-backendmvn -Dmaven.surefire.debug test

You can see a log as listening to 5005 from the test running terminal logs

6. Now come back for intellij idea and click on the Debug Icon or from the menu select Run > Debug option

When the debug point get hits you can see the required values through the intellij IDEA

How to run only one specific integration test

This can be done by editing the testng.xml file located in the below directory


Open the testng.xml file located in this path using a code editor

Remove the unnecessary classes and keep only the test class you want to get executed

When removing other tests you need to keep these mandatory tests which are prerequisites for any test without removing

Now you can replace the testng.xml file with the updated one for the location where you are executing the integration tests and run the integration tests.

How to run Integration Tests with Nohup

Navigate to below directory

cd product-is/modules/integration/tests-integration/tests-backend/nohup mvn clean installyou can tail the logs using below commandtail -1000f nohup.out

Checking the surefire Reports

Once you execute the tests navigate to below folder path

cd product-is/modules/integration/tests-integration/tests-backend/target/surefire-reports

Open the index.html file to view the test run results

Also you can get the test run log by reading the nohup.out file

Now You are almost done with testing the WSO2 Identity Server Product Compatibility using the Integration Test Suite

[1]. https://wso2.com/identity-server/



Shanika Wickramasinghe

Senior Software Engineer and Freelance Technical Writer. I write about any Computer Science related topic. https://www.linkedin.com/in/shanikawickramasinghe