Testing WSO2 Identity Server Product Compatibility with Integration Tests.
--
Are you familiar with Integration Testing. Lets See How we can verify a product stability or compatibility against different infrastructures with the help of integration tests.
What is Integration Testing
Integration testing — also known as integration and testing (I&T) — is a type of software testing in which the different units, modules or components of a software application are tested as a combined entity.
Integration testing involves integrating the various modules of an application and then testing their behaviour as a combined, or integrated, unit. Verifying if the individual units are communicating with each other properly and working as intended post-integration is essential.
Lets Run WSO2 Identity Server Integration Test Suite
If you want to run the integration tests against a pack which is below IS 5.11.0 then you need to follow the wum guide and get a GA pack updated to latest wum version.
If you want to run the integration tests against a pack which is equal and above IS 5.11.0 then latest updates are not available in the WUM channel. You need to get the GA pack updated from U2
In this blog post I will take IS 5.11.0 to describe the required steps.
- Get the IS 5.11.0 GA pack downloaded from the website (If you want to get the pack updated with U2 then dont download the pack from Github. Because update manager tool is not zipped in the packs released for Github.)
- Install latest maven version in the host machine where you will be executing the integration tests.
Note
If maven version is 3.6 and above
Then you dont need to build the dependency repositories
If maven version is below 3.6
You might face dependency issues when running the integration tests. To resolve them. Get the below repositories cloned from support branch and get them build in the host machine.
Navigate to https://github.com/wso2-support/product-is
Click on tags and check the latest tag for IS 5.11.0. In my case its v5.11.0.115
- Get the orbit repository cloned
git clone https://github.com/wso2-support/orbit.gitTake the master branch
Navigate to root pom file residing folder path
Execute the maven build skipping the tests
mvn clean install -Dmaven.test.skip=trueNote — You will get a build failure for this repo. Dont mind still you will get the required dependencies for your maven local repository.
2. Get the carbon framework
git clone https://github.com/wso2-support/carbon-identity-framework.gitgit checkout support-5.18.187.x-full ( You can check which branch you need to checkout by going through the readme of this repository. It has mentioned which version is compatible with IS 5.11.0.x. )
3. Get the kernel
git clone https://github.com/wso2-support/carbon-kernel.gitgit checkout support-4.6.1.x-fullmvn clean install -Dmaven.test.skip=true
4. Get the Identity Inbound Oauth
git clone git clone https://github.com/wso2-support/identity-inbound-auth-oauth.gitgit checkout support-6.4.111.x-fullmvn clean install -Dmaven.test.skip=true
Note
Other repositories such as kernel,framework,inbound-oauth should give a build success when you build the repository skipping the tests.
Now you are done with all the required dependencies
Prerequisites to run the Integration Tests
Java Installed and JAVA HOME set correctly
Maven installed
Required Dependency repositories built (if maven version is below 3.6)
How To Run Integration Tests
- Get the product IS repository cloned from support branch
git clone https://github.com/wso2-support/product-is.gitgit checkout v5.11.0.115cd product-ismvn clean install -Dmaven.test.skip=true
Note — First you need to build the repository from the root pom skipping the tests. Once you successfully build this there will be a folder generated as
product-is/modules/distribution/targetHere you can see a wso2is-5.11.0.zip file generated
2. Now delete the generated wso2is-5.11.0.zip and copy the wum updated wso2is-5.11.0.zip pack which you got updated with U2 to here.
3. Now navigate to below folder location
cd home/shanika/product-is/modules/integration/tests-integration/tests-backend4. Run the integration tests with nohup
nohup mvn clean install5. If you want to see the integration test running log open another terminal and navigate to the same folder location and tail the log with
tail -1000f nohup.outHow To Debug a Integration Test to identify test failures.
- Navigate to below file path
cd modules/integration/tests-integration/tests-backend/src/test/resources/automation.xml2. Open automation.xml from a code editor and comment out below line
<parameter name=”cmdArg” value=”debug 5005" />
Note :- If you want to quit out from debug mode you need to comment this line again after your debug is completed.
3. Then get the source code cloned for your host machine and open it using Intellij Idea
4. Click on Edit Configurations > + icon > Remote Debug option and add configure debug support > Apply > OK
4. Now open the java file related to the test you want to debug and add debug points in the required places by right clicking on the start location of each lines.
5. Now Navigate to below location and run the tests with debug mode using below command
cd product-is/modules/integration/tests-integration/tests-backendmvn -Dmaven.surefire.debug test
You can see a log as listening to 5005 from the test running terminal logs
6. Now come back for intellij idea and click on the Debug Icon or from the menu select Run > Debug option
When the debug point get hits you can see the required values through the intellij IDEA
How to run only one specific integration test
This can be done by editing the testng.xml file located in the below directory
product-is/modules/integration/tests-integration/tests-backend/src/test/resourcesOpen the testng.xml file located in this path using a code editor
Remove the unnecessary classes and keep only the test class you want to get executed
When removing other tests you need to keep these mandatory tests which are prerequisites for any test without removing
Now you can replace the testng.xml file with the updated one for the location where you are executing the integration tests and run the integration tests.
How to run Integration Tests with Nohup
Navigate to below directory
cd product-is/modules/integration/tests-integration/tests-backend/nohup mvn clean installyou can tail the logs using below commandtail -1000f nohup.out
Checking the surefire Reports
Once you execute the tests navigate to below folder path
cd product-is/modules/integration/tests-integration/tests-backend/target/surefire-reportsOpen the index.html file to view the test run results
Also you can get the test run log by reading the nohup.out file
Now You are almost done with testing the WSO2 Identity Server Product Compatibility using the Integration Test Suite
