| || |
, 05-16-2016 at 04:17 AM (92 Views)
How to do Interoperability testing?
We can follow the Deeming wheel (the PDCA cycle) to carry out the Interoperability testing.
#1 ) Plan
Planning is the most important phase of determining the strategy of doing almost anything in the software development. Before we actually plan for determining the procedure for doing the IOT, it is imperial that we understand each and every application or system deployed in the network.
We should know for all the applications – its functionality, behavior, input it takes and output it reveals.
I would also recommend that each and every application is fully functionally tested with no defects, before preparing it for the interoperability testing. So when you plan, don’t just think of 1 or 2 application, think of all the application as a single unit. You need to have a birds eye view when planning for this testing technique. Needless to say that – document your plan.
We can use our standard Test Plan document and tailor it a bit as per the requirement to document the planning of IOT. After your test plan is in place, move ahead to derive your test conditions.
The focus of deriving your test condition should not be limited to the individual applications; instead it should be based on the flow of data through all the applications. The conditions should be designed in such a way that, if not all, but most of the applications in the network are traversed.
Once your test conditions are identified, move ahead to design or script (in case you plan to automate) your test cases. You can create a RTM (Requirements Traceability Matrix) to map your test cases with test conditions and your test conditions with acceptance test conditions/requirements.
When you are working on a network, it is again important to plan for the Non Functional testing activities as well. This may not be written or documented anywhere, but it’s mandatory to check for the nonfunctional aspects of the system as a whole. These nonfunctional areas would include performance and security. If required you can create separate plan for Functional testing, performance testing and Security testing; or create a single plan and different document of test conditions for each of these testing types.
#2 ) Do
Do – is the span of time where you actually do your execution. Do plan your time accordingly to execute the functional and non-functional testing. We follow the testing cycle in this phase of executing the cases, logging the defects, following up with development team to get those resolved, doing the re-test and regression test of the system as a whole, reporting the test results and moving it to closure.
#3 ) Check
Check – Is the phase where we revisit our test results and try to map those with the RTMs and validate whether all the expected requirements are met and whether all the applications are traversed. We check that the data is traversed and exchanged correctly and smoothly between the applications/systems. We would also need to validate that the data which is traversed does not gets modified.
Also consider to do a retrospective of the entire process of interoperability testing. Identify the areas which worked well, those which did not go well and any action items that need to be taken care of.
#4 ) Act
Act – Is to act on the retrospective items. The points which were identified as “good practices”, continue executing those and the points which could be worked better, identify the steps to rectify those and act accordingly. Keep 1 thing in mind that the areas or steps which did not work well, should NOT be repeated. After all we should learn from our mistakes and not repeat them.
The 5 ½ Steps:
1. Identify all the applications that are part of the network.
2. Identify their respective functionalities.
3. For each application, identify the Input it takes and the output it returns.
4. Identify those data which would be traversing through all/most of the applications.
5. Identify the expected behavior for each combination of application and date that needs to validated
½ Document it.
Difficult to test all the application with all the permutations and combinations.
Applications are developed in different hardware/software combinations and are installed in different environments, so if any of the environment is down, it impacts the testing.
Because of the different software’s and environments, determining the testing strategy and executing it is itself a big task.
Stimulate the environment for conducting the test, is a big challenge.
In case of any defect, doing the Root Cause Analysis is a big challenge.
Because the applications are in a network, there would be times when the network is down. Because of this, testing also gets affected.
How can I mitigate these challenges?
1) Try to use the advance testing techniques like :
OATS (Orthogonal Array testing technique)
State Transition Diagrams,
Cause and effect graphs
Equivalence portioning and Boundary value analysis.
These techniques would help you to identify the interdependency amongst the application and identify the test cases/conditions that would ensure maximum coverage.
2) Try to identify some historical data like – under which circumstances the systems were down, how much time does it takes to be back in action. In that case try to execute those scenarios whose applications are not impacted, or utilize the time to document the scenarios and report results. Moreover whenever you plan or schedule the testing, always consider these historical data as an input for your estimation and plan accordingly.
3) PLAN – Use historical data, past experiences, skill of the team, environmental factors to identify the strategy of the testing. The better your plan, the better would be your execution.
4) Start working on preparing the environment much before than your actual execution starts. Needless to say- plan your steps when you are preparing the environment. Make sure that your environment is all set, ready and up & running when your execution starts.
5) Before starting with the IOT, ensure that the individual applications are fully functionally tested with no defects. Then in case of any defect, you would only need to look for the environmental factors that have resulted into some error.
6) As discussed in point 2, Plan your activity. If it’s a scheduled outage, you should be considering this downtime when you plan your testing.
Interoperability Test on Mobiles:
In Mobiles, we do interoperability test whenever a new App (Mobile Application) is launched. There are many areas that we have to consider when planning for this testing on Mobile devices:
Types of mobile devices available on market are huge. You would need to list down what all types of devices you would be considering for your testing. You would need to pair a device type along with the OS it supports.
All the Mobile OS are developed in different programming language. Hence, the app needs to be tested against all the variations of OS.
Understanding the legal factors and region related contracts.
Size / resolution of different devices are different.
Impact on the mobile inbuilt apps also needs to be considered.
So for doing IOT on mobiles, you would need plan and create an RTM just like we did for a computer-based application testing.
The intention, strategy, risks, and execution would be same but the tools and techniques would be different in case of mobiles.
Interoperability testing is a huge task. This technique requires proper planning which should start parallel when system test planning starts.
There are lots of factors which need to be considered while executing this technique. Keep in mind to have sufficient time for bug fixing and retesting, as this is a huge effort there should be provision for defect follow-ups.
This may happen that you may not attain 100% coverage, but we should be smart enough to select our cases in such a way that most of the applications are covered in a single flow by using good test case writing techniques.