SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 9 of 9
  1. #1
    Junior Member
    Join Date
    Jan 2003
    Location
    United Kingdom
    Posts
    7
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    In-house testing of outsourced development

    Would apprecaite any views on this situation:

    Corporation oursources bits development work - outsourcer does analysis, design, code, unit test and system test;
    then, corporation's in-house test group have to undertake "operational acceptance testing" - functional and non-functial to assure business user deaprtment and operations department that the delivered solution is fit for purpose.

    How much duplication of test effort do you think there could be?
    How well will the outsourcer test (if he's got delivery targets to meet)?
    How does the in-house test group influence the development testing?

    Thanks in advance for any comments

  2. #2
    Senior Member
    Join Date
    Jan 2003
    Posts
    1,898
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    There may be a great deal of duplication, but rerunning the same tests is desirable from environment to environment.

    The outsourcing organization may do a great job of testing and they may be terrible - just like any other company.

    The outsourcing organization and the in-house testing organization can both benefit from each other, but a great deal depends on how your contract with the outsourcer spells out quality requirements.

    You can require the outsourcing organization to submit test requirements, cases, and results to your in-house testing group. This may cut down on their workload, as they would not have to document cases already documented. Your contract can also specify that the outsourcing group's code pass a "smoke test" before you accept it for testing. In this case, the smoke test, produced by your in-house testing organization, should be submitted to the outsourcing organization well in advance of the turnover date. Your contract can also specify that that the outsourcing organization has to pass acceptance tests (or a pre-defined percentage thereof) run by
    your organization prior to payment. Again, if you would like to positively influence the quality of the code you receive, I would advise you to submit all of your test requirements/cases to the outsourcing organization as far in advance as possible. It has been my experience that when outsourcers know what tests will be run, they make sure their code passes those tests. This increases quality and cuts costs for your organization.

    And in case the in-house organization fears they will become superfluous, I have never found this to be the case. There are usually many last-minute fixes made to code and the final run-through by the in-house organization usually finds plenty of problems...

    - Linda



    ------------------

  3. #3
    Moderator
    Join Date
    Aug 2000
    Location
    Vancouver, BC, Canada
    Posts
    1,189
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    We have quite a bit of experience testing software developed outside. When we first started we thought that a fairly straightforward confirmation test/acceptance test would be enough. When we started testing however, we found that the external company had not done a very thorough job of testing and we quickly had to switch into system test mode. In just a few weeks time we reported over 350 defects some of which required significant rework to fix. The post-mortem of this project showed that earlier and more testing from our side was absolutely necessary.

    Later exercises greatly boosted our role and we basically do a full system test and then a full blown acceptance test with the business clients regardless of how it was tested before. Actually we do not even use the suppliers test scripts and tests because we found them fairly shallow and a bit much concentrated on technical issues, unit test areas and not the business logic.

    Hope this helps.

    ------------------
    Roland
    Roland Stens

  4. #4
    Senior Member
    Join Date
    Nov 2002
    Location
    Cincinnati,Ohio,USA
    Posts
    233
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    Having worked for Companies that built systems and software for the Government (DoD), this was common (haven't been there in a few years). First of all the Contract required that the Builder/contractor deliver all test test documentation (test plans, test procedures [test cases/scripts], and test reports) for review and approval. The Government also sent reps to witness the formal tests. This allowed the Government team to know what level of testing was being performed. On some major contracts the Government had an Independant Test Contractor also test the products. The Government Contracting Agency and the Government End User would develop a set of User tests to determine if the completed product supported the User. The Product would not be accepted until it passed this User test. Now the cost of such an effort may be a little more than most companies want to pay, but it does make sure the proper level of testing is performed.

    Many times software developed by outsourcing is done when ever the schedule said it was done or when ever the amount of funds allocated to the development is spent. Testing after delivery can be risky if the developer has not done the proper level of unit testing .

    ------------------
    Larry Schneller
    IT QCC

  5. #5
    Junior Member
    Join Date
    Jan 2003
    Location
    Melbourne, Victoria, Australia
    Posts
    2
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    I have found your comments very interesting, as I am currently in the position, of accepting a product developed and system tested by another customer, before passing it onto the customer for UAT.
    The strategy we are contemplating is:
    - checking all the documentation generated by the outsourced company(Test Cases, specs, test matrices etc)
    - build scenarios that would cover a broad range of functionality, to ensure it holds together, and is functionally stable.
    This is the first time I have done something like this, but you are suggesting that we do system testing, which seems like we are doubling up.

  6. #6
    Junior Member
    Join Date
    Jan 2003
    Location
    United Kingdom
    Posts
    7
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    Thank you to those who have given feedback so far. I'll trya na dadd a few more thoughts of my own.

    First, to rosap

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by rosap:
    I have found your comments very interesting, as I am currently in the position, of accepting a product developed and system tested by another customer, before passing it onto the customer for UAT.
    The strategy we are contemplating is:
    - checking all the documentation generated by the outsourced company(Test Cases, specs, test matrices etc)
    - build scenarios that would cover a broad range of functionality, to ensure it holds together, and is functionally stable.
    This is the first time I have done something like this, but you are suggesting that we do system testing, which seems like we are doubling up.
    <HR></BLOCKQUOTE>

    The strategy you're suggesting is teh classic one. The questions are:
    - how well will the external supplier test the solution?
    - have they actually scheculed time into their plans to provide you with the documentation you require,AND enough time to respond to your feedback?
    - does the overall Project Manager have enough time allocated in his project plan for this review to occur?
    - how experienced is your customer in conducting UAT?

    Yes there is a danger that you'll find you really need to do system testing to accept the solution form the supplier - this is a risk that needs to be identified to the Project Manager. If he says "no" then it's been recorded and you (the Test Manager) can fulfil the PM's expectations. IF it turns out that the delivered solution is not up to par, then it's exception planning time...

    What expectations does the customer who will do the UAT have? I'd suggest that you involve them in producing the "scenarios that would cover a broad range of functionaliy" that you refer to. This would mean (a) the business understanding is in place, (b) these tests would be a sub-set of teh full UAT pack, and (c) could be the basis of a regression test pack.

    regards,
    Allan

    ------------------

  7. #7
    Junior Member
    Join Date
    Jan 2003
    Location
    United Kingdom
    Posts
    7
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    Linda -
    thanks for the feedback; here's a few observations

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by ljeanwilkin:
    There may be a great deal of duplication, but rerunning the same tests is desirable from environment to environment.<HR></BLOCKQUOTE>
    Agreed - you need to ensure that the same code is doing the same thing in different environments.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>but a great deal depends on how your contract with the outsourcer spells out quality requirements.<HR></BLOCKQUOTE>
    Good point - but I've seen the cae where the contract detail is still being hammered out AFTER the development work has started and there is still disagreement over which deliverables the supplier and the in-house group are each responsible for.
    If this level of detail isn't worked out, invariably (a) the supplier will not plan enough time to provide you with the documentation you require, and (b) there will not be enough time for them to react to your feedback because that hasn't been allowed for either. Meanwhile the project Manager is pushing for things to be delivered...

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>In this case, the smoke test, produced by your in-house testing organization, should be submitted to the outsourcing organization well in advance of the turnover date.<HR></BLOCKQUOTE>
    Good plan - providing both sides appreciate the time involved and provision is made to address issues resulting from this test phase. I'd also suggest that it's useful to have a customer (i.e. end-user) along to witness this in order to give them advance warning of what the solution is really going to look like and to satisfy themsleves thatit's doing what they expected.

    regards,
    Allan




    ------------------

  8. #8
    Junior Member
    Join Date
    Jan 2003
    Location
    United Kingdom
    Posts
    7
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    Roland -
    Thanks for your comments.

    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by rstens:
    Later exercises greatly boosted our role and we basically do a full system test and then a full blown acceptance test with the business clients regardless of how it was tested before. Actually we do not even use the suppliers test scripts and tests because we found them fairly shallow and a bit much concentrated on technical issues, unit test areas and not the business logic.<HR></BLOCKQUOTE>
    Would definitely agree that it's betetr to use your own scripts and not the supplier's; and to use the business customer's input to build the acceptance scripts as well, to ensure they reflect the real world.
    I guess it's down to how well the supplier does his job and what level of risk the Project Manager and Business are willing to acceot. They're going to be wanting their solution delivered asap, and these Testers just keep finding problems ;}
    The problem I see is that because the supplier is external, it is usually difficult to influence how well they will test - by the time you get a chance to make an assessment, it's too late. At least with in-house development and in-house acceptance you have a greater opportunity to police and mange the testing strategy.

    regards,
    Allan


    ------------------

  9. #9
    Junior Member
    Join Date
    Jan 2003
    Location
    United Kingdom
    Posts
    7
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: In-house testing of outsourced development

    Larrys -
    Thanks for your observations. A thought...
    <BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by Larrys:
    Many times software developed by outsourcing is done when ever the schedule said it was done or when ever the amount of funds allocated to the development is spent. Testing after delivery can be risky if the developer has not done the proper level of unit testing .<HR></BLOCKQUOTE>
    Very true - the outsourcer has said "We will deliver the solution on this date,and you will be able to put it live 4 weeks later, because that's all the time you'll need to accept it". All the budgets are based around this, and all the launch activities are planned on this (with minimum contingency) and something gets implemented on deliverty day + 4 weeks. And the in-house IT people have to pick up the pieces becuase it's all as stated in the contract...

    regards,
    Allan



    ------------------

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 9.38%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 12:45 AM.

Copyright BetaSoft Inc.