SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 10 of 10
  1. #1
    Member
    Join Date
    Apr 2006
    Posts
    277
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Absence of SLA\'s or Benchmark

    Hi All,

    Most of the time the performance SLA's or the Benchmarks are provided. While in some cases the Benchmarks are absent.
    In absence of SLA's or benchmarks how do we carry out test.
    If the application provided to us is new and the client doesn't know performance testing and expects us to do it then how do we proceed ahead.
    Please guide me.

    Thanks
    Thanks

  2. #2
    Moderator
    Join Date
    Aug 2001
    Location
    NC
    Posts
    6,018
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    If I have no Hypothesis to test for then I cannot have a test. I can have an information gathering experiment, but not a test
    James Pulley

    Replace ineffective offshore contracts, LoadRunnerByTheHour. Starting @ $19.95/hr USD.

    Put us to the test, skilled expertise is less expensive than you might imagine.

    Twitter: @LoadRunnerBTH @PerfBytes

  3. #3
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  4. #4
    Member
    Join Date
    Dec 2009
    Location
    Alabama, USA
    Posts
    116
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    Is the client looking to identify places for performance improvement or are they trying to identify when and where the software will break as their business scales?

    You can perform stress tests and determine what the breaking points are for various points in the system. You can provide trending reports on how response times scale with ramping vusers or volumes.

    You can also identify bottlenecks in the system. Does the system write logs/messages to disk? What impact does this have on disk space or performance?

    Perhaps the client is looking for a performance analysis as opposed to a performance test.

  5. #5
    Member
    Join Date
    Apr 2006
    Posts
    277
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    First of all thanks a Lot to all of you for replying.

    In one of the case that I encountered, I was told to do Load testing for a New application with a certain load. The SLA'a for response times and TPH(transactions per hour) were not provided as the client didn't have it.
    In this case I was told to carry a baseline test and use the results of that test as benchmark for the Load test.
    Can the baseline test help us in getting the Benchmarks when they are not defined and what should be conditions for it i.e the test should be done for 20-30% of actual load etc.

    Please guide me.
    Thanks

  6. #6
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    What does "carry a baseline test" mean for you?

    If you mean "measure the response time and transactions per hour on the old system, and compare it to the new system", then it could help.

    Basically, you'd run the same experiments on both systems, and report your findings. Presumably the client would be interested whenever the new system was "worse" than the old system.
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  7. #7
    Member
    Join Date
    Apr 2006
    Posts
    277
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    Joe, that's what I meant "measure the response time and transactions per hour on the old system, and compare it to the new system".

    But then what would be load be with which the test needs to be carried?

    Thanks
    Thanks

  8. #8
    Member
    Join Date
    Dec 2009
    Location
    Alabama, USA
    Posts
    116
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    If I understand you correctly, you have an existing/old system. They are developing a new system. They want the new system to perform at least as well as the old system.

    If that's correct, process a set load over a set amount of time on the old system and that becomes your benchmark. Perform the same load/time set-up against the new system.

    If you are unsure what your load should be and the business unit cannot provide you with that information, you should try and capture that from the logs. Good system logs are an invaluable source for information. Ideally, they report start/stop or entry/exit date/time stamps. I would write a script to capture metrics from those logs and attempt to graph it based on time. For example, you may have 5 million transactions over 4 weeks but 60% of those happen on Fridays. Try to determine your peak times when the traffic is heaviest and use that as your load.

    This is assuming you are dealing with a system that processes large amounts of batch files or heavy messaging from queues. If you're dealing with a web/GUI system, your test would be slightly different. You can still check the logs to determine the peak time and figure out how many users are logged on concurrently.

  9. #9
    Member
    Join Date
    Apr 2006
    Posts
    277
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    I have 2 cases or rather 2 applications.
    First one is absolutely new with no logs or benchmarks available.
    The second one is modified one i.e it is in production and its new version has got some functional as well as scaling changes to accomdate more no of users.
    In second case, as joe had mentioned we can get the benchmarks by carrying out test(Baseline) on the application that is in production first and use its results as benchmark for test conducted on new version of appication.Also the logs can be useful in this case.
    The first case were application is absolutely new we can provide performance analysis i.e how much load the system can handle and what would be its response times. This can be done by conducting stress test with gradual increase of load.Running it for fixed time lets say say for 30 min or hour and then again increasing the load.

    Please correct my understanding if I am wrong.

    Thanks.
    Thanks

  10. #10
    Member
    Join Date
    Apr 2006
    Posts
    277
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Absence of SLA\'s or Benchmark

    Found a good pdf regarding this.
    http://www.perftestplus.com/resources/how_fast.pdf
    Thanks

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 02:20 PM.

Copyright BetaSoft Inc.