Thanks:  0
Likes:  0
Dislikes:  0

1. Boundaries Without Borders

Ok, so I was at this seminar this week about boundary testing. I thought, "GREAT!" I'm actually in the process of getting some test cases together for testing some serial commands (about 170 of them). So here were my initial specs for each command:

Min, Min+1, Min-1, Max, Max+1, Max-1, 0, Null, 3 more valid tests, Alpha input handling

I then began to read the spec (YAY Specs) for the serial commands, and realized that these ssame commands drive 3 or 4 different hardware architectures. Not only that, but there are different min/max values for each. So we might have one ranging 0-255, one going 0-128, and another going (-60)-59.

So now I've got 4 sets of the above initial specs that I need to run, since there may be boundaries or issues within the other ranges.

Then, I get to this seminar, stoked that I'm going to get all this simplified, and he lays onto me that bugs can exist anywhere within or outside a given boundary.

Man! And while I halfway expected it, it still made me cry a little. So let's see, an average of 5500 or so values per command and 170 commands brings us to nearly 1,000,000 total iterations.

So I guess what I'm trying to get at here is, are my initial tests a good starting point for an automation project? With the use of automation testing, are you still running boundary testing, or do you just run a full cycle of both valid and invalid inputs? What are some techniques you use for your own boundary testing?

2. Re: Boundaries Without Borders

Brent,

Of course bugs can exist anywhere within or outside a boundary.

(Imagine a function that accepts only integer values in the range 1-10, but happens to have a mathematical error inside it that only occurs at 5. Developers are very clever, and bugs can hide well.)

Still, analyzing boundaries can help construct a reasonably efficient set of test values, with a higher likelihood of finding bugs.

With automated testing, sometimes it's possible to run exhaustive tests in a reasonable period of time, and sometimes not.

You need to weigh the probability of finding a bug given your test data set, with the expense of adding more data points.

1,000,000 iterations might be necessary and sufficient.
Or it might just be desirable and affordable.
Or it might be too expensive.
Or it might not be enough (there are an infinite number of possible values, not just 1,000,000 - right?)

As far as your tests being a good starting point for an automation project - I can't really tell without knowing more about the risks of failure in your system.

I think they are good. But, I'm also not sure that automation has anything to do with it. Wouldn't you use this set during manual testing?

3. Re: Boundaries Without Borders

Hey Joe, thanks for the feedback. You might be right, actually. I guess I'm sort of in an enviable position here because I already developed an automation framework for a similar technology during a previous project. So I have the tools to get the automation in the pipeline within a reasonable timeframe.

So maybe I'm attacking this in the wrong way. Since I do have this previous experience in automating these commands, maybe I should be sending this through a manual run, using standard boundaries and maybe some ET, and then generate the automation suite in parallel. Then, I can actually choose to run the minimum boundary condidtions or every value via the automated scripts.

You are also right, there are possibly an infinite number of possibilities since some of the commands range from 0-10 and others are from -32768-32767, so there, really, are greater boundaries outside of the ones I'm hinting at because each set of boundaries for the 170 commands could possibly accept values from any one of the other boundaries, incorrectly, when an invalid value is passed. Likewise, there could be values which are misinterpreted at the code-level, meaning it might accept 400000 as a valid value when it's way off.

I suppose this could be used as a great example of how automation and manual testing should be used in conjunction. While speed is certainly required, cognitive thought is also something that is required in this case.

4. Re: Boundaries Without Borders

When you want to increase the volume of test data (perhaps because you suspect that the basic boundary values don't lend enough data points), it can be very handy to use your automation to throw pseudo-random values in.

Many times, I've driven my application-under-test through its paces repeatedly overnight or over the weekend, using a "Smart Monkey" style test script.

Occasionaly, this produces unexpected results.

5. Re: Boundaries Without Borders

Yeah, my thought was possibly to run some unattended automated scripts also. Thanks for the input Joe.

Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.