How do the Best Test?
Hey everyone, I'm a noob here, and am wondering if anyone can share any insight on your test methodologies, how you test, what tools you use, etc.
I'm curious about Apple, EA, Google, Microsoft, and more.
Anyone have anything on:
1. Nature of work in one product life cycle? i.e. level of involvement in inception, elaboration, detailed design, etc.
2. What kind of hardware is available to perform your work? i.e. VMWare, physical machines?
3. What is the general testing strategy? i.e. test-driven development, orů?
4. Any use of open source software in the test framework, if so which?
5. Percentage of white-box vs. black box testing?
6. Any UI automation?
7. Strategies for performance, load, reliability, and stress testing?
8. Does your company use, and thereby test, the products that it sells?
9. Ratio of test developers to application developers?
10. Greatest challenge in developing/testing?
11. and of course, what company you work at!
I know some people may have reservations about divulging company secrets. So if you're shy or worried about that then that is understandable. But I'm looking to share and gain any knowledge about what are the best practices out there so that we may all mutually benefit! Plus, I have a feeling that it's almost all the same across the board. =) Any feedback is greatly appreciated, thanks!
Re: How do the Best Test?
To start off, here's my information. I'm currently a software test developer at CD Soft!
1. We're highly involved in meetings that deal with the scope of the feature that we want to test. We gather around with developers and other product managers to discuss what is feasible and what is not. At this point we can always interject and speak about testing challenges. The detailed design is left to the developers, so we only get involved afterward. Then, we go through the process of analyzing requirements, writing specifications, coding our test drivers, testing, troubleshooting, etc.
2. We use VMWare, as well as have (usually) one extra machine dedicated to testing.
3. I would describe our general test strategy as waterfall.
4. We currently do not use any open-source software. Our test framework and tools are written in-house. Test tracking software is Rational but that is changing.
5. White-box: 95%. But that's only because of my specific team.
6. UI automation: some of our groups use Selenium
7. I don't have any visibility into this, but I know some of our guys have built their own tools to analyze metrics
8. We use and test our own products prior to release. I'm not sure if we provide for beta testing.
9. There is probably 3 times as many application developers as there are test developers
10. Greatest challange in testing: finding the time to come up with new test strategies and improving our existing processes. Most of the time is spent on actual troubleshooting and daily administrative work. [img]/images/graemlins/frown.gif[/img]