SPONSORS:






User Tag List

Thanks Thanks:  0
Likes Likes:  0
Dislikes Dislikes:  0
Results 1 to 4 of 4
  1. #1
    Senior Member
    Join Date
    Dec 2000
    Location
    Pune, Maharashtra, India
    Posts
    1,037
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Testing Compilers

    Probably the most challenging test engineering task must be to test a C/C++ compiler. Just wondering what kind of approach is taken to test software like that. My personal take is that one would want to write a sample program for every rule in the CPP Standard. That would amount to close to a 10,000 test cases at a minimum. Then combinations of various things. Can someone share an experience on this ?
    Suresh Nageswaran, CQA, CSTE
    Lead Consultant
    Kanbay Inc.
    Pune, India.
    sureshnageswaran@yahoo.com

  2. #2
    Moderator Joe Strazzere's Avatar
    Join Date
    May 2000
    Location
    USA
    Posts
    13,170
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    1 Thread(s)
    Total Downloaded
    0

    Re: Testing Compilers

    I haven't tested a compiler, but have tested several interpreters (which were themselves components of much larger toolsets). One of the interpreters I tested was part of an Automated Test Tool. I used the tool to test itself.

    10,000 test cases is not likely to be enough.

    It is a challenge to test interpreters, but an interesting one.
    In my case, I set up a framework for generically executing each test case quickly and analyzing the results. The results would be automatically compared against a baseline, and if any differences were found, they would be flagged for further analysis.
    Then, I built the individual test cases and expected results for each statement, function, opreator, feature, etc. Due to the combinatorics, this can get quite extensive.

    I usually ended up with a test suite that would run on multiple operating systems in 24 hours or less - sometimes overnight. Some of the volume/stress tests would take much longer, of course.

    I really enjoyed it, and miss it now (I'm working at a dot-com right now - not nearly as much fun). Maybe one of these days I'll have the opportunity to test a compiler, too.

    -joe

    [This message has been edited by jstrazzere (edited 03-20-2001).]
    Joe Strazzere
    Visit my website: AllThingsQuality.com to learn more about quality, testing, and QA!

  3. #3
    Junior Member
    Join Date
    Mar 2001
    Posts
    4
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Testing Compilers

    For one possible approach, you might want to study DejaGnu and the testsuite for gcc/g++.
    It has 16000+ test cases. The testsuite has been removed from the latest distributions of gcc while they sort out copyright issues, but older versions of it (see link) are still available.
    http://myweb.worldnet.net/~stcarrez/m68hc11_tests.html


  4. #4
    Junior Member
    Join Date
    Mar 2001
    Location
    Santa Cruz, California
    Posts
    6
    Post Thanks / Like
    Mentioned
    0 Post(s)
    Tagged
    0 Thread(s)
    Total Downloaded
    0

    Re: Testing Compilers

    Years ago we used to use Perennial and Plum Hall test suites to validate the language. COnsider also that a validation of the optimization was and probably is still a challenge. Someone here in the CA Bay area managed to write a test suite that had hand optimized assembly code that would compare to the generated code of the compiler.

    -CZ
    -CZ

 

 

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 10.71%
vBulletin Optimisation provided by vB Optimise v2.6.4 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
User Alert System provided by Advanced User Tagging v3.2.8 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
vBNominate (Lite) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Feedback Buttons provided by Advanced Post Thanks / Like (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
Username Changing provided by Username Change (Free) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.
BetaSoft Inc.
Digital Point modules: Sphinx-based search
All times are GMT -8. The time now is 08:21 AM.

Copyright BetaSoft Inc.