1) Can it be used by multiple departments (QA, Development, Product Managment, Project Management, etc.)
2) Does it contain modules/features that are relevant to your organization
3) Ease of installation. Will your IT department have to spend a lot of time with its upkeep
4) Will an internally created tracking tool be more efficient (My company started out using Remedy and realized we did not need all of its features. We ended up creating a simple in house product of our own. Works fine)
5) The company that created the tool will be around in the next few years. Don't pour a lot of money into a product where support will disappear
6) Speaking of support. You MUST have good support to assist with installation issues and product bugs (A defect tracker with a defect , it happens!)
2) what is the user interface (ie, installed exe on client machine, browser interface)?
3) what options are there for adding user defined fields, changing value lists.
For the evaluation report, come up with a ranking system for the criteria (something like 1 = low to 5 = high would do). You can weight the ranking by having low weight range from 1 to 3 and high from 1 to 10. Then to work out how each tool meets your needs, set a value for each of the criteria and you end up with an overall number than can be used to compare tools.
It's not perfect but it's a way for doing it without driving yourself crazy. Of course, it helps to get evaluation copies of the tools in your own environment. Hope it helps
A starting point is to define what you are looking for, how many users will you have, will it be part of a formal control system or something just to keep track of bugs. Bug trackers have a wide range of capabilities as well as costs. You can get a freeware bug tracker and then try to fit that into your process. Big thing is what do you want the tool to do? I have seen several place that build their own, but the long range cost of these may be more than you want to spend.
Below are a few questions I ask of a defect tracking tool.
Hope it helps
Interface with the Automated tool
Description and Summary Fields
Bug Fix Assignment Fields
Automatic Defect Reporting
Email Defects to Team Members
Sort Defects by various criteria
Assign Defects to Developer
Provides the ability to report defects throughout the test planning and analysis phase
Provides the ability to report defects during script execution phase
Report Listings can be created
Graphical report can be created
Proviides the ability to report defects from a file
Provides the ability to report defects using from e-mail
Provides the ability to send defect reports using E-mail
Provides the ability to send a defect report via the Web
Provides the ability to attach files
Provides the ability to attach screen images
Provides the ability to customize what defect data is displayed
Provides the ability to sort defects within the log
Provides the ability to filter defect reports
Provides the ability to view a 'history' of changes
Provides the ability to link test requirements to a defect
Requirements/Test Case Coverage
Requirements with Associated Defects
Tests with Associated Defects
Tests with Design Steps
Test Run - pass/fail
Ability to create Custom Reports
Life should NOT be a trip to the grave with the intention of arriving safely in an cool and well preserved body, but rather to skid in, chocolate in one hand, beer in the other, body wrecked, totally worn out and screaming WOO HOO what a ride!