| || |
WebService Testing, VS Ultimate & Team Foundation Server
I will be starting a new effort which will include web service testing using MS Visual Studio Ultimate and Team Foundation Server for test results. Neither of these systems are familiar to me and I'm just starting to do the research. I have several questions:
1) Is it possible to test web services (both .NET & Java) with VS Ultimate?
2) If it is possible, what do I need to do to get my test results into TFS for metrics & reporting?
I know that my questions are fairly broad, but thought I would start here.
2) If you have your tests linked to a TFS project and test cases it will happen automatically. You should have Microsoft Test Manager available with VS Ultimate - you use that to create test suites and test cases, then VS to associate the test cases to your automation. After that, it all links together more or less seamlessly.
How do I go about specifying my endpoint & running a request/response?
It works just as if you were programming it - you create yourself a test project - you can use either a unit test or a coded UI for this. Your [TestMethod] decorations and linking the routines so decorated to test cases with VS are what triggers the linking of test results and reporting to TFS (it can be done programmatically via the TFS API, but trust me that's a LOT more complex).
From there, you can define a data source (the method for doing this differs between 2010, 2012, and 2013 - it's available online with a bit of googling). From that (again there's plenty of examples online for the exact code structures) you can pull your end point, your requests to send, and your expected responses. The connection is something that would be created the same way any application creates one (another google). How to: Create a Data-Driven Coded UI Test has the information for connection to a data source for VS2012.
You'd end up with something approximately like this (I prefer to use the Arrange/Act/Assert pattern and explicitly comment it in, and C#-ish structure) with a CodedUI setup:
[DataSource(connection parameters go here), TestMethod]
public void CodedUITestMethod()
/* This assumes columns in the data source named ConnectTo, RequestText, and ResponseText */
/* The TestContext is where Microsoft stores all the general information about the test - it allows access to information in the method decoration */
string ConnectionString = TestContext.DataRow["ConnectTo"].ToString();
string RequestText = TestContext.DataRow["RequestText"].ToString();
string ResponseText = TestContext.DataRow["ResponseText"].ToString();
/* declare the connection to the service, and connect with the connection string etc */
/* This is where you send the request text - you'll probably do something like
ActualResponse = ServiceConnection.Send(RequestText);
Assert.AreEqual (ResponseText, ActualResponse);
Unless I'm missing something, this appears to only be comparing an expected, static xml response (stored in DB, csv, etc) to the actual response from the system. Using VS Ultimate, is it possible to data-drive the various elements in each xml document? I was looking for something a bit more sophisticated with this tool. SoapUI Pro has a great feature for this, but I'm not sure it integrates with VS Ultimate &/or TFS.
Yes, you can data drive the elements. It will take more programming to do this, but it's absolutely possible.
To do it, you'd be working with C# or VB XML core classes to build XML and parse the response. I gave the simplest possible scenario - the more sophisticated your scenario is, the more complex your code is going to be. You'd probably need to create at least two new methods, one to build the XML to send, and another to parse out the response against what you're expecting to receive. In that case, your original data source would point to the "key" you're using to build your XML and (optionally, depending on how dynamic you intend to be) the "key" to parsing out the response.
The final assertion would then be whether or not the response parser returned the correct result (most likely a Boolean OK/not OK). The response parser would need to log extra information for incorrect responses so that you had it there (asserts will do this: there's a reasonably large selection of possible asserts you can use, or you can simply code your logging).