Thanks:  0
Likes:  0
Dislikes:  0

# Thread: Help with understanding an automation question

1. ## Help with understanding an automation question

I am recruiting for a tester with automation experience to help us improve our existing manual testing program. No-one in our organisation has any experience in this area (although I have done some development but it was several years ago), so I am struggling with how to assess a candidate's knowledge.

My manager has produced the test below that 'one of his colleagues used to use in a previous firm' which he thinks we should give to candidates. Basically, that means we have to use it. The problem is, neither he nor I know what a good answer would look like! I am particularly struggling with how to assess the last 3 points.

Any advice on what would make a good solution to this test? Thanks in advance. (sorry about the table formatting).

------------------------------------------------------------
Scenario:

Table 1 represents a number of products, their varieties and their prices in pounds sterling.

Table 2 represents the same products and varieties with their prices converted to Euros. The value in each cell of Table 2 is calculated from the value in the corresponding cell in Table 1 as shown:

Value (Product1,Variety1) in Table 2 = (Value (Product1,Variety1) in Table 1) * conversion rate.

The conversion rate in this scenario is 1.7.

Not all products come in all varieties; hence there are some cells in the tables which do not contain values (represented by N/A).

The bottom row of each table shows the sum of the values in the corresponding column.

Table 1: Prices in Pounds Sterling

Variety 1 Variety 2 Variety 3 Variety 4
Product 1 12 15 18 52
Product 2 30 20 26 N/A
Product 3 24 58 N/A N/A
Product 4 32 N/A N/A N/A
Total 98 93 44 52

Table 2: Prices in Euro

Variety 1 Variety 2 Variety 3 Variety 4
Product 1 20.4 25.5 30.6 88.4
Product 2 51 34 44.2 N/A
Product 3 40.8 98.6 N/A N/A
Product 4 54.4 N/A N/A N/A
Total 166.6 158.1 74.8 88.4

Write a test to check the correctness of the values in the Euro table.

The test should ideally be written in a high-level language (C# or Java), but commented pseudo-code is also acceptable.

The implementation of the pricing system includes a Table object which represents a pricing table. In addition to the test code, also describe the properties of the Table object that are needed to write the test.

The solution will be assessed in terms of
• Error condition coverage
• Structure
• Robustness
• Ability to be compiled without the need for the AUT

2. ## Re: Help with understanding an automation question

[ QUOTE ]
My manager has produced the test below that 'one of his colleagues used to use in a previous firm' which he thinks we should give to candidates. Basically, that means we have to use it. The problem is, neither he nor I know what a good answer would look like!

[/ QUOTE ]
As a recruiter, you should strongly advise your manager that it is never wise to ask questions where you don't know and understand the answer.

If you are recruiting for a technical area in which you have no knowledge, and the candidate must be assessed technically, it's your responsibility to find someone competent to ask technical questions, and evaluate the answers. Look for someone outside your company who can assist you if you have nobody competent in-house.

Winging it, based on a few random answers you might find on a forum is no substitute for real capability - both in interviewees and in you.

3. ## Re: Help with understanding an automation question

I thought there might be some mileage in this test though as I do understand basically what it's getting at and having been a developer I thought I could probably make use of it given a few pointers.

Plus I need to get to grips with this sort of stuff anyway if we're actually going to implement any automated testing.

Not sure who I would look for 'outside my company' to assist me? I kind of thought that was what I was doing on a forum! My company is a small outfit which doesn't have a huge amount of resources for QA and hiring an expensive consultant isn't an option.

4. ## Re: Help with understanding an automation question

Should have added - any pointers to books etc that might help me also would be appreciated!

5. ## Re: Help with understanding an automation question

Perhaps someone else could provide some useful comments.

But other than suggesting that you have the candidates write out the solution on a whiteboard in pseudo-code and walk you through it, I can't imagine how helpful this could be. Even then, it doesn't sound like you are technically competent to assess the candidate's solution. Sorry, not trying to be harsh here, just honest.

It would be like me trying to hire an automobile mechanic for my software company without getting baffled by bullsh!t during the interview. I am not competent to work on cars, nor to assess the capabilities of someone who is. If someone on a forum told me to "ask the candidate how often you should change the framis junction rod or the zerble seal" would that be useful?

Do you have friends who do test automation? Do you have friends who are developers, and could judge the quality of a C# or Java program?

Whenever you are undertaking a job for which you don't have sufficient knowledge/background you should enlist an agent. In this case, perhaps you should be working with a recruiting agency.

6. ## Re: Help with understanding an automation question

I could ask the developers in the company - don't know why I didn't think of that! Obviously they can help with the structure/robustness aspects, but would the 'ability to be compiled without the need for the AUT' mean much to them or would that be more specific to the testing domain (I really don't understand this bit).

I don't think the situation is analagous to your mechanic example - I do have some knowledge in this area but it's rusty and I need some help getting back into it. Plus, I've already got questions to ask, it's the answers I'm struggling with!

Regarding agencies - we already do work with agencies. They just churn out CVs, often with no apparent consideration for whether the candidate matches what we are looking for. I've yet to meet a recruitment agent who knows even as much (and that's not much) as me...

7. ## Re: Help with understanding an automation question

[ QUOTE ]
would the 'ability to be compiled without the need for the AUT' mean much to them or would that be more specific to the testing domain (I really don't understand this bit).

[/ QUOTE ]
I would hope your developers would understand that. I suspect it means that the automation code must be able to compile without including any of the code from the application-under-test.

[ QUOTE ]
I don't think the situation is analagous to your mechanic example - I do have some knowledge in this area but it's rusty and I need some help getting back into it.

[/ QUOTE ]

[ QUOTE ]

[/ QUOTE ]
Right. Questions are easy if you don't need to understand the answer.

When a candidate gives you an answer to one of your questions, but then asks you for more clarification, are you going to be able to have a technical conversation?

Remember that while you are interviewing candidates, they are in turn assessing your company. If I interview at a company where the person asking me test automation questions doesn't really understand test automation well, it might give me pause. ("Is this a company where I can bluff my way through?" "Do they really know what they want?" "Are they going to have any real way to assess my work?")

[ QUOTE ]
Regarding agencies - we already do work with agencies. They just churn out CVs, often with no apparent consideration for whether the candidate matches what we are looking for. I've yet to meet a recruitment agent who knows even as much (and that's not much) as me...

[/ QUOTE ]
Apparently, you are working with the wrong agencies. The agencies I deal with are capable of bringing specific expertise to bear when the search merits it.

Why would you put up with an agency that has "no apparent consideration for whether the candidate matches what we are looking for"? What are you paying for here?

8. ## Re: Help with understanding an automation question

I was having "a little visit" with a "technical resource search expert" (recruiter looking for people apparently). He was asking questions and had a set list of formulaic answers. About 2 minutes in it became clear that he knew as much about software development as I did about nuclear physics (NIL!)

If you have a "range of acceptable" answers, recognize that you can be eliminating many top-flite candidates whose background and experiences lead them to different approaches than people with other experience. I find many people are looking for cookie-cutter responses. If that is what the search filters are doing, do not be surprised if the results are less than optimum.

9. ## Re: Help with understanding an automation question

So how do small companies with no skill in the area they are attempting to recruit skill for normally go about it? Surely this isn't an unusual situation? You don't have the skill in-house, you get someone in who does have it.

Or should we be looking at a consultancy/training option here, rather than taking on an employee?

Regarding agencies - of course I don't want to work with the sort of outfit that doesn't know what it's doing, but they ALL seem to be like that. Plus, I don't pick the agencies, HR does, I just feedback that they don't seem to understand what we're looking for. I have on one occasion gone straight to an agency to discuss why we weren't getting any suitable candidates and spoke to a couple of people who clearly didn't understand the field, and just said 'that's the state of the market at the moment'. This is one of the highest-regarded agencies in our area! I'd love to find a better one (I am in the UK by the way).

10. ## Re: Help with understanding an automation question

Personally I think a programming quiz will probably be a waste of time. Scripting/Programming is very easy to learn and is only a very small piece of the skill set necessary to become a successful test automation engineer. I think the time you'll take for him to complete the exercises, it's best spent interviewing on his experiences and his vision, and leave that coding exercises as homework.

One of the problems of hiring a strong coder who's not strong test engineer is after 6-months to a year, or about 2 major releases, you'll start to hit some issues with test maintenance and reliability issues. This is especially important for your first test engineer as everyone you hire after him will essentially be maintaining his code and following his examples.

Some other qualities you should also look for:
- Software Architecture and Design Pattern experience.
- Ability to design for reuse.
- Ability to design for reliability.
- Ability to design for portability.
- Ability to incorporate frameworks into the design.

Early decisions on how your test architecture is setup will determine the future pain the software automation project will have to endure.

You can try recruiting one of your lead developers to helpling out with the interview process. He can probably help you assess his ability in the software engineering disciplines like healthy programming patterns, layers of abstractions, and working with the different technologies involved.

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•
Search Engine Optimisation provided by DragonByte SEO v2.0.36 (Pro) - vBulletin Mods & Addons Copyright © 2016 DragonByte Technologies Ltd.