The online community for software testing & quality assurance professionals
 
 
Calendar   Today's Topics
Sponsors:




Lost Password?

Home
BetaSoft
Blogs
Jobs
Training
News
Links
Downloads



Miscellaneous Forums >> General Discussion

Pages: 1
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Testing Database Patches
      #714935 - 08/21/12 07:36 AM

We have a centralized database cluster (an Oracle RAC), which serves virtually all of our applications on behalf of all of our clients.

We have always been very reluctant and very careful when making any changes, as any downtime could affect everyone.

Consequently, we have only applied emergency patches. By our definition, the need for emergency patches outweighs the risk of the potential side-effects. We only do this when needed - a very rare event.

Now, as we continue to integrate with a larger organization, the company is hinting that we must get prepared to apply database patches on a regular basis - at least quarterly, and perhaps monthly.

I'm faced with trying to find the time to regression test several dozen applications periodically, then releasing the database updates off-hours. I estimate that it would take about a week of dedicated testing by at least 3 testers to accomplish these regression tests. And of course I simply don't have that kind of time in the schedule, so I'm currently negotiating for additional headcount.

I've spoken with other groups which were previously acquired by the parent company. They don't quite have the same problem, as they support only a single application each.

I'm wondering if others have faced a similar requirement, and if so, how you have handled it?

Do you steal time from other projects to test the effects of periodic database upgrades?

Do you have dedicated testers whose sole responsibility is to test infrastructure upgrades, rather than application changes?

Or something else?

Thanks for your help!

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Quantaga
Newbie


Reged: 02/14/05
Posts: 23
Loc: Perth, Western Australia
Re: Testing Database Patches [Re: Joe Strazzere]
      #719108 - 10/31/12 10:39 PM

Hi Joe

Did you get any response to your questions?

I am about to do something similar for a Project. We are about to upgrade a database.

I am thinking of the following tests (at least) using the application.

Any parms. views - make comparisons between pre & post migration - this is to make sure the SQL Queries actually do return the exact same record set.
I will also use various roles for the above as the system has a complicated user role hierarchy.

Display "all" for views and make sure the total count is identical for both.

One or two record creates, updates (no deletes in the system) to make sure the commits are working properly.

I am a little miffed that nearly all sites / blogs I looked at, there is hardly any mention of data migration testing.

Cheers
Jonathan.

--------------------
--
When the mind, one pointed and fully focused, knows the supreme silence in the Heart, this is true learning - Sri Ramana Maharshi


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: Testing Database Patches [Re: Quantaga]
      #719125 - 11/01/12 03:55 AM

Quote:

Did you get any response to your questions?



Sadly, nothing.

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
SimonFromLeeds
Member


Reged: 08/12/04
Posts: 139
Loc: Leeds
Re: Testing Database Patches [Re: Joe Strazzere]
      #719129 - 11/01/12 06:34 AM

The last time I was in an embedded IT team (I'm now back a s/w house), we tested upgrades by replaying a days activities through the dB. I'm going to sound vague about this, because I wasn't involved. The production dB's had transactional logging. We had two snap-shots, one from the end of business on day 1, one from the end of business on day 2. We had a test env with the new upgraded dB, which had been created from the first snap-shot. We re-ran the transactions from the transactional logging of that days business. Then we compared it to the 2nd snap-shot. The comparison was done using some tool (maybe from redgate, IKBA could have been hand-cranked sorry).

The test was primarily focused on the success of transactions. Date-time stamps and sequence numbers didn't match.

We chose the snap-shots from the day of our monthly business cycle which was most financial stuff happened.

The DBA team did all the tests above. It was all MS-SQL

The test team then just did tiny 5 min per system smoke test on the ~dozen systems which touched the dB. That was more connection testing than anything else.

I think we did about 3 tests in 2 years. We didn't find any bugs, during the tests, or afterwards in production.

So, most of the impact was on the DBA team. I think we documented the plan and did the smoke tests and reported, took maybe 1/2 day of my team.

Looking back (this was 4-5 years ago), this was a thin test, mostly just a big unit test of the dB. The potential impact of a problem is massive, the probability of a problem I'd judge as tiny. If I had to be involved in something similar again I'd be happy using the same approach, maybe getting a little closer to what the DBA team were doing.

Hope that helps


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: Testing Database Patches [Re: SimonFromLeeds]
      #719133 - 11/01/12 07:00 AM

Thanks, SimonFromLeeds - helpful information! I particularly like the idea of replaying logs. I'll have to chat with the DBAs about that one...

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
CPat
Member


Reged: 12/08/09
Posts: 102
Re: Testing Database Patches [Re: Joe Strazzere]
      #719135 - 11/01/12 08:29 AM

Joe,

Several years ago, I was a performance test manager for a middleware area. We had several core databases which fed multiple applications. First piece of advice is to treat a database upgrade/patch as an actual project. Also, we didn't support web-based applications as most of ours were batch interfaces and web services. Automation was our saving grace in this regard as most of our regression tests were automated.

I do not know why you would need a dedicated group for infrastructure changes since I assume you will ultimately be testing the actual application layer.

There are several unknowns here like do you have a middleware tier or do your application access the database directly? If you have a middleware tier, you may consider some level of automation to hit those services.

As I mentioned earlier, treat this as you would any other project. Perhaps when your leadership understands the costs and level of effort, they may consider increasing your staff or upgrading less frequently.

good luck,
chad


Post Extras: Print Post   Remind Me!   Notify Moderator  
Joe Strazzere
Moderator


Reged: 05/15/00
Posts: 12344
Loc: Massachusetts, USA
Re: Testing Database Patches [Re: CPat]
      #719137 - 11/01/12 09:15 AM

Quote:

I do not know why you would need a dedicated group for infrastructure changes since I assume you will ultimately be testing the actual application layer.



The point is that we have many applications here. The organization is hinting that the underlying database may need to be upgraded as often as each month - without any changes to the applications.

Quote:

There are several unknowns here like do you have a middleware tier or do your application access the database directly?



No middleware tier.

Quote:

As I mentioned earlier, treat this as you would any other project. Perhaps when your leadership understands the costs and level of effort, they may consider increasing your staff or upgrading less frequently.



So far, they have not agreed to consider increasing the staff, nor decreasing the frequency of database or server upgrades.

As far as I can tell, my only recourse at this time is to slow down work on all other projects, in order to free up time for this recurring "project". That's no going over so well, either...

--------------------
- Joe
Visit AllThingsQuality.com to learn more about quality, testing, and QA!

I speak only for me. I do not speak for my employer, nor for anyone else.


Post Extras: Print Post   Remind Me!   Notify Moderator  
Pages: 1



Extra information
0 registered and 26 anonymous users are browsing this forum.

Moderator:  Rich W., AJ, blueinatl 

Print Topic

Forum Permissions
      You cannot start new topics
      You cannot reply to topics
      HTML is disabled
      UBBCode is enabled

Rating:
Topic views: 3084

Rate this topic

Jump to

Contact Us | Privacy statement SQAForums

Powered by UBB.threads™ 6.5.5