Web app testing
I have a web app that we are testing on mobile devices. To date we have tested on 2 physical devices PPC (windows mobile 5) Palm and a blackberry emulator.
I'm getting push back from our CS dept that we must being testing on newer devices. I was wondering how others handled testing on mobile devices? Do you test on the platform or the device? Our customers seem to be getting alot of URL improperly formatted errors. My suspicion is that these are device configuration issues not bugs in the app. Any ideas?
Re: Web app testing
A couple of thoughts;
1) I would test on at least one physical device per operating system in use, connected using the intended end user network configuration. For example, Windows Mobile 6 broke an app of mine that worked fine on Windows Mobile 5. If it you target mobile phones, you will also need to check the main models from each of the major vendors and note which version of firmware you are running. Make notes of all of the details of the configurations tested so that when a bug is reported you have some chance of isolating it to hardware / configuration / network / or software.
2) Web applications on PDAs typically need to cater for quality of service errors, and other errors caused by bandwith limitations and time-outs. For a web application to work you need a reliable connection (duh). Wireless connections are subject to dramatic changes in service quality which in turn lead to numerous problems you would never see in an emulation environment, or in an environment where the physical device has a direct connection to your LAN. For example, if I view my own website on a mobile phone, it will appear normal if I have a good quality connection, but drop back to a simplified text version of the site on a weak connection. You need the weak connection to test the latter situation. Here in Ireland this is very noticable with 3G broadband, as the cells run out of capacity at peak times, most connections are dropped back to GPRS which equates to a factor of ten drop in bandwidth.
Interestingly, when I was in a position involving development of PDA solutions for mobile sales forces a number of years ago, we opted for a thick client rather than web based solution because of these issues.
Re: Web app testing
thanks for the reply.
We are currently testing on a couple of physical devices, which is what I thought we should do. The emulators seem to do the job but sometimes seem to produce unreliable results (e.g. the page renders in the emulator fine but on the device is jumbled and unusable). As for the URL improper format error it seems that this is specific to WM 6 and using relative paths. Go figure! We are planning on appending http://****.com to all paths to fix this. Seems like a kludge but we don't want to wait for the MS patch.
You mentioned firmware versions for mobile phones, in your experience has this been issue? This sounds like something we should train our tech support to ask for.
I appreciate your response and advice.