Hello! It’s been a while. I’ve been rather busy of late but whilst I’m waiting for the next piece of code to break, I thought I’d revisit my software testing blog.
It has recently struck me that I’ve been testing within the mobile arena for the last 5 and a half years. Don’t ask me where time goes but they do say it flies when you’re having fun and I must say I continue to enjoy mobile testing.
During this time I’ve tested many, many, wonderful (and not so wonderful) native mobile applications across iOS, Android and Fire OS. I’ve also tested mobile web offerings as well to include HTML5 based games most recently.
Invariably, a solid test approach will include testing across different devices, running different versions of operating system, featuring different screen densities, different methods of connectivity (Wi-Fi, Cellular, Offline, AirPlane Mode), different chipsets, navigation, looking at installation and upgrade paths, permissions, saving data, storage, security (e.g. DRM downloads), changing orientation, multi-tasking, controls, gestures, interrupt handling, performance, stability, external playback support (e.g. AirPlay, Chromecast), launching, relaunching, accessibility support, usability, layouts, statistics, gameplay, etc. The list goes on.
Despite all these different factors to consider when performing your mobile testing, a mobile related defect can be distilled to include a common set of criteria. I often champion the use of a defect template in order to promote ease of use and consistency. There is nothing worse when triaging defects to find a whole swathe of information (often crucial information) missing. When people start to assume which build version has been used or confusing the steps to reproduce for example this can generally cause lot’s of mis-understandings (Chinese whispers – not sure if this is politically correct or not) as well as ambiguity. So it’s better to get all the relevant information captured carefully balancing the needs of being detailed, informative and as concise as possible.
So let’s try to list some of the useful fields to include within a mobile related defect.
- Title – I like so follow some sort of naming convention. For example, if it’s an issue concerning a crash I’ll pre-fix the title with ‘Crash – …’
- Label – This depends on your defect management tool weapon of choice but classifying defects by either pre-fixing the title and/or adding a meaningful label may help when revisiting the particulars in future or when reporting. Again, with a crash you might label them with ‘app-crash’ and these can be reported on for stakeholders.
- Device – Self explanatory but record which model & generation of model you used e.g. iPad 2
- Operating System – Record the name & version of O/S you used e.g. iOS 9.3.2
- Build Version – Record the version of build you have tested. This is vital.
- Connectivity – Wi-Fi SSID? Cellular connection? Offline? AirPlane Mode enabled? Transition from one to the other?
- Steps to reproduce – I like to sequentially number each step. The reader should be able to understand exactly what you did (that’s assuming you even know what you did!).
- Expected Result – Try to avoid saying it should do this, etc. Say it WILL do something. Expected behaviour should be clearly defined and understood.
- Actual Result – Your big chance to explain what the hell happened. Try to include whether you could recover functionality and how or whether you crashed and burned.
- Frequency – Does this happen every single time? Or is it once in a blue moon? Indicating how prevalent something is often has an influence on the likelihood of developer time being allocated to investigate and/or fix a probem.
- Live Y/N – Another useful one. Is this already in live? If yes, then has the audience been making any complaints about this? If not then it’s likely to be a lesser priority to fix if at all.
- Screenshots – OMG. A picture can say a thousand words. If you have the ablity to provide exhibit A and evidence something has screwed up then it will not only serve to prove you haven’t lost your mind and are talking cobblers but it often cements the nature of a problem (particularly with stakeholders) and can influence decision making e.g. That looks terrible. We need to fix this.
- Video – Sometimes it isn’t always possible to illustrate something by way of a single screenshot e.g. Video playback is juddering, etc. In these instances consider providing a short video (remember to clean your fingernails beforehand).
Right, well the above should serve you and your team well. It’s not exhaustive obviously but if you start to routinely provide the above each time you capture a defect then it will pay dividends. Trust me. We’re all human (unless you’re some cyborg tester sent from the future) in which case you can’t remember everything. Not everybody can be co-located together so logging these on a shared system helps visibility. Moreover, the number of times somebody has come to me asking questions about an issue I raised weeks, if not months ago is reason alone to have something to refer to or else you’ll find you simply cannot remember all the relevant details….now….erm….what was I saying?
I’m experienced enough to realise you needn’t be overly process driven so perhaps consider only documenting things as and when appropriate. Not withstanding the merits of verbally discussing an observation with a member of the team. Sometimes a defect can be turned around much faster by having a conversation.