Testing a mobile project with actual users always presents invaluable feedback—it gives you an outside perspective directly from your target users. The more you can test your project, the more data and insight you gain into the potential success or failure of your work. As usability guru Jared Spool of User Interface Engineering (http://www.uie.com) says, “Talking to two users is better than talking to one.”
However, usability testing is not unlike the age-old question “If a tree falls in the forest and there is no one to hear it, does it make a sound?” In psychology this is called reactivity, sometimes referred to as the Observer Effect. When people are observed, their behavior changes, due to the fact that they are being observed. In usability tests, it is common for participants to attempt to please the person conducting the test. Some participants prefer to give overly harsh criticism, trying to find the faults.
The goal of a usability test is to identify actionable faults in the system from subjective behavior and opinion. The researcher must find what data is, or will be, indicative of how the target user will use the system. Unfortunately, this is more art than science.
In mobile development, the challenges of the device can add additional subjectivity. I’ve found that traditional usability practices fail to address many concerns that the normal user has with mobile technology. Though I’m a big fan of usability testing, especially for mobile products, I find ...