Tuesday, 10 March 2009

UI testing automation with Project White

Somehow I was assigned to testing work related to UI testing automation with a WPF application, and my teammates Jaksa and Darren have already spent more than 3 months developing a nice architecture and laying the basis for more intricate workflow before I joined. They both find it painful to deal with Project White. Yes, I have to admit that it's buggy and not as mature as expected in terms of functionalities it provides, but I find it much more interesting to play with than performing the acceptance test cases (sure it is, there are bloody 150+ of them!). Ok, BS is over, here are some points I want to address with Project White (refer to the offical page of Project White or the famous Ben Hall's blog if you want to set things up and say hello world):

1. UI spy is handy.
As we all know, UI testing automation is all about grabbing control of the UI elements in the application and do your stuff to them programmatically, e.g. clicking on a button or typing in some text in a text input. UI spy offers the capability of inspecting the properties of UI elements of almost every applications that are on your desktop, and you can also search for an UI element with the criteria you specify. I am not sure whether there is a stand alone distribution of UI spy, but you can certainly find it in the Windows SDK.
2. You can grab a UI element using the hierarchical structure or regardless of it.
Project White provides a nice mechanism for grabbing control of the children of an UI element, but you can always find the element by simply passing an AutomationID or text on the element to the search method no matter how deeply nested it is in the hierarchy.

3. Do not try to get the instance of any "static" element, it just doesn't work.
The so called "static" elements are elements that simply present something but have no significant events, e.g. a text block or a canvas. Project White typical favors those active elements in an application, like butons , expanders or list boxs, and tends to ignore "static" elements, which is somewhat annoying when you try to verify that there are some warning text message appearing when you perform some nasty operations, or you want to drag some thing and drop it on a beautiful canvas , or even your are unlucky enough to have some events that cannot be triggered unless you click on a text block. In this case, there is not much you can do with it alone, ping your fellow developers and see whether they can migrate the event to a button or at least put the static elements inside a container (a panel for example).

4. When you drag-and-drop using Project White, do it slowly.
Do not just pick a thing and drop it on the target element directly, give it a break or several breaks to make sure that it performs smoothly. You can always insert some threading code to make the mouse to wait for something like 5 millisecond so that the object you drag is with the mouse. I have been torturing both the application and my own little brain with this issue until Jaksa kindly point my way out.

5. Immediate Window is your faithful friend when you are lost in the jungle of bugs.
Joggle a break point, run your test code under the debug mode and press Ctrl+Alt+i, here comes your royal friend. Don't say that kinda "you should test instead of debug" shit to me unless you want to write test code for your test code (which makes it meta-test-code :o).

6. When you run an extra long test, do it when you are away for lunch.
Automatic UI testing hijacks your mouse, you can not do anything in the meanwhile, so grab a lunch or a cup of coffee, try to be sociable and come back to see all the green icons.

p.s. Project White should really have a more special name to be seachable in the jungle of information on the internet.