While working on a separate RoR project I’ve used Cucumber to drive acceptance tests. If you’ve not tried it already, it’s great give it a go! Also on a side note i also discovered the guys who wrote selenium-client put a great deal more thought into the API than the .NET counterpart. Lots of extensions for waits and JQuery/Prototype integration are included by default. Anyway back to the problem at had, Cucumber forces you to write plain text stories (or features) and one or more scenarios for that story. Scenarios are written with each line start with “Given”, “When” and “Then” (and optionally And). Each line of the scenario corresponds to an executable “Step” that in my case performed a selenium action (or it could also quite easily perform an action against an API).

Anyway this method of writing test with the emphasis on the behaviour of the user (not surprising for a BDD tool) meant i ended up with less focus on one bit of functionality on a single page and more on the overall journey of the users actions through the application. Each scenario taking a different path or journey through that story.

Overall cucumber forced me to stay true to testing behaviour, avoiding the pitfalls of the test written in a plain unit testing framework. So, try cucumber, even if you don’t use it in your project i hope it makes you think differently about how you write the same tests in your native language.

This post tries to cover some of the root causes of our problems and try to suggest practices to help avoid these problem in the future. While I talk about Selenium most of these practices in theory could be applied to any automation package…

What should I test with Selenium?

Actually this should really be where are my tests best placed? A lot of acceptance tests seemed to delve too deeply into the functionality of a story which partially leads to the problem of too many/long running acceptance test. Try to keep acceptance tests at a high level. Anything of the story that you can test in unit tests, test it there – don’t repeat yourself in the acceptance tests.

Try not to use Selenium as a tool to unit test Javascript, this again dramatically increases the number of tests you have and can become very brittle when waiting for expected DOM changes. Use an appropriate Javascript unit testing tool like JsTestDriver.

How should I organise my tests?

One of the key problems to organising tests as if they are single unit tests is they tend to navigate to a page, change something, assert something happened then the close page. Now write 25 tests for one page and for every tests you have to wait for the page to load, which as you many expect seriously increases the time taken to run the tests. This is especially a problem testing functionality based entirely in Javascript which is relatively quick to test.

Where possible group all tests for a page and browse to the page once. This does leave the problem of tests effecting each other but if you remember to follow a pattern of, change something, assert something and then reset the page you shouldn’t have any problems.

Something else that I came across was when testing functionality on a page that was in a sequence after others, you always ended up completing the first page and then moving to the second before starting your tests. Now think about organising your tests where for each test you are completing the first page to get to the second before performing your test, they start to take a very very long time to run. This leads to organising the tests into journeys through the system (or groups of related stories that can be run in sequence) allowing us to tests each criteria as we navigate through the system.

How should I organise the code that interacts with my web application?

Use a page model. Construct classes that represent pages or popup windows in your application that have an interface that reflects the user interface. For example a form page that takes a name may have a page model with properties for title, first name and surname that when set or got perform the relevant selenium actions on the browser to get or set the value. This has a couple of benefits, you can reuse these page models across tests and they can be easily replaced if a better automated testing tool comes along. Another point to note is to make sure your page model interface follows a single semantic, e.g. use properties to get/set values and methods to perform actions like clicking a button. Tests tend to become very confused and actions tend to get repeated if there is no common way of defining the interface.

How do I know when something has happened on the page?

It’s very important to know when something has happened on the page (especially on Javascript heavy pages), if you don’t, expectations become very brittle when they check to early or to late. Hopefully your first reaction is not to start putting Thread.Sleep()’s all throughout your tests, what happens when something takes longer than you expected? The best way to solve this problem in Selenium is to use the built-in WaitForCondition(condition, timeout) method. You can give a condition in Javascript which has access to the client side Selenium object. For example you can do things like this…

The important difference between this and Thread.Sleep() is that you can give longer timeout periods. A Thread.Sleep() will always make your tests wait for that period of time where as WaitForCondition will move on once the condition is met. This allow you to account for differences in the speed tests run without making your tests take longer and longer to run when increasing Thread.Sleep times.

Summary

• Test where it is most appropriate.
• Group tests into journeys where possible.
• Hide selenium browser interactions behind page models.
• Don’t Sleep just WaitForCondition’s to happen.
]]> http://www.christopherbird.co.uk/2009/11/lessons-learnt-from-acceptance-testing-with-selenium/feed/ 0