I don’t normally post about another blog article, but Agile-Friendly Test Automation Tools/Frameworks by Elisabeth Hendrickson is spot on about the challenges of Test Automation in an Agile world and well worth reading.
A couple year’s ago I worked on a LIMS system, and we had extensive regression tests written using Selenium. We used the Selenium HTML script, which was great, but maintaining them was a real challenge as over time we renamed buttons, reorganized menus, and were constantly fixing broken regression tests. One thing that I would disagree with Elisabeth on is that the maintenance burden for regression tests goes down if you are running them constantly. If you run them constantly, and the developers are responsible for them passing they same as unit tests, then that keeps them up to date, and adding more value then they might otherwise. But I agree, they are a burden, and sometimes really only worth it for certain high risk/high error portions of your application. Unfortunately putting something like Mercury Interactive based tests under CI is hard, and Selenium isn’t that much easier…
A lot of my interest about writing “Stories” using the RSpec framework comes directly from trying to make the tests easier for people to read, and separate out the logic of clicking on a button from the desire to perform an action that would require clicking on that button!
I met Elisabeth at CITcon 2007, and I still wear the Test Obsessed wrist band she gave me!