rss
twitter
    Twitter feed not available

How Do You Start Writing Tests?

I am beginning work on a personal project and I started writing unit/integration tests as one of the first steps. As I started, I realized this is the first project I am setting up from scratch and starting to write tests. (Other projects I have worked on I was brought in after the initial round of tests were already written. I simply needed to write new tests for new functionality we were adding.)

This lead me to think, how should I handle this? Should I write a bunch of tests, that I know will fail initially, and then go through and get them to pass? Or should I write one test, get it to pass, then write another test, get it to pass, and so on?

So, what do you think is the best way to handle this?

  • Write a bunch of tests and then go through to make them pass?
  • Write one test at a time and don't move on to writing another test until the current one passes?

8 comments

(Comment Moderation is enabled. Your comment will not appear until approved.)
Peter Boughton said...
Neither. Writing one at a time will drive you insane, whilst too many at once will bore you to death. :)

So write a handful of related tests, then work on the code until they pass, then do another handful, and so on.

(How big your handfuls are will of course depend on the sort of features your project has.)
Marc Esher said...
I tend to write one test at a time. It keeps me focused on the task at hand, and it helps me avoid thinking too far ahead. This isn't to say that I don't think about overall design... that's not what I mean. I simply mean that I try to stay in "simplest thing that works" land as long as possible
Steven Erat said...
To begin top-down, integration testing (i.e. UI testing of the web application by mimicking website traversal by a user), if the website was already quite comprehensive and in production, then you might want to identify the most critical user paths, those that would cost the most money if they encountered a defect or were unavailable, and write tests against those. Then define what the most heavily trafficked website areas are or those areas exposed to the largest user base, and test those paths. Then slowly try to expand the test base by prioritizing the remaining areas that are less frequented within the application. For this expansion stage, you could try to divide the application into logical areas, and complete testing as much as possible per logical area, one by one.

When writing Integration (UI) tests against a project in development, I would define test areas per feature then complete them only after the feature was frozen. Feature by feature, test area by test area.

To improve the ability to perform UI testing, I would develop with attention to naming objects as much as possible, using naming patterns that would be predictable. I have seen lots of areas in legacy code where objects were not named/id'd (divs, frame sets, you name it, table rows), and that causes the test developer to try to identify the object by its position with in the DOM, or by its XPATH location. The latter (DOM/XPATH) are somewhat more brittle than knowing an object's unique ID. Object location strategy is important in any integreation testing software such QTP or Selenium.

The ease of which you can write UI tests depends a lot on the ability to find all the objects of interest on a page, while making the tests flexible enough to anticipate unexpected page objects. Say for example, you dynamically create table rows by displaying a result set. As a developer it's easy to not think about making sure each table row as a unique ID, but if you were to do that, then test creation would be so much easier later. Again, you could write tests locate objects by DOM position, but it is much easier by ID. Use common sense, however, and know that it's not necessary to ID BR tag or other formatting markup, but try to uniquely ID all containers or data bearing parts of the page.

When individual test cases within a test suite interact with objects on the page like buttons or links they are prone to failure if the object ID or object position changed (say you moved a submit button from within one DIV to another DIV for better layout). If you have 100 tests all calling the button id="Home", and the button ID changes, you then have to go fix all 100 tests.

Test suites can be made much more robust and sturdy by using the Page Object Model. In brief, this is where you would (in ColdFusion) create one CFC representing one screen or one page in a web app, and that CFC would contain methods that abstract how interact with parts of the page.

For example, say you have a frame set with a navigation bar containing nav buttons across the top frame, and a frame set on the bottom with content matching the nav buttons clicked. The nav frameset could be modeled with a NavigationFrame.cfc which contained methods such as click_button_HOME() and click_button_LOGIN() (using a naming convention that works for you. I prefer a action_object_purpose convention). Those methods would encapsulate code that knows how to locate those specific buttons, such as by ID, by DOM position, or by XPATH. Individual test cases would access persistent objects representing the website pages, and then call the methods on those page objects in order to navigate through and interact with the application.

When developing tests, organize your Page objects in one area, and your tests in another. Both your page objects and your tests can refer to the driver (i.e. selenium), but only your tests should contain assertions.

Example:

import Pages.*;
...
NavPage = new NavigationPage(selenium);
LoginPage = NavPage.click_button_LOGIN();

The value of this is that if the button with id="LOGIN" ever changed to id="LOG_IN", then you could fix just that name on one line in one method in the Page Object, then all the tests that required clicking that button would be able to identify that button.

This gets a little better when Page Objects start returning something meaningful such "this" (itself) or an instance of another Page Object. Then you can start to use method chaining (within the limitations of what CF Builder can do).


Example:

import Pages.*;
...
UserHomePage = new NavigationPage(selenium).click_button_LOGIN().submit_form_DOLOGIN();

(I probably digressed a little (ok a lot) from the original question you posed, but I needed a break from watching Gnomeo or The Wild Krats for the 65th time.)

Hope something there is useful :)
Phill Nacelli said...
I say exactly what the test names imply.

Unit Testing - Once you have finished a given functional piece, test it by itself.

Integration Testing - Once you have finished a functional piece that needs to interact with another, test that integration piece.

Regression too.. and so on..

And remember, Soe Zarni is watching you! (hope you get the reference..)
Jim Priest said...
It sounds like you are starting from scratch? If so you could explore the whole TDD thing - write tests first - then write your code around the test. I always was intrigued by this but have never had the time to actually try it. If it's a personal project - and time isn't an issue it might be a fun experiment.

In the projects I've inherited I tend to take Marc's approach and do one at a time, with some of Steven's ideas - concentrating on the areas where I'm going to get the most bang for my buck for the time spent to write a test.
Mark Mandel said...
To be honest, I kind of go by gut.

I'll generally write one test at a time (which will often contain multiple assertions, just so I'm clear).

If I'm "in the zone", so to speak in regards to thinking through how this particular bit of code I'm writing will work, I'll keep hacking away at tests until I feel it's covered (i.e. multiple tests may be needed to cover off a piece of functionality).

Or if I'm still in the zone where I'm not 100% sure how the code will work, I'll carve off smaller chunks, test those specific bits, and work more iteratively towards my solution.

So.. uh.. it depends? ;)
existdissolve said...
Unit testing is something I'm just now getting into...so I'm interested to see what other responses come in (interesting reading so far).

Since this is first time you're doing this from scratch, have you thought about blogging your experiences? I know that something like that would be extremely helpful for people like me who are just getting into it.
Scott Stroz said...
Jim - I am approaching this as an exercise in TDD - I am not even attempting anything with the UI until the model is done.

The fact that this is the first time I am starting a project from scratch is what has be a bit confused as how to start. My 'gut' was telling me more of a mix - some tests may need to be done one at a time, others may be done in 'batches'. I just wanted to get the opinion of others who have more experience than I doing this.