25 September 2008

TDD. Killing the messenger?

Are we really doomed to another generation of procedural code masquerading as objects?

Roy Osherove has come out against "mocking" in favour of "isolating". One reading of his point is that mocking is just too hard for the bulk of the programmers he sees. Perhaps that's true (the others developers who discovered the technique are all very good) but I hope not (either that or I should increase my rates). And I'm pleased to see Oren, for one, has responded.

Roy addresses important issues which we need to think about, but we shouldn't pitch the whole industry at the level of its least skilled members; I've spent too much of my life cleaning up code that should never have been written in the first place. In the meantime, if his staff don't understand basic Object-Orientation, then that's what he should be teaching first, with a sprinkling of acceptance testing on top for regression.

Reading the post and its comments again, I see repeated "Doctor, it hurts when I do that" moments. Roy's response is to back off interaction testing, but that misses the experience and insight that led us into this situation in the first place. My call is that, like Object-Orientation, the technique is widely misunderstood, which is why the same objections keep coming back. This isn't a comfortable position.

To take the most obvious example, as Roy says himself, record/replay just doesn't cut it. That's why we've never supported it. It's easy to start with, but then my experience is that it encourages unmaintainable tests. It focusses the language of a test on the accident of the current implementation, rather than on the significant relationships between objects. It also encourages asserting every interaction, which leads to just the sort of problem that everyone complains about, brittle overspecified tests. That's why we don't do it, we only assert interactions that can affect the world around the object under test and stub the rest.

The other disaster (yes, I think record/replay has been a disaster) was our early attempt to mock the entire Java libraries. Apart from being unrealistic, it sent entirely the wrong message, emphasising isolating from databases over guiding the design. Later, Joe Walnes coined the notion of "Only Mock types you own", which really helped some people get the idea.

As someone posted somewhere in this exchange, TDD will punish you if you don't understand design. I'd argue that that's what it's for, and that it's essential to listen to what the tests are saying rather than turning down the volume.


Jamie said...

I wrote my comment on his page before I read yours.

Interesting, I would say great minds think alike, but I wouldn't want to drag you down to my level :-)

He's got a few points though.


Seb said...

Can you point me to any (informed) discussion on the relative merits of Record/Replay vs. AAA?

I'm trying to find a suitable Mock framework for C++ and had thought that MockItNow might be OK, but it definitely favours Record/Replay. Can you recommend any others?