Steve Tooke

Cucumber and Full Stack Testing

2013-01-18T07:55:00-08:00

18 January 2013

There has been two similar questions asked on two different mailing lists I subscribe to (Corey Haines’ BAWCH mailing list, and Ruby Rogues Parley list). Both of these lists are private so I thought it would be worthwhile posting my answer here.

Both of the questions were concerned with out-side-in development, full-stack integration testing, and how much of the application needs to be tested through the entire system.

Firstly consider why we write cucumber scenarios (or fitnesse test cases). These tests are business facing acceptance tests. They are a medium through which we can engage with the business people on our team and to help us understand how the system should behave. They give us an opportunity to check our understanding of what the system should do — to check the business’s understanding of what the system should do. We automate these tests to give the business confidence that the system behaves as expected.

Full-stack, end-to-end, integration tests are there to give us confidence that the system fits together correctly, that we have all the different pieces in place, and they are able to talk to each other.

It’s very easy to conflate these two concerns. I have worked on many systems where the business facing acceptance tests were also the end-to-end integration tests. The test runs end up being slow, and the tests are cumbersome to work with.

I’ve been talking about this with Matt Wynne and he drew the following diagram:

The circle on the left represents the tests that we would write in cucumber (or fitnesse). The circle on the right the tests which exercise the whole system end-to-end. In the centre we have the intersection — our cucumber scenarios which we run end-to-end against the whole system.

The key thing is that your business acceptance tests do not all have to drive the whole system end-to-end. We only a need a few scenarios to go end-to-end to give us the confidence the system as a whole is working. We can also write system tests, that aren’t part of the acceptance suite, to test specific integrations

Try to write acceptance tests that directly drive the domain objects. Use these to accurately describe your application’s behaviour. Focus them on the behaviour by not having them integrate the UI and the database.

Delegation is not inheritance

2012-08-08T06:16:00-07:00

08 August 2012

On the train home last night I watched the excellent Jim Weirich Play-by-play from PeepCode.

During the screencast Jim develops a library that “protects against unauthorized data model modification by users in less-privileged roles.” The screencast provides a great insight into the way Jim approaches problems, designs apis, and how he’s customised his environment to suit the way he works.

His approach is to build a proxy object which wraps the object to be updated, and provides a whitelist for fields which can be updated. He also inadvertantly demonstrates an easy mistake to make when using proxy objects.

Here is a simplified version of Jim’s solution - without any of the api for creating / finding proxies - which we will use to demonstrate this pitfall and its effects.

This approach works great for silently dropping calls to the accessor methods that are not in the provided whitelist. Here are some rspec examples which show how it works.

Now if we imagine we have a rails project, we can create a proxy to wrap our ActiveRecord object, and specify an attribute whitelist. This should then prevent mass-assignment of any non-whitelist attributes - it could be used in a controller like this:

Unfortunately this won’t work as we might expect.

Proxying like this is a great way to add new behaviour to existing objects, without modifying them - or creating new subclasses. but there is one thing to be aware of when you are using delegation in this way.

Methods called on the wrapped object have no knowledge of the methods in the proxy object.

So what happens when we call proxy.update_attributes?

The proxy object immediately delegates that method call to the user object, it will call user.update_attributes.

If you have used ActiveRecord, you will be aware of the way that ActiveRecord::Base#update_attrbiutes will make use of the accessor methods on its instances to set the field names.

So, user.update_attributes name: 'Joe' will call user.name = 'Joe', not the accessor methods on the proxy.

As we are not calling the accessor methods on the proxy, we aren’t filtering out the fields that don’t appear in the whitelist and our attribute protection won’t work when we use update_attributes.

Here is another example. Capitalise wraps an object and provides a upper case version of its name method.

Because greet is defined in the Person class, when it calls name it will always call Person#name.

This has caught me out a couple times. It’s so easy in ruby to create proxy objects or decorators that its easy to forget that you have a different object.

One solution is to implement a version of update_attributes on the proxy object.

Here we add an update_attributes method to the ProtectionProxy class - this only allows attributes allowed by the whitelist through to User#update_attributes.

The screencast ends with a note that Jim noticed this error later after recording of the screen cast finished. Jim’s complete solution, including the nice api, can be found on github.

Here is the whole of the ProxyProtection implementation, with rspec examples.

Tracking Project Emotions Using MercuryApp

2010-11-23T01:16:00-08:00

23 November 2010

I’ve been trying out MercuryApp as a way to track how I’m feeling about a project we’ve just started. I’m tracking once a day, usually at about 4pm.

We try to run regular retrospectives on our projects, this gives us a chance to reflect on how the project is going. We often include a segment where each participant draws a ‘sparkline’ of how they were feeling over the period of time leading up to the retrospective. This let’s us see how various events effect peoples emotions.

This time I had my sparkline already. MercuryApp had been tracking this data for me all week. The surprise for me though was that I seemed to be more upbeat about things than I remembered. If I hadn’t had MercuryApp I’m sure my sparkline would have been less positive.

I suspect that the negatives I perceived were something to do with my mood at the time of the retrospective. It as towards the end of the day so I was tired, and I’d had a tough day.

I’m guessing that had my day been better, and I was in a more positive mood then my sparkline may well have been influenced in the other direction.

Having real data is great, not only is it more accurate when thinking about the previous week, but looking at how it differs from your perception gives you the chance to reflect on your mood right now! Liking the results from MercuryApp so far.

Disclaimer: Eden Development have been working with the guys at MercuryApp, but I have not been involved in the project.

Fixing "There was a problem with the editor 'vi'" for Git on Mac OS X Snow Leopard

2010-04-08T15:31:00-07:00

8th April 2010

I have had an annoying problem with git and vi. I like to use vim to edit my commit messages, but I’ve been hit with this annoying message every time I write the message and quit vim.

error: There was a problem with the editor 'vi'

After a little bit of digging I found that this message is shown by git when the editor exits with a non-zero exit code. You can use $? to see the exit code of last script or application.

$ vim     # then exit vim with :q immediately
$ echo $?
1

I’m still not sure why vim is exiting with non-zero exit code, but it is definitely related to my .vimrc - moving it to .vimrc.bak seemed to fix the problem. I’m using the excellent pathogen plugin to manage my vimfiles, so I plan to go through that and my installed plugins to find the cause of the problem.

There is a fix though, I’m not sure what’s causing this, but I found a post on the vim-mac mailing list which shows this:

$ vim          # and exit with :q
$ echo $?
1
$ /usr/bin/vim # and exit with :q
$ echo $?
0
$ which vim
/usr/bin/vim

Running vim with /usr/bin/vim seems to make it exit cleanly. So to fix the problem with git commit you just need to run this:

$ git config --global core.editor /usr/bin/vim

I’d still like to get to the root of the problem, but this gets me my git commit messages back!

Why I think you should go to a code retreat

2010-03-17T15:51:00-07:00

17th March 2010

Last Saturday I went along to the UK leg of Corey Haines Code Retreat tour. Apart from the early start it was a really interesting day, and I really enjoyed the chance to pair with lots of new people.

The day was pretty tiring, pairing is always an intense experience, but I definitely learnt quite a lot. Its amazing how much fun working on the same problem several times in a row is, and how different approaches affect the way you think about it. By repeating the problem you allow your brain to concentrate on how you are solving it, rather than the problem itself. This gives you a really different perspective and is something I want to explore more.

There were so many things that I took away from the day, some of them are already changing my approach to building software:

45 minutes is a really short amount of time

The format of the code retreat is to work on the problem with a pair for 45 minutes. When the time is up, you delete all your code, and take break. Spend 15 minutes grabbing a coffee, reflecting on what you’ve done, and finding your next pair. Rinse and repeat.

Every 45 minute session flew by. Even so, with each pairing I was surprised at how far we’d managed to get. But what really surprised me, was how useful the short break and change of partner proved to be. The break gave you a real chance to reconsider your assumptions. That little bit of perspective was great in kick starting the next session.

It really surprised me how easy it was to swap pairs. Granted, everybody had been thinking about the same problem. But everyone was using a different approach, and sometimes a different language. The context switching didn’t seem to affect anybody. The new combinations brought new ideas and really contributed to the success of the day.

We’ve been trying the pomodoro technique at work. I know I’ve been a bit resistant to stopping when the timer goes. I always feel like I should just get the rest of my ideas out before I take break, but based on my experience at code retreat, I think the short break from the problem will turn out to be a real benefit. I’m determined to try and do it properly and see how it works out.

I also want to try and swap pairs more often. I think that the new perspective a new pair will bring to a problem will really help to come to the best solution. I’m not sure about every hour, but once or twice a day should be achievable.

Pairing is a great way to share insights and learning

Leading on from the new perspective a new pair brings is also the amount of shared learning that happens when your pairing. I learnt something from everyone I paired with. Not just how to approach the problem, but new things about the language, the tools. In a team, pairing will really help to bring every team member up to speed on any new part of the code base or library added. Switching often will make this happen even faster.

If you don’t need the infrastructure yet, don’t build it

Why do you need to build a class to make your first spec pass? Why not just write the code you need in the spec? Then write the next spec, and the code to pass it in that spec. As soon as you start to see shared behaviour extract a method. When specs are using the same state and the same methods extract a class.

Working like this is really hard, but its amazing how the design you need just starts to show itself.

Look at one behaviour at a time by isolating it using canned responses

Most of the code we write doesn’t split up nicely into discreet chunks of behaviour. We build systems that rely on several pieces all working together to produce complicated behaviour. Complexity is difficult to define, so to make it easier we need to try to isolate the part that we’re interested in right now. We can use simple objects that return canned responses, this allows us to consider only the behaviour we care about now.

Keeping things really simple is really hard

Corey was continually encouraging us to keep things simple. Its amazing how often you think your doing something as simply as possible, and then someone comes along and makes it even simpler. Simple is good, it allows you to work on one thing at a time, and not get bogged down in things that don’t matter right now.

A lot of the direction at code retreat was about ways to keep things simple, to specify only the smallest piece of behaviour. Writing the code in the spec at first, and using ‘doubles’ to isolate behaviour are both great techniques to help you do that.

One of the main things I’m taking away from code retreat is to work hard at writing smaller, more focussed specs.

If there’s a code retreat near you I really encourage you to go along. If there isn’t join the community and see if there’s anyone else who would be interested in helping get one organised.

I’d like to say thank you to Corey Haines, the sponsors RiveGlide and Eden Development, all of the attendees, and of course Bletchley Park and the The National Museum of Computing for making the day such a great success.

Exploring Harmony for javascript BDD with RSpec

2010-03-02T03:54:00-08:00

2nd March 2010

We try to BDD all of our production code, but the one area we always seem to struggle with is our javascript. There are various test/spec frameworks for javascript, but we’ve never quite found one we’ve been totally happy with.

There’s been a fair amount of interest lately in a new ruby gem which allows you to execute javascript against a DOM from within a ruby process. Harmony uses Johnson a ruby wrapper for the Mozilla SpiderMonkey javascript runtime.

To get started figuring out how I might be able to integrate harmony into my workflow, I’ve created a very simple project which uses rspec to make some very simple assertions about javascript behaviour.