Developer Blog

User-level Feature Specs With Cucumber

Summary

Applications are not just collections of technology. They are designed to provide meaningful functionality within the user’s domain of experience. To achieve that, they encapsulate complex technical implementations under intuitive, human-friendly user interfaces.

Congruent to that, the specifications for said application functionality should also be on the level of user experience, with their underlying technical implementation encapsulated.

Cucumber is often misunderstood as an unnecessary detour from expressing feature specs more directly in code. In this blog post I demonstrate that Cucumber’s code and language patterns emerge naturally when organizing/refactoring complex feature specs.

This substantiates the understanding of Cucumber as a set of patterns, tools, and programming languages specialized for expressing feature specs on the same semantic level as the functionality they describe, the level of user experience.

Introduction

An essential part of TDD are feature specifications (aka functional or integration tests). To verify that our application as a whole works, we fire up the complete application stack as well as a scriptable interaction device (browser or mobile application simulator). Using the latter we simulate users interacting with our app (clicking links and buttons, filling out forms etc) and check that our application as a black box exhibits the correct behaviors (displays the correct responses, sends the right messages to other apps etc). These feature specs can even drive the development of their features.

For simple feature specs we often don’t need anything beyond a fixture and mocking library together with a UI driver. As feature specs grow in size, however, expressing complex user interactions solely using only these intentionally low-level tools becomes increasingly cumbersome. Here is a representative example: the feature spec for changing the password of a user account in a typical web application. We use Ruby, RSpec, Capybara and Factory Girl.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
scenario 'changing the password' do
  user = create :user, email: 'tester@foo.com', password: 'old password'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully'
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: 'new password'
  fill_in 'Password Confirmation', with: 'new password'
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
  click_link 'Log Out'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'new password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully'
  click_link 'Log Out'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Login not successful, please try again'
end

Did you understand what this quite massive and cumbersome spec verifies? How does changing the password work? How long did it take you to understand all that? How much low-level source code did you have to read, parse, and execute in a virtual browser in your head in order to derive how the application is supposed to behave here? And that was still a relatively small, simple, and straightforward feature!

Although the spec nicely lists all the individual steps for changing a user’s password, it is too low-level. It is hard to see how the product actually works, and I am not confident from just looking at this that we didn’t forget to check something. This is merely what a developer thought the product should do, expressed in ways only a developer understands. But like all people, developers occasionally misunderstand requirements, or translate them incorrectly into code.

Commented groups

As a start, let’s group related steps together and add some comments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
scenario 'changing my password' do

  # Create a user with email "tester@foo.com" and password "old password"
  user = create :user, email: 'tester@foo.com', password: 'old password'

  # Log in as that user
  visit '/'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully.'

  # Change that user's password to "new password"
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: 'new password'
  fill_in 'Password Confirmation', with: 'new password'
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'

  # Verify that we can log in with the new credentials
  click_link 'Logout'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'new password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully.'

  # Verify that the old password doesn't work anymore
  click_link 'Logout'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Login not successful, please try again'
end

Great, this has already made more clear what we actually do here! But comments in front of blocks of code are an indicator that a method does too much (more than one thing), and that new methods want to emerge here. Also, this method is too long, and this code is not reusable. For example when testing other scenarios, we don’t want to duplicate the code for logging in.

Extracting reusable methods

Lets extract reusable methods. Doing so also gives us a chance to remove a now unnecessary comment, because the respective code piece is now self-describing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
scenario 'changing my password' do

  # Create a user with email "tester@foo.com" and password "old password"
  user = create :user email: 'tester@foo.com', password: 'old password'

  # Log in as that user
  login_with email: 'tester@foo.com', password: 'old password'
  verify_login_succeeded

  change_my_password_to 'new password'

  # Verify that the new password works
  logout
  login_with email: 'tester@foo.com', password: 'new password'
  verify_login_succeeded

  # Verify that the old password doesn't work anymore
  logout
  login_with email: 'tester@foo.com', password: 'old password'
  verify_login_failed
end


# Helper methods

def change_my_password_to new_password
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

def login_with(email:, password:)
  click_link 'Login'
  fill_in 'Email', with: email
  fill_in 'Password', with: password
  click_button 'Log In'
end

def logout
  click_link 'Logout'
end

def verify_login_succeeded
  expect(page).to have_content 'Signed in successfully.'
end

def verify_login_failed
  expect(page).to have_content 'Login not successful, please try again'
end

The scenario is now more concise and reads better. And the extracted methods make sense. But it feels like we aren’t quite there yet, and there is more we can do here.

I bet most of my feature specs have to create a user and then log in as that user. Let’s combine those steps into one.

Also, our spec contains two separate levels of abstraction now: comments describe higher-level end-user perspective, i.e. what people want to do with the product, and the corresponding code blocks represent the respective technical implementation, i.e. how to do these things. Our current feature spec mixes these levels inconsistently:

  • Comments and methods like change_my_password_to are on the high-level end-user perspective.
  • Code like create :user is on the technical implementation level.
  • Methods like login_with are in between: they already encapsulate pieces of end-user interaction, but need to be combined with other steps to form full end-user interactions.

All of that smells bad, so let’s keep refactoring.

Separate product perspective from implementation

Let’s make it so that our scenario solely describes the high-level end-user perspective, and all of the technical implementation is encapsulated in helper methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
scenario 'changing my password' do
  create_user_and_login_with email: 'tester@foo.com', password: 'old password'
  change_my_password_to 'new password'
  verify_login_works email: 'tester@foo.com', password: 'new password'
  verify_login_fails email: 'tester@foo.com', password: 'old password'
end


# High-level product-perspective methods

def change_my_password_to new_password
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

def create_user_and_login_with(*args)
  create :user, *args
  login_with *args
  verify_login_succeeded
end

def verify_login_fails(*args)
  logout
  login_with *args
  verify_login_failed
end

def verify_login_works(*args)
  logout
  login_with *args
  verify_login_succeeded
end


# Reusable building blocks

def login_with(email:, password:)
  click_link 'Login'
  fill_in 'Email', with: email
  fill_in 'Password', with: password
end

def logout
  click_link 'Logout'
end

def verify_login_succeeded
  expect(page).to have_content 'Signed in successfully.'
end

def verify_login_failed
  expect(page).to have_content 'Login not successful, please try again'
end

Some parts of our scenario try to sound a bit too much like English for being actual method names. They are too long. This isn’t well-factored, technically sound source code. We shouldn’t start naming our methods like that in the rest of the code base.

And it’s still doesn’t really come together. It doesn’t form a cohesive user story that makes me wave my credit card and say “Yes, I want that!” It’s not clear why we do all these steps, and what we are actually testing here. That creating users works? That passwords can be changed? That logging in still works after a password has been changed?

Part of that is because such concepts have to be explained, but this is still nowhere near real, intuitive English. Trying to make a general-purpose programming language sound like a natural language only gets us so far. In my experience it will always feel like putting lipstick on a robot, and there is no good solution here.

Describing the product part in plain English

Ultimately it is questionable whether a general-purpose programming language is the most appropriate tool here altogether. Feature specs don’t contain complex algorithms, loops, code paths, or inheritance. They don’t even require functions or variables per se. Feature specs just express a number of linear user interactions with an application, expressed from a non-technical human perspective.

We only described our scenario in code because its underlying implementation is technical, and as developers code is our hammer. But not everything requires code. Let’s try something more close to natural language: Gherkin

1
2
3
4
5
Scenario: changing my password
  Given I am a user with email: "tester@foo.com" and password: "old password"
  When I change my password to "new password"
  Then I can log in with email: "tester@foo.com" and password: "new password"
  And I can no longer log in with email: "tester@foo.com" and password: "old password"

Wow, that feels like a breath of fresh air. We expressed our interactions with the application in perfect English. For the first time, it’s absolutely clear what we are actually doing and verifying here, and why.

Gherkin is part of Cucumber. Let’s see how the corresponding step definitions look. If you wonder about Kappamaki below, it converts textual lists into collections.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Given /^I am a user with (.+)$/ do |user_data|
  user_data = Kappamaki.attributes_from_sentence user_data
  create :user, user_data
  login_with user_data
  verify_login_succeeded
end

When /^I change my password to "(.*)"$/ do |new_password|
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

Then /^I can log in with (.+)$/ do |login_data|
  logout
  login_with Kappamaki.attributes_from_sentence(login_data)
  verify_login_works
end

Then /^I can no longer log in with (.+)$/ do |login_data|
  logout
  login_with Kappamaki.attributes_from_sentence(login_data)
  verify_login_failed
end

These are the same high-level product-perspective methods we had before, just with more descriptive English names. The bodies are almost identical to the ones written in Ruby. The reusable helper files don’t change at all.

As we can see, Cucumber provides facilities to represent the abstractions that naturally emerge in well-factored, complex feature specs. And it allows to represent them in a more appropriate format than a general-purpose programming language can. Other advantages are:

  • Product experts can verify that feature specs describe the correct application behavior, resulting in better team play between the product and development departments.

  • User stories can be written directly in Gherkin. This means one less conversion step from product description to code, which means one less opportunity for things to get lost in translation. And less meetings.

  • Feature specs can be understood and executed by both machines and humans. Automation allows to catch bugs and regressions earlier, thereby making everybody’s life easier. Knowing that this happens, Quality Assurance (QA) personnel no longer have to do the boring and repetitive task of re-verifying already-tested functionality, but can instead focus on finding new issues and ensuring that the product looks correct.

I hope it becomes more clear that Cucumber as a platform for intuitive, user-level feature specifications provides value to the entire agile organization, including the development team. It allows for better functional testing than general-purpose programming languages, and should be a part of most serious agile projects.

Robust and mature Cucumber implementations are available for Ruby, JavaScript, the JVM, Python, .NET, and many other platforms. You can even develop cross platform Android and iOS specs with it.

No more low-level Gherkin that merely wraps individual interaction steps. That’s what Capybara is for. Cucumber is a high-level specification layer with end-user perspective, on top of the underlying technical implementation.

The future is green, friends!

Comments