Over the last seven years, our team at Originate has created countless iOS apps. Along the way we have continuously fine-tuned our iOS development process, creating best practices that we apply to every new app we build.

We’ve put some of this down on paper in the form of two checklists, one used when starting an iOS project, and one used when preparing to submit to the App Store.

Following these checklists has helped our team work together more efficiently, architect better solutions, reduce development time, and reduce the risks that come with publishing an app to the App Store.

We hope these checklists will do the same for you.

Starting an iOS project

• Repo/GitHub

  1. Start by creating a new repo in GitHub and adding the appropriate iOS gitignore file.

  2. Be sure to follow the git-flow workflow, with master holding your App Store builds, dev your latest development code, and feature branches holding the current work in progress.

• Xcode

  1. Make sure everyone in the team is on the same version of Xcode.

  2. Turn on “Analyze during build” and “Treat Warnings as Errors” (Set it under “Build Settings” in your application target).

  3. Turn off tabs and use spaces: XCode > Preferences > Text Editing > Indentation > Prefer Indent using Spaces, Tab width 2, Indent width 2.

• Jenkins/OS X Server/TestFlight

  Setup CI/CD (Continuous Integration/Continuous Deployment) making sure with every push to dev all tests are run and an Ad Hoc build is created/emailed to the team (with the commit logs in the email body). If the build or tests failed, a failure email should instead be sent out to the team. At Originate most of our iOS projects use OS X Server integrated with TestFlight.

• Coding Style/Standards

  1. Follow Apple’s recommended iOS coding style guide.

  2. In addition, follow recommendations here: http://qualitycoding.org/preprocessor/

  3. Keep your .h files short and simple, exposing only what is needed by other classes. Move all other methods, properties, instance variables declarations, etc. inside the .m file.

  4. Name your view controllers based on the page they are displaying (e.g. LoginViewController).

  5. Organize your project navigator and files using groups. Good group names are Data Models, Views, Controllers, App Delegate, Supporting Files, Tools, etc. Messy project navigators should not be accepted.

  6. Before submitting a pull request, go over the checklist at the bottom of our effective code review blog post and look for red flags.

• Architecture

  1. MVC (Model View Controller) is sometimes jokingly called Massive View Controller in iOS development. These massive controllers that do everything are common mistakes for beginners. They are not acceptable. As needed, split out table related delegates/data sources from the view controller into their own separate classes. Split out views (especially if they are reused) into their own view classes. Make sure helper methods are pushed out into helper classes. In addition, the View Controller should not make any calls to the server, instead the Model or a manager class should handle this.

  2. Some good sample code/tutorials/patterns: Lighter View Controllers, Viper and BrowseOverflow (from the iOS TDD book).

• Views/Nibs/Storyboards

  1. Be sure to use constraints/autolayout for your views and support different screen sizes. Otherwise you will have to manually configure the frame sizes/positions for each view to make sure it fits correctly for each screen size. PureLayout and FLKAutoLayout have been used on some projects at Originate.

  2. Determine if Nib files will be used. Originate recommends avoiding them, but leaves the decision to the Tech Lead. Storyboards are not used as they they are hard to manage with multiple developers (e.g. trying to merge a storyboard), slow down Xcode, and add a level of complexity to the code that is not necessary.

  3. Use FrameAccessor to modify frames if needed. This will make it very easy to get and set a UIView’s size and origin.

• Fonts and Colors

  Standardizing fonts and colors throughout the app (i.e. create helper classes) so that it’s easy to maintain/modify them as needed. This also makes for cleaner code that doesn’t have random RGB or font strings scattered in the views.

• Display text

  1. All strings displayed to the user must be placed in a localization file.

  2. Avoid using image assets that contain text, use a UILabel instead, pulling the text out of the localization file.

• Analytics

  http://replay.io is our go-to platform for analytics at Originate.

• Crash reporting

  Although Apple and iTunes Connect are making progress here, it’s still best to use a third-party tool like Crashlytics. They work much faster and have a better UI/reporting interface.

• Add AOP support if needed (e.g. for logging).

• Third party code dependencies

  Cocoapods can be used to maintain third-party dependencies. At Originate this decision is left to the Tech Lead.

• Server communication

  1. Add the ability to toggle between server environments (e.g. QA, dev, staging, etc.) in app settings.

  2. If needed, implement an app update notification system, where the server can inform the app a newer version is available and the app can display the appropriate message to the user.

  3. Make sure an activity indicator is displayed during server calls. MRProgress has been used on some projects at Originate.

  4. AFNetworking or RestKit (if Core Data is needed) should be used for network communication. See our AFNetworking cache blog post for optimizations/configuration.

  5. For debugging, make sure all server communications (i.e. web service requests/responses) are printed out to the console.

  NOTE: Use DLog to log all requests and response (in debug mode) to the console:

1
2
3
4
5
6

  #ifdef DEBUG
  #define DLog(fmt, ...) NSLog((@"%s [Line %d] " fmt), __PRETTY_FUNCTION__, __LINE__, ##__VA_ARGS__);
  #else
  #define DLog(...);
  #endif
  

• Polish

  Small tweaks to an app can go a long way. Make sure to include simple user expected features like pull-to-refresh, tap status bar for scroll to top, activity indicator during server calls, etc. to make the app user friendly. Your Design and Product team should be on top of this already. The iOS Human Interface Guidelines is a good reference doc for them.

Lastly, ensure that the process to create the needed accounts (e.g. iTunes Connect, Urban Airship, Analytics Accounts, etc.) is started.

Submitting to the App Store

Releasing to the App Store must not be taken lightly. Steps must be taken to ensure the submitted app follows Apple’s guidelines and will work once published to the store. Four to eight hours should be allocated to perform this task.

When your app has been tested and is ready to release, follow these steps:

• Apple Guidelines

  1. At a high level, Apple goes over everything related to App testing, Ad Hoc distribution, and App Store release in the App Distribution Guide. This document should be reviewed.

  2. And of course, most importantly, these documents should be memorized! App Store Review Guidelines and Common App Rejections.

  NOTE: Although not listed in the rejections link above, we’ve found that often developers forget to add content flagging for apps that create user generated data. Apple rejects these apps until content flagging is added.

• Core Data

  If you are submitting an updated app that uses Core Data, then you must make sure to write a migration script in order to properly propagate changes to the DB. If this is not done, then it’s possible that the app can crash once updated and the user will have to delete and reinstall the app. See Core Data Model Versioning and Data Migration.

• App Review Screen

  When you want your app to be reviewed by the user, UAAppReviewManager makes sure the review is done at the right time.

• Release Scheme

  In the “Edit Scheme” section of Xcode, “Archive” should be set to “Release”. Release builds must hide log statements and turn off any test code/test screens used for development.

  NOTE: Due to compiler optimizations, release builds can sometimes function differently than debug builds. It is best to start testing release builds a few days before the release to the App Store in order to capture any possible issues.

• Server

  Point your release build at the proper production servers (and use HTTPS)

• Release Candidate Ad Hoc builds / App Store build

  Make sure to use the correct Bundle Identifier, Apple ID, certificates (e.g. Push Certificates), etc. for the release candidate and App Store builds.

• Test Release Candidate configuration

  1. Make sure the app is communicating with the correct production servers (using HTTPS).

  2. Make sure all test screens are hidden.

  3. Make sure no sensitive data is printed to the console.

  4. Make sure Analytics is working with the correct production account.

  5. Make sure services such as Urban Airship are working with the correct production account.

• Push to master

  1. Update the bundle version number in the app plist file. This number does not have to match the version number you assign to the app on the App Store, but it’s best that it does.

  2. Push the code that is ready for release to the master branch and tag it.

• Setup the app in iTunes Connect

  1. In iTunes Connect create a new App, or if this is an update, select “Add Version”.

  2. Make sure to fill in all data and upload all necessary images (and video).

  NOTE: Only the fields “Description” and “What’s New In This Version” can be modified once the app is accepted and placed in the store. Other important fields like images cannot be modified.

• Submit

  Use Xcode to create the App Store build and upload to iTunes Connect.

Summary

Applications are not just collections of technology. They are designed to provide meaningful functionality within the user’s domain of experience. To achieve that, they encapsulate complex technical implementations under intuitive, human-friendly user interfaces.

Congruent to that, the specifications for said application functionality should also be on the level of user experience, with their underlying technical implementation encapsulated.

Cucumber is often misunderstood as an unnecessary detour from expressing feature specs more directly in code. In this blog post I demonstrate that Cucumber’s code and language patterns emerge naturally when organizing/refactoring complex feature specs.

This substantiates the understanding of Cucumber as a set of patterns, tools, and programming languages specialized for expressing feature specs on the same semantic level as the functionality they describe, the level of user experience.

Introduction

An essential part of TDD are feature specifications (aka functional or integration tests). To verify that our application as a whole works, we fire up the complete application stack as well as a scriptable interaction device (browser or mobile application simulator). Using the latter we simulate users interacting with our app (clicking links and buttons, filling out forms etc) and check that our application as a black box exhibits the correct behaviors (displays the correct responses, sends the right messages to other apps etc). These feature specs can even drive the development of their features.

For simple feature specs we often don’t need anything beyond a fixture and mocking library together with a UI driver. As feature specs grow in size, however, expressing complex user interactions solely using only these intentionally low-level tools becomes increasingly cumbersome. Here is a representative example: the feature spec for changing the password of a user account in a typical web application. We use Ruby, RSpec, Capybara and Factory Girl.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

scenario 'changing the password' do
  user = create :user, email: 'tester@foo.com', password: 'old password'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully'
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: 'new password'
  fill_in 'Password Confirmation', with: 'new password'
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
  click_link 'Log Out'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'new password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully'
  click_link 'Log Out'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Login not successful, please try again'
end

Did you understand what this quite massive and cumbersome spec verifies? How does changing the password work? How long did it take you to understand all that? How much low-level source code did you have to read, parse, and execute in a virtual browser in your head in order to derive how the application is supposed to behave here? And that was still a relatively small, simple, and straightforward feature!

Although the spec nicely lists all the individual steps for changing a user’s password, it is too low-level. It is hard to see how the product actually works, and I am not confident from just looking at this that we didn’t forget to check something. This is merely what a developer thought the product should do, expressed in ways only a developer understands. But like all people, developers occasionally misunderstand requirements, or translate them incorrectly into code.

Commented groups

As a start, let’s group related steps together and add some comments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

scenario 'changing my password' do

  # Create a user with email "tester@foo.com" and password "old password"
  user = create :user, email: 'tester@foo.com', password: 'old password'

  # Log in as that user
  visit '/'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully.'

  # Change that user's password to "new password"
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: 'new password'
  fill_in 'Password Confirmation', with: 'new password'
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'

  # Verify that we can log in with the new credentials
  click_link 'Logout'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'new password'
  click_button 'Log In'
  expect(page).to have_content 'Signed in successfully.'

  # Verify that the old password doesn't work anymore
  click_link 'Logout'
  click_link 'Login'
  fill_in 'Email', with: 'tester@foo.com'
  fill_in 'Password', with: 'old password'
  click_button 'Log In'
  expect(page).to have_content 'Login not successful, please try again'
end

Great, this has already made more clear what we actually do here! But comments in front of blocks of code are an indicator that a method does too much (more than one thing), and that new methods want to emerge here. Also, this method is too long, and this code is not reusable. For example when testing other scenarios, we don’t want to duplicate the code for logging in.

Extracting reusable methods

Lets extract reusable methods. Doing so also gives us a chance to remove a now unnecessary comment, because the respective code piece is now self-describing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53

scenario 'changing my password' do

  # Create a user with email "tester@foo.com" and password "old password"
  user = create :user email: 'tester@foo.com', password: 'old password'

  # Log in as that user
  login_with email: 'tester@foo.com', password: 'old password'
  verify_login_succeeded

  change_my_password_to 'new password'

  # Verify that the new password works
  logout
  login_with email: 'tester@foo.com', password: 'new password'
  verify_login_succeeded

  # Verify that the old password doesn't work anymore
  logout
  login_with email: 'tester@foo.com', password: 'old password'
  verify_login_failed
end


# Helper methods

def change_my_password_to new_password
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

def login_with(email:, password:)
  click_link 'Login'
  fill_in 'Email', with: email
  fill_in 'Password', with: password
  click_button 'Log In'
end

def logout
  click_link 'Logout'
end

def verify_login_succeeded
  expect(page).to have_content 'Signed in successfully.'
end

def verify_login_failed
  expect(page).to have_content 'Login not successful, please try again'
end

The scenario is now more concise and reads better. And the extracted methods make sense. But it feels like we aren’t quite there yet, and there is more we can do here.

I bet most of my feature specs have to create a user and then log in as that user. Let’s combine those steps into one.

Also, our spec contains two separate levels of abstraction now: comments describe higher-level end-user perspective, i.e. what people want to do with the product, and the corresponding code blocks represent the respective technical implementation, i.e. how to do these things. Our current feature spec mixes these levels inconsistently:

• Comments and methods like change_my_password_to are on the high-level end-user perspective.
• Code like create :user is on the technical implementation level.
• Methods like login_with are in between: they already encapsulate pieces of end-user interaction, but need to be combined with other steps to form full end-user interactions.

All of that smells bad, so let’s keep refactoring.

Separate product perspective from implementation

Let’s make it so that our scenario solely describes the high-level end-user perspective, and all of the technical implementation is encapsulated in helper methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

scenario 'changing my password' do
  create_user_and_login_with email: 'tester@foo.com', password: 'old password'
  change_my_password_to 'new password'
  verify_login_works email: 'tester@foo.com', password: 'new password'
  verify_login_fails email: 'tester@foo.com', password: 'old password'
end


# High-level product-perspective methods

def change_my_password_to new_password
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

def create_user_and_login_with(*args)
  create :user, *args
  login_with *args
  verify_login_succeeded
end

def verify_login_fails(*args)
  logout
  login_with *args
  verify_login_failed
end

def verify_login_works(*args)
  logout
  login_with *args
  verify_login_succeeded
end


# Reusable building blocks

def login_with(email:, password:)
  click_link 'Login'
  fill_in 'Email', with: email
  fill_in 'Password', with: password
end

def logout
  click_link 'Logout'
end

def verify_login_succeeded
  expect(page).to have_content 'Signed in successfully.'
end

def verify_login_failed
  expect(page).to have_content 'Login not successful, please try again'
end

Some parts of our scenario try to sound a bit too much like English for being actual method names. They are too long. This isn’t well-factored, technically sound source code. We shouldn’t start naming our methods like that in the rest of the code base.

And it’s still doesn’t really come together. It doesn’t form a cohesive user story that makes me wave my credit card and say “Yes, I want that!” It’s not clear why we do all these steps, and what we are actually testing here. That creating users works? That passwords can be changed? That logging in still works after a password has been changed?

Part of that is because such concepts have to be explained, but this is still nowhere near real, intuitive English. Trying to make a general-purpose programming language sound like a natural language only gets us so far. In my experience it will always feel like putting lipstick on a robot, and there is no good solution here.

Describing the product part in plain English

Ultimately it is questionable whether a general-purpose programming language is the most appropriate tool here altogether. Feature specs don’t contain complex algorithms, loops, code paths, or inheritance. They don’t even require functions or variables per se. Feature specs just express a number of linear user interactions with an application, expressed from a non-technical human perspective.

We only described our scenario in code because its underlying implementation is technical, and as developers code is our hammer. But not everything requires code. Let’s try something more close to natural language: Gherkin

1
2
3
4
5

Scenario: changing my password
  Given I am a user with email: "tester@foo.com" and password: "old password"
  When I change my password to "new password"
  Then I can log in with email: "tester@foo.com" and password: "new password"
  And I can no longer log in with email: "tester@foo.com" and password: "old password"

Wow, that feels like a breath of fresh air. We expressed our interactions with the application in perfect English. For the first time, it’s absolutely clear what we are actually doing and verifying here, and why.

Gherkin is part of Cucumber. Let’s see how the corresponding step definitions look. If you wonder about Kappamaki below, it converts textual lists into collections.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

Given /^I am a user with (.+)$/ do |user_data|
  user_data = Kappamaki.attributes_from_sentence user_data
  create :user, user_data
  login_with user_data
  verify_login_succeeded
end

When /^I change my password to "(.*)"$/ do |new_password|
  visit '/'
  click_link 'my account'
  click_link 'change my password'
  fill_in 'New Password', with: new_password
  fill_in 'Password Confirmation', with: new_password
  click_button 'Update password'
  expect(page).to have_content 'Changed your password'
end

Then /^I can log in with (.+)$/ do |login_data|
  logout
  login_with Kappamaki.attributes_from_sentence(login_data)
  verify_login_works
end

Then /^I can no longer log in with (.+)$/ do |login_data|
  logout
  login_with Kappamaki.attributes_from_sentence(login_data)
  verify_login_failed
end

These are the same high-level product-perspective methods we had before, just with more descriptive English names. The bodies are almost identical to the ones written in Ruby. The reusable helper files don’t change at all.

As we can see, Cucumber provides facilities to represent the abstractions that naturally emerge in well-factored, complex feature specs. And it allows to represent them in a more appropriate format than a general-purpose programming language can. Other advantages are:

• Product experts can verify that feature specs describe the correct application behavior, resulting in better team play between the product and development departments.

• User stories can be written directly in Gherkin. This means one less conversion step from product description to code, which means one less opportunity for things to get lost in translation. And less meetings.

• Feature specs can be understood and executed by both machines and humans. Automation allows to catch bugs and regressions earlier, thereby making everybody’s life easier. Knowing that this happens, Quality Assurance (QA) personnel no longer have to do the boring and repetitive task of re-verifying already-tested functionality, but can instead focus on finding new issues and ensuring that the product looks correct.

I hope it becomes more clear that Cucumber as a platform for intuitive, user-level feature specifications provides value to the entire agile organization, including the development team. It allows for better functional testing than general-purpose programming languages, and should be a part of most serious agile projects.

Robust and mature Cucumber implementations are available for Ruby, JavaScript, the JVM, Python, .NET, and many other platforms. You can even develop cross platform Android and iOS specs with it.

No more low-level Gherkin that merely wraps individual interaction steps. That’s what Capybara is for. Cucumber is a high-level specification layer with end-user perspective, on top of the underlying technical implementation.

The future is green, friends!

AWS OpsWorks is the next evolution of Amazon’s auto scaling, cloud deployment solution Elastic Beanstalk. OpsWorks attempts to abstract things a bit further, giving you more control over the various pieces of your deployment, see a tutorial here.

OpsWork has a pretty steep learning curve. To help with that, pre-made layers for various kinds of web apps are available (Rails, Node.js, PHP, Java, Static). This post will show you how to setup a Play! Framework app on OpsWorks in just a few steps.

I – Create Your Deployment Cookbook

An OpsWorks app uses Chef cookbooks to provision instances and deploy applications. For this example we’ll create a public git repository with a copy of Originate’s Play Framework cookbook. Creating a separate repository for your app’s cookbooks makes it easier to work with OpsWorks and also allows you to customize things as needed down the road.

Create a custom cookbook

First, install Berkshelf, which is available through ChefDK. Berkshelf makes it easier to manage cookbooks.

1
2
3
4

mkdir play-sample-opsworks
cd play-sample-opsworks
git init
berks cookbook app --skip-git # this creates the cookbook for our play application

This creates the basic structure of the cookbook. Next we need to add a reference to Originate’s play2 cookbook in the Berksfile:

1
2
3
4
5
6
7

# app/Berksfile

source "https://supermarket.getchef.com"

metadata

cookbook "play2", git: "https://github.com/Originate/cookbooks.git", rel: "play2"

One caveat of the OpsWorks Chef integration is that it doesn’t automatically fetch dependencies. Berkshelf can solve this for us by grabbing dependencies and vendoring them into our repository. Note that Berkshelf nukes the directory you point it at (including the .git folder), so use the following to download dependencies to a temporary directory and move the files afterwards:

1

berks vendor /tmp/cookbooks -b app/Berksfile && cp -rf /tmp/cookbooks/* .

Next create the recipes for the OpsWorks setup and deploy phases:

1
2
3
4
5
6
7
8

# app/recipes/setup.rb
#
# Cookbook Name:: app
# Recipe:: setup
#

# This installs the play framework
include_recipe "play2::setup"

1
2
3
4
5
6
7
8
9
10
11

# app/recipes/deploy.rb
#
# Cookbook Name:: app
# Recipe:: deploy
#

# This deploys the application
opsworks_play2 do
  app "app"
  deploy_data node[:deploy][:app] # This data comes from OpsWorks' JSON blob
end

In the next section, we’ll use what you’ve done here to setup a running OpsWorks system. If you’re following along, commit your work and push it to a repository that OpsWorks can access. Alternatively, you can use this sample repository.

II – Configure AWS OpsWorks

Now that you have your cookbook for your app setup, head to the OpsWorks Home Page to setup and deploy your app.

Create a Stack

The first step is to create a new stack. A stack is composed of one or more layers. A layer represents a group of instances with a specific purpose. For example, OpsWorks comes with a load balancer layer, a database layer, and some common web application layers such as Rails or Node. OpsWorks doesn’t provide a pre-made layer for Play! applications, so make sure to select “Use custom Chef Cookbooks” and fill in the URL of the repository you created in part I. If you’re using a private repository, don’t forget to add the SSH key in the textbox.

Next we need to tell the play2 cookbook to install version 2.1.3 of the Play! Framework and to bind the web server to port 80 instead of the default 9000. Add the following to the “Custom JSON” text box:

1
2
3
4
5
6
7
8
9
10
11

{
  "play2": {
    "version": "2.1.3",
    "http_port": 80,
    "conf": {
      "application": {
        "langs": "en"
      }
    }
  }
}

You can also add custom configuration for your application in the conf object. For additional information about which parameters can be tweaked, check out the README.

Create a Layer

Choose “Custom” as your layer type. This will represent the machines that run your Play application.

Edit Settings to Call Custom Recipes

When you edit your layer, you can add custom recipes to the different steps of your deployment. In order to get up and running, add app::setup to the “Setup” step and app::deploy to the “Deploy” step.

In general, the format for adding cookbooks to a specific step of the deploy process is <cookbook_name>::<recipe_name>.

Add the application

Add the sample application to your OpsWorks stack. Select “Other” for the application type, and use https://github.com/Bowbaq/sample-play-app.git for the repository URL.

Add an Instance

The next step is to add an instance to the stack. This will provision and deploy an EC2 machine into a context completely managed by OpsWorks.

Newly added instances are not started by default. Click on start, and OpsWorks will run the setup recipes for you (which will install Play!) and then the deploy recipes (which will deploy the app).

Once OpsWorks finishes, you should be able to visit http://<instance public ip> and see the following:

III – Profit

Now that you have your app setup on AWS OpsWorks, you can look forward to the following awesome features:

• One click deploy
• Automatic scaling based on various metrics
• Integrated instance monitoring
• Deployment tracking
• Easy environment management
• Easy Elastic Load Balancer integration

Conclusion

If you found this tutorial helpful and would like to contribute, feel free to fork the repository and submit pull requests. If you add any enhancements to your custom cookbooks that are worth sharing, contributions would be welcome. Additionally, I could use some help working on the enhancements listed here: https://github.com/Originate/cookbooks/issues.

Code reviews are a great way to improve code quality, catch bugs, and keep your whole team in sync with the latest changes. My favorite tool for doing a code review is still Github’s Pull Request feature, because of the easily-accessible diffs, inline comments, and commit-level and PR-level views of the file changes.

So, how do we make the code review process effective? I offer the following tips:

• Actually review PR’s; telling an author to “just ship it” is detrimental.
• Senior developers want you to review their code; don’t be scared.
• Close PR’s that don’t have specs
• If spec coverage is sparse, discuss it with the author (nicely).
• Spec descriptions should accurately reflect the behavior of the module/method they describe.
• Ask questions instead of making statements; the author likely put more thought into the code than you are aware of, and they’ve already solved the problem once.
• Look for design problems:
  • Does this change solve a real problem? is it worth solving?
  • Given the constraints, is the overall approach correct and reasonable?
• Look for code smells and suggest refactorings.
• Ask for help with reviewing code that is outside your area of expertise.

Ever seen a “code review” that looks like this?

LGTM :ship:

This is not a helpful code review. Rather than communicating the message, “good job,” it tells the author that you can’t or won’t take the time and effort required to properly review their work.

If you’ve ever done this, you’re not alone, and I’m not going to yell at you—I’ve done it too.

I often hear people say they aren’t qualified to review some portion of a pull request; generally, this is because of a lack of familiarity with the language used, or of the frameworks. While this is an understandable view, I argue that you are, in fact, qualified to review the code. That’s true even if it’s not your favorite language, or you’re not particularly good at programming in that language, or you’re not familiar with the framework. How do you review code you’re not well-versed in?

Remember that code reviews are a service, not an insult.

Please, please, review my code! I think everyone who has a title like “Senior X”, “Director of X”, “X Architect,” or whatever else knows the importance of having someone look over their code, ask questions, and point out mistakes. You don’t need to be afraid of reviewing someone’s code, no matter their position. It’s ok. I am not my code. Neither is anyone else. I also still write bad code sometimes. So does everyone else. We all need peer reviews, and we are all peers here.

Moreover, if you are reviewing code from a senior developer, don’t be a yes-man! Point out mistakes you see, ask about why specs are written a certain way, ask about architecture decisions, point out weird styling issues… “Senior” people need help too. If someone gets mad at you for nicely pointing out oversights or for asking questions, that’s a sign of immaturity, not seniority.

Ensure that there is only one feature in the pull request

Each organization does things a bit differently, but ideally, your pull requests should correspond to exactly one user story. If you can achieve this level of granularity, you gain a number of benefits:

• A feature spec can accompany the PR, and can be verified by your CI server.
• The code will be easier to reason about.
• If you use the nvie branching model (and you squash on merge), you can easily define different releases and cherry-pick features as you please.

Check that (nearly) all code has an accompanying specification.

If someone writes code without writing specs, they’ve introduced an unspecified, undocumented landmine into the code base. Your organization may have some policy about test-first, coverage level, etc.; I’m not going to get into that.

If you review code and there are no specs, talk to the author about it first. Discuss the importance of specifications, and offer to help them with writing some. I often suggest that the PR be closed and the code be rewritten using TDD.

Why do I recommend rewriting the code? Because if code was not driven by a spec, it is difficult to specify ex post facto. Bolting unit tests onto untested code results in brittle tests, encourages kludgy design, increases technical debt, and will make refactoring difficult in the future. If tests must be added on afterwards, they should be intergration tests.

I don’t say any of this to encourage dogmatic, or bully-like behavior. You and the author should discuss the specs, and their importance, and can come to a decision about the code together.

P.S.: Don’t close PR’s for purely cosmetic (i.e., only CSS/HTML) changes on this basis.

What about spikes? Aren’t they the exception?

Nope. Spikes are a really great tool, but their purpose is to provide a short-term proof-of-concept. They shouldn’t be merged; in fact, they shouldn’t even be reviewed. After a spike, should do a quick rewrite with specs.

Look for specs, and ask about the coverage level.

The first thing I do when I look at a PR is check the list of files changed (the github email digests are useful for this). If you see generic_modal.coffee was changed, then you should also see that generic_modal_spec.coffee (or something else that matches your naming scheme) was changed in the same PR.

Ideally, they should be changed in the same commit, or the spec should be changed first. This is not a rule, just a guideline; everyone commits a bit differently.

As mentioned above, keep an eye out for a feature spec (aka integration test). Optimally, there should be a one-to-one correspondence between a user story, a feature spec, and a pull request.

There should be relevant specs for all the code, both frontend and backend. If you don’t see something close to a 1-1 mapping between specs and source code, ask the author what’s going on. Some source code doesn’t need its own spec file: it may have been extracted from a spec’d module during a refactor; that’s ok.

If the author just neglected to write the specs, the PR should not be merged. This PR may be used as a reference for a spec-driven rewrite of the feature. If I see a PR without tests, I will close it. Specs are what give us the confidence we need to make changes later; we can’t afford to neglect them.

Read through the specs; they should make sense.

When you read through the specs, you should be able to quickly understand what each module does and what its public interface looks like. You should see relevant describe blocks, contexts, and it descriptions (or whatever your testing framework provides); they should describe, in English, what the module/API/UI does, and the code inside each should reasonably correspond to that description.

Ask the author to change non-obvious descriptions and tests; suggest a better description. For example:

1
2
3
4

describe '#fullName', ->
  it "works", ->
    person = new Person("John", "Doe")
    expect(person.fullName).to.equal("John Doe");

The above spec, “it works” makes no sense. The code inside is fine, but the description should be more like:

1
2
3
4

describe '#fullName', ->
  it "yields the full name of the person, space-separated", ->
    person = new Person("John", "Doe")
    expect(person.fullName).to.equal("John Doe");

If there is backend and frontend code, mentally separate them and review both.

In a web project, many times the backend and frontend components are written in different languages. When you’re only working on one part of the stack, it’s easy to just ignore code from the other parts. Instead, use PR’s as a learning opportunity; the other parts of the stack will affect the maintainability of the project just as much as the parts you’re working with, so make sure you look at both.

I find that it helps me to treat the backend and frontend code in a PR as if they were separate PR’s. It’s just a mental separation for me.

This works best when a pull request is for only one feature. The more features that are lumped together, the harder it is to review and, as a result, the largest PR’s end up with the worst reviews.

Be encouraging

The purpose of code reviews is to build up, not to tear down. That goes for both the code and the team. If you see something that was done well, say so! For example, “I like the way you were able to refactor that giant for-loop into a couple of map and reduce calls! I’ll have to keep an eye out for that in my own code!”

Everyone likes to receive positive feedback about their work. Plus, it will make you seem more appealing to work with, and it improves cohesiveness with the rest of the team.

Also, try adding some emoji, or some funny gifs to lighten the mood.

Have you tried asking questions, rather than making statements?

According to Dr. Albert Mehrabian, only 7% of our face-to-face communication is accounted for by the words we say. In the faceless, voiceless world of a text-based code review, it can be easy to offend someone, especially since people in our line of work tend to put so much care into their code. An easy way to diffuse this is to ask questions. For example, you see the following code:

1

rgba = hex.split(/\d{2}/).map (x) parseInt(x)

You could say,

This is wrong. Your split returns [“”, “”, “”], and parseInt needs a radix of 16.

Or, you could ask,

This doesn’t make sense to me; I think this will always yield [NaN, NaN, Nan]. Did I miss something?

Which would you rather someone post on your PR? I prefer the latter.

Asking questions is a powerful thing. When someone questions my code, sometimes that’s the first spark of thought about an alternative approach. Sometimes, it’s a gentle reminder that I left out a use case, or forgot to remove my debug lines, or left a swath of commented code.

My favorite is when someone new to my language of choice asks a question about my code. Sometimes it’s a chance for them to learn something new. Other times it’s a chance for me to break an age-old habit; you wouldn’t believe how many IE5/6 hacks I use out of habit…

If you’re not sure whether something is right, just ask the author. If it’s a mistake, they’ll recognize it and provide a fix. If it was intentional, they can justify whatever it is; you may even learn something.

Suggest refactoring in both the specs and the source.

Sometimes others point out code that I could refactor better; it’s easy to think I’m done refactoring when my head has been stuck in the same code for the last 6+ hours.

In the specs, point out where the testing tools could be better used. For example, I often forget about some of the features of a stubbing library. I sometimes write verbose, repetetive tests that could be extracted into a test helper. I also find that sometimes other developers can help make my tests less fragile. If you have an idea about improving the specs, share it! It may be helpful to add custom matchers to your spec suite; most tools support this in some fashion.

Do the same with the code! If you see where a specific refactoring can apply, say so. Look for code smells; you can refer to Cunningham & Cunningham’s wiki page on Code Smells.

We need to strike a balance here, too. The point of refactoring is to make the code easier to reason about, not to push indirection to its limits.

Ask for help.

It never hurts to have another pair of eyes on the code. Ask a senior person in your organization (or another open-source developer) to take a look; most will be glad to. If you know someone else with relevant experience, ask them! Most developers will gladly look over a PR with you.

If the PR contains a lot of code in a language you’re not great at, be proactive and find someone who is, and ask them to review with you! For example, if there’s a lot of CSS, and you’re not great at it, ask a frontend developer to help.

The Code Checklist

When it comes to most code (obviously this doesn’t apply to markup or styling code), try the following checklist:

• Is every piece of code in the right place, i.e. model code in a model, controller logic in a controller, app-specific helper code in a helper, generic helper code in a library?
• Do all classes have only one responsibility?
• Do all methods do only one thing?
• If a method has a side-effect, is it clear from the name, and otherwise documented?
• Are all classes/methods/variables named properly so that the code is self-describing?
• Is everything as private as possible, i.e. only the intended public interface is public?
• Are too many things private? This could be an indication that you should extract an object.
• Are all files within a reasonable size (e.g., less than 100 lines of code)?
• Are all methods less than 10 lines of code?
• No law of demeter violations (providing whole objects to methods when all that’s needed is the value of one attribute of them)?
• Is everything tested? Is each thing tested enough?
• Are there tests for private methods? This is a code smell.
• Every class should have a small comment describing what it represents/does. Public methods should have comments describing what they are for, or when to call them, if this isn’t obvious from the code. Comments shouldn’t describe what the method does (this is visible from looking at the code).
• Are there any obvious performance-stupidities, like making a database query for each loop iteration, rather than using a more optimized query that loads all data at once?
• Spacing errors like no empty line between methods, or too many empty lines
• There shouldn’t be any commented-out code.
• There should be no debug statements like “console.log” or the likes.

This blog post is inspired by some ideas from Avdi Grimm’s excellent book Confident Ruby. Definitely read it if this topic interests you. It contains a ton more best practices.

Modern software projects typically combine different code bases like frameworks, libraries, or toolkits. The problem is that these independent code bases usually comes with their own subjective domain models, each one specialized for the task the respective library is supposed to perform. Hence, combining different code bases forces developers to deal with several overlapping domain models within their applications.

If the individual libraries are too strict and narrow-minded about the types of parameters they accept as input, combining them into a cohesive code base becomes cumbersome. The code starts to feel like a bureaucracy where despite you having the right data, it is not accepted because it is not in a particular format, and just out of principle there cannot be any exceptions. Adding the necessary explicit type conversations to method calls makes them more cryptic than necessary. Extrapolating the converting of types into explicit adapter layers adds bloat to the architecture.

Wouldn’t it be nicer if libraries were a bit more casual, flexible, and “forgiving”? That with a pragmatic “can-do” attitude they accept parameters in any usable format, and do the right thing in any situation, like a friendly and helpful human operator would do?

This blog post describes a convention to accomplish this through transparently massaging input data dynamically into the needed formats. The examples are in Ruby, but the described techniques apply to most dynamically typed environments and even some reasonably sophisticated statically typed ones.

Implicit and Explicit Conversions

Let’s say we write a text processor application. As part of that we have a method that moves the cursor on the screen. It needs the number of characters by which to move forward, and given the discrete nature of characters this should be an integer.

But we want this method to be casual. If our data happens to be another type of number, i.e. a floating-point, decimal, or even our own custom classes that represent numbers, we still want to be able to use it here. Some examples:

1
2
3
4
5
6
7
8
9
10
11

move_cursor 10                              # Integer parameter

move_cursor column_width * 1.1              # Float parameter while drawing
                                            # a table

move_cursor account.balance * column_width  # BigDecimal parameter while
                                            # drawing a chart

move_cursor configuration.border_width      # We know its the right number,
                                            # but aren't sure which exact data
                                            # type configuration values have

Ruby provides a number of conventions that allow us to do this elegantly. The first one is converter methods. Let’s use one in our method to make it accept all types of numbers and convert them to the type it needs:

1
2
3
4

def move_cursor pixels
  pixels = pixels.to_int
  # pixels is now an integer, and we can move the cursor based on it
end

By calling to_int on the argument, our method now works not only with integers, but with any number.

Notice that we use to_int here, not to_i. The difference between the two is that the former is an implicit conversion, while the latter is an explicit conversion.

Implicit converter methods convert closely related types into each other. This means types which can easily stand in for each other. If you need an integer, receiving a float is probably okay. You can (and should) expect that any Ruby code base calls implicit converter methods for you, hence their name.

Explicit converter methods convert unrelated classes, i.e. classes that shoudn’t be converted without the developer knowing about it. As an example, DateTime does not implicitly convert into an integer, because points in time are not really numbers, and shouldn’t pretend to be. It should still be possible to convert DateTime into a reasonable number if necessary, but you have to do that explicitly. Don’t expect that other code does such far-fetched things for you implicitly!

The cool thing about these converter methods is that they allow for architectures that are open for extension. This means you can make your own objects behave like numbers everywhere, even in other libraries, simply by adding a to_int method to them. And bam! you can use instances of your classes anywhere numbers are required! No more manual converting!

Below is a list of Ruby’s built-in converter methods. Use them as much as you can, and don’t reinvent them!

Dynamic Type Casting

One side effect of these converter methods is that your classes have to be aware that they can be used as a number, and have to consciously implement the converter methods. That’s usually a good thing. But what if that’s not possible, like when you don’t have control over who calls your method? Or you use a library that doesn’t implement this protocol, and you don’t want to monkey-patch all of its internal classes?

Ruby provides another set of powerful converter methods for this case. These methods have the same name as the class they convert into. They can be seen as the bigger brothers of type-casting in statically typed systems. Bigger because these methods not only cast types, but also convert their parameter into the target class.

They are also siblings of constructor methods (funny how everything seems related here), with the difference that constructors always create a new instance based on the given value, while converters don’t create a new instance if that’s not necessary, i.e. when given a usable instance already. This makes them more efficient. Let’s play with Ruby’s built-in Integer method a bit:

1
2
3
4
5
6

Integer 100         # = 100  (ints get passed through)
Integer 100.1       # = 100  (floats are converted to int)
Integer "100"       # = 100  (parses strings)
Integer "0b100"     # = 4    (parses binary-encoded strings)
Integer Time.now    # = 133422 (calls to_i and to_int on the object)
Integer({})         # TypeError: can't convert Hash into Integer

As you can see, this method uses every trick in the book to convert its argument into an integer, and if that doesn’t work, it detonates an exception. Very convenient! Ruby provides such converter methods for many of the built-in classes: Array, Float, String, Integer, URI, or Pathname.

So if we have a print method that needs to convert its arguments into a string no matter what is given to it (everything that is given to it should be printed), we would best write it like this:

1
2
3
4

def print value
  value = String value   # Convert the parameter to a String, or fail here
  # print the string value now
end

Custom Converters

Let’s build such a converter method ourselves! Let’s say our text processor application should be modern and support colors. So we create our own Color class. Anytime a color is needed, developers should be able to provide one comfortably with whatever data about colors they have at that time. Lets take the method to set the current color as an example. We should be able to call it with parameters like these:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# HTML color strings
set_color '#102030'

# integer RGB values
set_color 10, 20, 30

# floating point RGB values
set_color 10.1, 20.1, 30.1

# existing Color instances
set_color red

# an array of RGB values
set_color [10, 20, 30]

# any object that has a 'to_color' method
set_color parent_container

That’s what I call a nice to use method that works with me! Here is how easily the method performs these conversions:

1
2
3
4

def set_color *args
  color = Color *args
  # do something with the 'color' variable here
end

As is visible, the method first massages its parameter(s) into the format it needs, in this case a Color instance, and then does its job with that value. Here is its implementation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# Converts the argument(s) into a Color instance
def Color *args
  case args.first

    when Color
      args.first

    when Integer
      Color.new(*args)

    when Float
      Color.new(*args.map(&:to_i))

    # Parse HTML color code string of the form '#112233'
    when String
      Color.new(*args.first.scan(/^#(\w{2})(\w{2})(\w{2})$/)[0].map{|x| x.to_i 16 })

    # Call to_color method if the parameter has one
    when ->(arg) { arg.respond_to? :to_color }
      args.first.to_color

    # Convert other color classes to our class by using their RGB values
    when ->(arg) { arg.respond_to? :to_rgb }
      Color.new args.first.to_rgb

    # Fail if there is no known type match
    else
      raise TypeError, "Cannot convert #{args.inspect} into a Color"
  end
end

Simple, straight, easy to understand Ruby, with clear expression of intent.

Best Practices

• Make your APIs casual, and thereby easy and intuitive to use. If there are different legitimate types of parameters, your API should do the work to support them, not your users. This makes your API robust, meaning your users can throw all sorts of different input at them, and it does the right thing.

• When unsure about the parameters given to your methods, for example when they are called by other people’s code, convert the parameters into the expected type or fail.

• Don’t give up too early. Try all suitable type conversions before you reject a parameter.

• You can take it easy on data types for parameters, but you should fail reliably for bad arguments. For example, if you allow strings to be provided for colors, and the string does not contain a color code, your code should fail, because that’s most likely a bug. If you get an id for a product, and the referenced product does not exist, there should be an immediate failure as well.

• Use integration tests (aka feature specs) to make sure everything is plumbed together correctly, and the application functions as it should.

• To keep things efficient, only convert types at the public boundaries of your code, not in each internal method. Similar to how the visa of tourists is only checked when they enter a foreign country at its border, and not on each bus ride inside the city.

• Document non-implicit conversions so that other developers know about them.

Conclusion

Done correctly, casual APIs are both robust and intuitive to use. Like a helpful human operator, they do the right thing in all situations no matter what input you throw at them. This allows developers to focus on calling libraries with the right data instead of the right data format.

There are several ways to implement this pattern, depending on how strict you want to be about your input. Done consistently, casual APIs allow for casual collaboration of independent libraries in modern, open-source based code bases.

Happy dynamic type massaging! :)

Idiomatic Scala: Your Options Do Not Match

Imagine you are reviewing a teammate’s work when you are confronted with the following piece of code. What would you think?

1
2
3
4
5

if (expr == true) {
  return true;
} else {
  return false;
}

Cast the first semicolon who never saw (or maybe even wrote) similar code. While it works and is strictly speaking correct, the code above disappoints for expressing what it does a little too literally. It is unsophisticated and ingenuous, a neophyte code smell almost. Experienced developers probably prefer the more compact idiom:

1

return expr;

Yet, it is very common to see even seasoned Scala developers decomposing Option via pattern matching:

1
2
3
4

def f[A](opt: Option[A]) = opt match {
  case Some(a) => ???
  case None => ???
}

Pattern matching on Option is almost never necessary as usually there are cleaner and more expressive ways to attain the same results with monadic functions. This is what the Option Scaladoc has to say:

The most idiomatic way to use an scala.Option instance is to treat it as a collection or monad and use map, flatMap, filter, or foreach […] A less-idiomatic way to use scala.Option values is via pattern matching

Any expression of the form

1
2
3
4

opt match {
  case Some(a) => foo(a)
  case None => bar
}

is precisely equivalent to

1

opt map foo getOrElse bar

Furthermore, since Scala 2.10 an even more concise alternative is available, fold:

1

opt.fold(bar)(foo)

fold even gives us the additional benefit of being more type-strict than either of the previous alternatives. For instance, while a and b in the example below have a type of Any (with value 2), c fails to compile with a type mismatch error:

1
2
3
4
5
6
7
8
9
10

val opt = Option(1)

val a = opt match {
  case Some(x) => x + 1
  case None => "a"
}

val b = opt map (_ + 1) getOrElse "a"

val c = opt.fold("a")(_ + 1)

On the not-so-bright side, fold may not be as readable as the alternatives and a little error-prone, too, since it is not clear what comes first, None or Some?

Readable or not, we still can do a lot better than fold: Option offers a few dozen methods, many of which can be used to better express more specific transformations. For example, opt.fold(true)(_ => false) is equivalent to opt.isEmpty. The converse is either opt.isDefined or its alias opt.nonEmpty.

The Ultimate Scala Option Cheat Sheet

We reproduce below a table highlighting the most relevant Option methods. Whenever you feel like writing case Some(a) => foo; case None => bar consult this table for the equivalent foo and bar in the Some(a) and None columns to find its corresponding method. It is also recommended that you refer to the Option Scaladoc for detailed descriptions and additional methods.

First, a few definitions. For arbitrary types A and B, let:

1
2
3
4

val a: A
def p(a: A): Boolean // a predicate
def f(a: A): B // a mapping function
def g(a: A): Option[B] // an optional mapping function

.exists and .forall

The most attentive readers may be baffled by why None.forall returns true.

If you have your favorite predicate logic introductory book at hand, you may want to read the sections on “universal quantification” and “vacuous truth”. If not, trust us, the math checks out :–)

Here is a colloquial explanation. Basically, it boils down to forall being the reciprocal negative (this may not be the correct mathematical term) of exists. This is easier to understand with an example:

1
2
3
4
5

def isPrime(n: Int): Boolean = ???

val a = Seq(2, 3, 5, 7)

val b = Seq(1, 2, 4, 8)

We can clearly see that a.forall(isPrime) is true, while a.forall(!isPrime) is false. However, for list b, both b.forall(isPrime) and b.forall(!isPrime) are false. Negating the predicate does not necessarily negates the result. The negation of forall(isPrime) is not forall(!isPrime), but exists(!isPrime). In other words:

1

forall(isPrime) == !exists(!isPrime)

It makes perfect sense in plain English: to say “all numbers are prime” is equivalent to saying “there does not exist a number that is not prime”.

Back to Option, None.exists(p) is always false, as is None.exists(!p). Therefore, by the identity derived above:

1
2
3

None.forall(p) == !None.exists(!p)
               == !false
               == true

As unintuitive and surprising as this result may be, it is true and correct, proven by much more rigorous mathematical derivation. As a matter of fact, this apparent oddity can be put to some pretty good use, as we will see next.

A more elaborate example

Let us start with a simple method: tracking events on a webpage. The only distinctive characteristic here is that whether we have a user logged in to the site, we record their ID.

1
2
3
4

def track(user: Option[User]) = user match {
  case Some(u) => Tracker.track(u.id)
  case None => Tracker.track(GUEST)
}

Now, due to privacy concerns, we have to allow users to opt out of tracking. Easy-peasy:

1
2
3
4

def track(user: Option[User]) = user match {
  case Some(u) if u.canTrack => Tracker.track(u.id)
  case _ => Tracker.track(GUEST)
}

All was going well, until one day we got a call from our lawyers: if a user opts out, they should never be tracked at all, not even anonymously. Hence, our final version:

1
2
3
4
5

def track(user: Option[User]) = user match {
  case Some(u) if u.canTrack => Tracker.track(u.id)
  case None => Tracker.track(GUEST)
  case _ => // do not track
}

Now let us convert everything to not use pattern matching. The first one is trivial:

1

Tracker.track(user map (_.id) getOrElse GUEST)

Not only did we replace the match with a more compact single-line expression, we do not have to write Tracker.track twice anymore. It may seem inconsequential, but over time such small things tend to grow in complexity, making maintenance harder.

The second version is more interesting. Whenever we see pattern guards, filter may be required. Overall, it does not change much from above:

1

Tracker.track(user filter (_.canTrack) map (_.id) getOrElse GUEST)

Finally, something more interesting. Now we have not two, but three case clauses. Clearly, there is no single method in Option that handles three possible outcomes. The answer, however, is surprisingly simple once you know it:

1
2

if (user forall (_.canTrack))
  Tracker.track(user map (_.id) getOrElse GUEST)

Knowing that None.forall is always true was crucial to allow us to come up with a simple expression that handles all cases.

Refactoring and the M-word

It is important not to forget that Option is a monad, the Maybe monad, and as such can be used in for-comprehensions:

1
2
3
4
5
6
7
8
9
10
11
12

case class User(name: String, contactInformation: Option[ContactInformation])
object User {
  def byId(id: Int): Option[User] = ???
}
case class ContactInformation(email: Option[Email])
case class Email(address: String, verified: Boolean)

def emailInformation(id: Int) = for {
  user <- User.byId(id)
  contactInformation <- user.contactInformation
  email <- contactInformation.email if email.verified
} yield (user.name, email.address)

Finally, higher-order functions and for-comprehensions offer another advantage over pattern matching that comes in handy when refactoring code. It is not uncommon for methods or data structures of type Option[T] to be revamped to something like Seq[T]. While code using pattern matching would have to be rewritten, code using only higher-order functions or for-comprehensions may not require any changes at all. Most higher-order function and for-comprehension idioms are consistent across all collection types.

Tip: Prefer Option(a) over Some(a)

You may have noticed in some examples that we wrote val a = Option(1) instead of val b = Some(1). Usually, one should prefer Option(x) over Some(x). First, a is inferred as type Option[Int], while b is Some[Int]. Secondly, and more importantly, is that Option(x) is more type-safe than Some(x). Again, let us see why with an example:

1
2
3
4
5
6

// We are enthusiastic about newcomers
def hello(name: Option[String]) = s"Hello, ${name getOrElse "stranger" toUpperCase}!"

def a(name: String) = hello(Option(name))

def b(name: String) = hello(Some(name))

Which is better: method a or method b? Call a(null) and b(null) and see the answer for yourself.

Trivia

All Option methods in the Scala standard library are implemented as single line methods that are slight variations of the form:

1

if (isEmpty) bar else foo

Most methods are also marked as @inline final, which means that using Option higher-order functions should be as efficient as safe-guarding for null in Java, especially after HotSpot does its magic. How cool is that?

Addendum: Do not abuse pattern matching

Pattern matching is a very expressive and powerful tool. However, as frequently is the case with the finest hammers, we must watch out for the proverbial nail.

Pattern matching “all the things” must be considered a Scala code smell. Especially for some common cases, pattern matching is not always as readable or concise as other constructs, and often the bytecode generated by the Scala compiler is less efficient, too.

In particular, do not pattern match on Booleans; use good old, true and tested if else:

1
2
3
4
5
6

cond match {
  case true => Ok
  case false => Error
}

if (cond) Ok else Error

Avoid pattern matching on a single value followed by a defaut case:

1
2
3
4
5
6
7
8

n match {
  case 0 => "Cart is empty"
  case _ => "View cart"
}

if (n > 0) "View cart" else "Cart is empty"

s"Your cart has ${if (n > 0) n else "no"} item(s)."

Finally, for most data structures like Seq and String, consider methods like isEmpty, headOption, tail, etc. And for enumerations, use withName.

Enjoy your Options!

Deep linking consists of using a hyperlink that links to a specific piece of content within an app. The specific content could be a specific view, a particular section of a page, or a certain tab.

To see an example, download the Twitter app. Log into it and close it. Now, try opening the following url in Safari Mobile on your device: twitter://timeline. iOS will switch to the Twitter app and go directly to your timeline.

Now try the following url in Safari Mobile: twitter://post?message=using%20deeplinking. This time Twitter will open up with the message “using deep linking” already composed and ready to be tweeted.

You can use deep linking for:

• Displaying a different landing page after launch.
• Inter-app communication by launching an app from another app and passing information.
• Creating a web-like URI based navigation scheme throughout your app.
• Enabling integration with other apps by letting them launch your app directly.
• Recording and analyzing user behavior to determine where your users launch your app from.

Given the above uses, let us look at a basic implementation of deep linking. In this tutorial we are going to:

1. Enable deep linking in an app.
2. Handle launch URLs to show a different landing page based on the url.

Note: This tutorial assumes a basic knowledge of creating simple apps in iOS

In this post we’ll be referring to the sample app, which is located here: https://github.com/vipulvpatil/deeplinking-in-ios.

Create an app and enable deep linking

Create a basic app in XCode consisting of a main ViewController pushed on a UINavigation ViewController. Also create some additional ViewControllers to be used later.

To enable deep linking, go to the Info tab in the Xcode project. In the URL Types section, click on the + button, and then add an identifier and a URL scheme. Ensure that the identifier and URL scheme you select are unique. Take note of the URL scheme you enter, as this is how iOS knows to open a link in your app. The sample app registers the following url scheme:

dlapp

To confirm that your URL scheme has been registered, check Info.plist for an entry named ‘URL Types’. Expanding it will show you the new URL scheme you just registered. You can check that this is working by typing the following url into Safari Mobile: your-url-scheme:// ( For the sample app, it would be: dlapp://). This should open up your app. If not, please go through this section again before moving on.

Handling the opening of registered urls within your app

Now that you have ensured that deep linking is working, we need to handle the url used to launch the app. In it’s current state, your app can be launched using a simple url, but it can’t do much beyond that. To do more, we need to override the following function in AppDelegate:

1
2
3
4

-(BOOL)application:(UIApplication *)application
          openURL:(NSURL *)url
          sourceApplication:(NSString *)sourceApplication
          annotation:(id)annotation

Note that this is not present by default, and needs to be added. This function gets called every time your app is launched using the registered url-scheme. The passed in arguments are:

• url: The complete url used to launch the app.
• sourceApplication: The bundle ID for the application from which the url was called.
• annotation: A property list object that can be used to pass additional info along with the url.

The format of the url is as follows: [scheme]://[host]/[path]

Breaking this down, we get:

• scheme: The url scheme tells iOS what app to launch. Note that the url scheme should be registered with the device for it to be able to handle this (this is what we did in the previous section).
• host: The host is analogous to a website/server name on the web. You can handle multiple hosts within your app.
• path: The path enables you to provide additional information regarding the location within your app.

In general, you would use the host and path parameters to determine what the user intends to do.

The exact contents of this method largely depend on your needs, but for the purpose of this post, we will check the host and then based on the path, load a particular ViewController.

1
2
3
4
5
6
7

-(BOOL)application:(UIApplication *)application openURL:(NSURL *)url sourceApplication:(NSString *)sourceApplication annotation:(id)annotation{
  if([[url host] isEqualToString:@"page"]){
    if([[url path] isEqualToString:@"/page1"]){
      [self.mainController pushViewController:[[Page1ViewController alloc] init] animated:YES];
    }
 return YES;
}

Line 2 checks to see if the host within the url is what we are expecting, i.e. ‘page’ in this case. After that, it matches the url path and loads the view controller for that page. In this way, you can handle, every expected url within the function and show a different screen for each. One thing to note is that the app can be launched with a url regardless of whether this function can handle it or not. In such cases returning NO from the method tells iOS that the url was not handled by the app. The result in general, is that the app just opens in the last state it was in.

In the sample app, we handle 4 different urls:

• dlapp://page/main
• dlapp://page/page1
• dlapp://page/page2
• dlapp://page/page3

Here is the complete code to handle the above urls:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

if([[url host] isEqualToString:@"page"]){
    if([[url path] isEqualToString:@"/main"]){
      [self.mainController setViewControllers:@[[[DLViewController alloc] init]] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page1"]){
      [self.mainController pushViewController:[[Page1ViewController alloc] init] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page2"]){
      [self.mainController pushViewController:[[Page2ViewController alloc] init] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page3"]){
      [self.mainController pushViewController:[[Page3ViewController alloc] init] animated:YES];
    }
    else{
      return NO;
    }
    return YES;
  }
  else{
    return NO;
  }

Conclusion

This post shows you how to respond to urls passed into your app, but you can do a lot more. By taking apart the url in the callback, you can do quite a lot. For example, you could:

• customize the look and feel of the page opened
• pre-populate certain elements on the page (the way the Twitter app did)
• track how users are getting to your app

Happy Coding!

TL/DR

I describe a technique that allows one to extract commits from a larger Git branch into separate branches. This “git branch refactoring” provides numerous benefits:

• A fast-track for integrating urgent changes (like refactorings or bug fixes) that were created as part of feature development into the main Git branch, before the feature is merged.

• Improvements to the quality, efficiency, and fun factor of code reviews, by allowing your team to review and merge individual changes within larger feature branches individually, by different reviewers, at different times.

Feature branches contain many different changes

When code classes or methods do too many things, we refactor them by extracting individual pieces of functionality into separate classes or methods. The same applies to feature branches in Git. Ideally a Git feature branch should also perform just one particular change, be it a bug fix, a refactoring, or adding a new feature.

We all know, however, that while adding the next cool feature to a code base, we come across existing code that needs to be improved along the way. So, in the spirit of Continuous Improvement and Kaizen, or simply because we depend on improvements to happen before we can continue developing our feature, we

• fix bugs in existing code that we depend on,
• add missing functionality to existing code that our feature requires,
• refactor existing code, for example by extracting pieces of it to make them available separately,
• do good in numerous other ways, for example by cleaning up messes, reducing smells, paying down technical debt, or improving code quality and spec coverage.

As a result of this process, many feature branches end up looking similar to the one depicted here (time flows from the bottom up here, i.e. new commits are added at the top):

Our feature branch called kg_feature_1 is cut from the development branch (we follow the terminology of the NVIE model here), which is our main branch that is shared by all developers here. This development branch only contains the framework so far. Our feature branch contains a number of commits for the feature itself (feature 1 – feature 4), a bug fix for an external code module (“bug fix 1”), and a more generic refactoring that was done in two separate steps (“refactoring 1a” and “refactoring 1b”).

Changes should be reviewed individually

We don’t want to just send this many-headed monster of a feature branch off as a single code review. It contains too many different things! This makes it hard for the reviewer to get an overview of what we are doing here. Its also pretty hard to have meaningful conversations about several different things at the same time. This is just a recipe for confusion, cognitive overload, and missing important details. It also reduces the fun and the power of code reviews.

We also want to review and merge the bug fix and the refactorings into the development branch right away, so that other developers can incorporate them into their work before their work deviates too much from our changes. If we waited with that until the whole feature is done, it will be too late, and we will have to deal with a lot more merge conflicts than necessary.

These different types of changes in different parts of the code base should probably also be reviewed by different people. The bug fix gets reviewed by the author/maintainer of the affected module, the refactoring by the architect or tech lead, and the feature by another team member.

Let’s refactor our Git branch!

Extracting commits into dedicated branches

The change that most urgently needs to get merged into development here is the refactoring, because it might touch code that other people are currently working on. Lets extract it into a separate branch.

1
2
3
4
5
6

# Cut a new branch for the refactoring off the development branch.
$ git checkout -b kg_refactoring development

# Move the refactoring commits into the "kg_refactoring" branch.
$ git cherry-pick [SHA1 of the "refactoring 1a" commit]
$ git cherry-pick [SHA1 of the "refactoring 1b" commit]

Our kg_refactoring branch now looks like this.

It only contains both refactoring commits. Perfect! Now we can push this branch to the Git repo, get it reviewed by the architect or tech lead, and merge it into the development branch.

Once this is done, we rebase our feature branch against the development branch to pick up all the changes that happened in it recently, like our extracted refactoring. We do this regularly anyways, since it is a good practice to keep our feature branches synchronized with ongoing development, and solve merge conflict as soon as they happen, rather than all at once when the branch is done.

1
2

$ git checkout kg_feature_1
$ git rebase development

If you know Git, you know that our branch situation looks like this now.

This looks much better already. The refactorings are now part of the development branch, and our feature branch still has access to them, and uses them!

The only foreign body left in the feature branch is the bug fix. Once we have extracted it using the exact same technique, our branch situation looks like this.

Both bug fix and refactorings have been individually reviewed and merged into the development branch, separately from the ongoing feature development. The refactored feature branch contains only feature-related commits, and can be reviewed now, or developed further. If we do more bug fixes or refactorings, we can repeat this procedure as needed.

When to do this

Extracting autonomous changes into their own Git feature branches can significantly improve the structure of your branches, the performance of your code reviews, and the efficiency of your workflows. In order to work well, however, it is dependent on a number of things:

• Each commit is well described and addresses a single change (feature, bug fix, refactoring, etc).

• You can rebase your feature branches. Only do that with branches that only you use! Read Linus Torvalds’ explanation if you are unsure what this means. In our case we marked our branches as private by prefixing their names with our initials, and according to our team policy this makes them private.

• You have enough specs, especially feature specs (aka integration tests) to verify that everything still works after your branch refactorings.

All these are good engineering practices to live by anyways, so give this a shot, and let me know what you think in the comments!

What is Harp?

Harp is a static web server built on Node. It serves Jade, Markdown, EJS, CoffeeScript, Sass, LESS and Stylus as HTML, CSS & JavaScript with no additional configuration or plugins. You get a powerful asset pipeline with a few extra perks. This makes it great for prototyping and as a frontend playground.

Why use Harp?

It’s blazing fast, easy to get setup, and dead simple. It’s also more powerful than things like CodePen or jsFiddle (more than just a “widget builder”). If you end up liking what you’ve built, you can then compile the static assets or actually deploy it to Harp’s own dropbox based hosting platform. Because of it’s speed and versatility, Harp is an efficient and effective tool for prototyping UI concepts and workflows.

Getting Started

• Install Node or brew install node
• Install Harp npm install -g harp
• Create a new Harp project harp init project-name (will create a folder with your project name in the current directory)
• Start Harp server harp server project-name
• Access your site at localhost:9000

Now you can write using your favorite frontend technologies! (This article will be using Jade/Sylus for all code examples.)

Note: Any time you would like to generate the static files for your project, you can run harp compile and your site will be generated in the /www folder in the root of your project.

Public vs. Private

In Harp, anything prefixed with an underscore will not be publicly available via the server or compilation. This is useful for partials and/or private layout files. Anything without an underscore will be compiled and publicly available. Making things private when possible will also improve compile time (not that you need it). Folders that are prefixed with an underscore will be private as well as anything that is contained within.

Layouts and Partials

You’ll have layout files which have the underscore prefix. The _layout file will be applied to anything on that folder level. eg: /_layout.jade would apply to anything on the root level. /posts/_layout.jade would be applied to any page view within the /posts directory.

Partials are just as easy. If you have a partial.jade file in the _partials/ folder you can simply:

1

!= partial("_partials/partial")

Static Data Structures

With Harp you also get flat JSON data structures to play with. You can create a _data.json file in any subdirectory and then have access to that data in your templates. Eg: if you have /posts/_data.json you can then iterate over that json collection.

with /posts/_data.json:

1
2
3
4
5
6
7
8
9
10
11
12

{
  "post1-slug" : {
    "name":      "Post 1",
    "author":    "John Doe",
    "content":   "Lorem ipsum..."
  },
  "post2-slug" : {
    "name":      "Post 2",
    "author":    "Kevin Smith",
    "content":   "Some Post Content"
  }
}

Then in your template:

1
2
3
4

each post, slug in public.posts._data
  .post
    h3
      a.link(href="/posts/#{ slug }")= post.name

This will enumerate over the top-level keys using the key name as the slug.

Output:

1
2
3
4
5
6
7

<div class="post">
  <h3><a class="link" href="/posts/post1-slug">Post 1</a></h3>
</div>

<div class="post">
  <h3><a class="link" href="/posts/post2-slug">Post 2</a></h3>
</div>

It should be noted here that the _data.json file is intended only for meta data. If you were blogging, like in this example, you would also need to include a view file in the posts directory (with a file name that’s the same as the slug) that contains the actual post itself. The _data.json would include things like the slug, title, tags, etc. It’s easy to see that you can use _data.json files to mimic database results for prototyping very easily.

You also get a global variables JSON file located at /_harp.json which is useful for layout variables (eg: site title) and other settings that you’ll need available within scope.

The Current Object

On any template or layout you will have access to current which will give you brief information about the current location of the app. The current object has a path and source property. Eg: if you were accessing /posts/post-name the current object may look like:

1
2
3
4

{
  "path":   ["posts", "post-name"],
  "source": "post-name"
}

This is very useful if you would like to have an active-state in your site’s navigation:

1
2
3
4
5

ul
  li(class="#{ current.source == 'index' ? 'active' : '' }")
    a(href="/") Home
  li(class="#{ current.source == 'posts' ? 'active' : '' }")
    a(href="/posts") Posts

A Boilerplate Project

If you prefer to learn by example, you can clone my harp boilerplate. It will set you up with a layout, a partial, some useful stylus mixins, icon fonts, and other helpful examples.

For more information you should check out the harp documentation

A World Wide War is being fought across the software world. The warring factions are engaging in battles in IT centers, corporate offices and your homes; between security experts on one side and cyber-criminals on the other.

This world we live in is getting increasingly connected and computerized. Consumers, organizations and governments alike are conducting business electronically at an ever increasing pace. This requires organizations to store information about their consumers. This includes financial, personal data as well as data about their online and spending habits.

Corporations need to secure this data because

• privacy laws demand it
• consumers demand it
• it gives them a competitive advantage

Naturally, this data is prized by hackers, criminals, competitive entities and unfriendly state agents. This battle of wits between the security experts and the cyber-criminals is an ongoing one and is only going to get bigger.

The systems that process and store data also become targets — either to disable them, or to access, corrupt or destroy data.

Securing the data and the systems that manage it is now a high priority and high profile task. Or, it should be if they mean business. It is no longer a nice-to-have or even a should-have. It is now a must-have.

Companies that don’t have their act together … are in for a crude awakening in the internet age, to significant damage to their business and reputation (http://www.businessweek.com/articles/2014-03-13/target-missed-alarms-in-epic-hack-of-credit-card-data), and in some cases to an end of their very existence (http://www.forbes.com/sites/cameronkeng/2014/02/25/bitcoins-mt-gox-shuts-down-loses-409200000-dollars-recovery-steps-and-taking-your-tax-losses).

The War Zone

The increasing mobility of data in corporations is one of the biggest challenges faced in the last couple of years. Users are feeling empowered to access data from anywhere. The BYOD (Bring Your Own Device) trend is bringing in new attack vectors. The data access end-point is also going through a transformation. No longer are these Windows-based, but now a diverse mix of different systems. Some of them are still in their infancy compared to long established platforms like Windows/Unix. These platforms are new targets, and malware for mobile devices is now a rapidly growing threat.

Employees are bringing in their smartphones, tablets and next-gen ultrabooks in to the corporate networks. IT professionals are being required to secure sensitive data on devices the have little control over.

The growing popularity of the “Internet of things” (smartphones, tablets, wearable gadgets, and interconnected devices) makes a fluid situation even more dynamic for security experts. They present ripe targets for cyber-criminals looking to either find access to your data or cause a large-scale disruption.

Cyber-criminals are getting sophisticated themselves. They are using underground online marketplaces to sell their wares and services. The Blackhole exploit kit accounted for almost a third of all malware infestations in 2012. And, it spawned a new generation of botnet exploits that drew on its source code for inspiration.

The Attacks

As the arms race unfolds between the cyber-criminals and security experts, here are some security threats that are at the top the list.

Cloud

Cloud computing continues to grow in popularity — and so too will the number of security threats targeting the cloud. It’s important that businesses realize that whilst they may outsource the handling and storage of their data, they can’t outsource the responsibility for the data itself. If their provider’s systems are breached, and data is exposed, the businesses are responsible.

Therefore, any risks need to be assessed in the same way as they would if they were holding the data internally. Other issues organizations should consider include: where will the data be stored, what happens to any data if organizations switch providers and what steps are being taken to secure the data on their provider’s systems, including how they prevent other customers from accessing it.

The bad guys will look to targeting the cloud in 2014, with cloud service provider employees the main focus. Successful phishing attacks on staff, especially if the password re-use jackpot is hit, have been the door to entry of many online member databases during 2013.

APT

APT or Advanced Persistent Threat is a set of well-camouflaged, focused, continuous hacking activities against a single target. APT activities require a high-level of stealth sustained over a period of time. These employ advanced malware to exploit vulnerabilities in the targets. And these are usually orchestrated for business or political motives. Flame/Gauss and Stuxnet are examples of recent APT attacks used to attack financial, government and military institutions.

Social Engineering

It’s a tried-and-true tactic in both the physical and digital worlds – social engineering. Before the digital age, this meant sneaking one’s way past the security desk with glib talk vs. a cleverly-worded email. Now social engineering has moved onto social networks, including Facebook and LinkedIn.

Attackers are increasing their use of social engineering, which goes beyond calling targeted employees and trying to trick them into giving up information. Cyber-criminals will try to hide malware using deceitful tactics to trick you into installing it.

The task of cybercriminal is a lot simpler — check if details are already posted on social networks. Social networks are about networking people; a convincing-looking profile of a company or person followed by a friend/connection request can be enough to get a social engineering attack started. Sarah Palin’s Yahoo email account was compromised using information that was publicly available on her social media profiles (birth name, date of birth etc).

Password Management

We live in a world where people are increasingly connected through social networking, cloud services & smart devices. This presents a password management headache for users and, by extension, the services they use. People tend to reuse passwords. More than half users reuse passwords for their online accounts. A data breach in one vendor/service provider can potentially put their accounts at other services at risk. If that weren’t scary enough, over a quarter tend to use easy-to-remember passwords such as birthdays or people’s names, opening the door for their online accounts to be hacked into by criminals. It’s worrying aspect for corporations that so many people are making life so easy for cyber-criminals hackers. Especially because simple to make strong password security a part of one’s everyday life.

BYOD (Bring Your Own Device )

This is one threat vector that will give security professionals many sleepless nights. BYOD (Bring Your Own Device) increases data leakage potential especially from devices unprotected by device specific counter-measures such as passcode/passphrase protection. etc. Most organizations going down the BYOD path aren’t implementing appropriate training for employees.

Attackers will try and circumvent app review and detection mechanisms on these devices. If possible their apps will mimic the UI of the native settings page and trick the user in to granting them admin privileges on the device. The device sensors such as GPS, microphone and cameras coupled with the ability to network over WIFI and mobile networks, could become tools for engineering further attacks.

The main areas of risk include: * Data loss — lost or stolen devices * ID theft — thieves logging in to your online accounts using saved credentials on your stolen device * Data leakage — via bogus WIFI access points and malware

Botnets

Essentially, a botnet is a collection of networked computers running a program to perform a task. Initially, most botnets were employed to perform legitimate tasks such controlling IRC channels. Later they were deployed to distribute spam, or perform DDos attacks.

Botnet operators are beginning to design systems that are more adaptive and redundant than many corporate and government networks. Botnets such as Gameover have replaced the traditional command and control link with a peer-to-peer network of infected systems. Controlling this agile attack vector before it can be used as an advanced persistent threat (APT) and migrates into smart mobile devices is crucial.

We’re also likely to see more mobile botnets, of the sort created using the RootSmart backdoor in Q1 2012. In order to prevent falling victim to mobile malware, businesses should install anti-malware protection on their Android devices, secure the data held on them and make sure that this can be wiped remotely if the device is lost or stolen. Businesses should also develop a policy for staff on how to reduce the risks from mobile devices. This should include not rooting the device, avoiding public Wi-Fi networks for confidential transactions, not relying solely on a simple PIN and only installing apps from trusted sources.

Spam

People still send emails. So do bad guys, and they will keep doing so long as people keep using email. You see spam that link to financial scams that are now mostly ignored, spam that link to malware designed to install botnet agents, spam that attempt to seem legit by linking to current events. Phishing via deceptive links in spam is a very common attack. Spammers are also wising up to having their bots and servers taken down. Snowshoe spam is their innovation against this countermeasure. This involves distributing their spamming across multiple ip addresses, spreading the load. Hence, the metaphor.

The Defense

So what are organizations to do to deflect security attacks? It used to be about securing the perimeter in the past. Now the perimeter is blurred due to BYOD and cloud services trends.

There is no one good answer. But most of them would include adopting treating security as a feature, good engineering practices, implementing systems, good employer training, a good plan to manage responses to attacks and finally transparency.

At Originate, when we work with partners, we emphasize security up front as a key element of the systems we build. In most cases, we have been able to educate and received buy-in from our partners. In one case however, we received push back on implementing a policy to enforce strong passwords — the partners feared having a strong password policy would impact user registration. Guess what, on the first day of launch, the first account to be compromised was the admin account. Fortunately, we were monitoring all activity and were able to retake control of the account and secure it. Needless to say, we got the go-ahead to implement the strong-password policy.

Make security a feature ( and not a Non-functional requirement )

When writing applications, security aspects should be first-class requirements. It cannot be a bolt-on later. Build the app with the best security practices from the start. According to security firms FireEye and Secunia, the real security problems in this decade are not in our operating systems but in the applications we run on them.

Design Software with Secure Features

Security issues in design and semantic flaws (ones that are not syntactic or code related), such as business logic flaws, cannot be detected in code and need to be inspected by performing threat models and abuse cases modeling during the design stage.

Threat modeling is an iterative technique used to identify the threats to the software being built. It starts by identifying the security objectives of the software and profiles it. It breaks the software into physical and logical constructs generating the software context that includes data flow diagrams, and end-to-end deployment scenarios, identifying entry and exit points, protocols, components, identities, and services.

Threat Modeling is performed during the design stage so that necessary security controls (safeguards) can be developed during the development phase of the software.

Develop Software with Secure Features

Follow the Saltzer and Schroeder list of principles for building good secure software:

Use secure-coding best practices

A stitch in time saves nine. It is always cheaper to build secure applications then to correct the security bugs later. Making security a check-off item for code reviews. OwASP and Apple have good checklists that software development organizations would do well to follow.

Use source code analysis tools

Whenever possible integrate source-code analysis tools (Fortify, FindBugs, etc.) in to your development process. However, it is not a substitute for testing. And it can generate a lot of false positives initially. But it can be a useful tool in locating security vulnerabilities.

Use testing techniques to stress security features

Focus on abuse/negative test cases as well: test the evil path in unit and integration tests using. Running penetration tests against new releases as part of the release process using tools like SkipFish, MetaSploit etc.

Password and Session Management

Passwords are the keys to a user’s account. It is often a target to gain unauthorized access. Password strength is key in thwarting such attacks. Enforce a strong password policy. However, it does present an inconvenience to users who now have to remember long complex passwords. A good strategy around that is to not have to manage passwords and authentication at all. Implement single sign-on, so that users need to only have to authenticate once. Or, offload it to an 3rd party authentication authority using OAuth. However, if you do implement authentication ensure that

• user credentials are never stored in clear text — makes sure that they are hashed and salted
• never send user credentials over an un-encrypted connection

Session tokens should always be generated on the server. Use existing application frameworks for session management as much as possible. For example, Rails, Play, Lift, J2EE/Spring, ASP.Net etc., come with features around strongly encrypted sessions, protections against attacks like CSRF, XSS, SQL injection, session replay, timeouts etc. That wheel’s been invented and rolls well.

The bad guys will still be targeting the cloud in 2014, with cloud service provider employees the main focus. Successful phishing attacks on staff, especially if the password re-use jackpot is hit, have been the door to entry of many online member databases during 2013. Ultimately, with innovation and planning, cloud services could reduce business risks by providing greater flexibility, resiliency and security

Input validation

One of the most effective safeguards against hacking attacks is to perform input validation. Input validation should take place on the server side for all data. Validation criteria should be set for input fields. All data should be encoded before use. Input validation techniques along with proper encoding help block against many attack vectors like XSS, SQL Injection and remote file inclusion.

Use open source cryptography

The only way to secure architectures is to use systems, algorithms, and implementations that are well known to be safe, and which have been peer reviewed in the open by lots of people, and which have withstood a lot of hacking attempts in production already. Custom or proprietary solutions often have not gone through this process, and as a result are often riddled with security weaknesses that can and will be exploited.

Configuring for security

What does this mean? It is simply managing and defining a process to ensure that the systems the software runs on have been secured adequately. Harden your infrastructure by:

1. Ensuring all software is updated to the latest security patches. This includes the OS, Web/App server, databases. And, any other component in the system architecture. Define a process to keep these components abreast of the latest updates and patches and deploying them to all deployed environments in a timely manner. The process should also include seemingly innocuous peripherals such as networked printers, VOIP phones
2. Turning off or disabling all unused and unnecessary features (e.g. ports, services, accounts )
3. Changing all default passwords and or accounts.
4. Ensuring that error handling does not reveal any stack traces or other sensitive information to the users.

In 2013, a third of the most serious threats facing enterprises were associated with worms (Win32/Conficker, INF/Autorun, Win32/Dorkbot) infecting windows systems. Worms are commonly spread through network drives, abusing the Autorun feature or exploiting weak passwords.

The Conficker worm exploited weak passwords. Once it compromised a system, it could steal the credentials of an IT administrator to spread on the internal network.

Manage data exposure

Analyze your data characteristics and determine which data is sensitive enough to require extra protection ( e.g. passwords, credit card numbers, health records, etc.). Ensure that such data is never backed up or transmitted in clear text. Ensure that strong/updated cryptographic algorithms are used to encrypt such data. Don’t store data unnecessarily. Data you don’t have is impossible to steal. For example, don’t store credit card numbers if you use a payment gateway, simply store only the last 4 digits for tracking purposes. Also, no sensitive information such as IP addresses in logs. Ensure passwords are stored with algorithms specifically designed for password encryption such as bcrypt, PBKDF2 or scrpyt.

Manage the end-points

BYOD is here to stay. CEOs want to use the latest in mobile technology — the President of the United States uses a tablet. It has it’s benefits – reduced capital expense, and productivity gains. While there are some businesses that may do well to steer away from this trend, most corporations can benefit if they manage this trend with their organization, rather than fight it. Put in a plan to train employees & raise awareness regarding the risks using their devices to access corporate data. Consider implementing a MDM (Mobile Device Management) solution to manage access to corporate apps and access on employee devices. MobileIron, Airwatch/VmWare, Citrix among others have MDM suites to manage BYOD issues. Also, consider reigning it in using a CYOD (Choose your Own Device) option — where the business has a list of approved devices the employees chose from, to gain access to corporate systems.

Employee Training

Education is a good risk-mitigator as well. Training developers to follow good design principles (including the ones mentioned above) helps build good secure software. Training operations to configure for security means there will be fewer if not none, changes that could compromise security, for example, inadvertently removing access restrictions on network drives, which happens more often than one would imagine.

Stay Paranoid

It is impossible to protect anything that is in the users hands from being read in its completeness. For example, the attempts to encrypt digital media like DVDs have all been hacked with ease. There is no platform or app that hasn’t been hacked and studied in detail by the community, even if it requires modifying the hardware that the code is running on, either for establishing a street cred or for financial gain. Sony’s Playstation was protected on the hardware level against hacks. Even that wasn’t enough protection apparently — you could buy extra chips (called flashers) could solder onto chips on the motherboard. You could then downgrade the firmware and load pirated games onto it.

Stay vigilant for any signs of abnormal activity on your network. No alarm should be treated as to insignificant to investigate. Just recently, a large store chain revealed that they did not take early warning signs of abnormal activity too seriously, eventually resulting in a data breach that impacted 70 million customers, and more importantly a loss of trust and goodwill.

Every piece of code, data, or hardware that is in the users hands must be considered read and understood in its completeness, even if that means grinding away a hardware chip layer by layer and photographing the internals. People do that. Obfuscation might be helpful to reduce the size of the shipped binaries, but it doesn’t prevent reading and manipulation of the compiled code.

No input from the outside world, even if it seems to come from our own client apps over secure channels, must be trusted. The clients might be (and will be) compromised, and will send malicious data, no matter how obfuscated or protected they are. If there are truly security-relevant things, they have to remain on the servers, and have to happen there.

Further reading

• http://en.wikipedia.org/wiki/Advanced_persistent_threat
• https://blogs.technet.com/b/security/archive/2013/11/26/microsoft-cybersecurity-report-top-10-most-wanted-enterprise-threats.aspx
• https://www.owasp.org/index.php/Main_Page
• https://www.owasp.org/index.php/OWASP_Testing_Guide_v3_Table_of_Contents
• https://www.owasp.org/images/0/08/OWASP_SCP_Quick_Reference_Guide_v2.pdf
• https://developer.apple.com/library/ios/documentation/Security/Conceptual/SecureCodingGuide/SecurityDevelopmentChecklists/SecurityDevelopmentChecklists.html
• http://www.forbes.com/sites/ciocentral/2012/12/05/the-biggest-cybersecurity-threats-of-2013-2/
• http://en.wikipedia.org/wiki/Blackhole_exploit_kit
• http://en.wikipedia.org/wiki/PlayStation_Jailbreak
• http://en.wikipedia.org/wiki/Sarah_Palin_email_hack
• http://www.pcpro.co.uk/features/386218/the-top-five-consumer-security-threats-for-2014
• http://www.zdnet.com/more-bugs-more-bucks-pwn2own-and-pwnium-2014-7000026177
• http://www.itpro.co.uk/643936/13-security-threats-to-counter-in-2013
• http://www.sophos.com/en-us/medialibrary/PDFs/technical%20papers/Yu-VB2013.pdf
• http://arstechnica.com/security/2013/11/how-an-epic-blunder-by-adobe-could-strengthen-hand-of-password-crackers/
• http://arstechnica.com/security/2014/02/how-to-turn-a-phone-into-a-covert-bugging-device-infect-the-printer/
• http://www.zdnet.com/malicious-apps-mobile-malware-reaches-1-million-mark-7000021371/
• http://passwordresearch.com/stats/statindex.html
• http://www.enterpriseios.com/wiki/Comparison_MDM_Providers
• http://www.concise-courses.com/security/top-ten-pentesting-tools/
• http://projects.webappsec.org/w/page/13246988/Web%20Application%20Security%20Scanner%20List