Developer Blog

Deep Linking in iOS

Deep linking consists of using a hyperlink that links to a specific piece of content within an app. The specific content could be a specific view, a particular section of a page, or a certain tab.

To see an example, download the Twitter app. Log into it and close it. Now, try opening the following url in Safari Mobile on your device: twitter://timeline. iOS will switch to the Twitter app and go directly to your timeline.

Now try the following url in Safari Mobile: twitter://post?message=using%20deeplinking. This time Twitter will open up with the message “using deep linking” already composed and ready to be tweeted.

You can use deep linking for:

  • Displaying a different landing page after launch.
  • Inter-app communication by launching an app from another app and passing information.
  • Creating a web-like URI based navigation scheme throughout your app.
  • Enabling integration with other apps by letting them launch your app directly.
  • Recording and analyzing user behavior to determine where your users launch your app from.

Given the above uses, let us look at a basic implementation of deep linking. In this tutorial we are going to:

  1. Enable deep linking in an app.
  2. Handle launch URLs to show a different landing page based on the url.

Note: This tutorial assumes a basic knowledge of creating simple apps in iOS

In this post we’ll be referring to the sample app, which is located here: https://github.com/vipulvpatil/deeplinking-in-ios.

Create an app and enable deep linking

Create a basic app in XCode consisting of a main ViewController pushed on a UINavigation ViewController. Also create some additional ViewControllers to be used later.

To enable deep linking, go to the Info tab in the Xcode project. In the URL Types section, click on the + button, and then add an identifier and a URL scheme. Ensure that the identifier and URL scheme you select are unique. Take note of the URL scheme you enter, as this is how iOS knows to open a link in your app. The sample app registers the following url scheme:

dlapp

To confirm that your URL scheme has been registered, check Info.plist for an entry named ‘URL Types’. Expanding it will show you the new URL scheme you just registered. You can check that this is working by typing the following url into Safari Mobile: your-url-scheme:// ( For the sample app, it would be: dlapp://). This should open up your app. If not, please go through this section again before moving on.

Handling the opening of registered urls within your app

Now that you have ensured that deep linking is working, we need to handle the url used to launch the app. In it’s current state, your app can be launched using a simple url, but it can’t do much beyond that. To do more, we need to override the following function in AppDelegate:

1
2
3
4
-(BOOL)application:(UIApplication *)application
          openURL:(NSURL *)url
          sourceApplication:(NSString *)sourceApplication
          annotation:(id)annotation

Note that this is not present by default, and needs to be added. This function gets called every time your app is launched using the registered url-scheme. The passed in arguments are:

  • url: The complete url used to launch the app.
  • sourceApplication: The bundle ID for the application from which the url was called.
  • annotation: A property list object that can be used to pass additional info along with the url.

The format of the url is as follows: [scheme]://[host]/[path]

Breaking this down, we get:

  • scheme: The url scheme tells iOS what app to launch. Note that the url scheme should be registered with the device for it to be able to handle this (this is what we did in the previous section).
  • host: The host is analogous to a website/server name on the web. You can handle multiple hosts within your app.
  • path: The path enables you to provide additional information regarding the location within your app.

In general, you would use the host and path parameters to determine what the user intends to do.

The exact contents of this method largely depend on your needs, but for the purpose of this post, we will check the host and then based on the path, load a particular ViewController.

1
2
3
4
5
6
7
-(BOOL)application:(UIApplication *)application openURL:(NSURL *)url sourceApplication:(NSString *)sourceApplication annotation:(id)annotation{
  if([[url host] isEqualToString:@"page"]){
    if([[url path] isEqualToString:@"/page1"]){
      [self.mainController pushViewController:[[Page1ViewController alloc] init] animated:YES];
    }
 return YES;
}

Line 2 checks to see if the host within the url is what we are expecting, i.e. ‘page’ in this case. After that, it matches the url path and loads the view controller for that page. In this way, you can handle, every expected url within the function and show a different screen for each. One thing to note is that the app can be launched with a url regardless of whether this function can handle it or not. In such cases returning NO from the method tells iOS that the url was not handled by the app. The result in general, is that the app just opens in the last state it was in.

In the sample app, we handle 4 different urls:

  • dlapp://page/main
  • dlapp://page/page1
  • dlapp://page/page2
  • dlapp://page/page3

Here is the complete code to handle the above urls:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
if([[url host] isEqualToString:@"page"]){
    if([[url path] isEqualToString:@"/main"]){
      [self.mainController setViewControllers:@[[[DLViewController alloc] init]] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page1"]){
      [self.mainController pushViewController:[[Page1ViewController alloc] init] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page2"]){
      [self.mainController pushViewController:[[Page2ViewController alloc] init] animated:YES];
    }
    else if([[url path] isEqualToString:@"/page3"]){
      [self.mainController pushViewController:[[Page3ViewController alloc] init] animated:YES];
    }
    else{
      return NO;
    }
    return YES;
  }
  else{
    return NO;
  }

Conclusion

This post shows you how to respond to urls passed into your app, but you can do a lot more. By taking apart the url in the callback, you can do quite a lot. For example, you could:

  • customize the look and feel of the page opened
  • pre-populate certain elements on the page (the way the Twitter app did)
  • track how users are getting to your app

Happy Coding!

Refactoring Git Branches

TL/DR

I describe a technique that allows one to extract commits from a larger Git branch into separate branches. This “git branch refactoring” provides numerous benefits:

  • A fast-track for integrating urgent changes (like refactorings or bug fixes) that were created as part of feature development into the main Git branch, before the feature is merged.

  • Improvements to the quality, efficiency, and fun factor of code reviews, by allowing your team to review and merge individual changes within larger feature branches individually, by different reviewers, at different times.

Feature branches contain many different changes

When code classes or methods do too many things, we refactor them by extracting individual pieces of functionality into separate classes or methods. The same applies to feature branches in Git. Ideally a Git feature branch should also perform just one particular change, be it a bug fix, a refactoring, or adding a new feature.

We all know, however, that while adding the next cool feature to a code base, we come across existing code that needs to be improved along the way. So, in the spirit of Continuous Improvement and Kaizen, or simply because we depend on improvements to happen before we can continue developing our feature, we

  • fix bugs in existing code that we depend on,
  • add missing functionality to existing code that our feature requires,
  • refactor existing code, for example by extracting pieces of it to make them available separately,
  • do good in numerous other ways, for example by cleaning up messes, reducing smells, paying down technical debt, or improving code quality and spec coverage.

As a result of this process, many feature branches end up looking similar to the one depicted here (time flows from the bottom up here, i.e. new commits are added at the top):

bloated feature branch

Our feature branch called kg_feature_1 is cut from the development branch (we follow the terminology of the NVIE model here), which is our main branch that is shared by all developers here. This development branch only contains the framework so far. Our feature branch contains a number of commits for the feature itself (feature 1feature 4), a bug fix for an external code module (“bug fix 1”), and a more generic refactoring that was done in two separate steps (“refactoring 1a” and “refactoring 1b”).

Changes should be reviewed individually

We don’t want to just send this many-headed monster of a feature branch off as a single code review. It contains too many different things! This makes it hard for the reviewer to get an overview of what we are doing here. Its also pretty hard to have meaningful conversations about several different things at the same time. This is just a recipe for confusion, cognitive overload, and missing important details. It also reduces the fun and the power of code reviews.

We also want to review and merge the bug fix and the refactorings into the development branch right away, so that other developers can incorporate them into their work before their work deviates too much from our changes. If we waited with that until the whole feature is done, it will be too late, and we will have to deal with a lot more merge conflicts than necessary.

These different types of changes in different parts of the code base should probably also be reviewed by different people. The bug fix gets reviewed by the author/maintainer of the affected module, the refactoring by the architect or tech lead, and the feature by another team member.

Let’s refactor our Git branch!

Extracting commits into dedicated branches

The change that most urgently needs to get merged into development here is the refactoring, because it might touch code that other people are currently working on. Lets extract it into a separate branch.

1
2
3
4
5
6
# Cut a new branch for the refactoring off the development branch.
$ git checkout -b kg_refactoring development

# Move the refactoring commits into the "kg_refactoring" branch.
$ git cherry-pick [SHA1 of the "refactoring 1a" commit]
$ git cherry-pick [SHA1 of the "refactoring 1b" commit]

Our kg_refactoring branch now looks like this.

refactoring branch

It only contains both refactoring commits. Perfect! Now we can push this branch to the Git repo, get it reviewed by the architect or tech lead, and merge it into the development branch.

Once this is done, we rebase our feature branch against the development branch to pick up all the changes that happened in it recently, like our extracted refactoring. We do this regularly anyways, since it is a good practice to keep our feature branches synchronized with ongoing development, and solve merge conflict as soon as they happen, rather than all at once when the branch is done.

1
2
$ git checkout kg_feature_1
$ git rebase development

If you know Git, you know that our branch situation looks like this now.

after refactoring merge

This looks much better already. The refactorings are now part of the development branch, and our feature branch still has access to them, and uses them!

The only foreign body left in the feature branch is the bug fix. Once we have extracted it using the exact same technique, our branch situation looks like this.

refactoring branch

Both bug fix and refactorings have been individually reviewed and merged into the development branch, separately from the ongoing feature development. The refactored feature branch contains only feature-related commits, and can be reviewed now, or developed further. If we do more bug fixes or refactorings, we can repeat this procedure as needed.

When to do this

Extracting autonomous changes into their own Git feature branches can significantly improve the structure of your branches, the performance of your code reviews, and the efficiency of your workflows. In order to work well, however, it is dependent on a number of things:

  • Each commit is well described and addresses a single change (feature, bug fix, refactoring, etc).

  • You can rebase your feature branches. Only do that with branches that only you use! Read Linus Torvalds’ explanation if you are unsure what this means. In our case we marked our branches as private by prefixing their names with our initials, and according to our team policy this makes them private.

  • You have enough specs, especially feature specs (aka integration tests) to verify that everything still works after your branch refactorings.

All these are good engineering practices to live by anyways, so give this a shot, and let me know what you think in the comments!

Lightning Fast Prototyping With Harp

What is Harp?

Harp is a static web server built on Node. It serves Jade, Markdown, EJS, CoffeeScript, Sass, LESS and Stylus as HTML, CSS & JavaScript with no additional configuration or plugins. You get a powerful asset pipeline with a few extra perks. This makes it great for prototyping and as a frontend playground.

Why use Harp?

It’s blazing fast, easy to get setup, and dead simple. It’s also more powerful than things like CodePen or jsFiddle (more than just a “widget builder”). If you end up liking what you’ve built, you can then compile the static assets or actually deploy it to Harp’s own dropbox based hosting platform. Because of it’s speed and versatility, Harp is an efficient and effective tool for prototyping UI concepts and workflows.

Getting Started

  • Install Node or brew install node
  • Install Harp npm install -g harp
  • Create a new Harp project harp init project-name (will create a folder with your project name in the current directory)
  • Start Harp server harp server project-name
  • Access your site at localhost:9000

Now you can write using your favorite frontend technologies! (This article will be using Jade/Sylus for all code examples.)

Note: Any time you would like to generate the static files for your project, you can run harp compile and your site will be generated in the /www folder in the root of your project.

Public vs. Private

In Harp, anything prefixed with an underscore will not be publicly available via the server or compilation. This is useful for partials and/or private layout files. Anything without an underscore will be compiled and publicly available. Making things private when possible will also improve compile time (not that you need it). Folders that are prefixed with an underscore will be private as well as anything that is contained within.

Layouts and Partials

You’ll have layout files which have the underscore prefix. The _layout file will be applied to anything on that folder level. eg: /_layout.jade would apply to anything on the root level. /posts/_layout.jade would be applied to any page view within the /posts directory.

Partials are just as easy. If you have a partial.jade file in the _partials/ folder you can simply:

1
!= partial("_partials/partial")

Static Data Structures

With Harp you also get flat JSON data structures to play with. You can create a _data.json file in any subdirectory and then have access to that data in your templates. Eg: if you have /posts/_data.json you can then iterate over that json collection.

with /posts/_data.json:

1
2
3
4
5
6
7
8
9
10
11
12
{
  "post1-slug" : {
    "name":      "Post 1",
    "author":    "John Doe",
    "content":   "Lorem ipsum..."
  },
  "post2-slug" : {
    "name":      "Post 2",
    "author":    "Kevin Smith",
    "content":   "Some Post Content"
  }
}

Then in your template:

1
2
3
4
each post, slug in public.posts._data
  .post
    h3
      a.link(href="/posts/#{ slug }")= post.name

This will enumerate over the top-level keys using the key name as the slug.

Output:

1
2
3
4
5
6
7
<div class="post">
  <h3><a class="link" href="/posts/post1-slug">Post 1</a></h3>
</div>

<div class="post">
  <h3><a class="link" href="/posts/post2-slug">Post 2</a></h3>
</div>

It should be noted here that the _data.json file is intended only for meta data. If you were blogging, like in this example, you would also need to include a view file in the posts directory (with a file name that’s the same as the slug) that contains the actual post itself. The _data.json would include things like the slug, title, tags, etc. It’s easy to see that you can use _data.json files to mimic database results for prototyping very easily.

You also get a global variables JSON file located at /_harp.json which is useful for layout variables (eg: site title) and other settings that you’ll need available within scope.

The Current Object

On any template or layout you will have access to current which will give you brief information about the current location of the app. The current object has a path and source property. Eg: if you were accessing /posts/post-name the current object may look like:

1
2
3
4
{
  "path":   ["posts", "post-name"],
  "source": "post-name"
}

This is very useful if you would like to have an active-state in your site’s navigation:

1
2
3
4
5
ul
  li(class="#{ current.source == 'index' ? 'active' : '' }")
    a(href="/") Home
  li(class="#{ current.source == 'posts' ? 'active' : '' }")
    a(href="/posts") Posts

A Boilerplate Project

If you prefer to learn by example, you can clone my harp boilerplate. It will set you up with a layout, a partial, some useful stylus mixins, icon fonts, and other helpful examples.

For more information you should check out the harp documentation

Secure Your Software for Battle

A World Wide War is being fought across the software world. The warring factions are engaging in battles in IT centers, corporate offices and your homes; between security experts on one side and cyber-criminals on the other.

This world we live in is getting increasingly connected and computerized. Consumers, organizations and governments alike are conducting business electronically at an ever increasing pace. This requires organizations to store information about their consumers. This includes financial, personal data as well as data about their online and spending habits.

Corporations need to secure this data because

  • privacy laws demand it
  • consumers demand it
  • it gives them a competitive advantage

Naturally, this data is prized by hackers, criminals, competitive entities and unfriendly state agents. This battle of wits between the security experts and the cyber-criminals is an ongoing one and is only going to get bigger.

The systems that process and store data also become targets — either to disable them, or to access, corrupt or destroy data.

Securing the data and the systems that manage it is now a high priority and high profile task. Or, it should be if they mean business. It is no longer a nice-to-have or even a should-have. It is now a must-have.

Companies that don’t have their act together … are in for a crude awakening in the internet age, to significant damage to their business and reputation (http://www.businessweek.com/articles/2014-03-13/target-missed-alarms-in-epic-hack-of-credit-card-data), and in some cases to an end of their very existence (http://www.forbes.com/sites/cameronkeng/2014/02/25/bitcoins-mt-gox-shuts-down-loses-409200000-dollars-recovery-steps-and-taking-your-tax-losses).

The War Zone

The increasing mobility of data in corporations is one of the biggest challenges faced in the last couple of years. Users are feeling empowered to access data from anywhere. The BYOD (Bring Your Own Device) trend is bringing in new attack vectors. The data access end-point is also going through a transformation. No longer are these Windows-based, but now a diverse mix of different systems. Some of them are still in their infancy compared to long established platforms like Windows/Unix. These platforms are new targets, and malware for mobile devices is now a rapidly growing threat.

Employees are bringing in their smartphones, tablets and next-gen ultrabooks in to the corporate networks. IT professionals are being required to secure sensitive data on devices the have little control over.

The growing popularity of the “Internet of things” (smartphones, tablets, wearable gadgets, and interconnected devices) makes a fluid situation even more dynamic for security experts. They present ripe targets for cyber-criminals looking to either find access to your data or cause a large-scale disruption.

Cyber-criminals are getting sophisticated themselves. They are using underground online marketplaces to sell their wares and services. The Blackhole exploit kit accounted for almost a third of all malware infestations in 2012. And, it spawned a new generation of botnet exploits that drew on its source code for inspiration.

The Attacks

As the arms race unfolds between the cyber-criminals and security experts, here are some security threats that are at the top the list.

Cloud

Cloud computing continues to grow in popularity — and so too will the number of security threats targeting the cloud. It’s important that businesses realize that whilst they may outsource the handling and storage of their data, they can’t outsource the responsibility for the data itself. If their provider’s systems are breached, and data is exposed, the businesses are responsible.

Therefore, any risks need to be assessed in the same way as they would if they were holding the data internally. Other issues organizations should consider include: where will the data be stored, what happens to any data if organizations switch providers and what steps are being taken to secure the data on their provider’s systems, including how they prevent other customers from accessing it.

The bad guys will look to targeting the cloud in 2014, with cloud service provider employees the main focus. Successful phishing attacks on staff, especially if the password re-use jackpot is hit, have been the door to entry of many online member databases during 2013.

APT

APT or Advanced Persistent Threat is a set of well-camouflaged, focused, continuous hacking activities against a single target. APT activities require a high-level of stealth sustained over a period of time. These employ advanced malware to exploit vulnerabilities in the targets. And these are usually orchestrated for business or political motives. Flame/Gauss and Stuxnet are examples of recent APT attacks used to attack financial, government and military institutions.

Social Engineering

It’s a tried-and-true tactic in both the physical and digital worlds – social engineering. Before the digital age, this meant sneaking one’s way past the security desk with glib talk vs. a cleverly-worded email. Now social engineering has moved onto social networks, including Facebook and LinkedIn.

Attackers are increasing their use of social engineering, which goes beyond calling targeted employees and trying to trick them into giving up information. Cyber-criminals will try to hide malware using deceitful tactics to trick you into installing it.

The task of cybercriminal is a lot simpler — check if details are already posted on social networks. Social networks are about networking people; a convincing-looking profile of a company or person followed by a friend/connection request can be enough to get a social engineering attack started. Sarah Palin’s Yahoo email account was compromised using information that was publicly available on her social media profiles (birth name, date of birth etc).

Password Management

We live in a world where people are increasingly connected through social networking, cloud services & smart devices. This presents a password management headache for users and, by extension, the services they use. People tend to reuse passwords. More than half users reuse passwords for their online accounts. A data breach in one vendor/service provider can potentially put their accounts at other services at risk. If that weren’t scary enough, over a quarter tend to use easy-to-remember passwords such as birthdays or people’s names, opening the door for their online accounts to be hacked into by criminals. It’s worrying aspect for corporations that so many people are making life so easy for cyber-criminals hackers. Especially because simple to make strong password security a part of one’s everyday life.

BYOD (Bring Your Own Device )

This is one threat vector that will give security professionals many sleepless nights. BYOD (Bring Your Own Device) increases data leakage potential especially from devices unprotected by device specific counter-measures such as passcode/passphrase protection. etc. Most organizations going down the BYOD path aren’t implementing appropriate training for employees.

Attackers will try and circumvent app review and detection mechanisms on these devices. If possible their apps will mimic the UI of the native settings page and trick the user in to granting them admin privileges on the device. The device sensors such as GPS, microphone and cameras coupled with the ability to network over WIFI and mobile networks, could become tools for engineering further attacks.

The main areas of risk include: * Data loss — lost or stolen devices * ID theft — thieves logging in to your online accounts using saved credentials on your stolen device * Data leakage — via bogus WIFI access points and malware

Botnets

Essentially, a botnet is a collection of networked computers running a program to perform a task. Initially, most botnets were employed to perform legitimate tasks such controlling IRC channels. Later they were deployed to distribute spam, or perform DDos attacks.

Botnet operators are beginning to design systems that are more adaptive and redundant than many corporate and government networks. Botnets such as Gameover have replaced the traditional command and control link with a peer-to-peer network of infected systems. Controlling this agile attack vector before it can be used as an advanced persistent threat (APT) and migrates into smart mobile devices is crucial.

We’re also likely to see more mobile botnets, of the sort created using the RootSmart backdoor in Q1 2012. In order to prevent falling victim to mobile malware, businesses should install anti-malware protection on their Android devices, secure the data held on them and make sure that this can be wiped remotely if the device is lost or stolen. Businesses should also develop a policy for staff on how to reduce the risks from mobile devices. This should include not rooting the device, avoiding public Wi-Fi networks for confidential transactions, not relying solely on a simple PIN and only installing apps from trusted sources.

Spam

People still send emails. So do bad guys, and they will keep doing so long as people keep using email. You see spam that link to financial scams that are now mostly ignored, spam that link to malware designed to install botnet agents, spam that attempt to seem legit by linking to current events. Phishing via deceptive links in spam is a very common attack. Spammers are also wising up to having their bots and servers taken down. Snowshoe spam is their innovation against this countermeasure. This involves distributing their spamming across multiple ip addresses, spreading the load. Hence, the metaphor.

The Defense

So what are organizations to do to deflect security attacks? It used to be about securing the perimeter in the past. Now the perimeter is blurred due to BYOD and cloud services trends.

There is no one good answer. But most of them would include adopting treating security as a feature, good engineering practices, implementing systems, good employer training, a good plan to manage responses to attacks and finally transparency.

At Originate, when we work with partners, we emphasize security up front as a key element of the systems we build. In most cases, we have been able to educate and received buy-in from our partners. In one case however, we received push back on implementing a policy to enforce strong passwords — the partners feared having a strong password policy would impact user registration. Guess what, on the first day of launch, the first account to be compromised was the admin account. Fortunately, we were monitoring all activity and were able to retake control of the account and secure it. Needless to say, we got the go-ahead to implement the strong-password policy.

Make security a feature ( and not a Non-functional requirement )

When writing applications, security aspects should be first-class requirements. It cannot be a bolt-on later. Build the app with the best security practices from the start. According to security firms FireEye and Secunia, the real security problems in this decade are not in our operating systems but in the applications we run on them.

Design Software with Secure Features

Security issues in design and semantic flaws (ones that are not syntactic or code related), such as business logic flaws, cannot be detected in code and need to be inspected by performing threat models and abuse cases modeling during the design stage.

Threat modeling is an iterative technique used to identify the threats to the software being built. It starts by identifying the security objectives of the software and profiles it. It breaks the software into physical and logical constructs generating the software context that includes data flow diagrams, and end-to-end deployment scenarios, identifying entry and exit points, protocols, components, identities, and services.

Threat Modeling is performed during the design stage so that necessary security controls (safeguards) can be developed during the development phase of the software.

Develop Software with Secure Features

Follow the Saltzer and Schroeder list of principles for building good secure software:

Design principle What does it mean? Example
Economy of mechanism Keep the design as simple and small as possible. Modular, minimalistic code developed using TDD (only write code to make the tests pass),centralized services
Fail-safe defaults Access denied by default, and permitted explicitly Firewalls are configured such that the default is to deny access
Complete mediation Every access to every object checked for authority. Rely as little as possible on access decisions retrieved from a cache. File permissions tend to reflect this model: the operating system checks the user requesting access against the file’s ACL.
Open design The design should not be secret, the implementation of safeguards is. Cryptographic algorithms
Separation of privilege More than one condition is required to authorize a task Bank lockers, nuclear launch systems (I hope!), that use two separate keys to operate
Least privilege Invoke minimum privileges running web servers using accounts with reduced privileges
Least common mechanisms Minimize the amount of mechanism common to more than one user and depended on by all users. Role based dynamic libraries
Psychological acceptability Requires the policy interface to reflect the user’s mental model of protection Help dialogs, intuitive iconography

Use secure-coding best practices

A stitch in time saves nine. It is always cheaper to build secure applications then to correct the security bugs later. Making security a check-off item for code reviews. OwASP and Apple have good checklists that software development organizations would do well to follow.

Use source code analysis tools

Whenever possible integrate source-code analysis tools (Fortify, FindBugs, etc.) in to your development process. However, it is not a substitute for testing. And it can generate a lot of false positives initially. But it can be a useful tool in locating security vulnerabilities.

Use testing techniques to stress security features

Focus on abuse/negative test cases as well: test the evil path in unit and integration tests using. Running penetration tests against new releases as part of the release process using tools like SkipFish, MetaSploit etc.

Password and Session Management

Passwords are the keys to a user’s account. It is often a target to gain unauthorized access. Password strength is key in thwarting such attacks. Enforce a strong password policy. However, it does present an inconvenience to users who now have to remember long complex passwords. A good strategy around that is to not have to manage passwords and authentication at all. Implement single sign-on, so that users need to only have to authenticate once. Or, offload it to an 3rd party authentication authority using OAuth. However, if you do implement authentication ensure that

  • user credentials are never stored in clear text — makes sure that they are hashed and salted
  • never send user credentials over an un-encrypted connection

Session tokens should always be generated on the server. Use existing application frameworks for session management as much as possible. For example, Rails, Play, Lift, J2EE/Spring, ASP.Net etc., come with features around strongly encrypted sessions, protections against attacks like CSRF, XSS, SQL injection, session replay, timeouts etc. That wheel’s been invented and rolls well.

The bad guys will still be targeting the cloud in 2014, with cloud service provider employees the main focus. Successful phishing attacks on staff, especially if the password re-use jackpot is hit, have been the door to entry of many online member databases during 2013. Ultimately, with innovation and planning, cloud services could reduce business risks by providing greater flexibility, resiliency and security

Input validation

One of the most effective safeguards against hacking attacks is to perform input validation. Input validation should take place on the server side for all data. Validation criteria should be set for input fields. All data should be encoded before use. Input validation techniques along with proper encoding help block against many attack vectors like XSS, SQL Injection and remote file inclusion.

Use open source cryptography

The only way to secure architectures is to use systems, algorithms, and implementations that are well known to be safe, and which have been peer reviewed in the open by lots of people, and which have withstood a lot of hacking attempts in production already. Custom or proprietary solutions often have not gone through this process, and as a result are often riddled with security weaknesses that can and will be exploited.

Configuring for security

What does this mean? It is simply managing and defining a process to ensure that the systems the software runs on have been secured adequately. Harden your infrastructure by:

  1. Ensuring all software is updated to the latest security patches. This includes the OS, Web/App server, databases. And, any other component in the system architecture. Define a process to keep these components abreast of the latest updates and patches and deploying them to all deployed environments in a timely manner. The process should also include seemingly innocuous peripherals such as networked printers, VOIP phones
  2. Turning off or disabling all unused and unnecessary features (e.g. ports, services, accounts )
  3. Changing all default passwords and or accounts.
  4. Ensuring that error handling does not reveal any stack traces or other sensitive information to the users.

In 2013, a third of the most serious threats facing enterprises were associated with worms (Win32/Conficker, INF/Autorun, Win32/Dorkbot) infecting windows systems. Worms are commonly spread through network drives, abusing the Autorun feature or exploiting weak passwords.

The Conficker worm exploited weak passwords. Once it compromised a system, it could steal the credentials of an IT administrator to spread on the internal network.

Manage data exposure

Analyze your data characteristics and determine which data is sensitive enough to require extra protection ( e.g. passwords, credit card numbers, health records, etc.). Ensure that such data is never backed up or transmitted in clear text. Ensure that strong/updated cryptographic algorithms are used to encrypt such data. Don’t store data unnecessarily. Data you don’t have is impossible to steal. For example, don’t store credit card numbers if you use a payment gateway, simply store only the last 4 digits for tracking purposes. Also, no sensitive information such as IP addresses in logs. Ensure passwords are stored with algorithms specifically designed for password encryption such as bcrypt, PBKDF2 or scrpyt.

Manage the end-points

BYOD is here to stay. CEOs want to use the latest in mobile technology — the President of the United States uses a tablet. It has it’s benefits – reduced capital expense, and productivity gains. While there are some businesses that may do well to steer away from this trend, most corporations can benefit if they manage this trend with their organization, rather than fight it. Put in a plan to train employees & raise awareness regarding the risks using their devices to access corporate data. Consider implementing a MDM (Mobile Device Management) solution to manage access to corporate apps and access on employee devices. MobileIron, Airwatch/VmWare, Citrix among others have MDM suites to manage BYOD issues. Also, consider reigning it in using a CYOD (Choose your Own Device) option — where the business has a list of approved devices the employees chose from, to gain access to corporate systems.

Employee Training

Education is a good risk-mitigator as well. Training developers to follow good design principles (including the ones mentioned above) helps build good secure software. Training operations to configure for security means there will be fewer if not none, changes that could compromise security, for example, inadvertently removing access restrictions on network drives, which happens more often than one would imagine.

Stay Paranoid

It is impossible to protect anything that is in the users hands from being read in its completeness. For example, the attempts to encrypt digital media like DVDs have all been hacked with ease. There is no platform or app that hasn’t been hacked and studied in detail by the community, even if it requires modifying the hardware that the code is running on, either for establishing a street cred or for financial gain. Sony’s Playstation was protected on the hardware level against hacks. Even that wasn’t enough protection apparently — you could buy extra chips (called flashers) could solder onto chips on the motherboard. You could then downgrade the firmware and load pirated games onto it.

Stay vigilant for any signs of abnormal activity on your network. No alarm should be treated as to insignificant to investigate. Just recently, a large store chain revealed that they did not take early warning signs of abnormal activity too seriously, eventually resulting in a data breach that impacted 70 million customers, and more importantly a loss of trust and goodwill.

Every piece of code, data, or hardware that is in the users hands must be considered read and understood in its completeness, even if that means grinding away a hardware chip layer by layer and photographing the internals. People do that. Obfuscation might be helpful to reduce the size of the shipped binaries, but it doesn’t prevent reading and manipulation of the compiled code.

No input from the outside world, even if it seems to come from our own client apps over secure channels, must be trusted. The clients might be (and will be) compromised, and will send malicious data, no matter how obfuscated or protected they are. If there are truly security-relevant things, they have to remain on the servers, and have to happen there.

Further reading

Using ScalaTest in a Play Framework Application

The Play Framework is configured out of the box for specs2. However, some people (including the author) prefer ScalaTest. Play actually doesn’t really care what testing framework you use, but there is one gotcha you need to know about.

The default Play configuration sets the test options sequential true junitxml console, which are specific to specs2. Because ScalaTest doesn’t understand these options, you need to specify testOptions in Test := Nil in your build definition.

Starting with Play 2.2, build.sbt is used for the build definition. Simply add the line:

1
testOptions in Test := Nil

For Play 2.0 or 2.1, project/Build.scala is used. You’ll have to do something like the following:

1
2
3
4
val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings(
  // other project settings here
  testOptions in Test := Nil
)

Also, don’t forget to add ScalaTest as a project dependency. At the time of writing, the latest version can be specified with the following:

1
"org.scalatest" % "scalatest_2.10" % "2.0" % "test"

Recursive Type Signatures in Scala

Have you seen a type signature like this before?

1
trait T[U <: T[U]]

If you’re like me, you’ve come across this type signature, and you’re wondering what the heck it means. You likely Googled something like “recursive type” or “self-referential type” and ended up here.

So what does this type signature mean? Why not use trait T[U]?

To understand the meaning of T[U <: T[U]], we’ll work through a simple example.

Example

Suppose we want database entities with CRUD methods. We could define them like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
// I don't know about you, but I like to name my fruit.
case class Apple(name: String, price: Double) {
  def create(entityData: String): Apple
  def read(id: String): Option[Apple]
  def update(f: Apple => Apple): Apple
  def delete(id: String): Unit
}

case class Bird(name: String, birthday: DateTime) {
  def create(entityData: String): Bird
  def read(id: String): Option[Bird]
  def update(f: Bird => Bird): Bird
  def delete(id: String): Unit
}

But we can see that these classes look nearly identical. In addition, any new CRUD entities we add will expose the same CRUD methods. Now, should we abstract the interface into a trait? Let’s see what happens when we do:

1
2
3
4
5
6
7
8
9
10
trait CrudEntity {
  def create(entityData: String): CrudEntity
  def read(id: String): Option[CrudEntity]
  def update(f: CrudEntity => CrudEntity): CrudEntity
  def delete(id: String): Unit
}

case class Apple(name: String, age: Int) extends CrudEntity

case class Bird(name: String, hobby: String) extends CrudEntity

Well this sucks. Our method signatures don’t fully express what we want. We’ve lost the ability to ensure that e.g. calling update on an Apple returns an Apple. As is, it can return any CrudEntity. Let’s try to regain some specificity by adding a type parameter to our CrudEntity trait:

1
2
3
4
5
6
7
8
9
10
trait CrudEntity_2[E] {
  def create(entityData: String): E
  def read(id: String): Option[E]
  def update(f: E => E): E
  def delete(id: String): Unit
}

case class Apple(name: String, age: Int) extends CrudEntity_2[Apple]

case class Bird(name: String, hobby: String) extends CrudEntity_2[Bird]

Okay, better. But we still haven’t locked this down. Our types don’t yet express exactly what we want. Do you see the problem?

The problem is that someone can extend CrudEntity_2 in a way we didn’t intend them to:

1
case class Orange(name: String, bankAccount: Double) extends CrudEntity_2[FloobyDust]

Whoa! In the code above, CrudEntity_2[E] does not restrict the type of E, so they can use anything they want, without complaint from the compiler — FloobyDust, Potato, BurritoAstronaut, you name it.

This is no bueno. Instead, we’d like them to get a big, fat compiler error if they try extending anything other than CrudEntity_2[Orange]. How do we ensure that E matches the class we’re defining?

Let’s try defining CrudEntity again. This time, we’ll use type bounds:

1
2
3
4
5
6
7
8
9
10
trait CrudEntity_3[E <: CrudEntity_3[E]] {
  def create(entityData: String): E
  def read(id: String): Option[E]
  def update(f: E => E): E
  def delete(id: String): Unit
}

case class Apple(name: String, age: Int) extends CrudEntity_3[Apple]

case class Bird(name: String, hobby: String) extends CrudEntity_3[Bird]

Better. Now we’ve constrained E to be a subtype of CrudEntity. No more FloobyDust. But there’s one last problem, and you can probably guess what it is. We haven’t yet ensured that E matches our class type, only that it subclasses CrudEntity. CrudEntity is still open for abuse:

1
case class Orange(name: String, age: Int) extends CrudEntity_3[Apple]

Yuck! To take care of this, we need a way to ensure that e.g. Orange extends CrudEntity_3[Orange]. For this assurance, we’ll use a self type.

Here is our final definition of CrudEntity, which uses a self type:

1
2
3
4
5
6
trait CrudEntity[E <: CrudEntity[E]] { self: E =>
  def create(entityData: String): E
  def read(id: String): Option[E]
  def update(f: E => E): E
  def delete(id: String): Unit
}

self: E => ensures that any concrete class that extends CrudEntity must be of type E and that code like

1
case class Orange(name: String, age: Int) extends CrudEntity[Apple]

will get rejected by the compiler because Orange is not of type Apple.

Now we can rest, confident that our definition of CrudEntity ensures that any subtype of CrudEntity[E] must in fact be an E. This definition gives us the semantics we desire, and enlists the compiler to reject all code in violation.

Speed Up Your Rails Specs by 10x

One of the primary reasons people end up being lax in letting specifications drive the development of their Rails applications is the time it takes to get feedback from running the suite of specifications. A number of tools have been built to help alleviate this pain like Spork, Zeus, and Spring. In fact Rails 4.1 will now come with Spring standard. Unfortunately, these tools are just crutches that tackle the symptoms of the problem rather than the problem itself. The actual problem is writing tightly coupled code that expects to have the full Rails framework always present, which is slow to start up.

Developing Decoupled Code

The solution is to write code that is isolated and decouple your components from as much of the system as possible. In other words, write SOLID Rails code. As a specific example, one might typically directly use a model class to create an instance. Instead we can use dependency injection to remove hard coded references to classes. We just need to make sure we safely reference the defaults using either block notation or a lazy evaluating ||=. Below we have a service that needs to create Widgets which happen to be ActiveRecord models. Instead of directly referencing the Widget class, we use lazy evalutation in our chosen injection method. This allows us to decouple our code and not need ActiveRecord loaded.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# A tightly coupled class. We don't want this.
class MyService
  def create_widget(params)
    Widget.create(params)
  end
end

# We can inject in the initializer
class MyService
  attr_reader :widget_factory

  def initialize(dependencies={})
    @widget_factory = dependencies.fetch(:widget_factory) { Widget }
  end

  def create_widget(params)
    widget_factory.create(params)
  end
end

# Or we can explictly inject via a setter with a lazy reader
class MyService
  attr_writer :widget_factory

  def widget_factory
    @widget_factory ||= Widget
  end

  def create_widget(params)
    widget_factory.create(params)
  end
end

# A specification injecting the dependency using the second method
describe MyService do
  subject(:service) { MyService.new }
  let(:widget_factory) { double 'widget_factory', create: nil }
  before { service.widget_factory = widget_factory }

  it 'creates a widget with the factory' do
    service.create_widget({name: 'sprocket'})
    expect(widget_factory).to have_received(:create).with({name: 'sprocket'})
  end
end

A Base Rails-free Configuration

When writing your applications in this way you can then start to restructure how you setup your specifications and minimize the required environment to run both your specification and your code fulfilling the specification. The typical spec_helper.rb will have a line like this:

1
require File.expand_path("../../config/environment", __FILE__)

This is what loads your entire Rails application and slows down the running of your tests. To make your specifications faster, you need to use a configuration file that does not contain this line. So let’s start by creating a very light weight base_spec_helper.rb:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
ENV["RAILS_ENV"] ||= 'test'
require 'rubygems'

RAILS_ROOT = File.expand_path('../..', __FILE__)
Dir[File.join(RAILS_ROOT, 'spec/support/**/*.rb')].each {|f| require f}

RSpec.configure do |config|
  config.mock_with :rspec
  config.order = 'random'
  # Your prefered config options go here
end

require 'active_support'
require 'active_support/dependencies'

We are requiring active_support and active_support/dependencies so we can have access to the autoloader Rails uses without actually loading up all of Rails. It is fairly light weight and the convienence outweighs the cost. In each spec helper which requires this base we will add the relevant portions of our app into the ActiveSupport::Dependencies.autoload_paths.

Plain Ruby Object Specifications

Depending on the part of application that you are specifying, you can create spec helpers specific to what you need in any one context. For example, the simplest would be one for specifying any type of pure Ruby class such as a service class. A sample services_spec_helper.rb might be:

1
2
3
require 'base_spec_helper'
Dir[File.join(RAILS_ROOT, "spec/support_services/**/*.rb")].each {|f| require f}
ActiveSupport::Dependencies.autoload_paths << "#{RAILS_ROOT}/app/services"

Decorator Specifications

For your decorators, you might choose to use Draper and your decorators_spec_helper.rb might look like:

1
2
3
4
5
require 'base_spec_helper'
require 'draper'
Draper::ViewContext.test_strategy :fast
Dir[File.join(RAILS_ROOT, "spec/support_decorators/**/*.rb")].each {|f| require f}
ActiveSupport::Dependencies.autoload_paths << "#{RAILS_ROOT}/app/decorators"

Model Specifications

Testing models needs a little bit more. Assuming you are using ActiveRecord you’ll need to include that as well as establish a connection to your database. We won’t include factory_girl or database_cleaner as most of your tests should not be actually creating database objects. In fact, the only place you really need to actually create an object in the database is when testing uniqueness validations. When you do need to create something you can just manually clean it up or use a transaction. So a sample models_spec_helper.rb can look like this:

1
2
3
4
5
6
7
8
9
10
11
12
require 'base_spec_helper'
require 'active_record'
# RSpec has some nice matchers for models so we'll pull them in
require 'rspec/rails/extensions/active_record/base'
Dir[File.join(RAILS_ROOT, "spec/support_models/**/*.rb")].each {|f| require f}

# Manually connect to the database
ActiveRecord::Base.establish_connection(
  YAML.load(File.read(RAILS_ROOT + '/config/database.yml'))['test']
)

ActiveSupport::Dependencies.autoload_paths << "#{RAILS_ROOT}/app/models"

Feature Specifications

Finally, when creating feature specs, we do need our full Rails stack and our feature_spec_helper.rb is going to look very similar to what your current spec_helper.rb looks like.

Summary

I found myself using varitions on the above spec helpers in projects I work on and decided I would write a set of generators to make it easier to bootstrap the project. The gem can be found at https://github.com/Originate/rails_spec_harness

While introducing these changes into existing projects I have found speed increases of 8-12 times. The worst project experienced a 27x increase once these changes and the corresponding changes in coding habits where applied. As an example I made a specification with 4 examples for a plain Ruby class. I then used the time command line utility to measure running rspec with the minimal spec helper as well as the full Rails spec helper and found the following:

Spec Helper Real User Sys RSpec Reported
Full Rails 4.913s 2.521s 1.183s 0.0706s
Minimal 0.492s 0.407s 0.080s 0.0057s

Write SOLID code, isolate your specifications and enjoy a fun and sane development experience.

TDD Is BS**

** BS = Behavior Specification

tl/dr

The misleading terminology around TDD is responsible for a good amount of confusion and hesitation in this area, as well as over- and undertesting. Calling tests specifications makes the concept more intuitively available in several ways.

One does not simply test something before it is built

testing thin air

One cannot test something before it is built. Asking people to write tests before they write code is like asking them to test-drive a new car before it even exists. This isn’t possible. Similarly, a piece of code can only be tested after it is completely written. We can’t test thin air.

Asking people to write tests before they write code is counter-intuitive and misleading. Resistance or discomfort with the idea is only natural.

specifications instead of tests

A better perspective is to treat tests as specifications for the application in the works, i.e. descriptions of what our code should do.

It is completely plausible to reflect upfront about what I’m going to build – and to write down these thoughts – before starting to build anything serious, costly, and/or sizeable. Especially when working within a team.

It is similarly intuitive that starting to think about these specifications – or writing them down – after a product has already been built makes very little sense, provides very little benefit to the development process, and is actually more a hassle than useful at that point. Specifications need to be defined, agreed upon, and considered before we build the actual system.

behavior-driven development

Specifications just define expected behavior. They don’t dictate how in particular to implement things. The idea of “testing”, on the other hand, naturally means checking a particular implementation in all its gory details.

This distinction is important. Ideally, our tests/specs should not be concerned with implementation details and private methods. Otherwise they become brittle, and will break unnecessarily each time we refactor or clean up something. All that we really care about when running our specs is that the code still works, not how it still works. The how should be discussed in code reviews.

user perspective

Developers tend to occasionally forget about the user perspective while being deeply immersed in the code. Specifications give them a natural place to muse over the user-visible behavior of the application, from the perspective of the user of their code. Specifications thereby form the missing link between business requirements and their technical implementation. Features specs (aka “integration tests”) are runnable user stories.

Tests, on the other hand, just assert implementation details from a technical perspective. Even integration tests just check that the different parts and layers of the application have been integrated properly.

conclusion

While there is often no real technical difference between “tests” and “specs”, the term “test-driven development” is misleading, while “specification-driven development” is intuitive and describes what we actually do here better. We should consider using the latter term in favor of the former. Let’s write “specs” instead of “tests”, and “feature specifications” instead of “integration tests” from now on.

Much love, and happy coding! :)

How Does Caching Work in AFNetworking? : AFImageCache & NSUrlCache Explained

If you are an iOS developer using Mattt Thompson’s ‘delightful networking framework’ AFNetworking (and if you aren’t, what are you waiting for?), perhaps you have been been curious or confused about the caching mechanism employed and how you can tweak it to your advantage.

AFNetworking actually takes advantage of 2 separate caching mechanisms:

  • AFImagecache: a memory-only image cache private to AFNetworking, subclassed off of NSCache

  • NSURLCache: NSURLConnection's default URL caching mechanism, used to store NSURLResponse objects : an in-memory cache by default, configurable as an on-disk persistent cache

In order to understand how each caching system works, let’s look at how they are defined:

How AFImageCache Works

AFImageCache is a part of the UIImageView+AFNetworking category. It is a subclass of NSCache, storing UIImage objects with a URL string as its key (obtained from an input NSURLRequest object).

AFImageCache definition:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
@interface AFImageCache : NSCache <AFImageCache>

// singleton instantiation :

+ (id <AFImageCache>)sharedImageCache {
    static AFImageCache *_af_defaultImageCache = nil;
    static dispatch_once_t oncePredicate;
    dispatch_once(&oncePredicate, ^{
        _af_defaultImageCache = [[AFImageCache alloc] init];

// clears out cache on memory warning :

    [[NSNotificationCenter defaultCenter] addObserverForName:UIApplicationDidReceiveMemoryWarningNotification object:nil queue:[NSOperationQueue mainQueue] usingBlock:^(NSNotification * __unused notification) {
        [_af_defaultImageCache removeAllObjects];
    }];
});

// key from [[NSURLRequest URL] absoluteString] :

static inline NSString * AFImageCacheKeyFromURLRequest(NSURLRequest *request) {
    return [[request URL] absoluteString];
}

@implementation AFImageCache

// write to cache if proper policy on NSURLRequest :

- (UIImage *)cachedImageForRequest:(NSURLRequest *)request {
    switch ([request cachePolicy]) {
        case NSURLRequestReloadIgnoringCacheData:
        case NSURLRequestReloadIgnoringLocalAndRemoteCacheData:
            return nil;
        default:
            break;
    }

    return [self objectForKey:AFImageCacheKeyFromURLRequest(request)];
}

// read from cache :

- (void)cacheImage:(UIImage *)image
        forRequest:(NSURLRequest *)request {
    if (image && request) {
        [self setObject:image forKey:AFImageCacheKeyFromURLRequest(request)];
    }
}

AFImageCache is a private implementation of NSCache. There is no customization that you can do outside of editing the implementation in the the UIImageView+AFNetworking category, directly. It stores all accessed UIImage objects into its NSCache. The NSCache controls when the UIImage objects are released. If you wish to observe when images are released, you can implement NSCacheDelegate’s cache:willEvictObject method.

Edit (03.14.14) : Mattt Thompson has gratiously informed me that as of AFNetworking 2.1, AFImageCache is configurable. There is now a public setSharedImageCache method. Here’s the full AFN 2.2.1 UIImageView+AFNetworking specification.

How NSURLCache Works

Since AFNetworking uses NSURLConnection, it takes advantage of its native caching mechanism, NSURLCache. NSURLCache caches NSURLResponse objects returned by server calls via NSURLConnection.

Enabled by Default, but Needs a Hand

An NSURLCache sharedCache is enabled by default and will be used by any NSURLConnection objects fetching URL contents for you.

Unfortunately, it has a tendency to hog memory and does not write to disk in its default configuration. To tame the beast and potentially add some persistance, you can simply declare a shared NSURLCache in your app delegate like so:

1
2
3
4
NSURLCache *sharedCache = [[NSURLCache alloc] initWithMemoryCapacity:2 * 1024 * 1024
                                              diskCapacity:100 * 1024 * 1024
                                              diskPath:nil];
[NSURLCache setSharedURLCache:sharedCache];

Here we declare a shared NSURLCache with 2mb of memory and 100mb of disk space

Setting the Cache Policy on NSURLRequest Objects

NSURLCache will respect the caching policy (NSURLRequestCachePolicy) of each NSURLRequest object. The policies are defined as follows :

  • NSURLRequestUseProtocolCachePolicy: specifies that the caching logic defined in the protocol implementation, if any, is used for a particular URL load request. This is the default policy for URL load requests

  • NSURLRequestReloadIgnoringLocalCacheData: ignore the local cache, reload from source

  • NSURLRequestReloadIgnoringLocalAndRemoteCacheData: ignore local & remote caches, reload from source

  • NSURLRequestReturnCacheDataElseLoad: load from cache, else go to source.

  • NSURLRequestReturnCacheDataDontLoad: offline mode, load cache data regardless of expiration, do not go to source

  • NSURLRequestReloadRevalidatingCacheData: existing cache data may be used provided the origin source confirms its validity, otherwise the URL is loaded from the origin source.

Caching to Disk with NSURLCache

Cache-Control HTTP Header

Either the Cache-Control header or the Expires header MUST be in the HTTP response header from the server in order for the client to cache it (with the existence of the Cache-Control header taking precedence over the Expires header). This is a huge gotcha to watch out for. Cache Control can have parameters defined such as max-age (how long to cache before updating response), public / private access, or no-cache (don’t cache response). Here is a good introduction to HTTP cache headers.

Subclass NSURLCache for Ultimate Control

If you would like to bypass the requirement for a Cache-Control HTTP header and want to define your own rules for writing and reading the NSURLCache given an NSURLResponse object, you can subclass NSURLCache.

Here is an example that uses a CACHE_EXPIRES value to judge how long to hold on to the cached response before going back to the source:

(Thanks to Mattt Thompson for the feedback and code edits!)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
@interface CustomURLCache : NSURLCache

static NSString * const CustomURLCacheExpirationKey = @"CustomURLCacheExpiration";
static NSTimeInterval const CustomURLCacheExpirationInterval = 600;

@implementation CustomURLCache

+ (instancetype)standardURLCache {
    static CustomURLCache *_standardURLCache = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        _standardURLCache = [[CustomURLCache alloc]
                                 initWithMemoryCapacity:(2 * 1024 * 1024)
                                 diskCapacity:(100 * 1024 * 1024)
                                 diskPath:nil];
    }

    return _standardURLCache;
}

#pragma mark - NSURLCache

- (NSCachedURLResponse *)cachedResponseForRequest:(NSURLRequest *)request {
    NSCachedURLResponse *cachedResponse = [super cachedResponseForRequest:request];

    if (cachedResponse) {
        NSDate* cacheDate = cachedResponse.userInfo[CustomURLCacheExpirationKey];
        NSDate* cacheExpirationDate = [cacheDate dateByAddingTimeInterval:CustomURLCacheExpirationInterval];
        if ([cacheExpirationDate compare:[NSDate date]] == NSOrderedAscending) {
            [self removeCachedResponseForRequest:request];
            return nil;
        }
    }
}

    return cachedResponse;
}

- (void)storeCachedResponse:(NSCachedURLResponse *)cachedResponse
                 forRequest:(NSURLRequest *)request
{
    NSMutableDictionary *userInfo = [NSMutableDictionary dictionaryWithDictionary:cachedResponse.userInfo];
    userInfo[CustomURLCacheExpirationKey] = [NSDate date];

    NSCachedURLResponse *modifiedCachedResponse = [[NSCachedURLResponse alloc] initWithResponse:cachedResponse.response data:cachedResponse.data userInfo:userInfo storagePolicy:cachedResponse.storagePolicy];

    [super storeCachedResponse:modifiedCachedResponse forRequest:request];
}

@end

Now that you have your NSURLCache subclass, don’t forget to initialize it in your AppDelegate in order to use it :

1
2
3
4
CustomURLCache *URLCache = [[CustomURLCache alloc] initWithMemoryCapacity:2 * 1024 * 1024
                                                   diskCapacity:100 * 1024 * 1024
                                                                 diskPath:nil];
[NSURLCache setSharedURLCache:URLCache];

Overriding the NSURLResponse before caching

The -connection:willCacheResponse delegate is a place to intercept and edit the NSURLCachedResponse object created by NSURLConnection before it is cached. In order to edit the NSURLCachedResponse, return an edited mutable copy as follows (code from NSHipster blog):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
- (NSCachedURLResponse *)connection:(NSURLConnection *)connection
                  willCacheResponse:(NSCachedURLResponse *)cachedResponse {
    NSMutableDictionary *mutableUserInfo = [[cachedResponse userInfo] mutableCopy];
    NSMutableData *mutableData = [[cachedResponse data] mutableCopy];
    NSURLCacheStoragePolicy storagePolicy = NSURLCacheStorageAllowedInMemoryOnly;

    // ...

    return [[NSCachedURLResponse alloc] initWithResponse:[cachedResponse response]
                                                    data:mutableData
                                                userInfo:mutableUserInfo
                                           storagePolicy:storagePolicy];
}

// If you do not wish to cache the NSURLCachedResponse, just return nil from the delegate function:

- (NSCachedURLResponse *)connection:(NSURLConnection *)connection
                  willCacheResponse:(NSCachedURLResponse *)cachedResponse {
    return nil;
}

Disabling NSURLCache

Don’t want to use the NSURLCache? Not Impressed? That’s okay. To disable the NSURLCache, simply zero out memory and disk space in the shared NSURLCache definition in your appDelegate:

1
2
3
4
NSURLCache *sharedCache = [[NSURLCache alloc] initWithMemoryCapacity:0
                                              diskCapacity:0
                                              diskPath:nil];
[NSURLCache setSharedURLCache:sharedCache];

Summary

I wanted to write this blog post for the benefit of the iOS community, to summarize all of the information I found dealing with caching releated to AFNetworking. We had an internal app loading a lot of images that had some memory issues and performance problems. I was tasked with trying to diagnose the caching behavior of the app. During this exercise, I discovered the information on this post through scouring the web and doing plenty of debugging and logging. It is my hope that this post summarizes my findings and provides an opportunity for others with AFNetworking experience to add additional information. I hope that you have found this helpful.

One Simple Trick to Level Up Your Code

As software developers we can’t stop bad code but we can make it better. There are a number of ways we try to make better code. We try to follow best practices like TDD. We’ve all read a number of books on design patterns, code standards, rules, and guidelines. It is a fact though, that the real world gets in the way. Deadlines are tight. Projects get rushed. Code is written quickly and sloppily. None of these things are going away but there is one simple thing we can do no matter how tight the deadline.

When you write a piece of code, take a brief pause and look at what you just wrote. Ask yourself:

Can I make this code better?

Chances are you’ll see at least one thing that could be changed. A large method that could be broken up. A convoluted logic expression. Or any number of small things. If you take just a moment to change that one thing before moving on, all of your code will be cleaner, easier to maintain, and in general much higher quality.