Recently, I was tasked to prototype a solution that was most easily solved by building a native Chrome Extension alongside a companion web application. I needed a way for these two applications to share data amongst each other, and from a bit of research I learned that Chrome Extensions have an API for interacting with Chrome Storage, but interacting with the browser local storage was cumbersome. To bridge the data-gap between these two applications, I brought in Firebase Real-Time Database (RTD). We have been using it for several projects in the Originate portfolio and it has proven to be a very valuable tool. Firebase is essentially a backend-as-a-service that hosts your data and allows multiple applications to hook into it. While both were interesting, in this post, I will walk you through how to integrate Firebase RTD into a Chrome App.
Configuring Firebase for Chrome Extensions
If this is your first time creating a Chrome Extension, I recommend this awesome boilerplate. Hooking up Firebase to your Chrome Extension requires an added layer of complexity due to certain restraints on security as well as architecture best practices. Chrome Apps have a specific Content Security Policy (CSP) which prevents inline scripting and instantiates that policy to mitigate against cross-site scripting issues. There are more compliancy rules to consider generally, but for the purpose of Firebase configuration, you will need to specify that certain Google/Firebase routes are in the clear. To configure your Chrome App, a great place to start is the Authentication Boilerplate as a guideline for the following two preliminary steps:
import the Firebase library as an inline script in your HTML template.
update the content_security_policy value in your manifest.json file with the following:
These rules let your Chrome Extension download the Firebase client library and connect to Firebase’s websocket server.
Note that the Firebase Authentication boilerplate has the configuration for protecting your Extension with a login screen. It is a great reference if you are building more than a rapid prototype and are looking into protecting your database with proper ruling.
As a result of the way the library needs to be imported, your Firebase app (as well as the RTD event hooks) will need to be instantiated in the background script. Data changes will alter state housed in the Chrome Storage which is accessible to your background script, the popup render code, and in content scripts. Each operates within its own scope as background scripts run on Extension load, content scripts are JavaScript files that run in the context of web pages, and popup render code runs when the menu is opened. In order to access your Firebase database, you will need to utilize the Chrome Message API to relay changes to the RTD.
Execution
What does this look like in practice? Let’s dig into some code:
// this file will run once on extension loadvarconfig={apiKey:"[insert api key]",authDomain:"[insert auth domain]",databaseURL:"[insert database url]",projectId:"[insert project id]",storageBucket:"[insert storage bucket]",messagingSenderId:"[insert message sender id]"};constapp=firebase.initializeApp(config);constappDb=app.database().ref();// instantiate global application state object for Chrome Storage and feed in firebase data// Chrome Storage will store our global state as a a JSON stringified value.constapplicationState={values:[]};appDb.on('child_added',snapshot=>{applicationState.values.push({id:snapshot.key,value:snapshot.val()});updateState(applicationState);});appDb.on('child_removed',snapshot=>{constchildPosition=getChildIndex(applicationState,snapshot.key)if(childPosition===-1)returnapplicationState.values.splice(childPosition,1);updateState(applicationState);});appDb.on('child_changed',snapshot=>{constchildPosition=getChildIndex(applicationState,snapshot.key)if(childPosition===-1)returnapplicationState.values[childPosition]=snapshot.val();updateState(applicationState);});// updateState is a function that writes the changes to Chrome StoragefunctionupdateState(applicationState){chrome.storage.local.set({state:JSON.stringify(applicationState)});}// getChildIndex will return the matching element in the objectfunctiongetChildIndex(appState,id){returnappState.values.findIndex(element=>element.id==id)}// if your Chrome Extension requires any content scripts that will manipulate data,// add a message listener here to access appDb:chrome.runtime.onMessage.addListener((msg,sender,response)=>{switch(msg.type){case'updateValue':appDb.child(msg.opts.id).set({value:msg.opts.value});response('success');break;default:response('unknown request');break;}});
That’s it! All your extension needs to do is load from the Chrome local storage before rendering itself. If it or a content script needs to manipulate the RTD, the following snippet should be invoked:
12345
chrome.runtime.sendMessage({type:'updateValue',opts:request.opts},(response)=>{if(response=='success'){// implement success action here }});
By following these steps, you should now have a Chrome Extension that will stay in sync with other companion applications! Now go celebrate the fruits of your labor. Happy hacking!
Our mobile team recently built a pair of content-driven iOS and Android apps for a large mixed-media company. Given how rich and extensive their media collection is, the designers provided us hundreds of cell layouts and rules for the media types as the basis for a dynamic and lively browsing experience. This presented us with an interesting architectural and design challenge.
We’ll focus on the iPad app in this post and how a few simple techniques helped us build and test our cells.
The Challenge
The app revolves around several primary types of media content—photos, videos, audio clips, and articles (think: digital magazine). These pieces of content are collected and curated by the backend server, which our app consumes and presents to users as a feed of cells arranged in grids.
Based on several parameters that we’ll be exploring, the cells need to adapt to a wide variety of physical form-factors and variations. We’ll refer to these as dimensions in this post.
Dimension #1: Media Type
A cell can represent one of a various number of media types (photo, video, article, etc…).
Dimension #2: Cell Size
Cells must occupy one of a handful of specific sizes (dimensions denoted in rows × cols notation) as they are laid out on a 3 x 3 grid that spans the screen. Some combinations of r x c are invalid.
Here’s an example of a screen full of cells:
Dimension #3: Cell Display
Cells also have a display property, which specifies a particular design variation of a cell.
For example, a photogallery cell might be rendered in several different ways: with two thumbnails side-by-side, or with 1 large thumbnail on the left and 2 smaller thumbnails on the right, or with 3 thumbnails etc… in all cases, the cells represent the same underlying content.
Most media types support a handful of display variations. However, some, such as a textual quote, may not support any variations.
Dimension #4: Device Orientation
Because iPad users tend to use their devices in both portrait and landscape (moreso than iPhone users), we also need to consider the device orientation before we can render the cell.
Some display types, depending on the cell size they’re embedded in, may undergo further layout changes depending on the device orientation.
In the following animation, notice how the (1 × 1.5) cell morphs into a (3 × 1.5) cell to fill the screen height once rotated in landscape. Because of the significantly altered aspect ratio, it makes sense for the cell to consider the device orientation.
Not all display variations consider the device orientation. The photogallery cell does, but the quote cell doesn’t (simple font scaling and image stretching doesn’t warrant any additional logic).
Dimension #5: Device Size
And finally, the actual device being used can introduce our final bit of variation. The iPad Pro 12.9” is significantly larger than the iPad 9.7”, giving us extra screen real-estate to work with.
Goals
Given the apparent design complexity of our cells, we needed to come up with an efficient process to verify that all our cells are built correctly, ideally as part of our CI and QA process.
With our problem space now defined, we have a few goals in mind.
How can we easily verify that all the cell variations look correct?
How can we reduce the complexity of handling these variations?
Our Solution
We’ll break down the problem a bit and examine our dimensions closely. For any given cell, we have a combination of:
If every combination is possible, this results in 7 × 7 × 8 × 2 × 3 = 2,352 unique cells to build! Some combinations aren’t valid though, so we can reduce the total set down to a few hundred—still a non-trivial problem.
Furthermore, our cells are highly visual in nature. Exact designer-specified proportions, fonts, dimensions, and subview layouts need to be built for our 5 dimensions. UI tests could be written to assert, for each combination, that the title label contains the right text, the background is the correct color, the thumbnail is the correct size, etc… but this would be an exhausting, expensive, and error-prone effort.
Enter: Snapshot Testing!
Using a concept called snapshot testing, we can replace code-based layout tests with visual tests performed by our eyes—or even better—a machine’s “eyes”. 👀
How It Works
The initial execution of a test case creates a screenshot of the view we’re interested in. It’s then saved to disk as a PNG “reference image”. Subsequent executions of the same test case compare a newly generated screenshot with the saved reference image. The test case fails if the images aren’t identical.
(Though this is generally referred to as snapshot testing, screenshot testing might be a more accurate term.)
Within our test suite, we utilized a well-crafted loop to iterate through all combinations of the 5 cell parameters, so that we could quickly and exhaustively generate views for all valid combinations, and utilize iOSSnapshotTestCase to take screenshots of the fully-rendered views.
Note: This exhaustivity is an important piece of the puzzle! Using Sourcery, a Swift code generation tool, we are provided a compiler-guaranteed mechanism for iterating through all enumeration cases, which ensures that any new scenarios we add are automatically tested and accounted for.
At a high-level, our cell tests look like this:
123456789101112131415
import XCTest
import iOSSnapshotTestCase
class CellSnapshotTests: iOSSnapshotTestCase {
func testCells() {
Cell.allCombinations.forEach { media, display, cellSize, orientation, deviceSize in
// 1. Instantiate and render cell view with these 5 dimensions
// 2. Compare rendered view with saved reference image, if in test mode
// or...
// Save rendered view to disk as reference image, if in record mode
// (This loop will iterate a few hundred times. One iteration for each valid combination.)
}
}
}
And here’s a collection of images automatically generated by our snapshot test for the completed photogallery cell:
These snapshot tests help us answer the two main questions we posed earlier.
1. How can we easily verify that all the cell variations look correct?
Quick visual inspection — They allow the developers and designers to quickly see how all combinations of the cells look.
Regression testing — They act as the reference image (or golden file) for testing. Once we have built up a test suite of hundreds of cell snapshots, any code change that results in a visual diff of any snapshot image will be flagged as a failed test, allowing us to quickly identify visual bugs and confirm visual changes.
Code review aid — The reference images are checked into the repository. Therefore additions, removals, and edits of images are immediately visible to reviewers of a pull request. GitHub and other tools even provide visual diffs to highlight exactly what changed in the images, making code review a pleasant experience. It takes, say, 10 seconds to look at a snapshot and comment that the colors and font size are way off, whereas pulling the feature branch, compiling, running the simulator, then poking around the app to find the right cell to comment on might take 5 minutes! Multiply these savings across a large team building hundreds of cells, and we can save weeks of developer time over the course of a project.
2. How can we reduce the complexity of handling these variations?
Shorten development cycles — Setting aside the compilation time (which is already a bit of an issue in Swift), viewing incremental changes during iOS development generally requires a complete simulator refresh. This makes development cycles quite expensive, especially when dozens or hundreds of cells will be updated on the latest build cycle. By creating snapshots, we can visually inspect the cells directly (and in all their hundreds of variants), eliminating the lengthy time needed to actually look for the cells in the simulated app.
Efficient cross-team iteration — Designers could quickly scan through all .png files and pass along feedback to the development team without ever running the app.
Keeping the development team in sync — The snapshots also help the team stay in sync during development. Once a project reaches a certain size, it becomes increasingly difficult for all developers to know each and every feature built by the team. In addition to lowering the barrier for code review for the cells, the snapshots provide a near-frictionless way to see what the rest of the team is building.
Caveats
Snapshot testing, despite all the benefits it afforded us, was not foolproof. There were numerous occassions where views that looked a certain way in the app were rendered sufficiently different in the snapshots to make the test useless. These were warts with the implementation, and not the technique though. As the tooling and frameworks mature and our experience with the technique grows, snapshot testing will become more reliable and is already a worthy addition to any mobile project.
Sourcery, which was critical to enforcing the exhaustivity of our tests, had issues too. Some upstream bugs caused sporadic build failures, which was a source of frustration that we had to work around. The good news is that beginning with Swift 4.2, the main Sourcery feature we rely on is built into the language.
Conclusion
The “secret sauce” behind this post is really just ketchup mixed with mustard. We took two simple concepts—snapshot testing and exhaustively iterable types—to create a delightfully tangy system for managing complexity.
We drastically minimized code review effort, saving precious time and keeping productivity and quality high.
We created a system that helped all teams—developers, designers, and the product team. The snapshots were visible to all and each team could use them in their own ways as necessary.
Achieving this required some diligence by the team, but by being thoughtful and organized with our code and using a few third-party tools, we were able to successfully build an extensible and safe system to manage hundreds of cell layouts and ultimately ship a lively and beautiful product to the App Store! 📸
The Go programming language and tool stack have been designed
to combine manual and automated software development in novel ways,
paving the path for how software will be written in the 21st century.
Humans will be slow-rolling the creative parts
while automated tools take over the repetitive parts
like upgrades, maintenance, and keeping it all working.
The Go makers have laid out their powerful vision for this in
a post on the Go blog
a few years ago.
They also made a tool called gofix
that updates existing Go code bases to new language features of the Go language.
This has helped use early versions of Go in meaningfully large code bases
while remaining able to evolve the language.
Library and framework authors can (and should) do the same for their users.
If your API changes or you introduce new features,
spend a few hours creating a fix that can safely update the millions of places
where your library is (or will be) called in code bases all over the world.
This helps not only your users, but also you, the author.
By making it easier for everybody in the world
to move to the newest version of your library or framework,
you can introduce breaking changes easier
and at the same time have to provide less support for older versions.
The existing go-fix code base is not a fully generic framework for writing custom fixes,
but it is a good start for writing your own fix tool.
Here are some pointers for how to go about that
using the existing state of the art in Go land:
main.go is the “outer framework”.
It parses CLI arguments,
locates source code files or IO streams to convert, parses them into an
AST,
provides that AST to the inner framework,
and serializes the changed AST back into files.
fix.go is the “inner framework”.
It allows to walk the AST, modify its nodes,
and it provides helper functions for typical modifications.
Look at some of the existing fixes in that code base,
for example the printerconfig fix.
The framework you have now allows you to change properties of existing nodes,
so if that is all you need you are good to go.
Replacing a node with a different one is not supported at this time, however.
Fortunately, Robert Griesemer (one of the three foregophers aka inventors of Go),
is working on an improved version
that provides this ability.
It is planned to hopefully become a part of the upcoming Go 1.10 release.
Here is an
easier to copy version
of it.
Copy-and-paste it into your code base as well.
Now you have an Apply function that allows replacing AST nodes.
Write your custom fixes.
Name your fix tool specific to what it does.
The gofix tool fixes Go-specific things.
If yours fixes things for your acme library,
a suitable name for it would be acmefix.
To see all of this together in a working example,
check out the exitfix tool
for Originate’s exit library.
It also contains tests via
examples.
A few weeks ago I was lucky enough to attend CukenFest in London. It was a series of 3 events, BDD Kickstart, CukeUp, and CukenSpace. BDD Kickstart was a two day workshop meant to teach the participants how to build software via Behavior Driven Development and how Cucumber can fit into that. CukeUp was a day of talks centered around BDD with speakers from the community. CukenSpace was an Open Space which was a combination of Cucumber contributors and community members sharing experiences, hacking, and planning the future of Cucumber.
Even as a maintainer of cucumber-js and an experienced user of Cucumber, the BDD kickstart class gave me a number of great takeaways (listed below). To me, cucumber had largely been a
testing tool and a form of living documentation though not always a very pretty form of documentation. BDD however is all about discussion and I learned quite a few things I want to apply on my future projects.
We had a version of CukenSpace last year with just Cucumber contributors, and opening it up to the community gave us some great new viewpoints. We were able to gain a few new contributors, got to hear peoples’ experiences with BDD and cucumber, and gained some valuable outside perspectives.
As with last year, getting some face to face time with other contributors also helps spur contributions and improvements to cucumber. I paired with Julien Biezemans (the original author of cucumber-js) on speeding up the cucumber-js test suite which resulted in the feature tests dropping from 9 minutes to 1 minute on TravisCI by avoiding using child processes whenever possible. I also got some great feedback from Matt Wynne (a core cucumber-ruby contributor) that will hopefully lead to a great improvement in the error output of cucumber-js. Essentially, scenarios that error will be printed in “pretty” format with the error inline.
Takeaways
Tests don’t tell you if your code works, they only tell you when its broken
I really like this way of thinking as tests help us ensure we don’t break existing features when we add new ones. Drawing a parallel to code coverage, I think that is best used to find sections of your codebase that aren’t tested. Those sections can then be deleted or updated with tests.
Use the word “should” in Then steps
I really like this simple rule as on a few projects we have had some back and forth on how to make the “Then” steps different from a related “Given” step. For example: “Given I have a file” and “Then I have a file”. I have accidentally used a “Given” step definition in a “Then” step many times. The use of the word “should” instead of “must” is also meant to help facilitate discussion as requirements can change overtime. The word “should” also parallels expectation libraries (like chai and rspec) which sometimes expose an interface that involves the word “should” (although I have always prefered the expect syntax).
Write your scenarios upwards (write Then, When, Given)
To do this write your “Then” step first. Next, write your “When” step above the “Then” step. Finally write your “Given” step above the “When” step. I like following this order because it focuses on the outcome (Then) and action (When) first and prevents you from getting sidetracked on what at times can be a complex context (Given).
What, not how. (Declarative, not imperative. No user interface in steps.)
I really like this as it focuses you on the business logic of the application. As an example the cucumber-js tests are very verbose and focus mostly on the how. For some apps the “how” may be important to demonstrate, but won’t be needed in every scenario. For example, having all the necessary steps for signing up and being granted admin privileges versus a step that says “Given I am logged in with admin privileges”.
Max 5 steps per scenario
I have not been able to achieve this on previous projects. I want to aim for this in the future though as I think it would be a good exercise to keep your scenarios shorter and easier to understand.
When testing user interfaces, the action (When) and outcome (Then) steps interact with the interface but the context (Given) does not.
This is not something I’ve followed in the past but something I feel would be really powerful. For example, this will hopefully prevent retesting the login flow on every scenario if you can login a user with an API and inject the credentials into the browser.
Follow the test pyramid
The test pyramid suggests having a large amount of unit tests, and a smaller amount of end to end tests. I think Cucumber is best used for the end to end tests as it allows you describe the core business logic in a manner non-technical people can understand. This keeps the test suite fast, as too many feature tests can easily slow down the suite. I’ve also found that having only feature tests can sometimes make the internals harder to refactor as without unit tests it can be hard to see what each part of the system is used for.
The successor to relish (used by rspec) which is being built as the place where all team members can discuss features. It has a simple UI that nicely displays your features and gives you the ability to comment on steps to discuss changes. Test result integration is also planned where results could be uploaded via CI.
Cucumber GUI
Built using electron, it will be a new way to display test rules. Developers can pipe the event protocol formatter to this app to have realtime updates of test results. No more scrolling through output in the terminal. A spike of this was demoed using cucumber-ruby which implements an initial version of the event protocol. The idea is that this tool can be used by all cucumber implementations (once they implement the event protocol).
Implementing several different unrelated code changes
at the same time
in the same feature branch
is like trying to have a conversation
about several completely different topics
at the same time with the same person.
Is never productive.
We end up mixing up issues or
forgetting to think about important edge cases of one topic
because we are distracted with the other topics.
We only have parts of our brain available to think about each topic.
Dealing with several issues at the same time
might create the illusion of multitasking and productivity,
but in the end, it is faster, easier, safer, cleaner, and less error-prone
to take on each topic separately.
This blog post describes a technique for highly focused development
by implementing code changes using a series of nested Git branches.
We use specialized tooling (Git Town)
to make this type of working easy and efficient.
Example
As an example, let’s say we want to implement a new feature for an existing product.
But to make such a complex change,
we have to get the code base ready for it first:
– clean up some technical drift
by improving the names of classes and functions that don’t make sense anymore
– add some flexibility to the architecture
so that the new feature can be built with less hackery
– while looking through the code base,
we also found a few typos we want to fix
Let’s implement these things as a chain of independent but connected feature branches!
The tables below shows
the Git Town commands –
as well as the individual Git commands you would have to run without Git Town –
to achieve that.
First, let’s fix those typos because that’s the easiest change
and there is no reason to keep looking at them.
Fix typos
We create a feature branch named 1-fix-typos to contain the typo fixes
from the master branch:
Git Town command
Git commands
description
git hack 1-fix-typos
git checkout master
we want to build on top of the latest version of master
git pull
git checkout -b 1-fix-typos
We do a few commits fixing typos,
and submit a pull request:
1
git new-pull-request
This opens a browser window to create the pull request on your code hosting service.
All of this only took us 30 seconds.
While the code review for those change starts taking place,
we move on to fix the technical drift.
Rename foo
We don’t want to look at the typos we just fixed again,
so let’s perform any further changes on top of branch 1-fix-typos:
Git Town command
Git commands
git append 2-rename-foo
git checkout -b 2-rename-foo
git append creates a new feature branch
by cutting it from the current branch,
resulting in this branch hierarchy:
12345
master
\ 1-fix-typos
\ 2-rename-foo
Now we commit the changes that rename the foo variable
and start the next pull request:
1
git new-pull-request
Because we use git append to create the new branch,
Git Town knows about the branch hierarchy
and creates a pull request from branch 2-rename-foo against branch 1-fix-typos.
This guarantees that the pull request only shows changes made in branch 2,
but not the changes made in branch 1.
Rename bar
This is a different change than renaming foo,
so let’s do it in a different branch.
Some of these changes might happen in the same places where we just renamed foo.
We don’t want to have to deal with merge conflicts later.
Those are boring and risky at the same time.
So let’s make these changes on top of the changes we made in step 2:
While renaming bar, we stumbled on a few more typos.
Let’s add them to the first branch.
123
git checkout 1-fix-typos
# make the changes and commit them heregit checkout 3-rename-bar
Back on branch 3-rename-bar, the freshly fixed typos are visible again
because the commit to fix them only exists in branch 1-fix-typos right now.
Luckily, Git Town can propagate these changes through all other branches automatically!
Git Town command
Git commands
git sync
git checkout -b 2-rename-foo
git merge 1-fix-typos
git push
git checkout 3-rename-bar
git merge 2-rename-foo
git push
Generalize the infrastructure
Okay, where were we? Right!
With things properly named
it is now easier to make sense of larger changes.
We cut branch 4-generalize-infrastructure
and perform the refactor in it.
It has to be a child branch of 3-rename-bar,
since the improved variable naming done before
will make the larger changes we are about to do now more intuitive.
Git Town command
Git commands
git append 4-generalize-infrastructure
git checkout -b 4-generalize-infrastructure
Lots of coding and committing into this branch to generalize things.
Since that’s all we do here,
and nothing else,
it’s pretty straightforward to get through it, though.
Off goes the code review for those changes.
Shipping the typo fixes
In the meantime, we got the approval for the typo fixes in step 1.
Let’s ship them!
Git Town command
Git commands
description
git ship 1-fix-typos
git stash -u
move things we are currently working on out of the way
git checkout master
update master so that we ship our changes on top of the most current state of the code base
git pull
git checkout 1-fix-typos
git pull
make sure the local machine has all the changes made in the 1-fix-typos branch
git merge master
resolve any merge conflicts between our feature and the latest master now, on the feature branch
git checkout master
git merge –squash 1-fix-typos
use a squash merge to get rid of all temporary commits and merges on the branch
git push
git branch -d 1-fix-typo
delete the branch from the local machine
git push origin :1-fix-typo
delete the branch from the remote repository
git checkout 4-generalize-infrastructure
return to the branch we were working on
git stash pop
restore what we were working on
With branch 1-fix-typos shipped, our branch hierarchy looks like this now:
We have been at it for a while.
Other developers on the team have shipped things too,
and technically the branch 2-rename-foo still points to the previous commit on master.
We don’t want our branches to deviate too much from the master branch,
since that can lead to more severe merge conflicts later.
Let’s get everything in sync!
Git Town command
Git commands
description
git sync
git stash -u
move things we currently work on out of the way
git checkout master
git pull
git checkout 2-rename-foo
git merge master
git push
git checkout 3-rename-bar
git merge 2-rename-foo
git push
git checkout 4-generalize-infrastructure
git merge 3-rename-bar
git push
git stash pop
restore what we were working on
Because we used git append to create the new branches,
Git Town knows which branch is a child of which other branch,
and can do the merges in the right order.
Build the new feature
Back to business.
With the new generalized infrastructure in place,
we can now add the new feature in a clean way.
To build the new feature
on top of the new infrastructure:
Git Town command
Git commands
git append 5-add-feature
git checkout -b 5-add-feature
Let’s stop here.
Hopefully, it is clear how to work more focused using Git Town.
Let’s review:
– each change happens in its own feature branch
– run git append to create a new feature branch
– run git sync to keep all feature branches in sync with the rest of the world –
do this several times a day
– run git ship to ship a branch once it is approved
Advantages of focused feature branches
Working this way has a number of important advantages:
Focused changes are easier and faster to create:
if you change just one thing,
you can do it quickly,
make sure it makes sense,
and move on to the next issue in another branch.
No more getting stuck not knowing which of the many changes
you did in the last 10 minutes broke the build,
and no need to fire up the debugger to resolve this mess.
They are easier and faster to review:
The PR can have a simple description to summarize it.
Reviewers can easily wrap their heads around what the PR is doing
and make sure the changes are correct and complete.
This is also true if you write code just by yourself.
You should give it a final review and cleanup before merging it into master!
Branches containing focused changes cause less merge conflicts
than branches with many changes in them.
They make fewer changes,
and the changes are typically more distributed across lines.
This gives Git more opportunity to resolve merge issues automatically.
In case you have to resolve merge conflicts manually,
they are also easier and safer to resolve.
You know exactly what single change both branches perform,
versus what five changes each of the branches perform in the unfocused scenario.
Others can start reviewing parts of your changes sooner
because you start submitting pull requests earlier.
Ultimately, using this technique you will get more work done faster.
You have more fun
because there is a lot less getting stuck, spinning wheels, and starting over.
Working this way requires running a lot more Git commands,
but with Git Town this is a complete non-issue
since it automates this repetition for you.
Best Practices
To fully leverage this technique, all you have to do is follow a few simple rules:
postpone ideas:
when you work on something and run across an idea for another change,
resist the urge to do it right away.
Instead, write down the change you want to do
(I use a sheet of paper or a simple text file),
finish the change you are working on right now,
and then perform the new change in a new branch a few minutes later.
If you can’t wait at all,
commit your open changes,
create the next branch,
perform the new changes there,
then return to the previous branch and finish the work there.
If you somehow ended up doing several changes at once in a branch anyway,
you can still
refactor your Git branches
so that they end up containing only one change.
go with one chain of branches:
When in doubt whether changes depend on previous changes,
and might or might not cause merge conflicts later,
just work in child branches.
It has almost no side effects,
except that you have to ship the ancestor branches first.
If your branches are focused,
you will get very fast reviews, be able to ship them quickly,
and they won’t accumulate.
do large refactorings first:
In our example we did the refactor relatively late in the chain
because it wasn’t that substantial.
Large refactorings that touch a lot of files have the biggest potential
for merge conflicts with changes from other people, though.
You don’t want them hanging around for too long,
but get them shipped as quickly as you can.
You can use
git prepend
to insert feature branches
before the currently checked out feature branch.
If you already have a long chain of unreviewed feature branches,
try to insert the large refactor into the beginning of your chain,
so that it can be reviewed and shipped as quickly as possible:
At Originate we use CircleCI for all of our Continous
Integration & Delivery needs. We’ve recently started to use DC/OS to deploy some of our
software. This blog post will walk through the steps required to deploy a simple
Marathon web service to DC/OS from CircleCI.
We’ll start with a very basic circle.yml and build it up as we go.
machine:environment:PATH:$HOME/.cache/bin:$PATH# <- Add our cached binaries to the $PATHdependencies:override:-scripts/ci/install-dcos-cli# <- Install the DC/OS CLIcache_directories:-"~/.cache"# <- Cache binaries between builds to speed things uptest:override:-echo "You'd normally run your tests here"
#!/usr/bin/env bashset -euo pipefail
if[[ ! -z ${VERBOSE+x}]]; thenset -x
fiBINS="$HOME/.cache/bin"DCOS_CLI="$BINS/dcos"DCOS_CLI_VERSION="0.4.16"# Returns the version of the currently installed DC/OS CLI binary. e.g 0.4.16installed_version(){ dcos --version | grep 'dcoscli.version' | cut -d= -f2
}# Downloads the DC/OS CLI binary for linux with version $DCOS_CLI_VERSION to the cacheinstall_dcos_cli(){ mkdir -p "$BINS" curl -sSL "https://downloads.dcos.io/binaries/cli/linux/x86-64/$DCOS_CLI_VERSION/dcos"\ -o "$DCOS_CLI" chmod u+x "$DCOS_CLI"}# Install the DC/OS CLI if it's missing. If it's present, upgrade it if needed otherwise do nothingif[ ! -e "$DCOS_CLI"]; thenecho"DC/OS CLI not found. Installing" install_dcos_cli
elseINSTALLED_VERSION="$(installed_version)"if["$DCOS_CLI_VERSION" !="$INSTALLED_VERSION"]; thenecho"DC/OS CLI has version $INSTALLED_VERSION, want $DCOS_CLI_VERSION. Upgrading" rm -rf "$DCOS_CLI" install_dcos_cli
elseecho"Using cached DC/OS CLI $INSTALLED_VERSION"fifi
Now let’s setup a basic Marathon service. We’ll use a stock nginx image as a stand in for a web service. Our
service will have one instance with 1 CPU and 512MB of RAM and will map a random port on the host to port 80
in the container.
The next step is to add a small script to send the manifest to DC/OS using the CLI tool. The Marathon API has
separate endpoints for creating and updating a service so we have to check if a service already exists before
doing the right thing. We’re using --force when updating to override any previous, potentially faulty
deployment.
#!/usr/bin/env bashset -euo pipefail
if[[ ! -z ${VERBOSE+x}]]; thenset -x
fiDIR="$( cd "$( dirname "${BASH_SOURCE[0]}")" && pwd )"SERVICE="$1"SERVICE_MANIFEST="$SERVICE/marathon.json"# Marathon automatically prefixes service names with a /. We use the slash-less version in the manifest.SERVICE_ID="/$(jq -r '.id' "$SERVICE_MANIFEST")"# This returns true if the service currently exists in Marathon (e.g. needs to be updated instead of created)service_exists(){local service_id="$1" dcos marathon app list --json | jq -r '.[].id' | grep -Eq "^$service_id$"}if service_exists "$SERVICE_ID"; thendcos marathon app update "$SERVICE_ID" < "$SERVICE_MANIFEST" --force
elsedcos marathon app add < "$SERVICE_MANIFEST"fi
We can now add the deployment section to the circle.yml file.
machine:environment:PATH:$HOME/.cache/bin:$PATHdependencies:override:-scripts/ci/install-dcos-clicache_directories:-"~/.cache"test:override:-echo "You'd normally run your tests here"deployment:# <- Add the deployment sectionmaster:branch:mastercommands:-scripts/ci/marathon-deploy services/hello-world
We’re almost there, the last thing we have to resolve is how to authenticate the DC/OS CLI to the cluster.
Without that we can’t call the APIs we need to deploy our service.
DC/OS Community Edition uses an OAuth authentication flow backed by Auth0. This
is used for both the browser-based authentication to get access to the admin dashboard as well as for the CLI
tool. In the latter case, the user has to follow a slightly different browser-based flow yielding a token that
is then provided to the CLI.
In a CI/CD setting, anything that requires a manual user intervention is a non-starter. Enter
dcos-login, the tool that we’ve created to solve this problem.
Given a set of Github credentials it will replace the human component in the login flow and let you run the
DC/OS CLI in your CI environment.
We recommend creating a separate “service” Github account just for that purpose. Once that’s done you can set
GH_USERNAME and GH_PASSWORD environment variables in CircleCI to the username and password for that account.
Just like the DC/OS CLI, we need to pull down dcos-login to our build container.
#!/usr/bin/env bashset -euo pipefail
if[[ ! -z ${VERBOSE+x}]]; thenset -x
fiBINS="$HOME/.cache/bin"DCOS_LOGIN="$BINS/dcos-login"# Returns the version of the currently installed dcos-login binary. e.g v0.24installed_version(){ dcos-login --version 2>&1
}# Returns the version of the latest release of the dcos-login binary. e.g v0.24latest_version(){ curl -sSL https://api.github.com/repos/Originate/dcos-login/releases/latest | jq -r '.name'}# Downloads the latest version of the dcos-login binary for linux to the cacheinstall_dcos_login(){ mkdir -p "$BINS"LATEST_RELEASE="$(curl -sSL https://api.github.com/repos/Originate/dcos-login/releases/latest)"DOWNLOAD_URL=$(jq -r '.assets[] | select(.name == "dcos-login_linux_amd64") | .url'<<<"$LATEST_RELEASE") curl -sSL -H 'Accept: application/octet-stream'"$DOWNLOAD_URL" -o "$DCOS_LOGIN" chmod u+x "$DCOS_LOGIN"}# Install dcos-login if it's missing. If it's present, upgrade it if needed otherwise do nothingif[ ! -e "$DCOS_LOGIN"]; thenecho"dcos-login not found. Installing" install_dcos_login
elseINSTALLED_VERSION="$(installed_version)"LATEST_VERSION="$(latest_version)"if["$LATEST_VERSION" !="$INSTALLED_VERSION"]; thenecho"dcos-login has version $INSTALLED_VERSION, latest is $LATEST_VERSION. Upgrading" rm -rf "$DCOS_LOGIN" install_dcos_login
elseecho"Using cached dcos-login $INSTALLED_VERSION"fifi
Next we need to use dcos-login to authenticate the DC/OS CLI. You’ll also need to provide the URL to your
DC/OS cluster in the CLUSTER_URL environment variable.
#!/usr/bin/env bashset -euo pipefail
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}")" && pwd )"# Check that all required variables are setfor NAME in GH_USERNAME GH_PASSWORD; doeval VALUE=\$$NAMEif[[ -z ${VALUE+x}]]; thenecho"$NAME is not set, moving on"exit 0
fidoneif[[ ! -z ${VERBOSE+x}]]; thenset -x
fiCLUSTER_URL="$1"# Setup the DC/OS CLI## Point it to the DC/OS cluster. CLUSTER_URL is the URL of the admin dashboarddcos config set core.dcos_url "$CLUSTER_URL"## Set the ACS token. dcos-login reads GH_USERNAME and GH_PASSWORD from the environment automaticallydcos config set core.dcos_acs_token "$(dcos-login --cluster-url "$CLUSTER_URL")"
Finally, tie it all together in the circle.yml file.
machine:environment:PATH:$HOME/.cache/bin:$PATHdependencies:override:-scripts/ci/install-dcos-cli-scripts/ci/install-dcos-login-cli# <- Install the dcos-login toolcache_directories:-"~/.cache"test:override:-echo "You'd normally run your tests here"deployment:master:branch:mastercommands:-scripts/ci/login "$CLUSTER_URL"# <- Authenticate the DC/OS CLI-scripts/ci/marathon-deploy services/hello-world
That’s it, you can now deploy your services from CircleCI to DC/OS Community Edition. The
dcos-login tool is free and open source. All the code in this blog
post can be found in this example project.
FAQ
What about private docker registries?
Marathon lets you pull docker images from private registries with a bit of configuration. You need to tar up
the .docker directory of an authenticated host and instruct Marathon to pull down that archive when launching
a new instance of a service:
How do I pass configuration / secrets to my service?
Couple of strategies here:
Set the environment variable in CircleCI and then use sed or envsubst to replace placeholders in your
marathon.json.
Store a subset of your marathon.json with your configuration / secrets on S3 or similar. Pull down that
file (which might be specific per environment) at deploy time and use jq
to merge it into the main marathon.json manifest for your service. You can use something like
jq -s '.[0] * .[1]' marathon.json secrets.json.
How do I assign a DNS name to my service?
If you’re using Marathon LB you can add the following section to your marathon.json:
The HAPROXY_0_VHOST value instructs Marathon LB to map the first port in your port mapping (index 0,
HTTP in our case) to that virtual host. You should have an entry in your DNS zone pointing *.yourdomain.com
to the public DC/OS nodes running Marathon LB
How do I make sure that my service is alive?
You can instruct Marathon to perform a health check on your behalf by adding the following to your
marathon.json:
This is an opinion piece by one of our employees,
and not an official statement of Originate.
At Originate we celebrate diversity of opinions around technology
as one of our greatest strengths,
and we encourage our employees to share their ideas on our blog.
These are amazing times to be a software developer.
We have access to a vast and multi-faceted ecosystem
of very well thought out programming languages
crafted by masters.
More are added regularly,
and they are typically free to use and adapt.
As somebody who looks at a lot of source code every day,
needing to evaluate its essence, architecture, and structure quickly,
I have learned that less is often more when it comes to syntax.
In my opinion, this is especially true for
infrastructure elements like
paretheses, curly braces, or semicolons.
This blog post is an attempt at comparing their costs and benefits somewhat objectively.
Before we dive into this, let’s remind ourselves that
the #1 reason for using a programming language
is enjoying using it and getting stuff done.
Shipping things.
Most languages achieve that,
truly awesome software has been written in many different stacks,
and that’s a very good thing.
With that said,
if you are into a bit of recreational bikeshedding about programming language syntax,
and you don’t just want to stick to what you and others are used to,
read on!
Upfront some simple rules that I think we all can agree on,
which we will use to judge the different approaches:
simpler is better:
if there are two alternatives that provide more or less the same functionality,
and one is simpler, easier, or smaller,
then that is the preferred alternative
robustness: the alternative that provides less rope for people to hang themselves with,
i.e. less possibilities to mess up wins
the signal-to-noise ratio of code should be high.
This means most of the source code should describe business functionality.
Boilerplate (code that is just there to make the machine work)
should be minimal.
Said another way,
code should express developer intent as directly as possible,
without too much infrastructure around it.
With that in mind,
let’s look at a few common syntax elements
and see if we can bring some clarity into the debates around them.
Maybe we can even come up with an idea for
how an ideal programming language syntax looks like,
if there is such a thing.
Or maybe it’s all a wash.
To keep things easy, the examples are focussed around small and simple languages
like JavaScript/CoffeeScript, Go, Python, Java, etc.
Let’s dig in!
Curly braces
Curly braces replaced the even older begin and end statements in Algol, Pascal, and friends.
Which was a big step forward since it saves a good amount of typing without losing expressiveness.
The question is, are curly braces the final step,
or can we optimize things even more here?
Let’s look at some code.
What does this snippet do:
Did you spot the typo?
Hard to read, right?
That’s because this code solely relies on delimiters
like curly braces, parentheses, and semicolons
to describe code structure.
Let’s format this more human-friendly
by writing each statement on its own line
and adding identation:
So much more readable! And it’s much easier to see that a closing curly brace is missing at the end.
This experiment demonstrates
that people use indentation as the primary mechanism to infer code structure,
and use curly braces only as the backup,
in edge cases like when the indentation is messed up and not obvious.
Braces also introduce extra categories of errors.
Forgetting to close a curly brace will cause a program to behave in unexpected ways,
even though the actual statements in it (the code that computes arguments into results) is sound.
Indenting code correctly but misplacing curly braces
results in code that does something else
than what most programmers would expect when only briefly looking at it.
Code with curly braces still has to be indented properly.
When we do that, this code uses two different ways of expressing code structure:
indentation and curly braces.
Humans mostly look at the indentation, and “filter out” the curly braces.
Parsers look only at the braces and ignore indentation.
Because both of these stakeholders should always agree 100% on what the code is supposed to do,
these two ways of expressing code structure must always be in perfect sync with each other.
As an example, this code here compiles and runs fine,
but is misleading and hard to maintain
since it is improperly formatted, i.e. the indentation is wrong:
So if indentation is the most important aspect for humans,
humans filter out curly braces most of the time,
and any time indentation differs from code structure we end up with misleading code,
what would happen if we got rid of curly braces at all and only used indentation?
This is still clear and unambiguously structured code,
understandable by both humans and parsers.
It is also horizontally and vertically more compact, i.e. uses less lines.
It is more clear,
and it avoids a number of issues like forgotten opening or closing curly braces,
or whether the opening curly brace should be on a separate line or not.
Because indentation errors are parse errors for white space sensitive languages,
we can rest assured the indentation is always correct,
and we don’t need formatters or linters to correct it for us,
which people might or might not run at all times.
At best, when used correctly, curly braces are just there, don’t add much readability,
and get filtered out by humans,
since readability is mostly provided by indentation.
At worst, when curly braces don’t match indentation,
human developers can be seriously mislead.
While proper indentation is a necessity, as we have seen above,
curly braces are redundant and unnecessary.
That’s why we call them line noise.
They are just more stuff to type,
more stuff to verify,
and exist mostly to satisfy our habits at this point.
They are a legacy,
and according to our rules above
we are better off simply leaving them out moving forward.
Semicolon to terminate lines
What is the difference between these two code blocks?
Nothing. Nobody needs semicolons at the end of lines.
Nobody misses them if they aren’t there.
Their only use is to separate multiple expressions on the same line.
So they should be optional.
Parentheses around expressions
Next, what about parentheses around expressions?
Let’s remove them from our example code block and see what happens:
123456
functionmaxa,blogaifa>=breturnaelsereturnb
It’s still clear that a and b are parameters to the function max,
that we log a on the next line,
and then check if a is larger or equal to b.
Similar to curly braces,
parentheses are a redundant way of expressing code structure
and developer intent.
The real world isn’t always as simple as this example,
so more complex code can definitely benefit from parentheses to group sub-expressions.
But there seems no need to enforce them being there for simple cases.
Let’s make them optional,
so that we can simplify our code where possible,
without giving up the ability to structure more complex expressions unambiguously.
Comments
The most widely used ways to add comments to source code are via //, /* */, and #.
Let’s look at C-style comments first:
12345678910
// a single-line comment/* a * multi-line * comment *//** * a JavaDoc or JsDoc comment */
Now let’s look at comments via a single character:
Both code snippets do the same, and are equally well readable.
The first version uses 19 comment characters
and requires indentation of subsequent lines in multi-line comments via a space
(which I count as characters here, since they need to be typed and be verified as well).
The second version only uses 7 comment characters,
without any need for indentation,
and results in less lines of code.
According to our rules the second version wins.
Spaces vs Tabs
The arguments for using tabs to indent code are:
because it saves you characters (1 tab vs 2 or 4 spaces), making the source code file smaller
because it allows different people to indent their code in different amounts,
by configuring the tab size of their text editors
it avoids bikeshed debates how deep code should be indented
The first argument comes from limitations of computer systems 60 years ago
and is no longer valid.
The arguments against tabs are:
the default tab size (8 spaces) is clearly too much.
This means EVERY PERSON in the world who looks at code
now has to configure EVERY TOOL and EVERY WEBSITE they use to write or view code
to their preferred tab size.
On EVERY MACHINE they use.
there are tools that don’t allow to configure the tab size,
for example many websites with embedded code snippets
or many low-level terminal commands.
These things that are often used to dispay source code.
the tab character is hard to insert into input fields,
for example in search-and-replace forms,
since it is also used to switch to the next input field.
People often have to copy-and-paste a tab characters from their source code
in order to search for it.
not all keyboards have a TAB key.
For example most keyboards on mobile devices are lacking it.
Mobile devices play a larger and larger role in our lives,
including in software development.
I review a good amount of code on my mobile device,
and code reviewers sometimes need to provide code snippets as examples.
the tab character was not invented to indent source code.
It exists to make it easier to format numerical content into columns and tables
using tab stops.
There are no columns, tables, or tab stops in source code.
Formatting using tabs doesn’t support many ways of formatting code in readable ways.
Let’s look at a few examples around the last point.
One is formatting function calls with larger lists of complex parameters:
This code draws a circle and calculates the parameters for it inline.
Because the arguments are so complex,
we want to put each one on a separte line to separate them visually.
Putting them behind the function name
makes it visually clear that they are parameters for this function.
The equivalent code using tabs needs to move all arguments to the next line:
The problem with this way of formatting is that the arguments to draw_circle
now look too much like an indented code block.
This is confusing, especially if there would also be an indented code block right below it.
Another – quite common – use case where tab-based formatting falls short is method chaining:
With spaces the code can be formatted nice and clear:
This makes it clear that we do a bunch of things with a Square instance.
With tabs,
we again have to move the call chain to the next line,
making it look too much like a nested code block:
Based on these examples, we can conclude that using tabs to indent source code
may sound good in theory,
but only works for simple code in environments where only few tools are used.
Let’s also evaluate the pros and cons of using spaces. Pros:
code looks exactly the same in all tools and environments
more flexibility around expressing complex code structures elegantly
works with all devices and keyboards
Cons:
opens up bikeshed debates about how many spaces to use to indent code
opens up bikeshed debates around what “elegant” code formatting looks like
formatting code with tabs feels a bit more semantic in nature,
while using spaces feels more presentational
To finish this,
let’s talk about how many spaces should be used to indent code.
The most common candidates are 2 and 4,
since we can (hopefully) agree that 8 is too large
and 1 is not enough.
Given that many communities like to limit the width of code
(to something like 80 or 100 characters),
2 seems preferrable since it leaves more horizontal space for code.
So the question is, are 2 spaces enough indentation?
Let’s look at our initial example to find out:
123456
functionmaxa,bloga,bifa>=breturnaelsereturnb
vs
123456
functionmaxa,bloga,bifa>=breturnaelsereturnb
Both styles work, 2 spaces uses less horizontal space, so it wins.
Final thoughts
Following our rules has led us on a journey into minimal syntax,
i.e. syntax that minimizes line noise and tries to get out of the way
to let the business logic and developer intent shine through better.
This is something that most of us deal with every day,
but often don’t spend much time thinking about.
Many modern languages (i.e. those designed in the last two decades)
are adopting many of these elements in some form,
with certainly more to come in the future.
Hopefully this was an enjoyable read,
and some food for thought
when you decide which language to learn next,
or design your own language or data format.
No matter what language you end up using,
happy coding! :)
…but afterwards the nightmares got worse, since now they included clowns and burning pandas. I stayed up for days.
The goal of this post is to help you become more familiar with variance so that it is easier to understand in code
you are reading, and so that you can use it when appropriate. The goal is not to promote the usage of variance (or overusage).
Variance is a tool that can be powerful, but keep in mind that most parameterized types you define will be invariant.
Note that if you already know what variance is, and you just want a quick reference to remind you how to tell the
difference between co/contravariance, refer to the cheatsheet at the bottom of this post
Why do we even need variance?
Variance is how we determine if instances of parameterized types are subtypes or
supertypes of one another. In a statically typed environment with subtyping and generics, this boils down to
the compiler needing to determine when one type can be substituted for another type in an expression.
For simple types, this is straightforward:
A Dog is an Animal, and so is a Cat, so anything that expects an Animal can also take a Dog or Cat. This is a
classic example of the Liskov substitution principle.
What are we really doing here though? We’re trying to determine when some type T can safely be substituted for a type U
in an expression that expects a U. The expression that expects a U only cares that it gets a U for 2 reasons:
The expression calls methods on a U
The expression passes a U to another expression that expects a U
An expression that passes a U along eventually makes its way to an expression that does call methods on a U (or stores it
for later method calling). So we’re left with only caring about whether something can be substited for another thing,
because we want to make sure that it is safe to call methods on that thing (and by “safe” I mean “will never fail at
runtime”). This is what we want our compiler to do: make sure that we never call the method something.sound on a
type that does not have the method sound defined.
A wild variant appears
Looking at a type that has parameters, it is no longer obvious when substitution within an expression is allowed. In other
words, if a function takes an argument of type ParametricType[T], is it safe to pass it a ParametricType[U]? This is
what variance is all about.
Covariance
Container types are the best example of something that is covariant, so let’s look at an example:
Our method whatSoundsDoTheyMake expects a Seq[Animal], and it calls the method .sound on those animals. We
know that all Animals have the method .sound defined on them, and we know that we are mapping over a list of
Animals, so it’s totally OK to pass whatSoundsDoTheyMake a Seq[Dog] or a Seq[Cat].
Dog <: AnimalimpliesSeq[Dog] <: Seq[Animal]
Notice where the method call on the animals actually happens. It doesn’t happen within the definition of Seq.
Rather, it happens inside of a function that receives the Animal as an argument.
Now consider what would happen if we tried to pass a Seq[Life] to whatSoundsDoTheyMake. First off, the compiler
wouldn’t allow this because it’s unsafe: error: value sound is not a member of Life. If it were allowed though,
then you could attempt to call bacterium.sound, even though the method doesn’t exist on that object. Note that in
a dynamically typed language you could try to do this, but you’d get a runtime exception like
TypeError: Object #<Bacterium> has no method 'sound'.
Interestingly, the real problem doesn’t occur within Seq; it occurs later on down the chain. The reason is that a generic
type makes guarantees to other types and functions that it interacts with. Declaring a class as covariant on type T is
equivalent to saying “if you call functions on me, and I provide you with an instance of my generic type T, you can be damn sure
that every method you expect will be there”. When that guarantee goes away, all hell breaks loose.
Contravariance
Functions are the best example of contravariance (note that they’re only contravariant
on their arguments, and they’re actually covariant on their result). For example:
Should we be able to pass weinerosity as an argument to isDogCuteEnough? The answer is no,
because the function isDogCuteEnough only guarantees that it can pass, at most specific, a Dog to the function f.
When the function f expects something more specific than what isDogCuteEnough can provide, it could attempt
to call a method that some Dogs don’t have (like .weinerness on a Greyhound, which is insane).
What about soundCuteness, can we pass that to isDogCuteEnough? In this case, the answer is yes, because even if
isDogCuteEnough passes a Dog to soundCuteness, soundCuteness takes an Animal, so it can only call
methods that all Dogs are guaranteed to have.
Dog <: AnimalimpliesFunction1[Animal, Double] <: Function1[Dog, Double]
A function that takes something less
specific as an argument can be substituted in an expression that expects a function that takes a more specific argument.
Conclusion
Enforcing safety by following expression substitution rules for parameterized types is a complex but super useful tool.
It constrains what we can do, but these are things that we shouldn’t do, because they can fail at runtime. Variance
rules, and type safety in general, can be seen as a set of restrictions that force us to engineer solutions that
are more robust and logically sound. It’s like how bones and muscles are a set of constraints that allow for
extremely complex and functional motion. You’ll never find a boneless creature that can move like this:
Cheatsheet
Here is how to determine if your type ParametricType[T] can/cannot be covariant/contravariant:
A type can be covariant when it does not call methods on the type that it is generic over. If the type
needs to call methods on generic objects that are passed into it , it cannot be covariant.
Archetypal examples: Seq[+A], Option[+A], Future[+T]
A type can be contravariant when it does call methods on the type that it is generic over. If the type
needs to return values of the type it is generic over, it cannot be contravariant.
Archetypal examples: Function1[-T1, +R], CanBuildFrom[-From, -Elem, +To], OutputChannel[-Msg]
Rest assured, the compiler will inform you when you break these rules:
1234567891011
traitT[+A]{defconsumeA(a:A)=???}// error: covariant type A occurs in contravariant position// in type A of value a// trait T[+A] { def consumeA(a: A) = ??? }// ^traitT[-A]{defprovideA:A=???}// error: contravariant type A occurs in covariant position in// type => A of method provided// trait T[-A] { def provideA: A = ??? }// ^
Over the years, “Callback Hell” has been cited as one of the most common anti-patterns in Javascript to manage concurrency. Just in case you’ve forgotten what that looks like, here is an example of verifying and processing a transaction in Express:
1234567891011121314
app.post("/purchase",(req,res)=>{user.findOne(req.body,(err,userData)=>{if(err)returnhandleError(err);permissions.findAll(userData,(err2,permissions)=>{if(err2)returnhandleError(err2);if(isAllowed(permissions)){transaction.process(userData,(err3,confirmNum)=>{if(err3)returnhandleError(err3);res.send("Your purchase was successful!");});}});});});
Promises were supposed to save us…
I was told that promises would allow us Javascript developers to write asynchronous code as if it were synchronous by wrapping our async functions in a special object. In order to access the value of the Promise, we call either .then or .catch on the Promise object. But what happens when we try to refactor the above example using Promises?
12345678910111213
// all asynchronous methods have been promisifiedapp.post("/purchase",(req,res)=>{user.findOneAsync(req.body).then(userData=>permissions.findAllAsync(userData)).then(permissions=>{if(isAllowed(permissions)){returntransaction.processAsync(userData);// userData is not defined! It's not in the proper scope!}}).then(confirmNum=>res.send("Your purchase was successful!")).catch(err=>handleError(err))});
Since each function inside of the callback has its own scope, we cannot access the user object inside of the second .then callback. So after a little digging, I couldn’t find an elegant solution, but I did find a frustrating one:
Just indent your promises so that they have proper scoping.
Indent my promises!? So its back to the Pyramid of Doom now?
12345678910111213
app.post("/purchase",(req,res)=>{user.findOneAsync(req.body).then(userData=>{returnpermissions.findAllAsync(userData).then(permissions=>{if(isAllowed(permissions)){returntransaction.processAsync(userData);}});}).then(confirmNum=>res.send("Your purchase was successful!")).catch(err=>handleError(err))});
I would argue that the nested callback version looks cleaner and is easier to reason about than the nested promise version.
Async Await Will Save Us!
The async and await keywords will finally allow us to write our javascript code as though it is synchronous. Here is code written using those keywords coming in ES7:
12345678
app.post("/purchase",asyncfunction(req,res){constuserData=awaituser.findOneAsync(req.body);constpermissions=awaitpermissions.findAllAsync(userData);if(isAllowed(permissions)){constconfirmNum=awaittransaction.processAsync(userData);res.send("Your purchase was successful!")}});
Unfortunately the majority of ES7 features including async/await have not been implemented in Javascript runtimes and therefore, require the use of a transpiler. However, you can write code that looks exactly like the code above using ES6 features that have been implemented in most modern browsers as well as Node version 4+.
The Dynamic Duo: Generators and Coroutines
Generators are a great metaprogramming tool. They can be used for things like lazy evaluation, iterating over memory intensive data sets and on demand data processing from multiple data sources using a library like RxJs. However, we wouldn’t want to use generators alone in production code because it forces us to reason about a process over time and each time we call next, we jump back to our generator like a GOTO statement. Coroutines understand this and remedy this situation by wrapping a generator and abstracting away all of the complexity.
The ES6 version using Coroutines
Coroutines allow us to use yield to execute our asynchronous functions line by line, making our code look synchronous. It’s important to note that I am using the Co library. Co’s coroutine will execute the generator immediately where as Bluebird’s coroutine will return a function that you must invoke to run the generator.
1234567891011
importcofrom'co';app.post("/purchase",(req,res)=>{co(function*(){constperson=yielduser.findOneAsync(req.body);constpermissions=yieldpermissions.findAllAsync(person);if(isAllowed(permissions)){constconfirmNum=yieldtransaction.processAsync(user);res.send("Your transaction was successful!")}}).catch(err=>handleError(err))});
If there is an error at any step in the generator, the coroutine will stop execution and return a rejected promise. Let’s establish some basic rules to using coroutines:
Any function to the right of a yield must return a Promise.
If you want to execute your code now, use co.
If you want to execute your code later, use co.wrap.
Make sure to chain a .catch at the end of your coroutine to handle errors. Otherwise, you should wrap your code in a try/catch block.
Bluebird’s Promise.coroutine is the equivalent to Co’s co.wrap and not co on it’s own.
What if I want to run multiple processes concurrently?
You can either use objects or arrays with the yield keyword and then destructure the result.
With the Co library:
123456789101112131415161718
importcofrom'co';// with objectsco(function*(){const{user1,user2,user3}=yield{user1:user.findOneAsync({name:"Will"}),user2:user.findOneAsync({name:"Adam"}),user3:user.findOneAsync({name:"Ben"})};).catch(err=>handleError(err))// with arraysco(function*(){const[user1,user2,user3]=yield[user.findOneAsync({name:"Will"}),user.findOneAsync({name:"Adam"}),user.findOneAsync({name:"Ben"})];).catch(err=>handleError(err))
With the Bluebird library:
1234567891011121314151617181920
// with the Bluebird libraryimport{props,all,coroutine}from'bluebird';// with objectscoroutine(function*(){const{user1,user2,user3}=yieldprops({user1:user.findOneAsync({name:"Will"}),user2:user.findOneAsync({name:"Adam"}),user3:user.findOneAsync({name:"Ben"})});)().catch(err=>handleError(err))// with arrayscoroutine(function*(){const[user1,user2,user3]=yieldall([user.findOneAsync({name:"Will"}),user.findOneAsync({name:"Adam"}),user.findOneAsync({name:"Ben"})]);)().catch(err=>handleError(err))
We are excited to release Git Town today!
Git Town is an open source command-line tool that helps keep software development productive as project and team sizes scale.
It provides a few additional high-level Git commands.
Each command implements a typical step in common software development workflows.
Check out the screencast to get an overview of what these extra Git commands are and how to use them.
The problem: Software development doesn’t scale well
Many software team runs into a few typical scalability issues:
More developers means more frequent commits to the main branch.
This makes feature branches outdate faster,
which results in bigger, uglier merge conflicts when
finally getting the LGTM and merging these feature branches
into the main branch.
Time spent resolving merge conflicts isn’t spent writing new features.
These merge conflicts increase the
likelihood of breaking the main development branch. A broken main branch affects
the productivity of the whole team.
Old branches are often not deleted and accumulate, making it harder to maintain an overview of
where current development is happening, and where unmerged changes are.
These are only a few of the many headaches that diminish developer productivity and
happiness as development teams grow.
These issues are almost completely independent of the particular workflows and how/where the code is hosted.
The approach: Developer discipline
Fortunately, most of these issues can be addressed with more discipline:
Always update your local Git repo before cutting a new feature branch.
Synchronize all your feature branches several times per day with
ongoing development from the rest of the team. This keeps merge conflicts small and easily
resolvable.
Before merging a finished feature branch into the main development branch,
update the main development branch and merge it into your feature
branch.
Doing so allows you to resolve any merge conflicts on the feature branch, and tests things before merging into the main branch.
This keeps the main branch green.
Always remove all Git branches once you are done with them, from
both your local machine as well as the shared team repository.
The solution: A tool that goes the extra mile for you
It is difficult to follow these practices consistently, because Git is an intentionally
generic and low-level tool,
designed to support many different ways of using it equally well.
Git is a really wonderful foundation for robust and flexible
source code management, but it does not provide direct high-level support for collaborative software development workflows.
Using this low-level tool for
high-level development workflows will therefore always be cumbersome and inefficient.
For example:
updating a feature branch with the latest changes on the main
branch requires
up to 10 Git commands,
even if there are no merge conflicts
properly merging a finished feature branch into the main development branch after
getting the LGTM in the middle of working on something else
requires up to
15 Git commands.
Keeping
feature branches small and focused
means more feature branches.
Running all these commands on each feature branch every day easily leads to each developer
having to run a ceremony of hundreds of Git commands each day!
While there are a number of tools like Git Flow
that focus on supporting a particular Git branching model,
there is currently no natural extension of the Git philosophy
towards generic, robust, high-level teamwork support.
We are excited to release exactly that today:
Git Town!
It provides a number of additional high-level Git commands.
Each command implements a typical step in most common team-based software development workflows (creating, synching, and shipping branches).
Designed to be a natural extension to Git, Git Town feels as generic and powerful as Git,
and supports many different ways of using it equally well.
The screencast gives an overview of the different commands, and our tutorial
a broader usage scenario.
The awesomeness: Made for everybody, as Open Source
Git Town is for beginners and experts alike.
If you are new to Git, and just want it to stay out of your way
and manage your code, let Git Town provide the Git expertise and do the legwork for you.
If, on the other hand, you are a Git ninja, and want to
use it in the most effective manner possible,
let Git Town automate the repetitive parts of what you would type over and over,
with no impact on your conventions, workflow, and ability to do things manually.
Git Town is open source, runs everywhere Git runs (it’s written in Bash),
is configurable,
robust, well documented,
well tested,
has proven itself on everything from small open source projects to large enterprise code bases here at Originate,
and has an active and friendly developer community.
Please try it out, check out the screencast or the tutorial, let us know how we can improve it,
tell your friends and coworkers about it, or help us build it by sending a pull request!
FAQ
Does this force me into any conventions for my branches or commits?
Not at all. Git Town doesn’t require or enforce any particular naming convention or branch setup, and works with a wide variety of Git branching models and workflows.
Which Git branching models are supported by Git Town?
Git Town is so generic that it supports all the branching models that we are aware of, for example GitHub Flow, GitLab Flow, Git Flow, and even committing straight into the master branch.
How is this different from Git Flow?
Git Flow is a Git extension that provides specific and opinionated support for the powerful Git branching model with the same name. It doesn’t care too much about how you keep your work in sync with the rest of the team.
Git Town doesn’t care much about which branching model you use. It makes you and your team more productive by keeping things in sync, and it keeps your Git repo clean.
It is possible (and encouraged) to use the two tools together.
Is it compatible with my other Git tools?
Yes, we try to be good citizens. If you run into any issues with your setup, please let us know!