Originally posted on Carbon Five’s Blog.

Recently, Jonah and I have been exploring test-driven development and automated deployment on the iOS platform. As we were both attending iOSDevCamp 2011, we decided to give a lightning talk summarizing our discoveries and to generate excitement within others in the community to start their project on the right foot by testing right from the start. While it wasn’t recorded, here is some of the ground we covered in the brief time we had.

tl;dr - Too Long; Didn’t Read »

Download: .ppt | .pdf

Unit Testing

After briefly reviewing why we test first (you are doing that, aren’t you?) we started with how to unit-test the behavior of classes in Objective-C. To Apple’s credit, they’ve bundled a unit-testing framework and the tools to use it in XCode; SenTestingKit. It follows the xUnit style of writing a series of tests with assertions, with an optional setup and teardown that can be executed before and after each test.

SenTestingKit works but is by no means our favorite, having issues with readability, mocking, and running from the command-line. Luckily, there are already a number of great tools to address these issues.

OCHamcrest is the Objective-C port of the popular matching library Hamcrest. Simply including the framework in your project gives you many useful matchers to allow you to write with more readability:

Likewise, adding the OCMock framework provides a wealth of mocking functionality including stubbing of methods, verifying execution of expected methods, partial mocking allowing passthrough to real instances, and even the ability to swap in new implementations for methods.

Finally, the Google Toolbox for Mac and GHUnit are excellent test runners, extending functionality including the capability to run headlessly.

Behavior Driven Development

For a few years now the testing community has been moving towards a more BDD or “spec” style of test-writing, allowing for better structure, more readability, and less repetition of code. The classic example of this is the Ruby based RSpec framework:

For those interested more about writing tests in this style, the RSpec Book is an excellent introduction and reference. While its examples are language specific, its principles can be applied to similar frameworks.

Cedar from Pivotal Labs, takes full advantage of the new Objective-C blocks to mimic RSpec’s structure:

Interestingly, instead of plugging into SenTestingKit, Cedar runs the specs as an app, allowing it to be compiled for the simulator or hardware device. Additionally, Cedar has baked in support to provide colorized output when run from the command-line; look into the accompanying project Rakefile to see how it is done.

A more recent entry into the game is Kiwi. Kiwi is structurally similar to Cedar but provides much cleaner assertions:

Using Kiwi is as simple as including its source in your project. Since it sits atop of SenTestingKit, you can include running all tests as part of your XCode build process. Our brief usage of Kiwi has shown it has some issues running UI components but it does hold promise.

Integration Testing

More recently our explorations have taken us into looking for an integration and acceptance testing framework to include in our process. While we could write test or specs to do end-to-end testing, it’s best to separate the concerns; in fact, we should start by writing integration tests that focus on the user experience!

Again, using our experiences with web development in Ruby, we looked for something similar to Cucumber. Its use of plain text and the tools to parse it are an ideal way to specify what a system is expected to do:

Is there a way to write Cucumber specs for iOS? There is! In fact there are two; iCuke and Frank.

However, there are problems with both projects. iCuke launches the application in the simulator and automatically launches an HTTP interface to interact with the screen. The project also hasn’t seen a commit for over a year, though a number of forks seem to be actively developed. Frank, the other Cucumber based library, is actively maintained and follows a similar strategy of embedding an HTTP server.

A Frank Cucumber driver then communicates with the server to fire-off user events. Our hesitation to use it comes in the amount of setup it requires with care needed to ensure you don’t include it in a release build of the application. And for both iCuke and Frank, the idea of running an entire server feels incredibly heavy for what we want to do. Is there another option?

Strangely enough, a possible candidate was released into the wild the day before our talk. KIF, short for Keep It Functional, from Square aims to minimize the layers and load in order to test like a user. Each step in a scenario focuses on a single user action, targeting an interface component through the accessibility attributes. Well documented, and with the ability to capture screenshots (and video come the release of Lion), KIF has a lot of potential and we look forward to exploring it in the future.

Continuous Integration and Automated Deployment

We close the loop on the full agile process through continuous integration and automated deployment, ensuring code stability and getting it out in the hands of testers. We’ve previously documented how to do so in other posts, including how to build XCode projects and run tests from the command-line, as well as rolling your own Over The Air distribution. Jonah provided a full script that carries out all these steps from start to finish.

Using Jonah’s script and following the lead of Cedar, I started creating a set of Rake tasks to carry out each step, from building to headlessly running specs to signing - all from the command-line (it can be found here). I also decided to use TestFlight to distribute and target builds; it provides a intuitive interface for our pre-release users to access the latest builds and easy to use tools for developers to manage those releases. Even better, their upload API allowed us to write a Rake task to deploy right from the command-line.

What’s Missing

While we’re excited to see all these great tools emerging for iOS development, there is still a LOT to be done to get us anywhere near the ease of writing tests that the Ruby world currently enjoys. While the nested spec-style of code alleviates duplication, you can still end up with large blocks of code that look like this:

This doesn’t really tell us anything about what we are trying to do! It would be far more pleasant to have a library like Factory Girl and simply write:

Similarly, while CoreData provides a way to update schemas from one release to another, there is no proper migration path that can move forward and back to ensure compatibility. Add to that time-based testing like Timecop or automatic testing via Guard and you have a slew of great projects that we could work on.

In Conclusion

As you can see, there are a number of solid options to test and deploy your iOS projects. Personally, I’m looking forward to bringing KIF into my current stack of Cedar, OCHamcrest, and OCMock. More importantly, there is plenty of room for new tools to help these processes. I plan on releasing a gem of the Rake tasks we’ve whipped up, and hope to write an interpreter of gherkin (the language cucumber features are written in) to KIF. What would you like to see?

tl;dr

• Test-driven principles still apply in iOS development.
• While out-of-the-box unit testing in XCode comes in SenTestingKit, it is somewhat limited. Use OCHamcrest, OCMock, GTM, and GHUnit to expand functionality
• Alternatively, for a more BDD approach, use Cedar or the newer Kiwi.
• Cucumber driven integration tests have been implemented with iCuke and Frank,while the recently released KIF provides for doing the same in Object-C.
• Command line builds, running tests, and Over The Air distribution are all possible and documented. We’ve done it as a bash script, or rake tasks.
• TestFlight provides not only a great way for your pre-release users to get your builds, but an API to automate sending it to them.
• There is much work to be done; factories, fixtures, time manipulation, … why not dive in?