The recent events on flight 253 have us all thinking about airline security. I think Bruce Schneier, as usual, has said it best:
For years I’ve been saying this:
Only two things have made flying safer [since 9/11]: the reinforcement of cockpit doors, and the fact that passengers know now to resist hijackers.
This week, the second one worked over Detroit. Security succeeded.
EDITED TO ADD (12/26): Only one carry on? No electronics for the first hour of flight? I wish that, just once, some terrorist would try something that you can only foil by upgrading the passengers to first class and giving them free drinks.
Bruce is referring, of course, to the new, rumored security procedures said to be rumbling their way out of the TSA’s nightmare bureaucracy and onto your next airline flight.
In a nutshell: planes must disable their seat-back in-flight entertainment, passengers can’t use electronics, get up or access their bags during the last part of a flight. Oh, and you can’t have anything in your lap.
Keep in mind, this is in response to a dim-witted “terrorist” who snuck a weak explosive onto a plane… inside of his pants.
Remember when shoe bomber Richard Reid tried to blow up his Reeboks? That resulted in a limit of one carry on bag per passenger, despite the fact that Reid’s plan had nothing to do with carry on bags. Then there’s the whole liquid limit for carry on bags, which also makes no sense given the simple reality that liquid re-combines very easily, even if you do happen to carry it aboard in small containers instead of big ones.
So the recent rumors of new policy, while wildly stupid, are just stupid enough. They carry enough non sequitur authenticity to be utterly believable. I was ready to believe them. Then a source contacted me. He’s inside the TSA and was desperate to leak the internal memo that brought the new rules into existence. Now it all makes sense: the non sequiturs, the absurdity, the utterly incomprehensible creation, amendment and abandonment of these policies.
The good news, if you can call it that, is that in a few places, it would seem the TSA exercised forbearance when it seemed like, even by their standards, they’d crossed the line. Here’s the document, reproduced without further comment:
When I was a kid I had two passions: Lego and the Macintosh.
Lego was an instant bullet train to any world I could imagine. Space ships, robots, lunar colonies, pirate treasures, ancient castles, you name it. These were mine to explore. I could spend days at a time perfecting some imaginary construct made real through the magic of Lego bricks and the exertion of my crude abilities. I treasured the rarer pieces, protecting closely my little snap-on magnet linkages and battery-driven light bricks. Regardless of whatever turbulent nonsense might have been happening elsewhere in my little world, Legos were an inviolable source of joy.
Joy, you know, that feeling that lives somewhere between the pit of your stomach and the tip of your smile. That vague something that builds a simple, contented glow inside of you that’s like a thousand perfect, extra-gravy-save-me-some-pie thanksgiving dinners with none of the bloated aftermath. Maybe you just saw the most beautiful vista in all of the world. Maybe you just fell in love. Maybe you’re ten years old and you see exactly what you were hoping for under the Christmas tree. You know what I’m saying, right?
Joy.
The Macintosh came a bit later. At age 7, I got my hands on a borrowed Macintosh SE. And the joy was there, too. It could do so many things. It could produce clean, perfect type that was huge! I made a lot of paper signs. It could store all of this information and then show it to me again later. It could show me pictures and organize them into this tidy scrapbook.
And sounds! It made all these noises. I was most enamored with the quacking duck.
It was this whole world inside there that I could barely understand. I knew only one thing for certain: I wanted more. So much more of it.
Eventually, through about three years of begging and cajoling, I convinced my mom to plunk down the tidy sum necessary to secure a Performa 6116CD for our exclusive home use. (The fact that I kept spending a lot of time at the home of a neighbor kid who had his own Mac and whom my mother intensely disliked probably sped things along, too.)
The 6116 was an even greater magnitude of joy. An 8x CD-ROM drive and a huge bundled library of multimedia content like encyclopedias and interactive atlases. Plus creativity applications and, wonder of wonders, Sim City 2000. I had so much fun exploring this new world. I spent an inordinate amount of time learning every piece of software I could get my hands on.
It was joy.
And then, I grew up. Like so many, I lost my capacity for the discovery of simple joy. It became the exception rather than the rule of life. Go to school, then go to work, do your job, go home, repeat.
Then I found programming. It occurred to me tonight, as I struggled, quite happily, to grasp how the hell it is block arguments in Ruby work, that I’d rediscovered a simple, consistent source of joy. Programming languages are infinite bins of Lego blocks, waiting to be assembled to my liking. Programming is a limitlessly fascinating Performa, waiting for me to learn and harness any language for any task I can imagine. There’s just so much to learn and enjoy in programming computers.
Even after a few years of it, programming makes me feel joyfully like a kid again.
Here’s some code from awhile back. It’ll take a CSV and turn it into a Plist file.
The structure works like this:
The first row of your CSV is considered to be the header. The code will use content from the header row to create key names. For every subsequent row, the code will create a dictionary, associating the content of each column with the key names from the header. The dictionaries live in an array which is written out to a Plist file.
This is handy if you’ve got a bunch of spreadsheet-type data you’d like to use as the basis for a static reference in your app. Say, data about the periodic table, airport listings, trivia questions, whatever. You can sift and sort your data in a user-friendly environment like Numbers or Excel, then dump it out to CSV and put this code to work.
It uses cCSVParse to read the CSV file.
// NSLog(@"Converting %@",pathAsString);
CSVParser *parser = [CSVParser new];
[parser openFileWithPath:pathAsString];
NSMutableArray *csvContent = [parser parseFile];
[parser closeFile];
if (pathAsString != nil)
{
NSArray *keyArray = [csvContent objectAtIndex:0];
NSMutableArray *plistOutputArray = [NSMutableArray array];
NSInteger i = 0;
for (NSArray *array in csvContent)
{
NSMutableDictionary *dictionary = [NSMutableDictionary dictionary];
NSInteger keyNumber = 0;
for (NSString *string in array)
{
[dictionary setObject:string forKey:[keyArray objectAtIndex:keyNumber]];
keyNumber++;
}
if (i > 0)
{
[plistOutputArray addObject:dictionary];
}
i++;
}
// NSLog(@"Plist output array %@", plistOutputArray);
NSMutableString *mutableString = [NSMutableString stringWithString:pathAsString];
[mutableString replaceOccurrencesOfString:@".csv" withString:@".plist" options:nil range:NSMakeRange([mutableString length]-4, 4)];
NSURL *url = [NSURL fileURLWithPath:mutableString];
// NSLog(@"Write to URL %@",url);
[plistOutputArray writeToURL:url atomically:YES];
}That’s from a simple desktop app I built for the purpose. It takes the path of your CSV, via drag-and-drop, then spits out the Plist beside it. You can grab the source at Google Code. You can also download this compiled binary if you want to get crackin’ right away.
The App Store has over 100,000 apps available for purchase.
That’s a big number.
I can’t tell you the secret to success in the App Store. I haven’t discovered any magic formulas. The one thing I can tell you with 100% certainty is this: if you ship a product with bugs that genuinely piss off your customers or, worse, make them unable to use some portion of your hard work, the road toward success just got much longer and a lot more miserable. Even with “prioritized review” requests, once your buggy app is on the store, the review queue means you could be waiting a long time before your customers can enjoy your bug fix.
You will not sleep well at night.
The good news is that this problem has been solved a thousand times over. A thorough beta test can save you from this terrible outcome and the sleepless nights that come with it. With a bit of diligence, you can ship an app with awesome stability to match your great ideas.
I’ll share how I run my betas. This is about process — the technical side of Ad Hoc distribution is thoroughly documented by Apple and others. The tools I use are simple and free. They work. If you have suggestions for better ones, I want to know about them before my next test! Please share in the comments.
Gather Your Testers
You’re going to need testers. If you use a volunteer testing pool, figure that 80% of testing will be done by 20% of your volunteers. That means to have any hope of gathering meaningful feedback, you’ll need at least 10 volunteers. 15 or 20 would be the ideal minimum.
Next, you’ll need to start early — but not so early that your testers’ enthusiasm cools by the time you ship your first beta build. Try to get the word out between three to six weeks before you’re ready to test. If you have a particularly strong network, you may need less time. If you’re relying more on the kindness of strangers, though, make sure to give yourself plenty of lead time. Use your website, your blog and your Twitter to solicit volunteers but don’t be a pest.
Most importantly: if you have contact with existing customers from previous versions or perhaps other products entirely, send a quick, polite email explaining what you’re going to test and encouraging them to join up. Customers who have emailed you in the past with feedback or feature requests may be especially keen on seeing your latest work.
Understand Your Testing Pool
You need to know things about your testers. What kind of hardware will they be using? What version of iPhone OS? What’s their geographic region? (International testers are crucial for finding your screwups with localized formatting and other geographic issues.) Most importantly, you’re going to need to collect their UUID.
I use Forms for Google Docs to collect most of my feedback. It’s an easy-to-configure way to store structured data. Since you can designate required fields, you can ensure your testers don’t forget to include anything crucial when reporting issues. Best of all, you can embed the forms into existing web pages, allowing you to include them alongside testing guidance, version history or other information that might be useful.
Here’s the form I use to collect beta tester info:
It’s not short. The other benefit to using a form is to ensure you only get testers who are reasonably skilled at providing detailed information. Everyone who makes it past this filter has a decent shot at actually telling you everything you need to know later in the beta.
Betas Worth Testing
If your app doesn’t look somewhat close to the final product, don’t distribute it for testing. If you’re relying on a volunteer testing pool, you want them eager to use your stuff. If your overall UI is incomplete, you’re not ready to test. You’ve got a limited amount of time and attention with your testers. Don’t squander it on early builds you could easily test yourself. The closer you think your app is to shipping, the more useful your testers will be to you.
You do yourself no favors if the last bit of functionality you add affects previously-solid features that now need a whole new round of testing.
Believe me.
Distributing Your Binaries
So you’ve toiled away and you’ve finally got an app you feel is ready to be tested.
Don’t email the binary.
Depending on the size of your app, the attachment may not be properly delivered to the recipient, especially if they’re using Exchange-hosted email. Instead, host your app and the necessary provisioning profile on the web.
I also recommend a simple checkout system. When you’ve got a new build ready for testing, send your testers a link to a one-question form that asks for their email address. On the form confirmation screen, you can provide a link to the binary. This lets you get an idea of who is participating. It also lets you answer an important question: am I getting zero feedback because there are no bugs or because there are no active testers? With a checkout process, you’ll know for sure.
Use clear versioning with every distributed build. You need to be certain you and your users are talking about the same build when they give you feedback. Beta phases, version numbers, dates, whatever you need to make things clear for everyone involved. Make sure that somewhere in your app, your code pulls the version number out of your app’s Info.plist file and displays it in the UI. Here’s some code to get that string:
NSString *currentVersion = [[[NSBundle mainBundle] infoDictionary] objectForKey:@"CFBundleVersion"];
Update your Info.plist’s version number religiously before distribution. Even better, update it automatically.
Communication
You need to communicate with your testing pool. Keeping them regularly updated, without being spammy, is crucial to keeping them excited about what’s going on.
When you send out a new build, explain what’s new. Give specific testing guidance so that your testers know where they should focus their efforts. New method of rendering cells you strongly suspect has a bug or two? Share your hunches. Built a new import subsystem you’re suspicious might break with legacy data? Provide an example data set or encourage existing users of the old version to give it a whirl. If you’re manipulating data that lives only on the user’s device, tell them to back up their iPhone before running your beta. And tell them why. Don’t be a douche on this: never take for granted the importance of your testers’ application data, especially if you’re lucky enough to have your existing customers helping you. Yes, we all know, betas are risky. Still, the iPhone isn’t the desktop and if you’re testing an upgrade to an existing app, your customers may have data they created before accepting the risks of a beta.
If it has been more than seven to ten days since your last build, send a quick update to the pool. Call out awesome testers and let them know they’re making a difference. Share the score of resolved issues you were able to fix thanks to the pool’s feedback. Most of all, thank them. Their efforts are helping you to not look like an idiot.
Sandboxing Helps
If you’re making an all-new app, you can skip this section. If you’re testing an upgrade to an existing app, read on.
Your early beta builds should use a different app identifier than your shipping app. This way, your beta will be installed as a distinct application alongside your currently shipping version on the user’s device, instead of overwriting it. This lets you ensure that no shenanigans happen to legacy data if you’ve got customers from previous versions in your testing pool. Toward the end of your beta, after much internal testing, you can start distributing builds that overwrite previous shipping versions. You’ll want to ensure that the app properly imports legacy data and preferences and, of course, ensure nothing otherwise disastrous happens when iTunes distributes your updated app to existing customers with existing data.
Gather Useful Feedback
You need to know things about how your application is failing. The forms I use are inspired heavily by what I saw once upon a time during the iPhone OS 2.0 Enterprise AppleSeed beta test. I think it covers the basics. Of course, don’t be afraid to add other questions specific to your application’s circumstances. For bugs, measure how well you can reproduce the bug and its overall severity while capturing the reproduction steps. For feature or enhancement requests, measure the tester’s impression of the request’s importance along with what they’re asking for.
However you gather your feedback, ensure that every single message you send to your testing pool includes brief instructions on how your testers should report their bugs. Never make them hunt around for it.
Ship
You haven’t added or changed anything for a couple of builds. You’ve fixed plenty of bugs and no one has reported regressions. You’ve sent emails to your top-performing testers and they’re giving you a thumbs-up.
Give it one last test yourself. Thirty minutes, minimum, actually doing things your users are likely to do. Is it kosher?
Then ship it.
Then, while you wait for approval from Apple, test it some more, informally, yourself. If somehow a show-stopping bug pops up while you await approval, you can reject your binary and submit a fix before it ruins anyone’s day.