Practical Get Started With Swift

Recently, I had to get started (fast) with Swift on a series of existing iOS projects. I would classify myself as being able to get to fourth gear in a Honda Accord with a manual transmission when it comes to using and applying Swift. I look forward to the day when my Swift knowledge is equivalent to running full out on the Nurburgring in a Bugati Veyron.

My background is C, C++, C#, Objective-C, with a side of JavaScript along the way.

I will never admit to any classic Visual Basic, or VB.NET, code ever being typed by my fingers, ever!

When it comes to learning a new language, I find myself in ‘Use it or lose it’ mode. Learning Swift comes very naturally if you have a concrete task which allows you to add a feature, or refactor an existing iOS project, with brand new Swift code.

I found that becoming proficient with Swift was very easy given the general C language progression in my career. In very short order, I was able to do almost all the tasks in Swift that I would normally do in Objective-C against the CocoaTouch / UIKit API surface. More advanced Swift language constructs exist, which I have barely touched, and hope to be able to learn and apply some day—I am looking at you, protocol extensions and generics.

Swift is an expansive language with many hidden gems and great language constructs. The Swift Programming Language (Swift 2.2) iBook is over 1,000 pages (as paginated at my screen size, your mileage may vary) and has so many great samples and documented language features that your brain quickly fills up as you read through it. It may seem daunting, but, once the basics of Swift are mastered, the advanced techniques will follow.

This very rough guide is meant to expose you to the philosophy behind the Swift language and provide references to the point where you feel comfortable doing almost any task in Swift.

What follows are references, philosophy, history, and a general story of what I went through while revving up with the Swift language. You will notice I don’t have any Swift code in this post. At all. Apple does such an amazing job in the The Swift Programming Language (Swift 2.2) iBook that I recommend you just go there for all the startup samples you need. Rev up your Mac, install Xcode, rev up a Playground, put the Swift Programming Language book on Monitor 2, and get started learning Swift.

I sincerely hope that your experience is as good as mine was as I started learning Swift.

Reference and Learning

iBooks – The Swift Programming Language (Swift 2.2)

iBooks – Using Swift with Cocoa and Objective-C (Swift 2.2)

  • These 2 iBooks are the definitive guide to the Swift language. They are sweeping and comprehensive and serve to build you up gradually from basics to advanced Swift language constructs.

iTunes U – Developing iOS 9 Apps with Swift by Stanford

  • I cannot recommend these movies more highly. These serve as a great simultaneous companion to the Apple Swift iBooks.

Ray Wenderlich – Swift 2 Tutorial: A Quick Start

  • Nobody does amazingly useful and entertaining tutorials like Mr. Wenderlich and his companions. Dig deeper at this site to find even more useful iOS code snippets.

Github – Awesome Swift – Includes tutorials and frameworks

  • Yet another great list of maintained links related to Swift language ‘Get Started’ guides and Swift based Frameworks.

Subscribe to Swift Weekly

  • Swift Weekly does a great job of summarizing what is happening with the constantly changing Swift language development, in addition to providing some really great links that should help you be more ‘Swifty’ with your code and thinking.

Swift Frameworks I can recommend

Github – Decodable – Translate JSON to Swift structs

  • I like Decodable more than Argo + Curry and SwiftyJSON for JSON to Swift struct translation.
  • IMHO: Nothing beats C# + Newtonsoft.JSON (or JSON.NET) for seamless and awesome JSON translation. But Decodable makes JSON to object conversion way easier than anything in Objective-C.
  • Side note: In order to use these frameworks in your iOS project, you probably also need to become proficient with Carthage or CocoaPods based dependency management within an Xcode project.

WWDC Swift Sessions

I hate to say: “Watch them all”, but …

From WWDC 2015:

What’s New in Swift

  • Worth it to see how the language evolved between Swift 1.0 and Swift 2.x

Protocol-Oriented Programming in Swift

  • Worth it to get exposure to a series of more advanced language features
  • The focus on class vs. struct (or reference vs. value types) is also a great object lesson (pardon the pun).
  • For those of you with C# experience this video is fascinating when you realize that protocol in Swift is the exact same thing as interface in C#.

Building Better Apps with Value Types in Swift

  • Helps reinforce the concepts from the Protocol-Oriented Programming in Swift video.

Swift and Objective-C Interoperability

  • Much of the Interoperability rules are changing with Swift 3.0. Especially naming.

Swift in Practice

Improving Your Existing Apps with Swift

Optimizing Swift Performance

WWDC 2016 promises Swift 3.0 videos and much more Swift architecture goodness.

Swift Futures

Swift is now all open source out on Github with a very active community of developers. To stay on top of things be sure to join the community and the mailing lists.

  • To give you an idea regarding Swift language topics discussed:
    • The community has talked about completely removing the for statement from the language and requiring a closure like replacement along the lines of map.

The Talk Show Podcast – Episode 139Transcript

  • Craig Federighi discusses the Swift language history and future.
  • There is a great follow up by John Siracusa.

Key Concepts You Should ‘Grok’

I believe that the following things are the practical takeaways from all that reference material for any starting Swift programming experience:

  • Optionals and unpacking – Over the long haul, this will become the #1 thing that you are always thinking about as you program in Swift. For C programmers, Optionals and unpacking is the same as malloc and free. Every variable and member needs to be thought of in terms of whether or not to make it optional. You will get so tired of the red dot in Xcode telling you to add a ! here and a ? there. The more you grok optionals and unpacking, the less friction you will have day-to-day with the language.
  • if .. let – If let is a great helper for all the optional access and value unpacking you will need to do. If let is a huge helper as you interface with legacy CocoaTouch / UIKit constructs.
  • var and let – Every variable declared needs to be thought of as mutable or immutable. This concept coupled with optionals is a huge mind shift, especially for Javascript and C# programmers.
  • Class vs. struct or reference vs. value types – Knowing the difference between reference and value types can help improve performance and also give you a leg up on the mutation of your object across calls. The weirdness that results when the Swift String is actually a value type (and not a reference type such as NSString*) is also an interesting brain twister.
  • Closures – Closures in Swift are similar to blocks in Objective-C, however the cleaner (and possibly more complex) language handling of closures can be mind bending. This is especially true for closures as the last parameter to a function where the function call can be closed, but the closure can go on.
  • Threading / Async – Unfortunately, Swift has no native language construct for async or multiple thread handling. This means that you are on your own with GCD calls, or other thread handling techniques you already do with Objective-C, only in an all new language.
    • C# people will really start to miss async / await, and Javascript people will start to dislike the GCD nastiness around their closures. You can only type dispatch_get_main_queue() so many times before you start to lose more hair than a 42 year old with a bald maternal grandfather.

Philosophical and Design Takeaways

  • Swift is a perpetual Beta language – I am being kind of vicious with this statement. Apple continues to evolve Swift at a rapid pace (see also the community links above). Apple has stated that they have no issues with changing the language so your code won’t compile under future versions of Swift. This is both good and bad. It is good in that Apple is very dedicated to awesome Swift futures. It is bad in that future refactoring of your code could include things like rewriting every for loop as the for keyword is rendered obsolete by the compiler.
  • Swift is designed to eliminate null access crashes – Apple has tons of app crash metric data. From that metric data they learned that EXC_BAD_ACCESS is probably the #1 problem all apps suffer from. To eliminate the dreaded null access crashes, Swift is designed from the ground up to make sure that you know how every memory chunk is being accessed from each variable.
  • Apple has shipped very few applications that utilize Swift code on iOS as of January 2016Unfortunately, this is believed to be true. As Apple really starts to ramp up on Swift, there will be more and more changes to the language, and much more stability to the language and runtime.
  • Swift may not be for you, your company, or your project, yet – Your company probably has a ton of assets in Javascript, C, C++, C#, and Java. Those assets are crown jewels. You should treasure them. There are many ways to reuse those assets on iOS that won’t break the bank and require you to learn a whole new Swift based technology stack. Swift may not be for you if you don’t have any Objective-C based resources yet.
    • Javascript – Take a look at React Mobile, PhoneGap, or other Javascript based app frameworks.
    • C# – Take a look at Xamarin and Xamarin.iOS / MonoDroid, portable assemblies, and .NET Core futures.
    • Java – Take a look at the Android platform. If you need to be on iOS take a look at J2ObjC. With all that Objective-C code generated from your Java code, you can always bring in Swift on an as-needed basis.

I come from a diverse background of technologies across Windows and iOS, front end and server side. I can say that Swift is a dramatic improvement over Objective-C on many levels. However, as with all new things that are working against 20-year-old code (and the ideas behind that code), there are going to be bugs and changes as things evolve. Sometimes it is best to use what you have, be ready for what is coming, then jump in feet first into something like the current Swift stack.

Knowing the depths of Objective-C helps you learn Swift

It is 1000 times easier to use Swift if you have a firm grounding in Objective-C.

There, I said it. Apple doesn’t want to admit it, but you can become efficient with Swift much faster if you understand all the concepts behind Objective-C.

Message passing, reference counting, automatic reference counting, strong / weak references, protocols, delegates, the whole philosophical and technological surface of CocoaTouch / UIKit, function naming conventions—All of that Objective-C knowledge will only help you ramp up rapidly on Swift.

Swift is way easier to learn if you can intuitively grok that behind all those smooth Swift wrapped CocoaTouch / UIKit calls exist a ragged, crunchy, 20-year-old Objective-C core. There are classes that still start with NS in there! NS stands for NeXTSTEP! Yes, that black cube thing from the bad Steve Jobs movie [Rotten Tomatoes Score of 74%—Maybe not so bad], of which 5,000 total were sold!

There are a ton of great Swift based frameworks out there, but for the foreseeable future you will still have to pull down Objective-C libraries and frameworks to get your real work done.

For so many tasks, nothing can substitute an Objective-C, C, C++ based approach due to the need to go ‘to the metal’ for memory management. Being able to go ‘to the metal’ lets you get the best out of pure native APIs like OpenGL, CoreAudio, and the Accelerate Framework. I don’t doubt that someday Swift will be optimal when used at those layers too, but for now the code samples and libraries that maximize certain kinds of apps are still rooted in Objective-C, C, or C++.

The great thing that Apple has done with the iOS tooling is the seamless way a single app can contain Objective-C, C, C++, and Swift code all mixed together in a single project. This is a great development story, and an amazing technological feat.

There are whole swaths of APIs within CocoaTouch / UIKit which are easier to use in Swift if you know how to exercise them in Objective-C. There is a reason that Apple’s Using Swift with Cocoa and Objective-C (Swift 2.2) reference is 81 pages long (and also well worth reading from front to back).

From the Fringe- NSClassFromString – Using Swift from Objective-C

In a recent project I wanted to use a Swift based class as a transplant into some legacy Objective-C code. The Swift class had to be instantiated using the NSClassFromString call.

I did what anyone would do, revved up my Swift class within my Objective-C project. Then tried to call NSClassFromString(“MyCoolSwiftClass”)

It didn’t work.

Well after Googling and a series of Stack Overflow articles, I stumbled onto a function and reference code that properly takes your Swift class name and makes it work via NSClassFromString.

According to Apple’s Using Swift with Cocoa and Objective-C (Swift 2.2) reference you have to include the app name into the string to use it. So NSClassFromString(“MyApp.MyCoolSwiftClass”) is the way to go.

Unfortunately, the journey doesn’t stop there. It still didn’t work.

The project I was working on had multiple build configurations, and the app name would change depending on the build configuration. So after much sadness and gnashing of teeth, I stumbled onto this Objective-C helper function thanks to Kevin Delord on Stack Overflow:

- (Class)swiftClassFromString:(NSString *)className {
    NSString *appName = [[NSBundle mainBundle] objectForInfoDictionaryKey:@"CFBundleName"];

    NSString *classStringName = [NSString stringWithFormat:@"_TtC%d%@%d%@", appName.length,  appName, className.length, className];

    return NSClassFromString(classStringName);
}

In Objective-C we have to go and parse out the bundle name from the main app bundle, and append that with the Swift class name in order to use our shiny new Swift class within our legacy code.

In all honesty, I don’t know what happens if you try to instantiate a Swift class that was brought in via a Framework (most likely via Carthage). The above code may not work because we are just looking at the main bundle name, not the Framework bundle name. Further investigation is needed here.

This won’t be the last time that I will need to know Objective-C in order to really use Swift effectively. Many more examples exist, and many future examples will be created, as the Swift language continues to evolve and Objective-C slowly dissolves. We are already starting to see cracks showing up in the Swift and Objective-C interoperability story via the ongoing discussions of dynamism and the Application Binary Interface (ABI).

Mix / Match of Objective-C and Swift in the same project

I am going to seek to summarize Apple’s great developer documentation in the Mix and Match section of the Using Swift with Cocoa and Objective-C (Swift 2.2) iBook.

To use Swift classes in your Objective-C files the compiler will generate a header named:

YourAppName-Swift.h

At the top of your Objective-C .m files you #import this file to use your Swift classes within Objective-C:

#import <YourAppName-Swift.h>

To use Objective-C classes in your Swift code the compiler will automatically include all headers you add to the Swift bridging header file:

YourAppName-Bridging-Header.h

You do not need to import anything within your Swift code as the bridging header will take care of all class bridging from the #import(ed) Objective-C headers for you.

The 2 header files mentioned above are part of the Build Settings of your main app project.

The file name that you add Objective-C header #import(s) to so you can use Objective-C classes in Swift code:

  • Swift Compiler – Code Generation
    • Objective-C Bridging Header

The file name that you #import into Objective-C  files in order to use Swift types:

  • Swift Compiler – Code Generation
    • Objective-C Generated Interface Header Name

To use Frameworks that are referenced at the project level within your Swift code you do need to import the Framework by name at the top of your Swift file.

For example:

In the links at the top of the article I recommend the Decodable framework to translate JSON into populated Swift structs.

To use Decodable within Swift code you need to include the Framework within your project and you also have to import the Decodable framework at the top of your Swift file:

import Decodable

Good Luck!

In general, if you are starting an iOS application from scratch, and you have made the decision to go pure native with Xcode based tooling, I highly recommend learning Swift and using it from the start. Your software quality will be awesome from day 1 as you start to internalize if .. let, optionals, completion handlers, and other ‘Swifty’ techniques of doing things.

You lose nothing by starting your new application in Swift because you can always mix/match Objective-C, C, and C++ as needed throughout the development cycle.

In your existing application, I would start to try out Swift for certain non-essential components and ramp up Swift usage as your knowledge, the language, and tooling matures. I have started using Swift for wrappers around legacy Objective-C based static libraries, to standard UITableViewController implementations, and to simplify JSON to struct processing. The mix / match nature of Swift to existing Objective-C code is seamless and stable for all tasks within an existing application.

You should find that you will be writing much less code, getting more done, at a high stability level, with Swift within the first couple of weeks. The clarity and readability of your code should also be much higher as you fall into the groove of using the Swift language.

Swift really is the future of iOS and Mac OS X app development. The time is now to start learning and improving your Swift skills.

Adding Amazon Alexa Voice Services to Your iOS App with Swift

Major thanks to the MacLexa project out on Github for providing motivation, source code, and a great starting place for this blog post.

secure

Amazon Echo is an always listening device which acts as an always available assistant. Thanks to the deep integrations that the Amazon Echo has to online services you can issue a wide variety of questions and commands via voice and get real world responses.

Once your Amazon Echo is setup it is always listening for commands. The key word being listened for is “Alexa”. For example: “Alexa, play Taylor Swift.” If you have an Amazon Music account, and you have purchased Taylor Swift songs (who doesn’t have the 1989 album by this point?), then your Amazon Echo will start playing Taylor Swift.

Notice that I used “Alexa” as a prefix in the above audio command. Saying “Alexa” to an Amazon Echo is the trigger word. It is like saying “Hey Siri” to your iPhone, “OK, Google” to your Android Phone, or “Xbox” to your Xbox, then issuing a voice command. Once you have spoken “Alexa” the Amazon Echo device will record the rest of your voice command and relay it to Amazon’s Alexa Voice Service which will take specific actions and return a response from Amazon.

The Alexa Voice Service has another trick up its sleeve: Skills. Third-parties can create a Skill using Amazon’s Alexa Skills Kit which can extend the Alexa Voice Service via additional service integrations.

  • Fidelity has created a stock value skill that allows you to get answers to questions about values on the stock market. Once you have activated the Fidelity stock skill for your Amazon login via an Android or iOS app, then you can ask Alexa: “What is the value of Microsoft stock?” and get back an answer via the Fidelity stock skill.
  • There is a skill called Yonomi that integrates with Phillips Hue lightbulbs. It allows you to change the color of your lights with a voice command, so you could flood the room with red light whenever you say: “Alexa, set lights to ‘Game of Thrones’ mode.”
  • Of course, you can also buy stuff from Amazon. Just say: “Alexa, buy me some Moon Cheese” If you are an Amazon Prime subscriber, and you have already ordered Moon Cheese, it will be ordered automatically and FedEx will deliver it to your doorstep in 2 days or less (or the same day if you have Amazon Prime Now).
  • If you have a Spotify subscription, you can configure Alexa to integrate with the Spotify Skill and issue voice commands to playback all the music you can stream from that service.

Let’s review all the terms in this Amazon voice control and audio device realm:

  • Echo — The always-on, standalone device that listens and responds to voice commands.
  • Alexa — The keyword that the Amazon Echo uses to determine when a voice command is being listened for. It is also the brand name for the back-end voice services.
  • Alexa Voice Service — The backend service that receives voice commands and delivers audio responses to voice commands.
  • Skill — A third party add-on to handle special voice commands.
  • Alexa Skills Kit — The collection of APIs and resources developers use to create skills.

 

iOS App Integration of the Alexa Voice Service

The Amazon Echo is an interesting device, however, I have an iPhone that has a perfectly good microphone + speaker combination in it. Lets integrate Amazon’s Alexa Voice Service (AVS) directly within an iOS app.

Why would you want to do this kind of in-app Alexa Voice Service integration?

  • It is just plain fun to see this actually work.
  • It is an interesting in-iOS-app stop gap that addresses the limitations of SiriKit’s intent domains for your specific app type.
  • Promote and do in-app integration of your own custom AVS Skill.
  • Get insight into Amazon’s web API design methodology.

Side Note: Not to be underestimated is the amazing configurability that Cortana allows app developers for Windows 10 Universal Windows Platform apps. However, Cortana integration and LUIS are topics for a different Blog post.

Let’s go through the steps needed to perform AVS integration within your native iOS app.

If you have a Mac and want to try out Alexa Voice Services + Skills on your Mac right now check out MacLexa.

There is a 4 step in-app procedure to follow for integrating the Alexa Voice Service into your iOS application:

  • Authorize the app via Login with Amazon and retrieve an Access Token.
    • This requires a user account with Amazon.
    • It is used to associate future Alexa Voice Service requests with the Alexa settings that any Amazon user has setup on their own personal Amazon account.
    • This step also lets your in-app Alexa integration fully work with any Skills that are associated to an Alexa + Amazon account via Amazon’s Alexa configuration apps.
  • Record audio from the device microphone and store it locally to your app.
  • HTTP POST the Access Token and the audio from the device microphone to the Alexa Voice Service.
  • Playback Alexa’s voice response audio which is returned raw from the Alexa Voice Service.

 

amazon-voice-services

Sample Source Code

Feel free to pull from my AlexaVoiceIntegration Github repo for a full working sample. (Sorry, it is still in Swift 2.3)

The most fun chunk of code is the upload function which performs the upload to AVS and the playback of the Alexa voice response.

 

Registration of your Alexa Voice Service app integration with Amazon Developer Portal

In order for your app to integrate with Alexa Voice Services you need to go to the Amazon Developer Portal and get certain specific keys:

  • Application Type ID
  • Bundle Id
  • API Key

To get started:

The end goal for the in-app Alexa startup procedure is to get an access token string that we can send via HTTP to the Alexa Voice Service API.

We get the access token by using the LoginWithAmazon.framework for iOS feeding in the Application Type IDBundle Id, and API Key values you will configure and generate on the Amazon Developer Portal.

From your Amazon Developer Portal / Alexa configuration you need the following values:

  • Application Type ID from your created Alexa Application / Application Type Info section
    • ApplicationTypeID
  • API Key and Bundle Id pair that you will create in the Alexa Application / Security Profile / iOS Settings area
    • iOSSettings

Be sure to keep track of the Application Type ID, API Key, and Bundle Id. We are going to need these later on when we setup our iOS app.

 

iOS code and Xcode project setup to use the LoginWithAmazon.framework from Swift

Start by going through Amazon’s documented steps to get the LoginWithAmazon.framework file.

What follows is a fairly standard way of using a standard iOS framework within an existing Xcode project.

Copy the LoginWithAmazon.framework file to a folder within your iOS Xcode project.

Open Xcode and go to your iOS project General settings:

XcodeConfig

In the Embedded Binaries section press the + button.

Navigate the file chooser to where you copied the LoginWithAmazon.framework bundle. Press OK.

You should see something like the above where the LoginWithAmazon.framework file is in the Embedded Binaries and Linked Frameworks and Libraries section.

To fix an app deployment bug with the framework go to the Build Phases section and ensure that the Copy only when installing checkbox is checked:

CopyOnlyWhenInstalling

The final step is to ensure that the master header from the LoginWithAmazon.framework is included in your Objective-C to Swift bridge header file.

If you already have an Objective-C to Swift bridge header file, then include the following line:

#import “LoginWithAmazon/LoginWithAmazon.h”

If you do not have a bridge header file, then you need to configure your Xcode project with an Objective-C to Swift bridge header, then include the above line in it.

See also the official Swift and Objective-C in the Same Project documentation provided by Apple.

Test to see if all this worked:

  • Get a clean build of your app.
  • Go into a Swift source file and use the standard auto-complete to try and access the AIMobileLib static class.
    • The auto-complete should present the list of functions you can call.

 

Configure your app with the API Key, Bundle Id, and Application Type ID from the Amazon Developer Portal

First up is to ensure that Bundle Id, API Key, and other values are properly configured in your Info.plist and application.

Open up your app’s Info.plist within Xcode:

AmazonInfoPList

 

Bundle identifier

Whoa, that’s a lot of weirdness with $(…) stuff.

As you can see the core of our needed Login With Amazon values is the value of $(PRODUCT_BUNDLE_IDENTIFIER).

The value for $(PRODUCT_BUNDLE_IDENTIFIER) comes from your General project setting page within Xcode:

BundleIdentifier

The above value in the Bundle Identifier field has to match the Bundle Id value from the Amazon Developer Portal.

If the Bundle Ids don’t match, then it is easy to go back to the Amazon Developer Portal and add a new value. Just be sure to track the right API Key and Application Type Id with the updated Bundle Id.

 

URL Types in Info.plist 

The LoginWithAmazon.framework file uses an in-app UIWebView to handle user login scenarios.

Part of those login scenarios involve the UIWebView needing to navigate or redirect to a URL on login success / failure scenarios.

The redirect URL used is generated by the LoginWithAmazon.framework using your Bundle Id as a seed.

When the login result redirect happens within the UIWebView during login the main AppDelegate – openURL function is called in your app.

This boilerplate Swift implementation ensures that openURL portion of the login procedure properly routes through the LoginWithAmazon.framework file to call back on all the properly setup delegates:

 func application(application: UIApplication, openURL url: NSURL, sourceApplication: String?, annotation: AnyObject) -> Bool 
{
   return AIMobileLib.handleOpenURL(url, sourceApplication: sourceApplication)
}

Debugging Tip: If you place a breakpoint on the above function and it is never hit during the login procedure, then you have a misconfigured Info.plist –> URL Types area.

App Transport Security Settings

The settings shown above are the most liberal and allow all HTTP traffic from within the process to be sent. To really lock this down, you should follow Amazon’s instructions which only allow requests to be sent from your app to specific Amazon domains.

API Key

The API Key entry within the Info.plist is read up and processed by the LoginWithAmazon.framework to get all the right IDs for all the requests (i.e. Client ID, and others). The API Key has to 100% match what the Amazon Developer Portal provided. It will be a fairly huge blob of text, and that is OK.

 

Config is now done! Woo hoo! Let’s login with AIMobileLib

ViewController.swift up on my Github repo shows how login will all come together.

The AIMobileLib is your static gateway provided by the LoginWithAmazon.framework.

Side note: I have ‘Swiftified’ my calls into LoginWithAmazon.framework by using an AIAuthenticationEventHandler wrapper class that implements the AIAuthenticationDelegate and bridges the delegate calls to closures.

The call chain to AIMobileLib to successfully login to Amazon:

  • clearAuthorizationState – Clear out any stored tokens and values.
    • authorizeUserForScopes – Pops a web view, user logs in, retrieves an authorization token.
      • getAccessTokenForScopes – Takes all the cookies and keychain stored values from authorizeUserForScopes and retrieves the access token.
        • We then use the access token in calls to the Alexa Voice Service.

In this sample I chose to clear out any stored tokens and values by using the clearAuthorizationState function on every run.

AIMobileLib.clearAuthorizationState(AIAuthenticationEventHandler(
            name: "Clear Or Logout",
            fail: {() -> Void in
                NSLog("Clear Or Logout Fail")
            },
            success: {(result : APIResult!) -> Void in
                NSLog("Clear Or Logout Success")
        }));

 

Now that all tokens / cookies have been cleared, let’s have the user login via authorizeUserForScopes.

Finally, we are at the location where we need to use that Application Type Id from the Amazon Developer Portal.

We need to feed in the Application Type Id from the Amazon Developer Portal into the option scope JSON:

 let options = [kAIOptionScopeData: "{\"alexa:all\":{\"productID\":\"Application Product Id\",
 \"productInstanceAttributes\": {\"deviceSerialNumber\":\"1234567890\"}}}"]
       

Note: kAIOptionScopeData key value comes from the LoginWithAmazon.framework headers.

 

When authorizeUserForScopes is successful we then turn around and call the getAccessTokenForScopes function.

When getAccessTokenForScopes is successful we now have the access token string we can use in pure calls to the Alexa Voice Service.

 

We have the access token!  Let’s call into that Alexa Voice Service.

The Alexa Voice Service makes sending voice commands, and receiving voice responses, a very straight forward process:

  • Record the user’s question as sound data from the microphone as PCM data.
  • Send an HTTP POST request to the Alexa Voice Service that contains the sound data.
  • In the response to the HTTP POST will be the sound data to playback to the user which answers their question.

The Alexa Voice Service calls are all handled within the AVSUploader class.

  • startUpload – The master function that manages the upload process. Takes in completion handlers for success / failure.
    • start – Just a general internal helper function, checks parameters and other cleanup.
      • postRecording – Manages an HTTP POST operation to the Alexa Voice Service using the access token, some boilerplate JSON, and the raw PCM data recorded from the device microphone.
    • When postRecording is successful, the completion handler fed into startUpload will be called.
      • A parameter to the success completion handler is the raw PCM sound data (in the form of NSData) from the Alexa Voice Service that contains the voice response for the recorded user command.

The PCM sound data returned from the Alexa Voice Service can then be played through an AVAudioPlayer (From ViewController.swift):

self.player = try AVAudioPlayer(data: audioData)
self.player?.play()

 

Go out and play… and also take a look at making your own Skill

The sample Swift code to access the Alexa Voice Services is just meant to get you started. It allows you to have some level of control in how Alexa Voice Services are accessed and used within your own app. It can also be used to test any Skills you choose to integrate with your Amazon profile without needing an Amazon Echo device.

The techniques outlined above can also be fully replicated for any other other native device, platform, or front end.

Virtual Reality vs. Augmented Reality – Impact on Your Business

pexels-photo-166055

Virtual reality (VR) and augmented reality (AR) have come out of nowhere to become ‘the next big thing’.

With the failure of Google Glass as an augmented reality platform for consumers, the seeming success of the Oculus Rift as a gaming-based virtual reality platform, and the weird novelty of Microsoft HoloLens as a resurgence in the augmented reality realm, it can be hard to understand the scope, purpose, and worth of these new ‘worn on the head devices’.

  • Are they toys?
  • Are they the ‘next big thing’?
  • How could these directly impact my bottom line?
  • Can they add efficiency to current day to day work across an enterprise?
  • Is this VR / AR stuff even worth paying any attention to?
  • How did these things rise up and become a multi-billion dollar business, when I just seem them as weird head mounted toys?

VR and AR – A History

Everything starts with MEMS. MEMS stands for Micro Electromechanical Systems. MEMS allowed for the at-scale manufacturing of tiny, easily integrated, highly accurate, and super cheap sensors such as accelerometers and gyroscopes.

These MEMS based sensors can fit within a housing that is no larger than a chip on a tiny motherboard. The sensors are super accurate, and easily integrated into anything with a breadboard. The first consumer breakthrough MEMS device is the accelerometer within the Wii Remote from Nintendo. An accelerometer can measure the relationship of the sensor to gravity, meaning an accelerometer can tell you if you are moving up (i.e. negative acceleration to gravity) or moving down (i.e. positive acceleration to gravity). As Nintendo found out, this is all you need to create a multi-billion dollar home gaming business and revive your whole business.

Technology moves on from the Nintendo Wii game console and the simple accelerometer. Soon we have MEMS based gyroscopes that could do more than measure the relationship of the sensor to gravity, but could measure the relationship of the sensor to all spatial dimensions. We have super small magnetometers which can measure the relationship of the device to the true magnetic North Pole of the Earth. Barometers which can be used to measure air pressure and hence altitude. GPS sensors which can measure the latitude / longitude / time / altitude of the device using the Global Positioning System. Soon, you get the iPhone 5 that incorporates all of these sensors, and the Apple CoreMotion and CoreLocation iOS app frameworks, which allow any iOS app developer to discover and log the relationship of an iPhone to all of these real-world spatial dimensions, by regulating, smoothing, and aggregating input from all those sensors in real time for easy in-app consumption.

All of this using sensors that cost less than $10 to make and are as easy as soldering their dangling pins to a motherboard.

In addition to sensor technology, we also have the ARM and SOC (System On a Chip) revolution. ARM Holdings is an intellectual property and design company that produces simple, low-power chip plans. ARM happily toils away shipping out amazingly fast, super low-power, instruction set, and chip designs that anyone can purchase off the shelf and manufacture. ARM doesn’t make the chips, they are just masters of chip and instruction set design. Suddenly, seemingly out of nowhere, the ARM revolution comes full force. Suddenly we have Apple, Nokia, and Samsung taking these ARM designs off the shelf and starting to manufacture super cheap, low power, super fast, highly efficient full systems on a chip.

Those ARM SOCs start to make their way into hugely profitable smartphones. The high margin of smartphones relative to their cost cause a virtuous cycle of ARM SOC manufacturing improvements + cost reduction + ARM design improvements. Suddenly, we have all this high-end sensor and CPU power at super low-energy cost and monetary cost. It is now that we can put high-end ARM hardware + high-end sensor packages into the tiniest of shells with off-the-shelf battery power. Through the natural cycles of profitability driving innovation, we are at the point where we can incorporate actual supercomputers into the smallest of form factors. Even wearables.

At the same time as we are getting all this high-end ARM CPU and high-end super low form factor sensor packages, we get the capacitive touch revolution and the blue LED revolution. Now we can also create screens that sip power and are instantly relatable via the human power of touch.

On the side as part of the SOC revolution we also get the PowerVR line of high-end graphics chips which sip power and have a full hardware accelerated 3D API via OpenGL and DirectX.

It is from the shoulders of PowerVR 3D chips, all those MEMS sensors, LED screens, the ARM SOC revolution, and the virtuous cycle of aggregation + capitalization via the smartphone, that we now have a platform from which to go at full head mounted wearables, which are viable for early adopters to experience virtual reality and augmented reality applications.

The first round of these head mounted wearables falls into 2 camps:

  • Virtual Reality (VR)
  • Augmented Reality (AR)

Virtual Reality

Virtual reality works off the basis that the only thing the user can see and hear are the outputs of the virtual reality device. No outside stimulus is meant to enter the user experience.

Augmented reality works off the basis that the user needs their real world sensory experience augmented with virtual data and graphics. The scope of human vision while using an augmented reality device is meant to be completely real world, while in real time a digital overlay will be presented into the main sensory experience of the user.

The current mainstream implementation of virtual reality exists in the following major consumer products:

  • Google Cardboard
  • Samsung Gear VR
  • Oculus Rift

Google Cardboard
Google Cardboard is exactly what the name implies. It is just a simple cardboard (yes, just cardboard) mount for a smartphone.

The primary use of Google Cardboard is to present stereoscopic, 360 degree / spherical, video.

There are existing SDKs for Android and iOS which allow easy division of a smartphone screen for stereoscopic viewing of a specially created 360-degree video.

360-degree videos are easily created using special multi-camera rigs that film the full sphere of a given location. Software tools then aggregate the input from all the cameras on the rig into a single video. The single video is then projected onto a smartphone screen using special 3D spherical transforms with the 3D support of a PowerVR chip. 360-degree video playback is augmented further by aggregating the all the gyroscope, accelerometer and other position sensors present on a cellphone, to make the user feel immersed in the environment of the 360-degree / VR video.

Almost any smartphone can provide a VR experience for a user via Google Cardboard. It is simple canned video playback, with a few 3D tricks, to make the filmed environment seem more real to the user.

Gear VR
Samsung Gear VR ups the ante just a little over the Google Cardboard experience by providing a more dynamic experience than just a canned 360-degree video.

Oculus Rift
The ultimate VR experience is the Oculus Rift. Oculus Rift is a full division within Facebook.

The Oculus Rift completely shields the user from the real world via a full headset. The headset is wired into a high-end Intel based PC + high end graphics card. Less than 1% of all PCs shipped can power the Oculus Rift. The full power of the PC is pushed into the Oculus Rift to create fully immersive virtual worlds.

The revenue model for Oculus Rift is primarily gaming to start.  One can easily envision more live social interaction, and/or some enterprise uses (i.e. simulated training) for the fully VR worlds that can be created with Oculus Rift.

Most developers of VR have stated that they are glad that they get to work on VR and not AR (or augmented reality). The ability of VR developers to not have to bring in live computer vision from the real world, in real time, and augment real time human perception, is seen as way easier.

Augmented Reality

Augmented reality largely needs to solve all the problems of virtual reality, in addition to bringing in the live sensor input of human vision.

It is my opinion that augmented reality systems are much more suited to future enterprise use than virtual reality systems. This value within enterprise has largely been vetted by the weird Beta release and consumer level failure of Google Glass.

Google Glass
Google Glass was a problem looking for a solution. Google sought a solution in the consumer space hoping for some level of mainstream attraction and adoption. Instead, Google Glass has found a second life in the enterprise.

For those of you who haven’t used Google Glass, all Google Glass did was project a 32” flat panel TV into the upper right portion of your vision.

It was that simple. Any time you wanted to interact with the digital world, all you had to do was look up and to the right and there was a full virtual screen showing you something.

In many enterprise applications, the simple implementation of augmented reality via Google Glass can be extremely powerful. Imagine being an aircraft mechanic, and you would like to look up the design, plans, or details of a given part you are working on, but you are stuck in a fuel tank. Now you can get that information by simply looking up and to the right. The very simple projection of a single rectangle overlaid into human vision can lead to huge efficiencies in many areas of real work. The initial $1500 dollar price tag for Google Glass may have been a small one to pay for many in-the-field jobs. If all Google Glass did was allow a technician to view a relevant page of a PDF-based manual in the upper right area of their vision during tasks such as aircraft maintenance, automobile fleet maintenance, or assembly line work, the value of the simple augmented reality model that Google Glass presented may be realized quite quickly in increased quality, reduced repair times, or even increased safety.

It was unfortunate that Google went after the consumer instead of the enterprise markets.

Google Glass was just the start of augmented reality. We now have Microsoft HoloLens, which goes beyond the simple projected rectangle in the upper right of your vision and can fully incorporate fully 3D overlays onto any object that you can see within your field of vision.

Microsoft HoloLens
Microsoft HoloLens starts with simple gaming models for mass-consumer based revenue, but one can see a much larger enterprise vision as a dramatic improvement over the possible gains from Google Glass.

Imagine a supervisor looking out over a manufacturing assembly line while wearing Microsoft HoloLens and seeing real time stats on each station.

Imagine UPS, Amazon, or FedEx workers being able to get guidance to where something is in a warehouse overlaid directly onto their vision without needing any physical signs.

Imagine software engineers that can place status and development windows anywhere in virtual space for their work.

Imagine DevOps staff that can look out over a data center and see which machines have possible future hard drive failures, or  the real time status of a software deploy and onto which servers.

Realization of all the above with Microsoft HoloLens is aided by Microsoft’s vision of the Universal Windows Platform (or UWP). UWP allows businesses to reuse much of their existing C# / .NET / C++ code across a myriad of devices: Windows 10 Desktops, Xbox One, Windows 10 Mobile, and Microsoft HoloLens. In essence, many enterprises may already have logic that they can integrate into an augmented reality device so they can realize certain efficiencies with minimal software development overhead.

Augmented reality holds boundless promise for efficiencies within an enterprise, and we are on the cusp of being able to actually realize those efficiencies especially with Microsoft HoloLens and the Universal Windows Platform.

Microsoft has also recently announced the launch of Windows Holographic which allows third parties to create their own augmented (or mixed) reality hardware using Microsoft’s software.

Virtual reality and augmented reality each have their future killer applications and killer niches. Hopefully you can find them within your business and start to realize greater efficiencies and value with augmented reality, and possibly virtual reality, based solutions.

Increase Local Reasoning with Stateless Architecture and Value Types

It is just another Thursday of adding features to your mobile app.

You have blasted through your task list by extending the current underlying object model + data retrieval code.

Your front-end native views are all coming together. The navigation between views and specific data loading is all good.

Git Commit. Git Push. The build pops out on HockeyApp. The Friday sprint review goes well. During the sprint review the product manager points out that full CRUD (Create, Read, Update, Delete) functionality is required in each of the added views. You only have the ‘R’ in ‘CRUD’ implemented. You look through your views, think it just can’t be that bad to add C, U and D, and commit to adding full CRUD to all the views by next Friday’s sprint review.

The weekend passes by, you come in on Monday and start going through all your views to add full CRUD. You update your first view with full CRUD; start navigating through your app; do some creates, updates, and deletes; and notice that all of those other views you added last week are just broken. Whole swaths of classes are sharing data you didn’t know was shared between them. Mutation to data in one view has unknown effects on the other views due to the shared references to data classes from your back-end object model.

Your commitment to having this all done by Friday is looking like a pipe-dream.

Without realizing it you are now a victim of code that is not locally reasoned due to heavy use of reference types.

Local reasoning is the ability of a programmer to look at a single unit of code (i.e. class, struct, function, series of functions) and ensure that changes made to data structures and variables within that unit of code don’t have an unintended effect on unrelated areas of the software.

The primary example of poor local reasoning is handing out pointers to single references of classes to many different areas of the application. These pointers to single references are then mutated by all the clients that were handed the reference to the shared instance.

Adding to the pain is the possibility that a reference to your single instance was held by a client, then mutated by a parallel running thread.

Code that you thought that was one-pass, single-path, easy-peasy has now become a spider web of live wires blowing in the wind that short circuit and spark at random times.

With the recent rise of Swift, there has been a movement to use value types to avoid all that sparking caused by random mutation within reference types.

For so long, we were conditioned to use classes types for everything. Classes seem like the ultimate solution. Lightweight hand off of references (i.e. pointers), instead of allocation and copying of all internal members across function calls. The ability to globally mutate in one shot via known publicly exposed functions. It all looks so awesomely ‘object-oriented’. Until you hit the complex scenarios that a CRUD based user interface has to implement. Suddenly that awesome class based object model is being referenced by view after view after subview after sub-subview. Mutation can now occur across 10s of classes with 10s of running threads.

Time to go way too far up the geek scale to talk about possible solutions.

A classic trope in many Star Trek episodes was something sneaking onto the ship. Once on the ship, the alien / particle / nanite / Lwaxana Troi would start to wreak havoc with the red shirts / warp core / main computer / Captain Picard’s patience (respectively).

By using nothing but classes and reference types, even with a well defined pure OO interface to each, you are still spending too much time with your shields down, and letting too many things sneak onto the ship. It is time to raise the shields forever by using value types as an isolated shuttlecraft to move values securely between the ship and interested parties.

Apple has been emphasizing the use of value types for the past two years via their release of the Swift language.

Check out these WWDC 2015 /2016 presentations which emphasize the use of Swift value types as a bringer of stability and performance via using the language itself to bring local reasoning to code:

Apple has even migrated many existing framework classes (i.e. reference typed) to value types in the latest evolution of Swift 3.0. Check out the WWDC 2016 Video: What’s New in Foundation for Swift.

At Minnebar 11, Adam May and Sam Kirchmeier presented on Exploring Stateless UIs in Swift. In their presentation, they outline a series of techniques using Swift to eliminate common iOS state and reference bugs. Their techniques meld together stateless concepts from React, Flux, and language techniques in Swift, to dramatically increase the local reasoning in standard iOS code.

Riffing off of Adam and Sam’s presentation, I came up with a basic representation of stateless concepts in Xamarin to solve a series of cross-platform concerns.

The rise of using value types as a bringer of local reasoning to code is not just isolated to Apple and Swift. The recognition that emphasizing local reasoning can eliminate whole swaths of bugs is also burning it’s way through the JavaScript community as well. React.js and underlying Flux architectural concepts enforce a one-way-one-time push + render to a view via action, dispatcher, and store constructs. React + Flux ensure that JavaScript code doesn’t do cross-application mutation in random and unregulated ways. The local reasoning is in React + Flux is assured by the underlying architecture.

Even the PUT, DELETE, POST, and GET operations lying underneath REST based web interfaces are recognition of the power of local reasoning and the scourge of mutation of shared references to objects.

C# and .NET languages are very weak on the use of value types as a bringer of local reasoning. For so long Microsoft has provided guidance along the lines of ‘In all other cases, you should define your types as classes‘, largely due to performance implications.

Have no fear, you can bring similar concepts to C# code as well via the struct.

The one draw back of C# is the ease with which the struct can expose mutators in non-obvious ways via functions and public property setters.

Contrasting with C#, Swift has the ‘mutating‘ keyword. Any function that will mutate a member of a struct requires the ‘mutating‘ keyword to be attached in order for compilation to succeed. Unfortunately, there is no such compiler enforced mutability guard for structs in C#. The best you can usually do is to omit the property set from most of your struct property definitions, and also use private / internal modifiers to ensure the reasoning scope of your type is as local as you can possibly make it.

The next time you see a bug caused by a seemingly random chunk of data mutating, give a thought to how you may be able to refactor that code using stateless architecture concepts and value types to increase the local reasoning of all associated code. Who knows, you may find and fix many bugs you didn’t even realize that you had.

Practical Data Cleaning Using Stanford Named Entity Recognizer

I enjoy learning about all of the events in and around World War II, especially the Pacific theater.

I was reading the book Miracle at Midway  by Gordon W. Prange (et. al.) and started to get curious about the pre- and post- histories of all the naval vessels involved in the Battle of Midway.

Historically, so many people, plans, and materials had to merge together at a precise moment in time in order for the Battle of Midway to be fought where it was and realize its radical impact on the outcome of World War II.

I got curious as to the pre- and post-history of all the people and ships that fought at Midway and wanted a way to visualize all of that history in a mobile application.

As a software guy, I started digging into possible data sources of ship histories. I was looking forward to constructing my vision of a global map view onto which I can overlay and filter whole historical data sets in and around naval vessels such as:

  • Date of departure
  • Date of arrival
  • Location in lat / long of source / destination
  • Other ships encountered during mission
  • Mission being performed
  • People who commanded and served
  • People who may have been passengers or involved in the mission some way

I am not anywhere near constructing my vision of fully time and space dynamic filtered ship history maps yet. The keyword being: Yet.

This story details one tiny step I undertook to try and get to my goal against a found data source.

 

Finding a data source

I found a whole set of plain text ship histories as part of the Dictionary of American Naval Fighting Ships (DANFS).

An awesome Pythonista had already ripped the DANFS site content into a SQLite database which contains all 2000+ ship histories. – Thanks jrnold!

The ship histories are interesting to read but are not organized in a machine friendly format. The histories are largely just OCR’d text + a little HTML styling markup which was embedded as a side effect of the OCR process. I fully acknowledge that beggars can’t be choosers when it comes to data sources of this type. DANFS is as good a place to start as I could find this side of petitioning the National Archives for all naval ship logs. As any data scientist will tell you: You will spend 90% of your time cleaning your data, then 10% of your time actually using your data.

In this case, most of the ship histories are written in chronological order from past to present (once you separate out some header text regarding the background of the naming of the ship). In theory, if we just had a way to markup all the location names in the text, we could process through the text and create an in-order list of places that the ship has visited. Please note: All this is largely naive. It’s actually much more complicated than this, but you have to start somewhere.

Needless to say: I don’t want to do location identification by hand across 2,000+ free text ship histories. We are talking about the entire history of the American Navy. Having to classify and separate out locations by hand, by myself, is a huge task!

It turns out that there is a library that will markup locations within the text via an already trained machine learning based model: The Stanford Named Entity Recognizer (or Stanford NER)

Note: I continue in this post by using Stanford NER due to its C# / .NET compatibility. You may also want to check out Apache openNLP. Also keep in mind that you can also pre-process your data using NER / Natural Language Processing (NLP) and feed the results into ElasticSearch for even more server side search power!

DANFS History Snippet Before being run through Stanford NER:

&lt;i&gt;Abraham Lincoln &lt;/i&gt;returned from her deployment to NAS Alameda on
9 October 1995. During this cruise, the ship provided a wide variety of
on board repair capabilities and technical experts to 17 American and
allied ships operating in the Middle East with limited or non-existent
tender services. In addition, the Communications Department completed a
telemedicine video conference with Johns Hopkins Medical Center that
supported X-ray transfers and surgical procedure consultations.

 

DANFS History Snippet After Stanford NER:

&lt;i&gt;Abraham Lincoln&lt;/i&gt;
returned from her deployment to
&lt;LOCATION&gt;NAS Alameda&lt;/LOCATION&gt;
on 9 October 1995. During this cruise, the ship provided a wide
variety of on board repair capabilities and technical experts to
17 American and allied ships operating in the
&lt;LOCATION&gt;Middle East&lt;/LOCATION&gt;
with limited or non-existent tender services. In addition, the
&lt;ORGANIZATION&gt;Communications Department&lt;/ORGANIZATION&gt;
completed a telemedicine video conference with
&lt;ORGANIZATION&gt;Johns Hopkins Medical Center&lt;/ORGANIZATION&gt;
that supported X-ray transfers and surgical procedure consultations.

Pretty cool, huh?

Notice the ORGANIZATION and LOCATION XML tags added to the text? I didn’t put those into the text by hand. Stanford NER took in the raw text from the ship history of the U.S.S. Abraham Lincoln (CVN-72) (Before) and used the machine learning trained model to markup locations and organizations within the text (After).

 

Machine Learning: Model = Sample Data + Training

From the documentation regarding the Stanford Named Entity Recognizer:

Included with Stanford NER are a 4 class model trained on the CoNLL 2003 eng.train, a 7 class model trained on the MUC 6 and MUC 7 training data sets, and a 3 class model trained on both data sets and some additional data (including ACE 2002 and limited amounts of in-house data) on the intersection of those class sets. (The training data for the 3 class model does not include any material from the CoNLL eng.testa or eng.testb data sets, nor any of the MUC 6 or 7 test or devtest datasets, nor Alan Ritter’s Twitter NER data, so all of these remain valid tests of its performance.)

3 class: Location, Person, Organization
4 class: Location, Person, Organization, Misc
7 class: Location, Person, Organization, Money, Percent, Date, Time
These models each use distributional similarity features, which provide some performance gain at the cost of increasing their size and runtime. Also available are the same models missing those features.

Whoa, that’s an academic mouthful. Let’s take a deep breath and try to clarify the Stanford Named Entity Recognizer pipeline:

NER

 

All the parts at the top of the diagram labelled ‘Machine Learning’ (with black borders) are already done for you. The machine learning + training output is fully encapsulated and ready for use via models available for download as a handy 170MB Zip file.

  • When you crack open the model zip file, you will see the classifiers sub directory which contains the 4 class Model, 7 class Model, and 3 class Model:
    • Classifiers
    • It is one of these 3 models that you will use to initialize the CRFClassifier in your client code depending on your needs.

The parts at the bottom of the diagram in the ‘Using NER Classifier’  space (with purple borders) are what you will do in your client code to get person, location, and organization markup placed within your text. Check out the instructions on how to do this in C# / .NET.

The key to the client code is the CRFClassifier from the Stanford.NER.NLP NuGet Package. The CRFClassifier takes in your text and the trained model output file, from the above zip file, to do classification of person, location, and organization within your text.

Sample C# code:

 // Loading 3 class classifier model
            var classifier = CRFClassifier.getClassifierNoExceptions(
                classifiersDirecrory + @&quot;\english.all.3class.distsim.crf.ser.gz&quot;);

            var s1 = &quot;Good afternoon Rajat Raina, how are you today?&quot;;
            Console.WriteLine(&quot;{0}\n&quot;, classifier.classifyToString(s1));

 

Cleaning the data

Cleaning a set of OCR’d free form text files and converting it to a format that makes it easy for a developer to process it using code can be difficult. Data cleaning is an ad-hoc process requiring use of every data processing, and possibly coding, tool you have in your tool belt:

  • Regular Expressions
    • String replacement
  • XML DOM traversal
    • Node + attribute add
    • Node + attribute remove
  • Standard string split and replacement operations
  • Log to console during processing and manually create filtered lists of interesting data.
    • These lists can also be fed back as training data for future machine learning runs.
    • Filter values on created lists from source data.
  • Use of trained models

All that said, Named Entity Recognition gives you a fun and solid starting point to start cleaning your data using the power of models from machine learning outputs.

I highly recommend using Stanford NER as one or more stages in a pre-production data cleaning pipeline (especially if you are targeting the data for rendering on mobile platforms).

Beware: your data may have a series of false positive PERSON, ORGANIZATION, or LOCATION tags written into it by Stanford NER that may have to be filtered out or augmented by additional post-processing.

In my case Stanford NER also marks up the names of people in the text with PERSON XML tags. You may notice that Abraham Lincoln above is part of a ship name and also the name of a person. In my first run of this ship history text through Stanford NER, the text Abraham Lincoln was surrounded by PERSON XML tags. Having 1,000 occurrences of the person ‘Abraham Lincoln’ in a ship history about the U.S.S. Abraham Lincoln is probably not very useful to anyone. I had to run a post-processing step that used the XML DOM and removed any PERSON, ORGANIZATION, or LOCATION tags if the parent of those tags was an ‘i‘ tag. I found that ‘i‘ tag was written into the original history text from DANFS to indicate that it is a ship name so it was the easiest (and only) data marker I had to aid in cleaning. The same problem would occur for the U.S.S. Arizona, U.S.S. New Jersey, U.S.S. Missouri,  and other text where locations, people, or organizations were used as ship names.

I fully intend on trying to use machine learning, and alternate training sets for Stanford NER, to ensure that PERSON, ORGANIZATION, and LOCATION tags are not written if the text is within an ‘i’ tag (but haven’t done this yet).

As I was cleaning the data using Stanford NER I ran across instances where the rank or honorific of a person was not included within the PERSON tag:

Capt. &lt;PERSON&gt;William B. Hayden&lt;/PERSON&gt;

My initial implementation to include the honorific and/or rank of the person was problematic.

  • After the NER stage, I scan through the XML output using System.Xml.Linq (i.e. XDocument, XNode, XElement) looking for any PERSON tags.
  • Using the XML DOM order I went to the previous text node right before the start PERSON tag.
  • I then displayed the first 3 words, as split by a space, at the end of the preceding text node.

In my naiveté I figured that the honorific and/or rank would be at worst about 50 different 3 word variations. Things like:

  • Private First Class
  • Lt. Col.
  • Col.
  • Gen.
  • Maj. Gen.
  • Capt.

Well imagine my astonishment when I discover that the DANFS data doesn’t only contain the names and honorifics of military personnel; but of passengers and people in and around the history of the ship:

  • Senator
  • Sen.
  • King
  • Queen
  • Princess
  • Mrs.
  • Representative
  • Congressman
  • …. and many, many, many more

In addition I found out that almost every possible abbreviation exists in the ship history text for rank:

  • Lt.
  • Lieutenant
  • Lieut.
  • Commander
  • Cmdr.

My second pass at determining the honorific of a person may just involve a custom training stage to create a separate ‘Honorific’ model trained by the set of honorific data I have derived by hand from the whole DANFS text data set.

As stated above: Data scientists spend way too much time cleaning data using any technique that they can. On this project, I found that I was never really ever done extracting new things from the data, then cleaning the output data some more in a never ending loop.

I hope in the future to provide all the sample cleaned DANFS data, my data cleaning source code, and a sample mobile app that will interactively render out historical ship locations.

Universal Windows Platform – The Undiscovered Country for Windows Desktop Apps

It is unfortunate, but I think Dr. McCoy from Star Trek: The Original Series said it best: ‘Windows 10 Mobile is dead, Jim“…. and just to pile on with one more.

Buried underneath the red shirt like death of Windows 10 Mobile lies the amazing Universal Windows Platform (UWP).

The developer capabilities of the Universal Windows Platform have been documented in many a location.

UWP provides:

  • Full developer parity at the API and framework level across all flavors of Windows 10, Xbox One, HoloLens, and Windows 10 Mobile devices.
  • A simplified install model via APPX bundles.
  • A per-app separation of registry and file systems.
  • XAML controls ‘just work’ across all Windows 10 / UWP devices.
  • Targeting the UWP API set ensures that your app works across Windows 10 / UWP devices.

In my experience, there is nothing weirder than seeing your Windows 10 / UWP app just work on an Xbox One,  in a virtual projected rectangle via a HoloLens, and via mouse and keyboard on a standard Windows 10 PC.

 

NetToday-UWP

The above diagram is from the great Build 2016 presentation – .NET Overview given by Scott Hanselman and Scott Hunter.

UWP fits within a completely different vertical of development than the existing Windows Desktop technologies of Windows Presentation Foundation (WPF) and Windows Forms. As such UWP has access to the more modern feature set common to Windows Store applications. UWP has both duplication and subsets of API / XAML features that exist within WPF.

Right now may be a good time to take a look at your existing Windows Desktop apps and assess if they can be moved to UWP.

A UWP port of your legacy desktop app will allow access to modern UWP features:

  • Live tile support
  • Push notifications
  • Cortana
  • Background Downloader

There are 3 primary techniques to bring your current legacy desktop app to UWP:

  • Project Centennial
  • Straight UWP + Portable Library Breakout
  • Brokered Windows Runtime Components

Microsoft’s Project Centennial can be thought of as a UWP enabling wrapper for your existing Windows Desktop app. You may be able to repackage your existing Windows Desktop application and run it as a UWP app via a Project Centennial wrapper on the forthcoming Windows 10 Anniversary Update. As with all huge development efforts, there may be some caveats to using Project Centennial to package your Windows Desktop app as a UWP app (i.e. driver installation for custom hardware devices is not supported).

Update: Project Centennial based apps can go-live against the Windows 10 Anniversary Update as of August 2, 2016.

A straight UWP + Portable Library code breakout of your app could allow the features of existing Windows Desktop apps to be migrated in a straightforward way to Windows Store / UWP applications.

Note that the main UI layer of your existing Windows Desktop app is  not easily converted to UWP.

  • WPF XAML is not the same as UWP XAML with differences in control support, touch events, and lack of Triggers in UWP XAML.
  • UWP + XAML is a radical departure from WinForms. The headaches you may have encountered going from WinForms to WPF are only magnified in a WinForms to UWP conversion due to no real access to the underlying Win32 API set.

Brokered Windows Runtime Components allow a UWP app to tunnel between the UWP API and the Windows Desktop environment to run full .NET Framework 4.6 code on the Windows Desktop side. The downsides of this approach are many, the most limiting of which is that this technique can only be used on sideloaded UWP apps, not on UWP apps which are meant to be distributed via the Windows Store.

Interesting and non-obvious outcomes I have discovered during Windows Desktop app to UWP app code migrations:

  • You discover that there may exist possibilities for your app on Android and iOS with your existing code via third party Portable Libraries and your own code as Portable Libraries running via Xamarin.
  • I have found that having UWP compatible code, and Portable Libraries, allows you to create live prototypes that can bring your business services into an augmented reality future on Microsoft’s HoloLens, and a passable default experience on Xbox One.

We are at a true maturity point across the Windows, iOS, and Android ecosystems when it comes to repurposing managed code across all the .NET runtime variants via portable libraries. There is no better time than now to dust off that old-school Windows Desktop code and see how you may be able to put it to great new purposes on modern devices via UWP / .NET Core and/or Xamarin for iOS and Android.

I believe that the great things that Microsoft has shipped via their Universal Windows Platform have been lost in the 5 year sideways journey to catch up to Android and iOS. That sideways journey started with Windows CE variants in the form of Windows Phone 7; through the slight update of Windows Phone 8;  the flub of full Windows 8;  the long forgotten Windows RT / Windows On ARM strategy; through the full revamp (and necessary stepping stone) that was Windows Phone 8.1; and onto today’s real deal: Windows 10 / Universal Windows Platform.

Microsoft always nails something the third time they take a stab at it. In this case it feels like they have something real with UWP after 5 or 6 stabs and a whole bunch of missteps.

In a recent post to ArsTechnica, Peter Bright, master writer of all things Microsoft, wrote up a concise technical history regarding the path that Microsoft took in order to arrive at the current Universal Windows Platform destination.

With the maturity and solidity of UWP, and the positive mindshare linkage to Windows 10 via marketing and branding schemes, I think it is right to declare that UWP gets Microsoft out of the ‘Bull wandering through a china shop’ phase. With UWP, Microsoft is firmly into ‘We have something that is truly a firm third place contender, and primed for growth’ territory.

 

Windows Desktop app development has always been a major pain when it comes to:

  • Installation and Upgrades — The only clean way to do this was with Windows Installer, and the only clean way to do Windows Installer is Windows Installer XML (just look at that crazy XML schema, and bizarre otherworldly installer object model — Thank goodness for Votive though!)
  • COM + Win32 API / PInvoke — No matter what you do, it always seems like you have to resort to COM or  Win32 / PInvoke for something.
  • Device Access — It seems as if you need to know DIFx to install a driver, then have to know way too much about Windows Driver Model, Bluetooth sockets, or UMDF just to get read and write streams to your device from a program.
  • Security — How many times have you seen the ‘Red Banner Dialog of Doom‘ while installing something? Only to be told to click ‘Install this driver anyway’?

To solve the above platform deficiencies, the Windows platform needed a simplified app development model along the lines of Android and iOS. They have that scheme today in the form of UWP, with far superior tooling in Visual Studio 2015, and far easier app coding with C# / XAML.

On previous projects, the primary things holding back any sort of Windows Desktop to Windows Store / UWP migration were:

  • Third Party Software Libraries — Tied directly to full .NET Framework 4.x and usable only in Windows Desktop apps.
  • Specialized device access (i.e. USB / COM Port / Bluetooth) — Oft times these devices came from third parties as well as an access library written for the device that was only usable on Windows Desktop apps.

The third party software library realm has softened quite a bit. Many third party libraries that were directly tied to full .NET Framework 4.x have been open sourced (allowing you to pull needed bits out), or turned into Portable Libraries for use in UWP (and other mobile platform) applications.

If your existing Windows Desktop app requires integration with devices, you may have some issues with a straight Project Centennial wrap of your app. You may have to adapt and rewrite your device access layers against UWP APIs.

Specialized device access has matured greatly within UWP / Windows Store apps with the release of Windows 10 / UWP. There are now fully supported UWP API sets which allow:

A more friendly environment has been created in the world of Windows application development via a series of fortunate events:

  • The depths of despair that came with Windows 8 are gone with the rise of Windows 10.
  • The purchase of Xamarin by Microsoft.
    • Along with the support guarantees and savvy short term moves made since the acquisition.
  • Maturity of open source code + flexible licensing of that code
  • The rise of the UWP, iOS, and Android compatible Portable Library.

In general, the environment is such that it is a great time to look at a way to fracture and/or pivot your Windows Desktop apps onto UWP, and possibly even iOS and Android via Xamarin. Give it a try, you might be surprised how far your existing old-school code can stretch across modern devices.

Your pivot to get your Windows Desktop application working on Universal Windows Platform may even work on Surface devices from The Next Generation.

Mobile App Services

Starting on a mobile app can be a daunting proposition.

Your stakeholders at Rex’s Gym really need this app to help drive customer retention, promote the business, and make payments a breeze.

Getting the development environment set up, learning a new language, understanding a wholly different API for screen layouts per-platform, having to go to the Apple Store and buy a Mac (which is the absolutely last thing you thought you would ever do).

Stress, headaches, pain, mayhem, churn! You haven’t even started yet! Then suddenly a manager says that you need:

  • Source control
  • bug tracking + task tracking
  • A reliable build system
    • Have all Alpha and Beta builds be installable to any mobile device within the organization
  • in-app usage tracking and phone-home crash logging
  • push notifications
  • OAuth2 based login to existing back-end systems
  • in-app payment + in-app recurring subscriptions
  • … and to top it off your boss wants an integrated SMS based text message e-mail gathering t-shirt promotion

You go out and look at the Apple, Google, and Microsoft references for all that stuff. Have a slight heart attack after scanning through hundreds of pages of doco. Go home. Have a beer. Lose some sleep. Go for a run. Then you stumble into this blog post that will give you pointers to make all that stuff easier and more reliable than if you had to do it all on your own.

Please note: Where possible I have tried to make service choices that have libraries and connectors that work across iOS (Obj-C / Swift), Android (Java), UWP / Windows 10 / Windows 10 Mobile (C# / XAML), and cross-platform mobile app solutions like Xamarin or Ionic / Cordova. As with all things, your mileage may vary.

 

Source Control Options – Just use Git, but hosted where?

Source control for projects now has many cloud based options. Most of them also integrate to task tracking and documentation services in diverse ways.

Depending on your preferences and cost profile, there are many awesome options for source code hosting. I have found that the options tend to settle down to 2:

A dark horse here would be to checkout Visual Studio Team System to host your Git repo.

Be sure to check out our other SuperDevelopment article regarding source control best practices for a great overview.

 

Bug / Task Tracking

For bug / task tracking there are many options. These options may be dependent on choices you make for source control hosting.

If you want industrial strength task management + work items integrated with your source control, then consider Atlassian + Bitbucket + Jira.

If you want something more lightweight, then go outside of the box and take a look at Pivotal TrackerTrello, or Github issues.

I think that the key here is to create an always available, fully transparent, one-stop-shop for all bug and task tracking. Things like e-mail, Slack, and other communication schemes tend to fall apart really quickly when the number of people working on a mobile project is greater than (or even sometimes equal to) 1.

 

Mobile App Build Server Options

You took the morning and you settled on source control and a good task tracking system. The first task you put in your tracking system is: Setup a build server.

As a corollary to setting up your build server, you would also really like options for continuous integration and continuous builds as you commit changes. You are just one person, and you have 15 stakeholders. The more you can automate, and the more builds you can get out there, the quicker you will solidify the decisions of those 15 stakeholders.

For mobile app build systems + full app CI you have a series of good options:

  • Bitrise – A fully cloud hosted build system that lets you snap together your build process. Has integrations to Github and Bitbucket. It even lets you build those pesky iOS projects because they handle a virtual in-the-cloud Mac to handle the Xcode command line stuff. With Bitrise you can be up and running in under a day.
  • Jenkins – An on premise solution. Only tackle this if you have a spare Mac laying around. Could take 4-5 days to rev up Jenkins and make it reliable. Plus ongoing maintenance. Overhead is worth it if need a fully on site solution.
    • The recent shipment of Blue Ocean by the Jenkins team really starts to make the Jenkins build system look like a great option.
  • Visual Studio Team System — If you have Microsoft Azure credits, then take a serious look at VSTS for mobile app building.

 

 

Special Note: Fastlane Toolset

You start revving up your builds, and start to notice some badness: The Apple Developer Portal has all these crazy requirements for certificates, provisioning profiles, app IDs, Ad-Hoc provision, App Store provision … Arrrgh!

Fastlane match is here to solve all that crazy cert, provisioning profile, app ID stuff by tooling up against these guidelines.

Fastlane also solves many other problems:

  • Pilot – Command line publishing of your iOS builds to iTunes Connect / TestFlight
  • Snapshot – Creates localized screenshots of your app which you can use for per-country app submission to the Apple App Store.
  • and so much more.

To document all the greatness that is Fastlane would take hundreds of pages. Suffice it to say, relax, lean back, and learn to love Fastlane.

 

Alpha / Beta Build Distribution – HockeyApp

While revving up your build system, you took a look at your next line item: Easy distribution of Alpha and Beta builds to any mobile device, usage tracking, and phone-home crash logs. In this realm, I really recommend:

  • HockeyApp – It really does it all.
    • Easy integration path as an app binary output destination with all of the above build systems.
    • On-device app which creates a mini-app store for distribution of your Alpha and Beta builds to your internal customers.
    • Easily integrated usage tracking and crash log reporting API for iOS, Android, Windows, and Xamarin based applications.
      • You can even use all the usage tracking and crash log reporting for your shipping application, not just for Alpha and Beta builds.

Other app tracking services are out there, but the ‘one stop shop’ of HockeyApp, in addition to Microsoft backing the product, makes HockeyApp a very slick solution especially if you are in a hurry.

 

It has been a couple of days, and you have all your build infrastructure revved up. Your initial Xcode project is up in Git source control, the app is building automatically on every commit / push via Bitrise, all that provisioning profile / certificate stuff is on auto-pilot with Fastlane – match, and you even have your first ad-hoc build installing to your personal iPhone via the HockeyApp app. All is right with the world.

Now you tackle a couple of screens from your requirements, and meet your first sprint deliverable. Your stakeholders are amazed that you were able to deliver the app to their specific devices after only 2 weeks.

The new sprint comes up, and you look at some of those bigger items on that task list. The next task you know you need to tackle is push notifications.

 

Push Notifications – Urban Airship

IMHO the first place to go is Urban Airship.

There are other services out there, but nothing scales out and up with your business messaging needs quite like Urban Airship does.

In my experience, I was able to hand over a restricted role based login from Urban Airship to my primary stakeholders and watch as they created and sent push notifications out to the app with ease.

Urban Airship’s per-platform integration options are second-to-none as far as ease of app integration.

Everything may depend on your cost requirements especially as it relates to projected number of users and needed level of messaging.

It took a couple of days, but you now have Urban Airship integrated, a development messaging environment is now delivering push notifications to your app, and you are ready to tackle the next item.

Alternatives to Urban Airship for push notification handling include:

 

In-app OAuth2 Login

The customers login to the current web site using a series of web based OAuth2 login schemes against a custom back-end server. Your app has to integrate into those same login schemes. Even worse: They also want you to be able to handle login via Facebook, Google+, and Twitter.

Just to pile on, your stakeholders demand that you find a way to not ‘just embed a web view’ to do the primary OAuth2 based user login. It’s OK, they might understand you have to show a web view for Facebook, Google+, Twitter, and other social network pairing but they don’t want to see a web view for the primary app login. As much as possible your stakeholders want pure native views for all logins.

There are frameworks out there to help you do this, but they can seem scattered and hard to integrate per mobile platform per social network. Windows 10 / UWP especially seems very complicated as far as doing a native OAuth2 based login, getting back the user roles by cracking open a secure JWT, all without a web view.

There is a third-party service that may be able to help. In this case the service is Auth0 (pronounced Auth Zero). Auth0 provides a wrapper / helper depending on server login type as well as a full suite of per-mobile platform front end login API connectors.

The mobile app integration for Auth0 divides into 2 general schemes:

  • Lock — Auth0 presents all needed views, web views, and other login UI for you.
  • Client API — Allows you to code a fully native UI yourself and just passthrough username / password and other login configuration parameters via a pure API.

Worthy mention: Amazon Cognito.

 

Intermission

You are about 3-4 weeks in so far. You have major app screens up. Your boss loves the notifications, she even gets them on her Apple Watch. Your product managers are floored at the awesome login integration. You have already forgotten about all that build / signing / provisioning / app distribution stuff because all you do is commit, and the build pops out of HockeyApp automagically and installs to your internal test and stakeholder devices.

Well there was that whole crazy Ad-Hoc build headache with adding UDIDs of the designer’s main devices but Fastlane made that easy via register_devices

 

In-app Payment + In-app Subscription

Your stakeholders are getting itchy. They want $. They want to make sure that you are looking at in-app payment and in-app subscription schemes. Next sprint comes around, and you start to look around at options (I mean, you need to get $ too, right?).

Your business doesn’t have a payment system in place that you can easily integrate with. You start to sweat a little. Until you find Braintree Payments.

You look at the Braintree client list and realize that you have bought stuff through them and never even realized it.

You sign up for their sandbox. Do some preliminary app integration. The stakeholders are kinda ‘Meh.’ when it comes to a few issues. Then you go for the throat and show an in-app  Apple Pay integration with your sandbox. That demo just showed you buying a recurring membership to Rex’s Gym with your thumbprint. How is that not made of pure awesome!

You also looked around and found out that Urban Airship has a similar system.

 

T-shirt e-mail gathering promotion?

You pop your head up at 6PM on a Friday after an all day coding frenzy, go get a Diet Mountain Dew. Close your eyes. Stretch. Open your eyes. Wha… your product manager is dangling a t-shirt in front of your face.

It is a cool t-shirt. You are working for Rex’s Gym and the design is awesome.

Your product manager says she wants you to fully engineer a t-shirt promotion from within the app. The end user is going to text their e-mail address via SMS to a phone number, it is your job to receive the text, process the e-mail address out of the text, and send out an invite e-mail from Printfection that allows the user to get their t-shirt.

She hands you the number that receives the SMS messages, and the Printfection credentials, and tells you go get it done.

So you look around and realize that Printfection has an API. But the last thing you want to do is code up a whole bunch of the stuff in the app, or even on a hosted server, to hit that API and manage the whole workflow of this promotion.

It is 6PM. It is Friday. Code haze has entered your brain from the all day binge. A sunny weekend awaits you! Help!

So you look around some more and find Zapier.

Zapier lets you setup a ‘zap’ that fully automates the whole flow:

  • SMS is received
  • Scan SMS for e-mail address
  • Log e-mail address in a Google Sheet
  • Send Printfection invite to e-mail address.

… all of that with no code to create, maintain, or host.

Special thanks to Ben Peter of Zapier for his presentation at minnebar 11 for this one!

 

Worthy Mentions

Azure – iOS Developer Center – Can handle push notifications, single sign-on, and many other data handling scenarios.

Firebase – Owned by Google. Handles many back-end services, including push notifications, usage tracking / diagnostics, and much more.

 

Apple WWDC – The Worth of Being There

In this era of travel budget crackdowns and higher than normal oversight into technology budgets, the iOS software engineer can get lost in the shuffle when it comes to training time and opportunities to talk with other members of the iOS development community.

Due to the rise of the popularity of Apple’s iOS platform, Apple WWDC (Worldwide Developer Conference) has become the central rallying point for iOS developers. It seems that the entire iOS community gathers in San Francisco over the week of WWDC. Many developers go to San Francisco over the week of WWDC even if they don’t have a ticket to the conference.

A ticket to WWDC costs $1599. Since 2014 you only get to purchase a ticket if you get picked during the lottery phase of enrollment. In my opinion, this is a very fair system considering the high number of developers who wish to attend.

Due to the high number of people going out to San Francisco over WWDC week, the hotel prices during WWDC week have almost tripled since 2011. John Siracusa,  of Mac OS X Review and Accidental Tech Podcast fame, has equated a night of sleep in a hotel during WWDC as $ equivalent to an Apple Watch. You sleep 1 day, that’s 1 Apple Watch (Sport). You sleep 2 days, that’s 2 Apple Watches… From Sunday through Friday you may have to plan on spending $1600 just to sleep.

The airfare remains affordable. Well, as affordable as airfare can be from your home location to SFO.

Before you have so much as sipped even one $7 Blue Bottle coffee (totally worth it, just do it!), the conference has already cost around $5000 dollars.

To a cost conscious manager $5K as a baseline price for a week spent at a conference can seem like a pretty steep price to pay.

I would submit that the $5K cost is a really good value if you know how to attack WWDC.

Summary:

  • WWDC is not anything like Microsoft Build.
  • You get insight into the hierarchy at Apple that you can’t get anywhere else. Especially the relationship between you (the developer), the Apple Evangelist, and the Apple Engineer.
  • The all-day technology-specific labs that Apple runs at WWDC may be the only way you are ever going to get face time with Apple personnel that have specific knowledge in your app subject area.
  • You get inspiration and insight from the special lunch sessions.
  • You can take advantage of the 1 to 5 ratio of Apple personnel to attendees.
  • You agree to a Non-Disclosure Agreement regarding session content. This turns out to be a great thing because you get exposed to stuff that you never experience otherwise.
  • There is now plenty of good content presented at other nearby venues at the same time as WWDC.
  • Sometimes you are at the epicenter of a major shift in Apple’s technology stack.
    • Those who attended WWDC 2014 got the boost of being at the center of the Swift rollout. The Swift rollout surprised most Apple employees, not to mention many attendees.

Note: During this blog post you may notice that I am being completely vague and only giving you an impression or insight with no detail. When you attend WWDC you agree to a Non-Disclosure Agreement. This means that you can’t talk about details or technical information related to content presented there. Because of this, I can only give you my impressions of my experience at WWDC.  This NDA has loosened in recent years, but I think it is still good to keep the attitude: ‘What happens at WWDC, stays at WWDC’.

 

WWDC – General Structure

WWDC starts with a day 1 keynote. This is a classic Apple keynote.

The keynote is usually at 10AM Pacific Time.

For WWDC attendees who want to be ‘in the room’ this means that you may have to line up at about 7AM. You are let in the building around 8:30AM. Then you wait. For almost an hour and a half. In a hallway with no chairs.

They open the doors to the main keynote area, everyone finds a seat, and the keynote begins.

The keynote ends about an hour later. Everybody leaves for lunch. Then the remainder of the first day is spent on a more engineering focused deep dive set of talks called ‘State of the Union’.

The rest of the week has standard conference tracks scattered around the many rooms all grouped by technology.

Many of the sessions are repeats of sessions from previous years with a little more content added and/or clarified.

I am pretty sure I am not divulging too much info here as all of these sessions and the schedule are available via the WWDC app.

 

WWDC != Microsoft Build

In order to really ‘grok’ how Apple’s WWDC is dramatically different from other technology conferences you really need to attend. I have been extremely fortunate in that I have been at 2 WWDCs and a series of Microsoft conferences. I would also point out that the WWDC differences also show how Apple is a dramatically different company (especially from Microsoft).

Apple WWDC is dramatically different from Microsoft conferences in so many ways.

  • Degree of Preparation
    • WWDC is highly polished. Every session. All the time. It feels like all the session speakers have rehearsed their material for weeks. It feels like all the presented material in all the sessions has been vetted by the Evangelist (and possibly marketing) hierarchy at Apple.
    • Microsoft Build has well polished keynotes, but the per-session material can feel a little ‘slap dash’. In this way Microsoft Build sessions feel more genuine but aren’t as watchable as Apple WWDC sessions.
  • Openness
    • Contrary to the initial impression you may get, WWDC is fairly open. It is more open once you understand the role of the Apple Evangelist as a gatekeeper. Do not try to get the time or attention of a presenter at a given session. Go to the labs. The labs are largely a free zone where engineering conversations flow freely between attendees, Evangelists, and Apple engineers.
      • Apple presenters do not usually hand out their e-mail or contact information as their public exposure seems closely regulated by the Evangelists.
    • Microsoft Build is completely open with engineers and members of the Microsoft hierarchy handing out Twitter handles and work e-mail addresses.

 

Insight into hierarchy and roles – You, Apple Evangelist, Apple Engineer

The Apple WWDC session structure is very rigid. If you analyze a series of Apple WWDC session videos you start to notice this rigid structure:

  • Speaker shows title slide of session with their name on it for about 5-15 seconds. — No e-mail address, Twitter handle, or other contact info of the engineer will appear on the slides.
    • Note: Sometimes the presenter exposes some contact info, but it is incredibly rare.
  • Speaker proceeds through their material.
  • Last slide shows links to online Apple SDKs, additional WWDC sessions, and the name of an Evangelist for the technology stack that the session was about
  • If the session has a Q&A you will notice that the Evangelist does triage of incoming questions from WWDC attendees. The Evangelist will usually do their best to answer the incoming question, frame the scope of the answer, then will either trust an engineer to answer on stage or redirect you to a lab time to talk more.

The well coordinated Developer to Evangelist to Apple Engineer communication path is unique. The role of Evangelists as gatekeepers to the time and energy of the Apple Engineer leaks through in almost every session you attend at WWDC. Once you understand the rules of the Apple hierarchy in regards to exposure of Apple engineer face time to the developer community,  you are now ready to head into the lab spaces at WWDC get your questions answered and have great conversations with Apple personnel.

 

The Labs are invaluable

The per-technology lab areas at WWDC are invaluable. If you have had electronic contact with an Apple engineer at any time the previous year, you should be able to get face to face contact with that engineer in a lab space. WWDC is, most likely, the only time that you will ever get real face time with an Apple engineer throughout the year.

If you are signed up for any of Apple’s side programs (i.e. Made for iPhone/iPod [MFi], AirPrint, …) then the labs are the one safe space for you to talk engineering specifics in and around that program. The NDAs you sign in order to participate in those programs may largely preclude you from even talking with your peers about what you are really working on. At the specific lab times, and only at WWDC, can you get one-on-one time to talk about the proprietary NDA’d specifics of your application.

When I was at WWDC 2010 there was mention of a series of technologies during the keynote that seemed to overlap with a non-Apple project I was working on. I went down to the labs to talk with Apple people about their project, and related my experience on my non-Apple project. From that experience I  gained a ton of insight and validation regarding my project that I never would have received otherwise.

 

Go to the lunch sessions – They are the hidden gem of WWDC

You just spent a whole morning attending hard technology sessions. You have a numb brain. You have some messages and work to catch up on. A lunch to get eaten. The last thing you may want to do is go to a session over lunch.

The WWDC lunch sessions are where Apple invites people from outside of the company to come in and share how they are applying a specific technology stack to a given creative problem. Every lunch session I have attended have been the best sessions I have attended at any conference. The speakers are engaging, the content is insightful and inspirational. Don’t miss the lunch sessions.

 

There is a 1 to 5 ratio of Apple personnel to attendees

This is one of those stats that I heard then just kind of dismissed. I mean: How could I ever possibly be able to pick out Apple personnel from the 5,000 other attendees at the conference?

It turns out that Apple gives employees specially colored badges which stand out. In addition, most of the Apple employees wear an ‘Apple Store’ like t-shirt. You do end up seeing and talking to quite a few Apple employees. Many Apple employees seek out and engage with developers at WWDC because that is one of the very few times that they can engage with the developer community.

Over breakfast one morning I had struck up a conversation with another engineer about App Store policies. An Apple employee responsible for App Store policies, and some of the technology, overheard our conversation and joined in. The conversation that was sparked was great. In this case the Apple employee had a series of questions that he asked, and largely listened as we talked about a series of things. The feedback we got was invaluable regarding how and why Apple has the App Store policies that they have.

At each session there is a roped off area at the entrance: One side where Apple employees line up, the other side where non-Apple employees line up. All non-Apple employed attendees get into the session first, then the Apple employees get any remaining seats.

I mention this because when you combine the waiting lines in front of a session with the specially colored badges + t-shirts you can get insight into which sessions may be the most interesting.  You get immediate visual feedback for what the Apple engineers think the most important technology topics are just by looking down a wide conference room hallway and looking for clusters of colored t-shirts.

Way back in 2011 I was at WWDC and I was waffling between 3 different sessions being held at the same time. One was about Bonjour networking, one was about Xcode enhancements, and one was really out there (for me at least): LLVM / Clang compiler futures. I walked past the networking and Xcode enhancement sessions and there were people in line, but only 1 or 2 Apple employees. Then I went past the LLVM / Clang compiler futures session, the line of Apple employees waiting to get in was down the hallway and had practically turned the corner. I immediately ducked into that LLVM / Clang compiler futures talk.

 

Non-Disclosure Agreement – Not a bad thing

I hope that I am allowed to tell the above story, because you sign an NDA that states that you cannot talk about any session content from WWDC. This actually turns out to be a good thing.

Apple has relaxed the NDA in recent years. However, that NDA still applies for some sessions, labs, and conversations that you have with Apple employees. If you are under NDA for other Apple programs, then those NDAs are fully binding in the lab areas as well. You may be asked by an Apple employee in a lab area if you are under NDA before they can talk about certain technologies.

I can’t get specific, because I signed an NDA when I attended WWDC 2011, but I think I can dance around enough stuff to give you insight into why that NDA was a good thing.

Over lunch Apple has special sessions given by people from other companies that highlight how they use certain Apple technologies. These special sessions are skewed way more to practical application of technologies for specific businesses or creative endeavors.

At one lunch session we were able to see unreleased creative content, as well as how technology enabled that content to be created. The speaker stated outright that without the NDA we agreed to at the beginning of WWDC that there was no way his management would have allowed him to show this special content and inside info during his presentation.

C# vs. Swift – Iron Man vs. Captain America

In Captain America: Civil War we get to see the ultimate battle between Iron Man and Captain America.

It is a battle of simple gutty defense vs. smart weapons and flashy offense, humility vs. brashness, down in the dirt vs. up in the clouds.

To totally geek it up, the same kind of battle exists in the languages that software engineers use today and I believe this is especially true in the battle of C# vs. Swift.

Don’t worry, this really isn’t a versus type write up. If anything I seek to point out each language’s unique strengths, then show how software engineers can get into the right superhero mindset to really use those strengths, and be aware of the weaknesses, to create great solutions.

 

C# = Iron Man – Strong Flashy Offense

C# is truly the Iron Man in this comparison. It is amazing how much showy weaponry C# can bring to bear to solve problems.

  • Have a problem with your UI freezing up? Bring in Task<T> / async – await, and seamlessly refactor years old code to be fully asynchronous, even within native UIs where thread synchronization is key.
  • Have a super large dataset that you need to tame? Bring in IEnumerable<T> / yield return or Lazy<T> from the deferred execution realm.
  • Have a need to produce arrays and collections that are the result of super smart functional filtering? Bring in LINQ.
  • Need to do smart filtering of XML? Bring XDocument and its magic LINQ over XML powers.
  • Need to do JSON processing? Head on over to NuGet, reference Newtonsoft.JSON ,and get stable, powerful JSON serialize / deserialize behavior and be done in no time.
  • Need to hit a REST service with super specific HTTP header formatting and authentication? System.Net.Http has you covered.
  • Need a cross-platform way to retrieve data from remote sites + Model / View / View Model architecture? Check out ReactiveUI.
  • Need async but inline immediate responses to collection changes? Check out Rx.NET.
  • C# is strongly typed, but you can also use reflection and loosen that strong typing as needed.
  • C# can even be used to generate itself.

You can even do most the above across all platforms. Windows, Windows Mobile, Android, iOS, Linux servers, … C# has you covered, in any way, any time, any where.

There is every gadget and gizmo that you can bring in and bolt on to existing C# code to get your job done. Even the very oldest C# code from 2002 is still supported and easily transformed using Visual Studio, or Resharper tooling into something shiny and modern.

C# has all these weapons, but the lifetime of these weapons are sometimes mere seconds before they are replaced with the next version via NuGet. The masters of C#, just like Tony Stark, have no issue with throwing out just created stuff in favor of the next great thing.

C#’s weaknesses are the dreaded NullReferenceException and the indeterminate mutability of most data structures. C# is great at attacking problems, and hitting at the 90% of the solution in a lightning quick way. However, C# doesn’t play defense very well. If you go too fast too far with C# you can out run your air cover, leaving you vulnerable when attacked from the flanks by null members, timeout errors, weird data changes, unhandled exceptions, or not connected to network scenarios.

C# is a highly flexible, multi-weapon language that you can use to attack any problem.

Swift = Captain America – Strong Defense

Swift is the ultimate boots-on-the-ground and in-the-mud defensive language. In Swift you don’t solve problems by attacking them head on with whole armories of shiny weaponry, you hunker down, put your shield up, and start slogging up that hill one step at a time.

Every line of Swift code written requires the author to think of the worst case:

The inventors of Swift must have looked at the metrics of existing Objective-C iOS applications and realized that the number 1 and 2 problems were:

  • Coders not realizing that the data they are changing on one screen leads to changes in data on other screens.
  • App crashes due to trying to access the value behind a null pointer.

Defense was needed. The power of Captain America’s vibranium shield was needed, only this time wielded by coders to stop these major in-app issues.

Coding in Swift is highly defensive in nature. To see this in action take a look at Swift Optionalsif .. let syntax, and the class initialization rules. As you use them, Swift optionals will permeate your entire thinking as you rev up your data structures. Every variable you use and/or create needs to be immediately thought of in terms of optional vs. non-optional.

Within Swift the use of specific reference and value types allows you to defend against unwanted mutation to your data structures.

Each line of defensive code is one more brick piled on top of other bricks of defensive code. When done you end up with applications that resemble the Great Wall of China and are truly written for the worst case. This layered defensive code coordination leads to super solid software structure that can take a real beating when attacked by the invaders (or users) of the software.

When done solving the problem in Swift you have a beautiful shiny shield that can repulse almost any mutability or null data attack. Because you play so much defense in Swift, coding up solutions to things seems to take longer. In reality, you have taken all that defense that C# out ran and have incorporated it into the structure of the solution.

The constant nullability slog, coupled with the constant mutability slog, while coding in Swift leads the developer to wanting more weapons to go at problems. Constant defensive coding slog can lead to developer fatigue quite quickly. However, when the problem solution is achieved, the engineer can see that their defensive coding effort has been like using Captain America’s shield as their primary weapon.

With the future potential of Swift existing across platforms, it could mean more solidly crafted software across the board, from server side to mobile applications. The underlying thought process behind Swift is already starting to leak into other languages, especially in the area of nullability.

I know that Swift isn’t without its share of flashy bolt-on weapon systems. Comparing C# to Swift when it comes to third-party library support is like comparing Iron Man’s crazy spin laser and hand repulsors to any weapon Captain America has ever wielded (at least in the movies).

 

But who wins in the ultimate battle?

If you ask me, I think Captain America and Iron Man are both good guys who should be augmenting each other’s strengths and covering their corresponding weaknesses.  I bet the  *SPOILER* real big bad *SPOILER*  is hiding behind the scenes and making them battle it out.

As with all things in software engineering there is never a clear winner. There are drawbacks and advantages to the structure, approach, and capabilities of all languages and constructs. These drawbacks and advantages tend to fit or not fit for solutions to given problems.

Good ideas tend to bubble to the top (i.e. delayed execution, in-language async, nullability enforcement, mutability enforcement), whereas bad ideas tend to settle to the bottom (i.e. garbage collection in Objective-C).

Swift is so new, the in-language concepts so hardened, the community so outspoken and willing to move quickly, that the verdict is not yet out.

C# has been around so long, has so much weaponry, and is currently so incredibly flexible across even dynamic and functional concepts. There is no way C# doesn’t continue to flex with some of the core concepts surfacing from the Swift and Kotlin worlds.

Who knows, one day you just might see Iron Man wielding Captain America’s shield. Just like C# currently wields dynamic and functional constructs, C# could well start to take on deep nullability constructs as they exist in Swift.

At the end of the day, we all win as these 2 languages battle, take on, and wield the best parts of each other to surface great solutions to real software problems.

My Trek Through MinneBar 11

minnebar.logoOn April 23, 2016 I attended my first minnebar conference: minnebar 11

I am amazed that content of this quality was presented by community members for free. It was totally worth being inside and roaming the maze-like halls of Best Buy Corporate Headquarters on one of the few sunny Saturdays we get up here in the frozen tundra of Minnesota. The exposure to new technical topics was great, but more importantly experiencing the energy of the people who are active in the Minnesota tech community was the real core of the experience.

I will try to mirror the energy and great themes of each of the sessions that I attended. The keyword is ‘try’. I apologize in advance if the energy and competence of each session I attended doesn’t shine through. Hopefully, some of the links I am adding to each session will help you navigate to additional resources.

Minnesota: The State of Tech [Startups]

This session consisted of a panel of people active in the Minnesota startup community.

The general takeaways that I got from the panel were:

  • Get involved with the community
  • Don’t be afraid to share your ideas. Sharing your idea will only make it stronger.
  • Double down when it seems like you should quit, double down again when it seems like you should stop, and when it gets hard and you think you are truly at the end, double down again
  • There is help for anything you want to do with your startup. From business, to community, to technology, there is an amazing community here in Minnesota that is dedicated to helping you get your business started in some way.
  • In general the panel is encouraged at the state of the startup and general business climate in Minnesota.
  • The depth of participation in the MN Cup Startup Competition was cited as a great sign of the vitality of the startup community within Minnesota.

University of Minnesota Research is Generating Disruptive Tech Sector Inventions

I can’t say enough about how eye opening this presentation given by Russ Straate and Chris Ghere was for me.

The process and overall maturity of the way in which the U of M incubates and manages the movement of their technology portfolio from the university into the business sector is astounding.

I can’t do the whole presentation justice, but for future reference you should watch the U of M Entrepreneurs site. Especially great is the University Startup Pipeline Google Doc which shows the stages of the startups as they move from the university onto business.

Running your startup with APIs (accomplishing more with less code!)

Ben Peter gave a great presentation on how to use the Zapier service to create custom data workflows from existing third party Web APIs.

His best demo was a code-free custom workflow that would send a free t-shirt to an attendee given an e-mail address contained in a text message. The demo used Zapier to aggregate SMS, Google Sheets, and Printfection, all without code to create a custom business promotion on-the-fly.

There is real power in being able to aggregate the features of entire businesses via their API surface. This power allows startups to create easy solutions for repetitive business tasks in a fraction of the time and resources of having to write custom code or rev up whole hosts and custom service silos.

Kotlin vs Swift : Which language is more Modern, Concise, Safe and Functional?

Andrew Rahn of Iconfactory gave a great side by side talk that compared Kotlin and Swift.

I really enjoyed his openness, energy, and passion for this topic. His openness really made the talk a true two-way discussion between himself and the audience.

Personally, I have done some Swift, and absolutely no Kotlin. His session served as a great jump start for me as I learn more about the Kotlin language.

The general take aways were:

  • Both Swift and Kotlin are really new languages, and as such are in a state of flux as their owners tweak the compilers, syntax, and environments
  • Both Swift and Kotlin are modern languages that force the developer to use the compiler to check nullability and mutability rules
  • Swift and Kotlin suffer greatly from being bolted on top of their respective legacy heritage (Swift — CocoaTouch / Objective-C / Automatic Reference Counting — Kotlin – Java / Android)
  • I would be incredibly lucky to ever get a chance to work with Andrew Rahn on any project.

Panel: Why YOU Should Enter a Data Science Hackathon

I went out of my comfort zone of front-end UI development and attended this panel.

Please see the minnebar session page for the panel members and full description of the session.

My takeaways:

  • When it comes to data science you spend over 90% of your time cleaning up your data sets before you can actually do any analysis. Things like different date formats, missing or null values, punctuation, and differing data entry techniques cause the data scientist to spend a ton of time cleaning up data.
  • There is a debate within the data science community between those who are going hunting through data looking for patterns, and more hard science statisticians who seek to apply statistical techniques to data to achieve a less biased / less ‘hunt and peck’ methodology to looking at mass data sets.
  • The Analyze This! competition seems like a really great thing to take a look at. The Analyze This competition is a long term (3+ months) competition that goes from raw data through to creating a possible predictive model.
  • Another interesting resource that was mentioned was Kaggle. Kaggle is a source of competitions and sample datasets.

Exploring Stateless UIs in Swift

At heart I am a native front-end developer, and this talk was 100% in my wheelhouse.

Sam Kirchmeier and Adam May from Livefront, and helper / organizers of TC HACK, presented their discoveries while creating a well factored, stateless UI, for iOS.

Their work brings the architectural ideas of React and the concepts in Flux to front-end iOS development using Swift.

Their presentation was done so well that the core concepts they presented could be applied to clean up and simplify almost any front-end development you may be doing on any platform.

Their work has inspired me to start pursuing a similar framework for Xamarin.iOS and Mono.Android development. In fact, the more I think about, the more I realize that their work may be more powerful when expressed in a cross-platform framework as it could lead to incredibly thick UI code sharing between platforms, and incredibly thin per-platform code. The possibilities for eliminating MVCs (Massive View Controllers) or huge overwrought Android Activity code are awesome.

If you do any UI work at all on any platform: Run, don’t walk, to their presentation slides and see the future.

Looking into the Crystal Ball: Using Google’s Prediction API

This presentation by Justin Grammens detailed his exploration of the Google Prediction API.

Justin had some great examples of applying the Google Prediction API against the Nice Ride data set to try and predict where a bike would be dropped off depending on when and where it was picked up.

The main question from the audience during Justin’s demos was: What is the underlying statistical model that the Google Prediction API uses? The answer, unfortunately, is that Google isn’t telling.

I would file the Google Prediction API away as a future tool for your toolbelt. At least until Google takes it away, never to be seen again.