Writing Node Applications as a .NET Developer

As a .NET developer, creating modern web apps using Node on the backend can seem daunting.  The amount of tooling and setup required before you can write a “modern” application has resulted in the development community to display “Javascript Fatigue”; a general wariness related to the exploding amount of tooling, libraries, frameworks and best practices that are introduced on a seemingly daily basis.  Contrast this with building an app in .NET using Visual Studio where the developer simply selects a project template to build off of and they’re ready to go.

In addition to the tooling and environment differences, Javascript as a language is very different than most .NET languages, particularly C#; statically vs. dynamically typed, thread based vs event based async, compiled vs interpreted, the differences are myriad even though the syntax is very similar.  All of which can quickly get a .NET developer out of their comfort zone.  In this series of blog posts, I’ll describe some of the key differences between the two frameworks, my experience (and opinions) in setting up my development environment and my first impressions when writing a Node application.

Key Differences between Node and .NET/C#

Prior to starting a Node application, you should be familiar with some of the nuances of the framework (and Javascript as a language) that are different from C# applications.  While this list isn’t exhaustive, these items were the first three that stuck out to me when working with Node.

Javascript Modules

One of the major pain points in Javascript development for the web is the lack of browser support of a module system for Javascript files.  If you don’t use a module loader library in your browser, all libraries and code are available globally which can cause collisions and confusion.  Luckily Node supports a module system out of the box called CommonJS.  CommonJS has a very simple format, akin to function calls wherein the object returned is your exported module:

//exporting module with myService as a property of the exported module object
module.exports.myService = function myService() {}

//importing
var myServiceModule = require('./my-service')
var myService = myServiceModule.myService

//exporting module as default
module.exports = function myService() {}
//importing
var myService = require('./my-service')

Not only does Node’s module system support importing code that you write, it also supports importing third party dependencies that you install via npm (which are located in the node_modules folder).  The critical difference is in the string you pass to the require statement:

• Node modules – provide the name of the module you installed e.g. react would be require(‘react’)
• Application modules- provide a relative file path to the module e.g. a file located at src/app/main.js imports a service at src/app/service/myService.  The call to require would be require(‘./service/myService’)

When working with Node’s module loader, it’s best to drop all assumptions and comparisons to C#’s using statement and remember the following facts:

• Require statements are actually just function calls using a module loader that Node provides. The using statements in C# is a language feature that provides convenience to the programmer when they write code that references namespaces outside of their current code.
• Require statements do not affect the current namespace nor do they implicitly make available code to the execution environment.  They simply provide an object/module that you can access.  Again, this is in contrast to C# where the using statement is more of a convenience feature in that the programmer does not have to type out the fully qualified type name if that type’s namespace is referenced in a using statement.
• The CommonJS module loader is very similar to an IoC container wherein it caches references to modules that it loads and keeps them around for use by multiple dependencies.
• In a Node application, the first time the module loader encounters a require’d module, it will evaluate that module’s file and return the result to the dependent module.  The result of this evaluation will be cached by the module loader and retrieved for all subsequent imports.

I struggled with this last point when thinking about how to structure an application as I felt that explicitly requiring in a file via file path was coupling that dependency with the specific file.  I’ll discuss this feature in a future post that discusses issues and confusion that I had related to application structure.

Async Programming

Given Javascript’s single threaded, event based programming model, the language is inherently asynchronous.  However, compared to C#, the syntax to support async programming is a little behind other programming languages (albeit catching up with the next version of the standard and the TC39 proposal for async/await currently in the “Candidate” status). The primary means of async programming with Javascript is via callbacks (the old way) and Promises (the somewhat new way).

Callbacks have been a main staple in Javascript since its inception and are quite simple in their nature: when this action completes, call the function I have provided with the arguments I have specified.  They are used throughout the Node community and third party libraries for asynchronous actions, however the community has adopted an “error first” convention that dictates that the first argument passed to all callbacks should be an error object (if there is one) followed by the rest of the function’s arguments.  This makes for consistent (albeit repetitive) callback code:

//get information about a specific file
fs.stat('myFile.txt', function(err, stats) {
    if(err) {
        //if err is truthy (i.e. if it exists) handle error
        console.error('an error has occurred',err)
        return;
    }
    //rest of code that checks stats
})
 

While the callback pattern is simple to understand, consecutive asynchronous actions often results in a “pyramid” of callbacks whereby one callback calls another callback which calls another leading to unattractive and hard to follow code.  In addition, callbacks are not “composable” in the sense that you cannot group a set of callbacks together or chain them without writing imperative code that is painful on the eyes.  The latter is the main reason why Promises have gained traction over the past few years.

A Promise is a paradigm which at first glance seems very similar to callbacks but has a couple of distinct advantages.  With a Promise, the programmer can state the following in code: perform this action, then perform this action, then perform this action but if there is an error, catch it and perform this action.  This allows for declarative code that is easier to follow:

//third party library that supports db queries that return a promise
db.queryById({id: 1})
    .then(function(items) {
        //code that process items
        return db.query({name: 'some other thing we query for'})
    })
    .then(function(result){
        //code that handles this result
    })
    .catch(function(error) {
        //error thrown in any of the promise functions above
        console.error('an error has occurred', error)
    })
 

As shown in this snippet, a Promise is just a value that the user can register callbacks on that will be executed in sequence.  When the action related to the Promise resolves, the first “then” callback is invoked with the result of the action being passed as the first parameter.  If an error is thrown in any of the functions (or the original action) that is registered with the Promise, it will be caught in the next catch block that is registered.  The main benefit of Promises is that it allows for multiple “Promise” actions to be performed in parallel or serially in a declarative fashion. While Promises offer a step forward in asynchronous programming, error handling can be somewhat confusing to novice users. In addition, Promises can also suffer from the “pyramid effect” that callbacks incur if they are used improperly.

As mentioned above, the Node standard library seems to “prefer” callbacks as the async pattern of choice, however that stance seems to be changing in the community as Promises are generally the better tool for the job.  Libraries like Q and Bluebird offer utilities to convert asynchronous functions that accept callbacks into Promise bearing functions.  In addition, third party libraries are now designing their apis to work with both paradigms; if a callback is provided, call that function when the action is complete, otherwise return a Promise bound to that action.  

How does this all compare to C#’s async story? Introduced in C# 5, the async/await pattern freed the developer from writing painfully imperative code, allowing for asynchronous code to be written in a synchronous fashion:

private async Task DoSomethingAsync()
{
    var value = await getValueAsync();
}

In many ways, the Task type returned from getValueAsync is similar to the Promise object conceptually; both objects represent an asynchronous action that will eventually return a value (or throw an exception) . However, whereas a Promise’s “then” function is the manner in which you would register a callback on resolution, a Task’s value is “unwrapped” via the await keyword.   

The ability to write code in a declarative fashion using async/await really allows C# developers to write terse and easy to understand code.  While this feature is not officially standardized and available to Javascript programmers at the moment, one can use transpilation tools and libraries to be able to use future language features that run in most Javascript runtimes.

Using New Language Features via Transpilation

There is an interesting contrast between C# and Javascript when it comes to language features.  Since its inception, C# language features have been released at a somewhat regular cadence, with each release having purposeful feature sets that either built on or improved previous functionality.  Compare this with Javascript (formally known as Ecmascript) and you’ll see a stark difference.  During a 10 year period from 1999 to 2009, there was an essential deadlock on the language specification during which the 4th edition of the language was abandoned.  It was only until Ecmascript 5 was ratified (and implemented in browsers) that the language started to become more modern and feature rich. However, it wasn’t until talks of a sixth edition came about with a bevy of useful and much needed features that would completely remake the language.  

The problem that arose is that due to the seemingly snail’s pace at which browsers implemented these new language features coupled with the need to always be backwards compatible with existing language features, these features have not been readily available for use to the programmer.  This was the case until the advent of transpilation tools like Babel and Traceur which allow one to write cutting edge Javascript code that “compiles down” to ES5 syntax (or lower) that can run in most Javascript environments.  However, introducing transpilation into one’s toolchain carries with it complexities and frustrations that a .NET developer only encounters when trying to target an older .NET framework.

Language transpilation is a double-edged sword: one has access to the newest features of a language and allows one to “future proof” their code, but in doing so introduces dependencies and build steps that can complicate the development process.  While Node does a good job of releasing versions that support these newer language features, it’s still behind the bleeding edge of the spec which requires the developer to introduce complexity to their build toolchain in order to access these features.  One has to weigh these tradeoffs and compare the benefits of writing more powerful code (like in the example of async/await above) to the necessity of understanding how to setup a development environment to support such language features.

Summary

As detailed above, there are certain features and functionalities in Node that differ greatly from C# and the .NET runtime.  Many of these differences can be contributed to Javascript’s relative immaturity as a language as well as it transitioning from a scripting language used primarily on the web to a server side language.  While the language is experiencing a great leap forward in terms of features and performance, at the moment one must use a transpiler to access the language features that a C# developer gets out of the box.

In the next blog post, we’ll go in a different direction and look at the considerations and decisions I faced in setting up my development environment to run a Node application. Finally, in the third post, I’ll describe some of my pain points in developing a Node application in regards to application structure and setting up a test harness.

Footnotes

Quick note on ES6 module syntax

Per the ECMAScript 2015 specification, there is a new syntax for importing modules.  The concepts are the exact same; modules being imported are just objects, nothing is being added to the global namespace like in C#.  However, the main difference with the syntax is that importing modules is now best thought of as a language feature and not a function call.  This puts restrictions on where import statements can go (only at the top of a file before any other declarations) and their use with conditional statements (they can’t be conditionally called).  While it may sound like this is a step backwards, this new syntax offers potential benefits for optimization and static analysis.

jQuery Promises

If you are a developer who uses jQuery, it’s important to note that while jQuery.deferred returns a promise object, it is not Promises/A+ compatible.  This has been rectified in the latest major version of the library (3.0).

Async/Await and Promises

It’s interesting to note that async/await (a language feature related to writing asynchronous code in a synchronous fashion) will perform the same type of “unwrapping” on Promise objects as C# performs on Task objects.