A Dive into SystemJS – Loading and Translating

In the last article we took a look at some of the basic configuration options of SystemJS and also the high level workflow of what happens when you attempt to import a module. This article is going to walk through what happens from when a script has been fetched by the browser until a module instance is returned, as well as provide some information on creating plugins for SystemJS.

This is the second of three articles related to SystemJS. The first article covers the basics, and the third article talks about production considerations.

Source Translation
After SystemJS has successfully loaded the script file it will perform a translation step. Normally there is not any translation that needs to happen, but you can use custom loaders that hook into this step to add support for additional file types or modify the source of the file. For instance, if your company requires that every JavaScript file served up has your company’s copyright at the top or bottom of the file, then you could write a copyright plugin that appends it to every loaded file. This is just a contrived example to demonstrate a possible use and it is not intended to be a suggestion on best practices, but hopefully it demonstrates the purpose of the translation step.

/*
  Copyright plugin - plugins/copyright.js
*/
exports.translate = function(load) {
	return load.source = load.source + '\n/* Copyright by Joel Peterson */';
}

// SystemJS config
meta: {
    "src/*": {
        loader: 'plugins/copyright.js'
    }
}

In this example we just performed a simple string concatenation to add the copyright, although you could also do things like transpile your code or perform CoffeScript compilation in this step, or if you wrote a custom DSL then you could use that to compile your source back into JavaScript in this step. The next thing that we did was configure SystemJS to use this custom loader plugin for all of the files inside of the src package, and for that we used the meta configuration block. In this case the only configuration item that we needed was to set the loader.

An alternative to this would be to explicitly specify the loader to use when you perform the import of your file.

// Using an import
import someFile from "someFile.js!copyright";

// Using System
System.import("./someFile.js!copyright");

// Add alias for copyright
map: {
    copyright: "plugins/copyright.js"
}

This is the approach that is commonly used when importing things like CSS or JSON files with one of the SystemJS plugins, although you can also register the plugin for all files with a given extension.

The load object that is passed to the plugin’s translate function has a property for the source map of the code, so if your translation logic modifies the source code then you can generate a source map for your changes, and then set the sourceMap property on the load object so that the browser will be able to map the code back to the original source. If you do anything like transpilation or create a DSL then your developers will appreciate having the source maps.

Loading the Module
After all of the source has been loaded and translated, SystemJS will finally attempt to instantiate the module. This process involves detecting the module format used in the file, if any, and also discovering the dependencies of the module so that it can load those prior to executing the code in the current module. Once the module has been successfully instantiated then the output will be cached so that future imports of that module will be quicker.

SystemJS supports loading 5 different formats of modules: the ECMAScript module format, AMD, CommonJS, SystemJS register modules, and global javascript. It will do a fairly good job detecting the module format for a file, but SystemJS does have a way for you to specify the format for certain files in the SystemJS configuration file. If you use a code instrumenting tool like Istanbul it might affect the ability of SystemJS to detect the module format since SystemJS looks for specific patterns that might be broken by the additional code. In that case it would probably be good to explicitly add your module format information to your SystemJS config file.

Global Modules
Global files are probably the most interesting to consider because global files do not have a way to explicitly identify their dependencies and are also not necessarily formatted like a module. As a result, SystemJS provides the ability to shim globals via configuration elements.

// a global file
var logLevel = "INFO";
window.log = function(level, msg) {
    if (level === logLevel) {
        console.log(msg);
    }
};

This is an example of a global file. The variables and functions from this file are all created in the global scope so SystemJS will interpret them and expose them as module exports. SystemJS performs a comparison of the global scope before and after loading a global file to detect what the global exports should be for a file. By default it will export all globals from a file although you can configure it to only export specific variables.

// config file
meta: {
    'src/globalLogger.js': {
	exports: 'log', // this only exports the log variable
	format: 'global'
    }
}

If we use this configuration and import the global file from our previous example then only the log function will be exported because we defined an explicit list of exports.

It would be impossible for SystemJS to detect the dependencies of a global file since there are no import statements or register statements that list the dependencies. Because of this, if your global file has dependencies on other files then you will need to either make sure the dependencies are loaded first or you have to configure SystemJS to register those dependencies.

// config file
meta: {
    'src/globalLogger.js': {
	exports: 'log',
	format: 'global',
        deps: ['src/baseLogger.js']
    }
}

This configuration registers the file “src/baseLogger.js” as a dependency of the globalLogger.js file so SystemJS will load that first.

// globalLogger.js
function log(msg) {
    baseLogger.log(msg);
}

// ES2015 module baseLogger.js
export default let baseLogger = {
    log: function(msg) {
        console.log(msg);
    }
};

// config file
meta: {
    'src/globalLogger.js': {
	exports: 'log',
	format: 'global',
        globals: {
            'baseLogger': 'src/baseLogger.js'
        }
    }
}

This configuration setup will register “src/baseLogger.js” as a dependency and create a global variable “baseLogger” in the context of the globalLogger module using the “baseLogger” variable from the dependency. This approach lets you use a global JavaScript file and wire up some of its variables with values from other files.

AMD Modules
AMD modules are still able to use the require statement to load additional dependencies since the AMD require is asynchronous. SystemJS provides its own require statement though and exposes that to AMD modules for use instead of the normal require statement to make sure that any required modules are loaded via SystemJS.

SystemJS also makes it possible for you to explicitly identify your AMD modules.

// config file
meta: {
    'src/amdLogger.js': {
	format: 'amd'
    }
}

CommonJS Modules
SystemJS does support CommonJS modules but because the CommonJS require statement is synchronous these modules need to be wrapped so that dependencies are registered and loaded prior to the require statement being executed. A previous example showed the syntax for this, but here it is again.

// Legacy code codeFile1.js in CommonJS format
module.exports.something = 'test';
module.exports.nestedDep = require('codeFile1Dependency.js');

// New version as a System module wrapper around the CommonJS code
System.registerDynamic(['./codeFile1Dependency.js'], true, function(require, exports, module) {
    module.exports.something = 'test';
    module.exports.nestedDep = require('./codeFile1Dependency.js');
});

SystemJS Plugins
We already briefly covered how you can create a custom loader that performs source translation but there are 3 other plugin hook points of which you can take advantage: locate, fetch, and instantiate. The locate hook point is responsible for taking a module name and returning the URI that will load the resource. The fetch hook point is responsible for taking the URI to a resource and making the request to fetch the contents. The translate hook point has already been covered so the last hook point is the instantiate step. This hook point takes the module source, determines the dependencies, executes the code, and returns the module. The returned value from this hook will be used as the module instance.

Conclusion
We just walked through what happens when you load a module and also how to create a plugin for SystemJS. The information in this article and the previous article should be enough to get you going on a development system and get you started creating your application. However, this will not be good enough for a production system and so in the next article we are going to look at considerations to take into account when your code is ready to go live.