The story you are about to read is mostly true. The names have been completely omitted to protect the innocent. It is extracted from the caffeine addled brain of an old-school software engineer. Take from it what lessons you wish.
As software engineers we always seem to want to rewrite something. There is an immense sense of freedom that comes from clicking: File –> New Project…
The next time you have the rewrite urge, think instead on how you can execute an incremental improvement to the code without rewriting. I would submit that if you think that a rewrite would be just ‘one step’ and ‘super easy’, it really isn’t. Factors, actors, capital, and time on a scale you never dreamed of will enter into your rewrite effort eventually.
No matter how much you may deny it, that code you want to shred does something vital for your business. The thinking, effort, and valuable time of many people you will never be able to talk to went into that code. You will not be able to ever get something that does 100% of what that older code did, ever. During your rewrite you are doomed to asymptotically approach 100% of the previous functionality forever. However, if you just take the initiative to perform a few incremental improvements to the existing code over time, you will have many successes, gain credibility, build trust, loosen up your organization, and allow the original code to go beyond what it originally did for your organization.
Over the long term, with little improvement after little improvement, you will actually achieve the engineering freedom you thought you would get from that ‘super easy rewrite’ and far surpass the fleeting glee of clicking ‘File –> New Project…’.
Got Dramatic on that ‘no rewrite’ stance above – Time to pull back?
It has been brought to my attention that tech debt, bugs, age, technology improvements, and relevance of a given chunk of code may lead to a rewrite being more optimal.
Feel free to disagree with me, but at the very least have a rewrite be the last thing you consider as a solution to your problem.
As with all things there are exceptions to the rule. If you have an awesome plan, a great new technology stack, a fully agreed upon reason for a rewrite up and down the tech and business stacks, and champions in the management ranks who can run interference when things go bad (and they will go bad).
I would submit that by refactoring your existing code you will find whole chunks that need to be refactored, then old code deleted, not rewritten in whole. In my experience deletion via refactoring is almost always a better alternative than File –> New Project…
When Microsoft started creating the Edge browser, they didn’t start with File –> New Project…, they took the Internet Explorer code base, branched it, and started deleting.
The base for my personal ‘Avoid a rewrite at all costs‘ stance is in the very old-school story which follows. What follows ends up as mostly a success story, my career is littered with many rewrite failures which are not documented here.
Back in my day….
In 2012 I had 1.5 million lines of 7 year old VB.NET code, C# code, proprietary XML configs, HTML / CSS / ASPX, and corresponding old school build scripts dropped on me and was told: “Get it to work. Fix it. Add all this other stuff too“.
The code was all from Visual Studio 2005 / .NET Framework 2.0. Actually, much of it still worked against .NET Framework 1.1. — Does anyone remember ArrayList?
The code spanned almost every possible Windows project type:
- 11 individual VB WinForms based apps — Anyone want to bet on how many lines of code were in each ‘Click’ event handler in each Form1.vb ?
- Backing business logic assemblies for all 11 apps.
- About 5-6 XML based config files per app which controlled everything from UI control text, placement, and function
- A C# ASP.NET master site which used .NET Remoting to talk to…
- 3 C# based Windows Services
- Matching old-school Deployment and Setup projects for MSI / MSM generation across the board.
- It was all built using CruiseControl.NET
- There were a set of common assemblies referenced across the 11 individual VB WinForms based apps which had to be GAC’d — To the non-technical: GAC’ing (Pronounced: ‘Gacking‘) an assembly is as bad as it sounds.
I found out that we had purchased this software suite + source code from another company. After looking at the code for a day or two, I asked my boss if we could renegotiate the contract and get our quite substantial amount of money back. Unfortunately, the answer was no.
When other developers were presented with the code base they all said: “Rewrite it all.”
When other developers stated ‘Rewrite it all’, I asked them:
- What would a rewrite look like?
- Who in the company will decide what the new product would do?
- We have 11 distinct semi-working apps, and semi-working admin, config, and backing web services. How much time and $ would it take to document specific requirements (or even get to a valid starting point) and start iterating on a whole new functionally equivalent product suite?
I asked all of these questions with a full understanding of our product / business / sales side. The product / business / sales staff was stretched incredibly thin and had totally different goals than the developers. Our sales people needed to ‘flip that house‘! They wanted to land a sale of this purchased software suite so they could make some of the purchase price back in the form of software + paired hardware sales.
We didn’t have any product people interested in creating a whole new software suite from scratch.
We didn’t have any product people who could dictate what a final product could/would/should look like.
Our business bought this software suite believing it would solve all the business side problems of having to create one from scratch.
What almost everyone didn’t realize was that we didn’t buy functioning code, we bought all the thinking behind one possible implementation of a whole product line. We didn’t buy code, we bought a technology shell and some supporting documentation which crystallized millions of tiny decisions to create a series of fulfilled business requirements for an entire software suite. We didn’t buy the code, we bought all the thinking, research, documentation, blood, sweat, tears, and overhead that went into ideation behind a functioning and deployed software suite. When it comes to the actual cost of purchasing the software suite, we didn’t get the value we wanted out of the code (not by a long shot!), but we definitely got our money’s worth via the software demonstrating (even partially) fulfilled business rules all learned the hard way. I learned real quickly that there is an incredible amount of value in a semi-working software product in that it can get whole departments of people into a unified mindset.
From an political perspective a rewrite was not in the cards. There was no management, sales, marketing, or capital support for a rewrite. But a whole chain of refactoring and specific bug fixing would allow us to get this up and running. Once up and running, perhaps the politics would start to gel. From an engineering perspective, at least, we would have some semblance of order and possibly a platform from which to jump to other things later on.
My job was engineering, and from the engineering perspective all I could do was try to get this whole suite up and running, then start to fix all the stuff to get it into alignment with what the business thought they purchased.
We were given the source in the form of a set of Visual SourceSafe files.
So in 2012 we revved up Visual SourceSafe and CruiseControl.NET to build a set of Visual Studio 2005 solutions + old school Package and Deployment projects to create a series of MSI / MSM files.
Amazingly enough, after filling some gaps, and some old-school Windows dev magic, it all installed and worked. We now had 1.5 million lines of code + configurations all compiled, installable, and ‘working’. Woo!
Well, it was ‘Woo!’ for about 20 minutes until we demoed our work to the business people. We started showing the apps and asking what capabilities certain parts of this purchased system were supposed to have. From those talks and demos we found an incredible pile of bugs and unfulfilled requirements across the entire system.
The bugs were numerous and spaced out across almost every part of the interlocked system. We could’ve started going at the bugs. But the bugs were systemic against almost 10 year old code. Instead of going straight at the bugs, we had to tool up, contain, then sneak up and squish them. We took on the following tasks first:
- Ditch Visual SourceSafe — At this time we used TFS. (Oh! To go back in time and use Git!)
- Ditch CruiseControl.NET — Because we used TFS, we used Team Build / MSBuild.
- Ditch Visual Studio 2005 + .NET Framework 2.0 in favor of Visual Studio 2012 + .NET Framework 4.0 — VS2013 wasn’t out at the time we started, and we couldn’t go to full on .NET Framework 4.5 because we needed Windows XP support across the suite.
- When we went from Visual Studio 2005 to Visual Studio 2012, all of those old school Package and Deployment Projects were no longer supported.
- Migrate all Package and Deployment Projects to WIX + Votive projects.
We did it!
The tooling to migrate Visual SourceSafe to TFS even migrated the history that was stored in Visual SourceSafe so we could see how all that sausage was made.
We now had separate Team Build servers and a build process we could fire off from within Visual Studio, no more CruiseControl.NET stuff.
We now had .NET Framework 4.0 instead of .NET Framework 2.0. We got some WinForms fixes, ability to use WCF instead of .NET Remoting, and more importantly for future features and squishing bugs — LINQ support!
We got a way better installer and product distribution experience in the form of WIX + Votive.
We re-ran all of our manual smoke tests against our new shiny .NET Framework 4.0 products. Everything seems good. We now have .NET Framework 4.0 features to use, and now we can reliably attack bugs, and refactor all that old school .NET Remoting stuff used for interapp communication between apps 1-11 using Windows Communication Foundation (WCF).
All was going OK, until management came by and wanted apps 12,13 and 14 added to the suite.
The timing couldn’t have been better for 12,13, and 14. We found that we could create a new Windows Presentation Foundation based UI framework + reuse of the refactored WCF based interapp communication scheme which would allow the new apps to integrate with Apps 1 – 11 via a very slim assembly.
I figured that I could silence the engineers who wanted a ‘full out rewrite’ because now they had the tasks of apps 12,13, and 14. They could build those apps in a clean room using the full surface of C# / XAML / WPF (no more VB based WinForms like apps 1-11) + a very thin shim library which integrated the new apps into the entire suite in a fairly clean way.
The engineers of apps 12,13, and 14 didn’t have to maintain or worry about any legacy stuff. All the legacy integrations were on me. You would think that those ‘rewrite it all’ engineers would be pretty happy. They got pure WPF projects, XAML, C#, Prism, full .NET Framework 4.0, and all the goodness that comes with it.
What was fascinating was that the engineers who effectively got their clean rewrite, and never had to deal with legacy code, were way more miserable than I was.
I had the job of shoring up all the old school infrastructure, supporting legacy to new code adapters, adding new admin and suite wide features, and discovering / squashing the never ending pile of undocumented bugs throughout the 1.5 million lines of legacy code and config data. I had a pile of tasks 30 miles high, and all the tools I needed to go at those tasks.
The engineers who had to attack apps 12,13 and 14 had little to no real instruction from business / product people. All they had was a ‘1 sheet’ (in one case, it was one line) of what apps 12,13, and 14 were supposed to do. At no point did anyone on the product side even really know where to start. The engineers were responsible for taking the ‘1 sheet’ and trying to spec out everything from UI design, UI flow, data storage rules….
I was lucky, I had specific direction in the form of the intent of the legacy code and a whole new infrastructure that I could use to apply fixes to that code. I could see day by day improvement across the whole suite of legacy applications and infrastructure.
The engineers, who wanted to do a full rewrite, were literally lost in the wilderness. They had what they had always wanted from a technology perspective, but no real business driven direction. The engineers had all the tooling to go any direction they wanted, but no roadmap on where to go.
Let’s get more concrete. App 13 was an app that was supposed to be a full on editor for a data set which was defined via a known XSD Schema written by a standards body.
The XSD was fairly complex and had 20-30 entities which needed a full on touch capable, WPF based, suite integrated Windows app to allow a user to edit, create, and delete those entities.
The engineer got the XSD and immediately tried to turn to non-existent, uninvolved, and apathetic product owners to try and waterfall out a full design for editing all 20-30 root entities.
As you can predict, the engineer didn’t get a whole lot done. The full editor design for all 20-30 root entities didn’t magically appear from all the uninvolved product owners.
Given the lack of a waterfall of requirements, I figured that the engineer needed to attack the problem from the bottom up. We had a good base:
- Dedicated, integrated, WPF app host
- Full Team Build + Visual Studio Test support
- An XSD Schema from the standards body, along with sample XML from other apps that implemented that XSD Schema.
The one problem we could solve at the engineering level was that we needed to turn that XML / XSD into some sort of object model in .NET so we could read / write all of the sample XML. I had the engineer start on basic XML / XSD processing in C# using the WPF app harness and starting with test and read/write assemblies. After some trial and tribulation we got a valid read / write scheme working against all the sample XML and validation of newly created XML from our object model against the XSD.
Now we had a data model expressed in C# code that could help the engineer determine how the editor UI for App 13 could flow.
Once the engineer was immersed in the details of the XML data, we got knowledge we didn’t have previously. That knowledge could now be communicated to the product owners, in their own terminology and language.
We were also able to start creating some primitive View Models and corresponding WPF based Views against a few of the entities. Now that we had a start on a working data driven WPF app, and progress that people could see, product owners started taking notice and were able to communicate (even if only partially) some real requirements for App 13.
If you notice, we actually shifted product development mindset, not technology mindset. We went from waterfall type requirement gathering to a more Agile approach. Using Agile we were able to identify and attack one or more technical problems with tools and knowledge we had on hand. Then use the technical outcome of that effort as a stepping stone to attack the next set of product requirements.
During this process the engineering staff started to see how a rewrite of the suite just wasn’t ever really in the cards.
Super Development 
Leave a Comment