How to deal with the Frankenstein Software Monster

Posted on July 24, 2024 • 16 min read • 3,209 words

A discussion on the dynamics that lead to a Frankenstein software monster. How it occurs, why it occurs and how to navigate the situation.

Foreword  

First, we have to salute the achievement. Creating life is no small feat! Let’s engage in this conversation with the utmost respect for all the Victor Frankenstein out there.

Keep in mind that with complex systems, answers are highly contextual. Nuance is our friend. Understanding the dimensions of the problem is more important than trying to compare our situation to the latest tech story.

The approach is definitely not limited to software. Every city is a Frankenstein monster, Every house of a certain age is one as well. Home improvements projects can leave our home looking like a patchwork of eras, architectures and interior designs. Picture a 1980’s Bathroom in a 1920’s house lit with 2020’s LED bulbs and digital dimmers on 1900’s knob-and-tube wiring .

It works but there are caveats.

Take Away  

Frankenstein software is the result of complex interactions between many variables. (see diagram )

Frankenstein software has its place for the purpose of concept validation and bootstrapping.

Frankenstein software can appear gradually, over time. What was a deliberate desired state can become a liability.

Software is never finished, always in flux as the world changes, technology changes and we change. All those changing conditions can lead to the creation of a Monster if not managed properly.

Navigating out of Frankenstein requires experience and effort. In most cases, it can be done but don’t go alone!

The story of Frankenstein could have likely had a better ending if Victor had faced the situation and properly cared for his creation.

Let’s define “Frankenstein Software”  

“Frankenstein ”, written by English author Mary Shelley , follows the story of Victor Frankenstein, a scientist captivated by the mysteries of nature. Driven by an ambition to create a new species, he succeeds in bringing a sentient creature to existence. However, this accomplishment sets the stage for tragedy as the creature, longing for companionship and acceptance, faces a world that shuns and fears him.

Frankenstein software is the result of the multiplication of heterogeneous software components with little to no investment in consistency and consolidation. Done deliberately or not.

To build our analogy to the original story, let’s consider some parallels with software development:

Nature  

In our context, Software is “Nature”. Software is more complex than we often understand. Software engineering can be a highly difficult task. It is an attempt to understand and model the “World” (our Problem Domain), and harness Technology in the hope to create a solution to a given problem.

The Doctor  

We are all Victor Frankenstein. We have inside us this strong desire to create. This desire, combined with our Hubris and the complexity of Nature, sometimes result in a monstrosity unleashing pain and suffering onto our team, the company, the world.

The Monster  

The collection of programs, processes or services cobbled together is the Monster. It works, but it is already hard to control and is likely to cause great pain as it grows and evolves if not accepted for what it is.

How does Frankenstein software come to be?  

It is important to understand how it happens to discover how to influence the outcome.

It’s often the same story  

A shortcoming in one area leads to mistakes which are never completely fixed due to little leftover time and the resulting product becomes the starting point for the next iteration. The cycle repeats, with the likelihood of mistakes increasing as the complexity of the system grows.

A Model  

The following diagram is a simplified model of the causal relationships between some of the variables of the software development equation.

This diagram is a lot to digest so let’s talk through a few points.

Business Context  

There’s a wide spectrum between cobbling together off-the-shelf software and writing everything from scratch. There are many variables to consider when gauging where to be on that spectrum at any given time. Things change as the product and business evolves. Let’s discuss how the business context can lead to this situation.

Business Strategy  

The stage of the company is a factor and will drive budget availability, time and the team’s size and experience.

Early companies will initially benefit from cobbling software together quickly to “prove their point” cheaply. They might need several attempts to find product market fit and the cheaper those attempts, the more attempts they can make. The key being to know and be ready to switch gear when the time for growth comes. I would argue that this remains true even for established companies. The amount of investment should be a function of the product fit, not the amount of “disposable” resources available.

Clear goals are paramount, too many teams have wasted precious resources running in circles due to a lack of Business strategy and lack of clarity on what the goals were. Failed attempts will eat resources leaving the next attempt constrained and without a clear reset, the existing solution will be used as the starting point for the next iteration. Effectively starting the amalgamation process.

With those clear goals established, the evaluation of Business opportunities should drive allocation of resources such as money and people. The opportunity could also carry an expiration date, limiting the amount of time available to complete the work.

Resources  

We need time. Unless we already have the knowledge of the domain and relevant technologies, we will need to acquire it. This is often the case.

Time is also necessary to build sustainably, to build the product along with the necessary infrastructure to maintain development velocity as the team and scope grow.

We also need time to find the right product fit, to clean up after failed attempts and changes of direction. Failure to do so will see the leftover stitched into the whole.

We also need a team with the right skills and amount of experience for the job. Not allocating the right folks is likely to drive inferior decisions which will in turn impact the outcome. With every poor decision leaving behind an out of place “part”.

We also need money to pay for required services and infrastructure. Free tiers only go so far.

Domain Complexity  

The complexity of the domain for which we are building a solution will influence the fit of our solution. Finding relevant technologies, understanding and defining what to build will be harder for more complex domains.

Maintaining quality will be challenging as it will be easier to make mistakes. This inherent complexity will also drive the introduction of accidental complexity whenever we fail to understand the domain enough.

Technology  

Off-the-shelf  

Although it’s not immediately obvious in the picture below, Off-the-shelf Technology in itself carries the seed of Frankenstein.

Any domain of a certain complexity will include lots of use cases. Existing technologies in this domain are likely to address some but not all use cases. Businesses might position themselves in novel ways and not fit exactly with the most frequent use cases.

All these factors will lead to a new business requiring something of a compromise between several offerings. “We need this feature here and this one there”. If they decide to move forward, they will find themselves with the task of integrating those existing solutions. The alternative is to build on top of one, the other or the whole thing. That might not make sense for reasons we discuss in this article.

Note that the narrative above represents the perfect case. In which requirements and offerings are perfectly communicated and understood. What is more likely to happen is as follow:

• The right people are not involved because we don’t want to “distract” them. Or they just don’t exist in the organization.
• The requirements are flawed due to lack of domain understanding
• They are inexactly communicated because of the inaccuracy of language and inexperience of the individual
• They are inexactly understood for the same reasons
• There’s an incentive to “get something going”
• There’s an incentive to sell the service.

With no ill-intent from anyone, a match made in heaven is definitely not guaranteed. And only once the annual contract is signed will the teams discover how the technology behaves in their context. That technology could be a Frankenstein Monster in itself!

Gaps and Overlaps  

As more technologies are introduced, from SAAS all the way down to Programming Languages, gaps and overlaps will exist.

How to deal with each will be left to the team. Their skill will determine the outcome of the stitching process.

The Team  

From Leadership to Engineering, there are many bodies of knowledge to explore and learn. Not knowing is not a crime but certainly has an impact on the outcome.

Lack of Domain understanding  

If we don’t understand the problem domain, we will forget requirements and we will fail to see zones of complexity and fail to forecast future need for change.

If we fail to talk to customers and users, we might misunderstand the scope of the project. The initial understanding is typically just a starting point we build upon. Often, there are many incorrect assumptions made.

All these opportunities for error will drive the selection of solutions which will be revealed incomplete/inadequate after more is learned.

Lack of experience building Software  

It is important to remember that Software engineering is Research and Development wrapped together. Most of what needs to be known to succeed is not known when the project starts.

It is also important to realize the depth of Software engineering as it is easy to be lured by the promise of being productive after a 6 weeks bootcamp. A lot of the quality aspects of a solution is observed over a longer period of time and is driven by knowledge which goes beyond language syntax.

There are advantages in UI driven tools and their lower barrier of entry. They might make it easy, with little training, to perform a small number of operations without the overhead of coding but those approaches will often fail to scale economically. For example allowing for automation and testing as the activity grows.

It is important to know what to do. It is even more important to know when not to do it.

Properly evaluating vendors requires an understanding of what technology can do to be able to ask the right questions.

Lack of software engineering experience will often lead to complexity being missed resulting in the wrong solution being selected at a higher cost of change in the future.

Lack of experience Leading  

The wrong goals being set will nullify any chance to achieve success.

It is rare for someone to have an exhaustive knowledge of the entire scope, we’ll be most often looking to build and manage a team of people.

Limiting the team to the leader’s level of understanding is a sure way to miss critical pieces of information. Similarly, failing to effectively share critical pieces of information with our team will have the same result.

We will have to manage the builders’ emotions.

Failure to consider how we will attract and hire the next generation of team members will leave us with orphan systems surrounded with duplicated components. One of the pitfalls of Legacy software is the difficulty to keep folks interested in it. Especially if we’re talking about Adhoc in-house frameworks that are only exciting to their creators and provide no marketable skills.

Emotions  

Desire to create  

I believe that every builder wants to create. Entrepreneurs and software engineers are no exception. This desire is strong and we are compelled to express that desire regardless of its relevance to a given situation. And in that case, yes, the creation will bring misery upon the world and upon ourselves.

If the problem is not interesting enough, the developer will be tempted to make it interesting by creating a solution to the bigger problem they hoped to find. Hire a nuclear scientist to build a coffee machine and we’re likely to end up with a Nuclear powered coffee machine.

Hubris  

In a culture full of super-heroes and one-person armies, it’s easy to succumb to the desire to identify and see ourselves as one.

What if we’re not that resourceful? Should we wager the success of our endeavor on our perceived limitless power?

Others have been there before. Learning from them will save us from wasteful meanderings. Collaboration is a power boost, the group is more likely to achieve a greater outcome than one person alone.

Things might be more complex than what we understand right now.

Maybe we’re the one who’s in the wrong.

A good dose of humility will protect us from stopping our learning too early. If we start to find failings in all things that surround us, we might be the problem.

Time  

It takes time to create  

Strong strategy and teams will certainly get us there faster but all things take time. There’s an incompressible amount of time required to achieve great outcomes. Denying an initiative the time it needs will increase the chances of a Frankenstein Monster appearing now or soon in the future.

Passage of Time  

The Monster might simply appear through the passage of time and the incremental process of evolving software. As parts get replaced over time.

Generational Endeavor  

Generations of developers come and go. Leadership comes and goes. Each wave brings with them change, deliberately or not. New Technologies are introduced, creating duplication. Technologies disappear leaving gaps. This happens over many corporate lifetimes.

The Future is a Far Away Land  

Strong business inertia leaves no time to look back or too far ahead.

Inexperience across the organization creates focus on short term rewards at the expense of long term goals.

More importantly, the most impactful consequences are in the future, and therefore not felt.

The Monster waits for his revenge  

Short Term  

The act of creation has succeeded, the creature is alive and everything is good in the land. We’ve shortened the feedback loop. Achieved the ideal of Lean product development. Whe feel good!

There’s definitely a place for this minimalist stitching approach of cobbling technologies together to prove a point. When the goal is to validate a concept, the cheaper the better. The “floor” of this minimalist approach is of course a function of existing commitments to our users and clients.

But assuming that we’re doing it well (Hubris warning), we get some valuable benefits: Early Concept Validation Early Customer feedback Faster changes in some cases Costs saving (but mostly because we’re skimping on quality)

We might even be able to break even for a long period of time if the use case does not change. We could increase our margin by maturing the solution but in some cases the savings might not be greater than the benefits of doing something else. Or at least, not early enough. I have seen processes being run more economically by paying someone to do the work manually rather than implement software to do the task.

Long term  

If we consider the success scenario of the product gaining some popularity. Now, the load increases, the number of users increases. Requirements expand and multiply. If we continue with the same approach bad things are likely to happen:

Quality Issues  

What was initially a quick and straightforward process because the absence of many safety mechanisms becomes a path with pitfalls and traps.

The number of issues increases as the scope to test grows and it becomes impossible to reason about side effects of changes. The easy to use and other “No-code” and UI focused technologies often do not support automated testing. There’s no time to test everything manually. The product becomes Brittle.

The performance suffers as the business logic cannot handle an increase in data size or number of users.

Availability Suffers  

The initial solution might have relied on some ad-hoc resources. For example, part of the system could be running on someone’s computer. If that person is sick or on vacation, the capability becomes unavailable. It might have been fine when usage was low but now it’s being felt.

The product might even start to fail under some circumstances.

Things Slow Down  

There might not be any permission system in place as one user was initially running the whole thing. It becomes hard if not impossible for several users to collaborate.

For many of the reasons listed, progress slows as we hit the limitation of the weakest link in the system and the brittleness of the system which forces us to slow down to avoid making mistakes or spend time fixing them.

If it’s the combination of several external services we might find ourselves being endlessly redirected to the other provider for a resolution.

It becomes harder to adapt to the requirements.

Business Risks  

Some key components supporting the core of our business might end up being out of our control or not adequate in terms of their guarantees. For example the increased usage might bring us beyond the Compliance and Security guarantees of the existing solution.

Help! How do I tame my Monster?  

In Business, as long as there’s Money there’s Hope. Once we’ve run out of money, there’s still Life so there’s Hope still.

It’s rarely an “all-or-nothing” scenario. Dump the bath water, but keep the Baby!

Assess the Strategy  

Core Business  

Identify and accept the nature of the business. If it’s software technology, invest in software technology. If it’s not software, partner.

Utility Vs. Differentiator  

Let’s ask ourselves: is this Utility or a Differentiator? Would my customers buy more of my products if I do this in-house?

ROI  

What is the extent of our continued investment so far? Sales, Maintenance, Feature Dev, Support, Runtime, etc. How much are we getting in return? What else could we be doing with those resources?

Assess the Tech  

Inventory  

Review the tech stack. It’s worth going deep and performing an inventory of the technologies, capabilities and considering them in the light of our core business, ROI, etc.

Have an Architecture  

Model the domain. Document the technology decisions that will best support the business goals? Have a diligent process to introduce and remove technology. Reassess on a regular basis.

Clear the clutter  

Clean up failed experiments. Eliminate unprofitable products. Upgrade the tech.

Get some help  

Folks who created the situation are unlikely to be the ones getting us out. They are likely committed to previous choices and unable to see alternatives. This happens when one pour months or years into building something, even if it’s ultimately for the wrong reasons.

Get a fresh pair of eyes to help assess.

Inject a good amount of fresh thinking into the existing teams.

Conclusion  

I hope this was helpful to you. Check out the take-away again. I would be happy to hear more about your specific situation and assist you with any aspects of this endeavor. Feel free to book a free consultation.