Meatware
In previous articles, I touched very briefly on the fact that engineering can be a social activity. A bridge that engineers could use to connect to the rest of the world then use it to make the world a better place by solving its most complex problems.
Everyone today has some need or other that is directly or indirectly dependent on software. A mobile app, a website or just simply a piece of paper being processed through a printer with a person waiting on the other side for it to finish.
In any software engineering process, there are three aspects that have to be in direct contact at all times throughout the entire process of building software. These aspects are the software, the infrastructure, and the people both the engineers building these systems, and the users that are going to utilize them which are also known as Meatware.
Let’s talk about the historical evolution in the communications between the engineers, the users, the software, and the hardware.
Engineers & Software
Engineers and their development environments at the early ages of designing software weren’t in direct contact the way we see software being built today. An engineer today can easily download a compiler with an integrated development environment in a matter of minutes and at once, they are in direct contact with their software to build whatever they please.
But this wasn’t always the case. From the invention of computer programming languages up the mid-1970s, most computer programmers created, edited, and stored their program line by line on punch cards. Engineers would take the card deck to be executed. Depending on backlog, anywhere from an hour or so to a day or so later, you got your deck back along with a printout of the results.
You can imagine at these early ages, the communication between engineers and their development environment was crippled. Engineers in some cases couldn’t even monitor the execution process, they had to submit their punch cards and come the next day to see the results.
Most of the software that was written at a time was done manually, then punched on a card – the feedback process between the engineers and the execution of their code was blocked by the technology of that time. And therefore, an evolutionary step needed to be made to remove these blockers to allow engineers to get the fastest feedback on the software the write today.
This is when the era of desktop computers and desktop apps started to take shape, engineers were able to have their own compilers to write and execute their code – this coincided with the dawn of the open-source era, Linus Torvalds and Richard Stallman and others began advocating for open-source and free software, allowing hundreds of thousands of engineers at a time to have the ability to write, compile and run software with the least cost possible.
Today, there are hundreds of integrated development environments, compilers, and programming languages, available at the tip of every engineer’s fingers to download, write, compile, and run their software wherever they want.
This evolution contributed directly into expediting 33% of the software development processes we have today.
Engineers & Hardware
Writing, compiling, and running software on your local machine could be great to verify your code is doing what it’s supposed to in the shortest time possible. But that isn’t enough to build something someone else could benefit from. Especially if you are targeting a large scale of users who could potentially benefit from your application.
Engineers required a new level of evolution, a new issue that needed to be resolved when engineers needed to take their software to the rest of the world and share it with others who would benefit from it.
For a while there, engineers could write their software on floppy disks then CDs to sell their software – using a computer at a time was a very lonely activity, you felt disconnected from the world, and the best you could do is just use your desktop apps or buy magazines with CDs attached to get newer apps to try. This lasted for a while until the internet became mainstream, and a new age of distributed connected systems began with endless number of possibilities even before the cloud era.
Prior to the cloud era – software engineers couldn’t have direct access in most cases to the hardware or the servers where their code is being deployed and published. An engineer would need to submit a request to what today we would call DevOps or Service engineers to schedule a time for their bits to be deployed and later engineers would receive results on their deployment process.
I’ve lived personally through times where Administrators (as they were called at a time), had to be contacted, and they would sometimes give you the attitude if you didn’t provide an executable or a detailed form for explaining how to deploy your software.
This used to be another huge blocker, the time consumed between developing and deploying software used to be measured in the days – concepts such as continuous deployment and integration were just dreams, something engineers would only hear about but never see in reality.
Then the age of the cloud came in, engineers had full access through both build pipelines and cloud infrastructure to provision, build, monitor and deploy their software without any issues at the least cost possible.
An entire new culture around cloud computing was built, you would hear it on the tongues of founders and CEOs who advocated for the ideas of failing fast, failing cheap and failing soon. Being able to write software, compile it, build it, and deploy it was so streamlined that an engineer could go through the entire software development & design lifecycle within 20 minutes or less! And then their work is already accessible for both contribution and utilization by millions around the planet!
Enabling engineers to own the pipelines for the build and release of their software has removed another 33% of the blockers that impeded the development process from moving further, faster and more efficiently.
Engineers & Meatware
Having an overall control over the design & development components of software is crucial to the success of any business. The world of software design and development have gone through so many iterations to remove the blockers between engineers and the tools they need to finish their work.
And while there’s still a remainder of all of these blockers here and there in some smaller, mid-size and larger companies, a lot of engineering teams have already moved beyond the Lab punch card runner and the Service Administrators decades ago.
But the new challenge that the engineers face today isn’t very different from the one they had to deal with yesterday. Another wall between them and the resource they need to communicate with to get more information about their software.
Today, engineers have to have a middleman (let it be a PM or a PO or whatever the title maybe) to be able to communicate with the customers who would be using their software.
A huge misconception that a lot of engineers bought into, where they thought that they shouldn’t be communicating with the customers or the clients who would be consuming their services. Engineers were told that communicating with customers requires a special set of skills to be able to perform a natural activity, such as talking to people!
I always tell people that engineering software is so much like tailoring a suit, you need to be there, face to face with the client so you can ask them questions, get their feedback and do exactly what they are looking for all in one session!
Imagine if tailors had to have a middleman to schedule meetings with the clients, then other meetings with the tailors to help tailors understand what the clients really need from a size, color, and shape perspectives? How many meetings would need to happen to get exactly what the client needs? Here’s a timeline visual for you:
The red spots there is a time a middleman has to schedule yet another meeting with each party simply to take notes and relay information. Like telephone games! And the midst of all of these meetings some information will be lost, because for a person that relays that information and isn’t necessarily practicing design and engineering on daily basis, they may not really know which parts important and which parts are secondary to the whole conversation.
Here’s the little summary of the outcome of this process:
Broken communications: Engineers and clients are disconnected – there’s no visualization of what the client actually wants, what they are actually saying versus what they are saying.
Loss in Translation: Information is lost, details from the client and specifications and limitations from the engineers might be missing the translation.
Revenue Loss: On both the time wasted to communicate ideas which negatively impacts delivery dates and the extra personnel on the team playing telephone games between engineers and customers.
Some engineers were sold on the idea that customer meetings are boring, and they need to have additional skills to be able to listen and communicate with customers to gather requirements. And I have to say that this is the most absurd idea one could possibly think of when it comes to developing software.
Software engineering is a social activity, it requires engineers to connect to people, then connect people to technology. Opening their eyes to the possibilities and how technology could server their purposes better and take them where they need to go faster and more efficiently.
Now, let’s flip the coin and visualize the opposite of the chart above, where engineers and customers are on the same table at the same time discussing requirements:
You can see in the figure above, there’s as little as one single meeting versus seven meetings in the one before it, and engineers and customers are able to get to a final decision and a conclusion in the shortest time possible.
This visualization is actually optimistic – because it assumes that the customer’s questions and the engineer’s answers are relayed properly word by word between each party. Which isn’t usually the case. In fact, sometimes some middlemen would chime in with a suggestion or a solution that may be perceived by the customer an engineering suggestion or to the engineers as a customer requirement which results in an even greater frustration on both ends.
So, what do we get when we have engineers and customers communicating directly on the same table?
- Transparency: Both customers and engineers know exactly what is intended on both requirements and technical possibilities & limitations standpoints.
- Strong Partnerships: Engineers and customers will develop a strong partnership that earns the engineers the trust from the customers to invest in more features and more projects as they feel their concerns and heard, and their vision is clearly understood.
- Expedition of Productionization: faster feedback and shorter circuit of communications inevitably results in faster delivery and a better return of investment on every party’s time and money.
Given my history as both an entrepreneur and an engineer, I’ve always found it to be extremely beneficial for engineers to engage in direct customer-to-engineer communications to increase their ability to negotiate features, propose newer approaches and deliver better values.
Learning to communicate with customers should be as important as learning a programming language, a new pattern or framework to engineers – without that direct contact a tremendous damage to the design, engineering and development process would be sustained which at the end greatly harms the engineering experience.
It’s time for the decision-makers in every engineering process to start working on removing that last hurdle in achieving 99% efficiency in the design and development of their software.
It’s also time for individuals who built their entire careers on playing telephone games between engineers and customers to start seriously thinking about learning how to develop software, and use their existing communication skills to become a part of the crew that drives both requirement gather and engineering processes all together.
It’s no accident that most the successful software companies in the world today were founded, established, and operated by 100% engineering crews. Engineers who did all the work from requirement gathering to innovation, and from design to development, and from deployment to delivery.
A successful company like Tesla today invests more time into perfecting their product than wasting it on marketing and public relations. That’s simply because Musk believes that a good product is self-marketing – You don’t need someone with an MBA to drive your innovation as an engineer – if you are smart enough to solve some of the most complex engineering processes then you should be smart enough to talk to people who would benefit from your solution and understand their issues then help them solve it.
Now, someone might ask, what about the 1%? How can we achieve 1% efficiency in our engineering process? Here’s my two cents:
Engineers & Engineers
Engineers working on the same team could have every potential to succeed, they could have direct communications with their customers and faster feedback for their products, but they would still fail. Why? Engineer-to-Engineer communications.
Engineers working together is a huge aspect of ensuring the success of any development process. Pair Programming for instance, ensures engineers are cooperating, brainstorming, and prioritizing the distribution of knowledge and quality of software all at the same level.
Up until very recently, engineers were very obsessed with the savior model – the one guy who works by himself and knows the system end-to-end – no one knows how the system works but him, and no one dares to push any code to the source without his consent.
This is what I call SAM (Savior Architecture Model) – it’s a team architecture that gives an opportunity to silos and isolation of knowledge to exist which results in a poor team communication and even poorer engineering experience.
Without engineers perfecting the communication with each other, there would be no strong foundation for the development process to stand on. Regardless of the budget of the project or any other benefits engineers may get by working on that project.
When engineering experiences are put as the main goal and the highest priority, a spirit of success and excitement shall engulf the overall process which empowers the engineers to take ownership of their product and push it to places never thought would be even be possible to imagine.
The Future
Systems design and engineering processes have gone through so many different iterations since the 70s. But the one aspect that maintained throughout all these years was growth and evolution. With every iteration there was always those technologists who pushed the industry beyond it’s limits and opened the doors for the rest of us to look forward to the possibility of one day we would do it ourselves and change how everything works forever, once again!