Companies unavoidably have to go through digital transformations from time to time. Technologies get outdated and business requirements evolve, making your tech stack obsolete.
What does an engineering leader have to do in this situation?
How do you build the right tech stack for your business to help you solve your problems?
Steven Lopez, VP of Engineering, Technology and Operations at Deem, talks about the three elements of digital transformation that help you choose the right tech stack. He gives valuable advice on defining the problem and the success criteria, getting the leadership and the engineering teams on board, and finding the right tech stack through careful planning.
This blog post was written based on episode 72 of the Level-up Engineering podcast hosted by Karolina Toth.
This post covers:
You have to consider the three elements of digital transformation to know how to choose the right tech stack. The major aspects of digital transformations are the following:
The spot where these three areas overlap is where digital transformations come together. Leaders tend to make the mistake of trying to find the right technology first. I recommend starting with the people. Digital transformation is about people processing technology.
Think about the problems you’re trying to solve. Understand the people you work with, visualize them working with the technology that solves those problems, and choose the tech stack based on these. If you don’t have the right people to install and utilize a tech stack, then it won’t help your business.
I consider the entire software development cycle in the tech stack and all the tools that support it. This holistic approach differentiates digital transformations from regular projects.
For example, CI/CD includes what technologies and software programming languages you use. But it doesn’t end there.
Your tech stack also includes the type of version that your users have and their toolsets. If you’re doing CI/CD, you need a strong set of tools and disciplines around it to keep up with new versions as you put them into production.
You also need to make sure to customize the tech stack according to the company’s industry. Each industry has different needs when it comes to choosing the right tech stack. For example, in fields that use credit cards, you have to take PCI security standards into account.
Ten to 15 years ago, digital transformations were different. People came up with a tech stack, and they would implement it right away. Technologies are moving so fast nowadays that I recommend defining the tech stack only after you do your due diligence and digital transformation experts have looked at your company thoroughly.
In the age of agile implementations, we have to be flexible enough to try new things. We set hypotheses, and we try out whether they work, and we make necessary updates.
Even here at Deem, we started with a tech stack three years ago. Only afterwards did we identify the type of CI/CD tools, linking tools, and scanning tools we’d use.
As we started expanding those transformations to our frontend, middleware, and backend, we noticed that we needed to adjust some of the scanning tools, the linting roles, and the standards we’d put into place.
These all became part of the solution. When we put that stack together, and we used a more holistic view.
Digital transformations occur when a company or an IT department hasn’t invested enough into the tech stack they’re in. They may use outdated technology, which hinders their ability to keep up with market trends. In such cases, hiring experts to help with digital transformations can help.
Experts will first analyze the current state of the business. They help you determine what problems you’re trying to solve. It could be a quality or product-related problem, a difficulty with monitoring issues, the speed delivery, or a mix of various problems at the same time.
After addressing the current challenges, experts will ask you to describe your strategic vision for the product. It helps them understand how to put changes into production to get the desired results.
At Deem, we wanted to ensure that our customers got the proper care, quality, and response time, so that we could follow market trends and demands. Although the travel industry in itself hasn’t innovated much, they’ve done a lot of integrations between different partners. The biggest challenge was to react to a personalized environment.
We tried to find a way to give a quality product and a highly personalized experience to our customers, so that we knew who they were and what they were looking for. If we know that you’re on a business trip and you like Italian food, we will give you a list of Italian restaurants near the hotel you’re staying in. We can tell you what time these are open and the easiest way to get there, and we can offer to make a reservation for you.
We didn’t have the technology in place to give such a personalized experience at first. We needed to install a brand-new set of technology for these goals, but we didn’t start by getting the new tech stack; we looked at the people and the problem we were trying to solve first.
Working on new tools and building and fixing problems are exciting tasks, especially for engineers, but you have to know the entire context of the problem first. If you don’t, you might end up solving a different problem, not the one you have.
Take a step back and make sure everyone understands the problem you want to solve. It might get complicated, for example, if you decide to switch to a new platform from an antiquated one, because your team will need time to get used to it.
That’s what happened with us here at Deem. We went from a 10- to 15-year-old technology to a very current cloud development and CI/CD. It took us a few weeks to figure everything out, but in the end, we were able to implement it.
The biggest difference between picking new tech stacks for projects and digital transformation is that projects focus on implementing one asset or tool, while digital transformation focuses on a more holistic approach. In digital transformation, you have to look at everything from how, what, and where you develop, and what tools you’re using.
Digital transformations aren’t about specific projects, they’re a series of integrated components. You have to consider them as one unit, but you don’t have to solve them all together. The key is to understand the context of the components, bring them together, and solve the problem.
For instance, we wanted to prioritize speed at Deem. We immediately moved to a CI/CD model.
The idea was to make small changes quickly, but this type of approach comes with a cost. When you use microservices with a distributed architecture, you need more people. It keeps you from doing large releases and implementing them immediately. If you have a bunch of moving pieces in a microservices technology, you can test them one by one, but testing can also be automated.
If you go with a distributed environment, then you have to install test-driven technology. You have to put more in the pipeline, so things are tested as you develop, and you have a fairly good idea of the quality of the software once the developer has finished the code.
It takes a lot of work because you have to take the requirements into account, too. You have to have product managers who understand test-driven development. They know how to put together qualitative tests that can determine your team’s success.
We put those test cases into the actual pipeline development so we can validate the quality of the code as the developers are working on it. These tools and techniques require the right people, the right technology, and a deep understanding of the problem you’re solving.
Communication and project management tools are critical when you choose a tech stack. The right stack isn’t just about the programming language and the cloud services that a business can use; it is also about the project management and communication tools. Tools like Jira can help you in managing the processes of the transformation.
The monitoring components are also part of the tech stack. Some companies prefer to test as they run, especially when conducting a digital transformation. You have to monitor how well automatization runs and what the fail rate is.
You have to be able to see issues before the customers face them. If your monitoring components are in place, what the customer will see is that the product works the way they expect it to work.
Decide whether you prefer a Kanban approach for error detecting and resolving issues, or if you prefer to do it via scrum. These are all part of the solution. You have to track these issues and maintain them as part of the product life cycle.
Transformations can impact and benefit company culture. At Deem, we have clear values and cultural components associated with the company itself. Our aim is to align our processes with the company’s values.
One of our values is to put customers first, which is a value proposition. Engineers should know the company values and understand how they can contribute to them in their work. For me, the reason why I chose CI/CD or rapid development is specifically to ensure that we place customers first.
We want to be able to give them the quality they’re looking for. We want to detect and resolve any mistakes in production before they see it. These practices are all part of our “customers first” approach, and we have to choose our tech stack accordingly.
Choosing the right tech stack is about understanding the values of the company, knowing your team, and being able to align these two aspects. The tech stack should support this alignment.
At a former company, I was asked to come in and lead a digital transformation. By that point, they had already made a large investment in certain software licenses. It’s best to do these transformations when you have control or at least flexibility in all three aspects of digital transformation.
In this case, however, the technology was already determined.
In such situations, you have to use the other two aspects the best you can to influence the use of technology. It ties you down in multiple ways and might hinder the success of the transformation. You’re trying to fill in the gaps, but you might not find the perfect solution because the tech stack has already been determined.
Companies going through digital transformations should know that purchasing technologies that don’t support their problem-solving processes can cause more problems in the long run. In the case of my former company, we made the transformation as successful as we could, and they were happy with the solution. However, it could have been more successful if they were more flexible on the technology.
When I have control over all three aspects of the transformation, that’s Nirvana. It’s very fortunate and very unusual to have the people, the technology, and the processes all as flexible as possible.
As exciting as it may be, don’t jump into solving problems right away. Assess the current situation first because it’s going to be different at every company. Look at the people, the technology, and the processes, and understand the problem you have first.
Once I have a better understanding of the problem, I evaluate the leadership I’m working with. Sometimes the leaders and stakeholders don’t understand digital transformation, and they’re hard to collaborate with, especially when you’re transforming something they’re used to.
Make sure that leaders and stakeholders understand the problems you’re trying to solve. It’s also important to understand what type of talent you need to solve the problems.
In the last three companies I’ve worked with, we were moving from on-premises development to cloud solutions. Being able to code in the cloud means you can have developers anywhere, which is often part of the success criteria in digital transformation. COVID taught us that resources need to be distributed, and cloud environments help you achieve that.
At each company, the same type of issues kept coming up.
They wanted to work on new features for their customers. To achieve that, it was necessary to move to the cloud, and we had to convince the leaders that it was a critical step.
We needed to get them on board with cloud because we couldn’t move without their collaboration. They had to give us 30% of the engineering teams’ capacity so we could make the digital transformation work.
These moves are expensive, so you often need multiple conversations with leadership to get them on board. If they have the money and the fortitude to get through these phases, you can start implementing it without problems. Tension generally happens when people don’t understand these ideas, so they resist it. Getting everyone on the same page often gets everyone on board.
Once these components are identified in the first 4-6 weeks, I look at how I want to solve the problem, and I set up our success criteria.
When you have clear success criteria, it’s easier to plan each step and to have alignment throughout the transformation process.
Digital transformations are similar to marathons. When you’re running a marathon, you don’t want to think about the 26.2 miles you have to complete. You want to break it down to smaller, doable milestones.
You can do the same with a digital transformation.
Think about the current problem you’re trying to solve, break it down to small pieces, and keep your plans flexible. You don’t know what mile 16 is going to look like when you start the marathon. Similarly, you can’t predict everything that’s going to happen during the process of digital transformation.
Even though you have a plan for each phase of the transformation, you need to be flexible enough to adjust along the way. You may have to adjust to changes in the technology and the process, and it becomes a step-by-step transformation. Focus on the next mile, the next phase.
As long as you have the storyboard of the problem, the solution, the success criteria, and a vision of how you’ll get there, you can be flexible in your approach.
My success criteria always includes security.
I make sure that the tech stacks are in compliance with PCI security standards, and IT audits are also a part of this process. My approach was to institute it as part of the software development process. As you’re developing, it automatically gives you the documentation you’ll need, so you can plan ahead as you go through the transformation.
Once I understand the problem we’re trying to solve, and I know what success looks like, I look into the tools. At Deem, we wanted to make sure we understood the quality components, so we decided to use a distributed architecture. That was our first step.
In the first six months of the digital transformation, we defined the entire stack. We looked at all the data stores, the security, and the technologies of the frontend. We looked at how we wanted to transform the frontend and the backend because we have customers who want to use our platforms throughout the transformation, too.
As I’m implementing strangler patterns to take functionality, rewriting them, and plugging those things back in, I have to make sure that nothing happens to our customers around the platform.
The analogy I like to use is that you’re driving an airplane full of people 5000 feet high in the air. You’re switching out the engines, the wings, and the communication channels, and you keep the plane 5000 feet high in the meantime, but sometimes, you can go up to 7000 feet. This translates to you adding more revenue and more customers to your business. Keep this information in mind and make sure that the infrastructure helps you achieve your objectives.
Make sure that you have the right people to work with. When you’re doing digital transformations, you might need new people who are focused on the new technology, and people who have the domain knowledge of the old platform.
My secret to choosing the right tech stack has always been careful planning. We identified the functional pieces we wanted to replace. We combined the team that knew the most about the old platform with those who knew the new platform, and we let them rewrite a solution that was better than the current one.
In digital transformations, the team on the older platform may feel vulnerable because the functionality they specialize in gets replaced. By having them rewrite the program with the new people, they’re also part of the solution, and they don’t have to worry about losing their work.
I believe in partnerships, collaboration, open-mindedness, and transparency. It makes the working environment easier, and it helps us stay consistent with trust, which is one of our values at Deem.
Build psychological safety while working on a digital transformation in your organization. The whole transformation usually takes 3-5 years, and it takes a lot of collaboration and trust.
One of the most common problems in digital transformations is tech debt. It occurs when you have an older version of the stack, and then you have the new stack that you’re building. The company wants a new approach and a new stack, but in most cases, they still have to maintain the existing platforms.
This means that the company has tech debt, which is the debt they haven’t paid down on the old stack. Working on tech debt takes away some of the capacity we could be using to concentrate on the new tech stack, and it can create some tension. In these cases, I ensure we don’t get misaligned because of this issue.
Make sure that the company understands the journey you’re about to embark on. This helps you align, create product roadmaps and a technology roadmap, and shows them what we’re working on.
Moving forward takes coordination and compromises. Conflicts can still occur. A third-party vendor can force the company to upgrade their APIs; otherwise, they shut it off. You can’t always plan those situations, so you need to be flexible enough to handle them.
At Deem, we decided on React, and we went with Google Cloud Platform for our Cloud solution. We also decided to use a lot of the native components of Google’s platform.
We used gRCP calls, and we created an abstraction layer using GraphQL to be able to run the frontend and the backend at different speeds. By abstracting that, we were able to use some of the old APIs as we updated the frontend, and we could show the new pages and new mobile apps to our customers as soon as possible.
We created a lot of mobile apps and websites using GraphQL to abstract. You couldn’t always tell whether it was the old or the new version because they coexisted. We started changing out old APIs while adding in other pieces to solve other issues, such as monitoring, logging, error detection, and test automation.
When we looked for testing tools, we selected multiple tools to make sure we had the entire stack fully tested. We were flexible and adjusted as we moved forward.
It has been a great experience for everyone involved. Sometimes, lacking excitement can also lead to problems. If you can show your team the plans, and you can connect it to the company’s values, it tells the entire story, which can motivate them.
Your chance to succeed is much higher if you first focus on understanding the problem at hand. Don’t stay in the problem zone forever, but spend enough time with the problem to ensure your success. Stay in the problem zone long enough where you really understand the issue as well as the success criteria.
You may find that the newest technology you’re excited about won’t help you find the solution you’re looking for. As engineers, we often start with technology in hand, and then we try to find a problem. Do the reverse: define the problem first, and then choose the best technology to solve it.
Give yourself time and don’t rush the solution. Connect the dots and find the best way to solve the problem.
When it comes to digital transformation, the faster you fail, the better. Don’t dwell on technologies that didn’t work out. These transformations can get messy, and that’s okay.
The idea is to try new things. Test your hypotheses and learn not only from your successes, but also learn from your mistakes. It’s all part of digital transformation.
Stay flexible, recognize if something wasn’t a good pick, and adjust as quickly as you can. Technology changes every day. It keeps growing, and that’s the beauty of it.
Steven has been VP of Engineering, Technology and Operations at Deem for almost 3 years. His mission is to help people travel easier using Deem’s products.
In the past twenty years, he has helped multiple organizations establish a digital vision. He has extensive knowledge on how to choose the right tech stack that reflects each company’s values, brings out their employees’ talents and maximizes their efficiency.
He has completed an Ironman Triathlon and ran more than 25 marathons so far.
👉 If you are serious about becoming a great engineering leader, you should take a deep dive into the State of Engineering Management 2022 report.
🚀 Need developers for your team or project? Hire our experienced Angular, React or Node.js developers! Click here for a FREE consultation.
About the author:
Dorottya Csikai is a content marketer at Coding Sans. She has experience in journalism, interviews, management and content marketing. She researches, writes and edits posts for the Engineering Leader's blog.