Azenqos is a product offered by FreewillFX solutions that uses mobile handsets that interfaces with a web backend to measure mobile phone signal quality. This allows telecommunications companies to identify coverage areas where their subscribers are not getting good service.

The problem with most consultants has been that they have no real world experience in the technologies they’re consulting on and have no stake in the outcome. I like working with Mishari Muqbil because he not only recommended and helped us solve our technology problems but also patiently help us improve the way we work with it, allowing us to dramatically reduce our turnaround time for developing client features and the time we’ve spent troubleshooting significantly.

– Kasidit Yusuf CTO

The ChallengeAzenqos has an architecture that comprises of two parts: a mobile front end that collects signal quality data and a backend that performs an analysis. As the number of clients grew, the volume of data grew, and the overhead and complexity of managing the services for each client increased. Scaling up the processing to handle big data became an uphill battle, with unexpected heavy workloads causing the system to crash repeatedly. Also, the pressure to deliver features to users means manual code tests were not being performed thoroughly, resulting in issues that ranged from minor annoyances to significant malfunctions that led to delays to feature launches. The issues were also starting to have a demoralizing effect on the team who are now spending significant time in trying to keep the system running instead of adding value for their customers.

The solution

The first step is to make an assessment of the system, this was done simply with weekly phone calls with the CTO of Azenqos who would express what went well during the week and what didn’t go so well. This is a simple and effective process that over the course of several weeks would reveal much about the organization, it’s aspirations, culture as well as the challenges it faces. Then solutions were developed by Zymple, extended by Freewill, tested then deployed. This was accompanied with process improvements at Freewill in order to improve the way their organizations interacts with the technology.
The services included:

• CTO Coaching – help the CTO in improving processes incrementally by changing the way to look at the problems and aligning technological goals with business outcomes. One percent per day adds up to one hundred percent after 70 days
• PostgreSQL DB Troubleshooting and Optimization – to remove bottlenecks and increase performance
• Writing Ansible scripts – to automate the process of bringing up a new server
• Interfacing with clients to procure on premise servers – explaining to clients what the resource constraints are
• Implement monitoring system with application metrics – this allowed Freewill to monitor business metrics in real time
• System hardening – enhancing security and mitigating security issues
• System architecting – decrease interdependencies of different modules so that each module can be developed and tested independently; this allowed better scaling and decreased complexity in development
• Help with vendor procurement – helped with a vendor selection process for an advance GIS development project

Results:From discussions with the CTO, it was established that a major problem being faced was the length of time for deployments was very long and error prone. Different servers also had different code base which made it difficult to manage. Deployment time was reduced by automating the process and making it repeatable, reducing the deployment time for new systems down from 3 days to 1 hour. This freed up man hours to work on client facing features and dramatically increased the rate at which Azenqos can onboard new clients. The team now also does less work manually, using automation as a proxy so that interactions with the system is repeatable and scalable. To take advantage of the new deployment system, the team’s behavior changed to standardizing the codebase to one version, making it easier to manage and increasing productivity.
Another recurring issue that came up was the frequency of crashes, which is now reduced from several times a week to effectively zero, giving clients a better experience and improving their confidence in the products. This is achieved through a rapid feedback using monitoring that was added in place, allowing the Azenqos team to be able to monitor key indicators of service quality, allowing their team to know when a system is starting to malfunction before the client is aware that a potential incident is taking place. With the monitoring, usage patterns that trigger problems have been identified and mitigated.
Once the distractions from an unstable system was removed, it’s time to move on to enhancements and growth. The next priority identified with the CTO is improvement in code quality, so automated tests were implemented, so any new code submitted has a high degree of confidence that it didn’t create any negative effects in other parts of the system. Team morale improved as conflict between team members reduced because there was no more blaming each other for breakage. This enabled the team to take on more ambitious projects like adopting a Saas model, which will allow Azenqos to multiply their profitability.
In summary, because of good systems and process design introduced Azenqos workflow, the way the developers interact with the system improved, creating happier development teams and happier clients, while making the entire system simpler in the process. We make it Zymple.