Integrating Legacy System With monday.com
Our client had an internal custom system to manage processes and data in their organization for years. The system is rather big and complex, capable of gathering and combining data from multiple sources.
They discovered they could further optimize their processes with the monday.com platform. Managing data inside monday.com turned out to be much easier. They could customize the platform to their specific needs without waiting for software developers. However, they had to migrate the data from the legacy system and keep it in sync with upcoming updates from the different sources this system could handle.
At first, our client tried doing it manually. They soon realized this task would take a lot of time and is error-prone with a massive amount of repetitive human labor. While initial data migration is still manageable, constantly keeping it in sync with updates would be a nightmare.
They started looking for a software partner to help them automate this process and found our in-depth article on how to build monday.com apps. After an introduction meeting, we started working on a possible solution together.
Discovery
The client planned on using monday.com for everything and had no intention of keeping the legacy system and supporting it any further. Still, re-implementing all the functionality from scratch as part of a new application would be pretty expensive and time-consuming.
Keeping the legacy system for automatic data gathering and normalization seemed the best solution. Exporting already processed data to monday.com would be quicker and more cost-effective. We would develop an app to import this data to the platform and keep it in sync with the updates.
Updates to this data often started specific processes in the organization. Notifying their employees of any changes was critical. So was ensuring that the synchronization logic runs smoothly, without any errors.
Solution
With the rough high-level plan in place, we started discussing more details with the client. Synchronization frequency was not that critical - two times a day would be more than enough and much better than doing everything manually. We decided to go with the most straightforward solution - their developers would extend the legacy system to dump all the data in a CSV file and upload it on an FTP server where our app could pick it up for processing.
Importing the data wasn’t that simple. They had multiple monday.com boards for different data and complex subjective rules to decide where each item belongs. While automating those is possible, it would require additional development when those rules change in the future. Since new records aren’t added very often, we decided to keep it manual. Each board has a different integration for additional control of item mapping from CSV columns to board columns.
The most important feature was to notify the employees of the synchronization results. For convenience, all notifications appear inside the monday.com platform. After updating items - they would know what changed and can react accordingly. When new data appears in the CSV - they know they have to manually create an empty item with just the legacy system’s item ID on a corresponding board. Our app would immediately populate it with the latest data from the CSV. And, of course, if synchronization fails for some reason, like a malformed CSV file, they would get a notification.
The client planned on using the app for years. To save on the infrastructure cost and maintenance, we decided to go serverless on AWS. The app consists of only three lambda functions:
- Download and parse the CSV
- Upload the data to monday.com
- Handle new items on the boards and populate them with the latest data.
It uses a few DynamoDB tables with the latest data from the CSV file to populate new items and save on the monday.com API calls when uploading the data. All the necessary infrastructure would cost a few dozen dollars a month and requires zero maintenance.
Results
The client is already using our monday.com app for more than a year, and the development paid off with saved costs on manual labor. As a bonus, automation produced zero data bugs while humans doing tedious, repetitive tasks easily could.
After our successful cooperation and achieving outstanding results with this app, the client asked us to build another one. The development stage is already completed. They are using it internally and thinking of publishing it into the monday.com Apps marketplace soon.
The client is happy, and hopefully, we will work together on other helpful solutions in the future!
