Backend Optimization for Real-Time Monitoring
Project Details:
- Client: SENTRY Water Monitoring & Control Inc.
- Industry: Water Treatment
- Solutions: .NET
SENTRY™ Water Monitoring & Control Inc. develops cutting edge wastewater monitoring and treatment solutions. The Sentry solution by SENTRY™ is a unique bio-electrode sensor technology that allows for real-time monitoring of microbial metabolic activity in anaerobic and aerobic wastewater treatment systems. SENTRY™’s clients—usually water treatment centers—can view the bacterial activity in the water in real time, as well as manage their sensors through SENTRY™’s dashboard. Clients are also able to configure various alarms in case measurements record unusual activity. SENTRY bio-electrode sensors are a commercially validated solution for measuring wastewater quality in different units of a wastewater treatment plant (WWTP). Their clients span across each continent, with the exception of the Antarctic.
Client Key Facts
650+
Operators Benefiting from SENTRY’s Data
$75k+
Average Annual Savings Using SENTRY
100+
WWTPs Served
The Challenge
SENTRY's online real-time monitoring dashboard is a critical aspect of their service and it allows clients to view sensor signals live and analyze historical performance. As SENTRY continues to grow, the volume of stored data and the number of concurrent users have increased rapidly, which has begun to slow down dashboard performance.
TORQ was asked to optimize the backend infrastructure to improve responsiveness, enhance scalability, and reduce data storage costs, all while maintaining uptime for their active clients.
The challenge was optimizing the large amount of readings. TORQ couldnt just optimize the current the table as it was contiously being wrote to. Each sensor writes a reading every second. This had to be done with little to no downtime, so it had to take place on the weekend to prevent that.
Key Issues Faced:
- Dashboard response time was slowing down due to increased data volume and concurrent usage.
- Legacy database structures were not optimized for large-scale queries and daily user access.
- The optimization process needed to be completed with minimal downtime to avoid disrupting client access.
- Historical data migration needed to be handled securely and without data loss.
"Collaborating with TORQ has been incredibly smooth. They helped us maintain and improve our backend infrastructure without disrupting our live services. The improvements are immediately visible: faster dashboards, more efficient storage, and a solid foundation for scaling our data platform.”
Kash Delsoz Bahri
Lead Data Scientist & Client Manager, SENTRY Water Monitoring & Control
Our Solution
TORQ designed and executed the full optimization plan, including database restructuring, query optimization, and storage compression. The main solution was to change the way SENTRY stores some critical parameters. This had to be done without interrupting the online dashboard (receiving data) while keeping the downtime limited (less than 12 hours) for historical values. There were a number of things that TORQ had to get done to solve this.
The Solution:
- Create a newly optimized table
- Stop IoT Devices from writing to the current table
- Transfer data into a new table with the newly optimized columns
- Update the code to load readings from the newly optimized table
- Update IoT Devices to write any new data to the newly optimize table
- Turn IoT devices writing back on
Solution Highlights:
- Consolidation of historical datasets into optimized tables for faster retrieval.
- Improved data compression and partitioning to lower overall storage costs.
- Real-time monitoring of performance metrics during and after deployment to verify stability.
- Updated the table without intrupting receiving the new data and with minimum downtime to transfer historical data (within couple of hours)
The Results
The optimization resulted in a faster, more reliable, and scalable monitoring dashboard. Data retrieval times improved significantly, allowing clients to access reports and historical visualizations without delay. It also enabled faster queries with external applications such as PowerBI.
Key Benefits:
• Faster dashboard
• Faster queries
• Reduced required storage capacity
350M+
Reading Sensors Optimized
Faster Exports
Render a years worth of data in 5 seconds
