How Spectrio cut costs and boosted analytics speed with DoubleCloud

For more than twenty years, inReality has been leveraging technology to help offline retail stores boost their revenue. It started as a marketing firm but evolved to offer retailers advanced analytics on how customers interact with their products and advertising.

InReality uses radar technology (instead of cameras or cell phone trackers) to build an IoT application that can collect data on store visitors' movements and how they engage with advertisements, enhancing customer privacy. The data gathered from various sources — including radar, existing security camera footage, online activity, weather patterns, transaction records, and CRM systems — equips retailers with a comprehensive view of consumer behavior.

Additionally, inReality focuses on KPIs to enhance their data-driven strategy. InReality’s platform is designed for real-time triggers, enabling personalized content display based on current data analysis. This approach allows for more targeted marketing, improving customer engagement and helping retailers optimize their marketing strategies. The integration of diverse data sets enables inReality to provide a nuanced understanding of consumer behavior, aiding retailers in making informed decisions for product placement and advertising.

Spectrio, one of the nation’s leading providers of comprehensive digital signage systems, recently acquired inReality in July 2023. This acquisition complements Spectrio’s technology stack by combining dynamic digital signage with AI-powered analytics technology. Together, they empower customers to make data-driven marketing decisions that enhance the customer experience.

InReality’s previous data infrastructure was built on Spark and Databricks, systems that were modern at the time but later struggled to adapt to the increasing demand for real-time, efficient data processing. However, after being acquired by Spectrio, inReality’s team faced a few challenges that compelled them to seek an evolved analytical approach. For instance, accommodating GDPR-like data subject requests (DSRs) became increasingly difficult, as data had to be meticulously tracked and erased upon request. The team encountered this scenario in a project where they collected temperature screening and COVID-related data.

The shift to Snowflake was a strategic move to consolidate data processing and analytics in one platform. Snowflake’s performance and the convenience of a unified architecture appealed to inReality, providing them with the agility they needed. As data volumes surged, however, operating costs increased dramatically along with them, and the team faced complexities in managing the entire data infrastructure. The data stack’s high cost became a financial strain, especially when real-time updates became necessary.

Moreover, the Snowflake infrastructure required complex data handling, including multiple stages of processing and transformation of data from radars transmitted through MQTT, which became cumbersome and time-consuming.

In search of a technology that could match Snowflake’s high performance at a fraction of the cost, inReality discovered ClickHouse and DoubleCloud. The DoubleCloud platform built on open-source and cost-effective components such as managed ClickHouse presented a more efficient yet effective alternative that allowed them to process and visualize hundreds of gigabytes of data daily. During the proof of concept stage, inReality received substantial help from support engineers and solution architects, playing a critical role in the testing and tuning of their data pipeline, which included, for instance, the use of materialized views in ClickHouse.

InReality focused on 'dashboard loading time' as a crucial metric, knowing that a good user experience (UX) depends on quick access to information, as users typically have little patience for long loading times.

After a thorough evaluation and a series of optimizations, inReality successfully reduced dashboard load times from 17 seconds to a remarkable 1.38 seconds, achieving this with 250 million records representing 1.5-2 months of data. The transition to Managed Clickhouse not only addressed the financial constraints but also significantly improved query speeds, enabling a faster decision-making process for their clients.

Currently, DoubleCloud facilitates inReality’s real-time insights into advertising content, significantly enhancing the consumer experience by tailoring displays to demographic profiles such as age and gender. inReality by Spectrio has leveraged the DoubleCloud platform’s capabilities to create a variety of visualizations and KPI dashboards, which now efficiently report traffic and engagement by the time of day, dwell times, and interaction with specific products. This comprehensive analysis has led to an increase in the product’s margin and has made it more competitive in terms of pricing.

As the company continues to scale, the affordability and scalability of the Double Cloud platform will play a pivotal role in supporting Spectrio’s ambitious growth plans. The data lake solution built on open-source Managed ClickHouse by DoubleCloud has become instrumental in their internal strategy, allowing them to add to their roadmap by creating internal sales dashboards that integrate disparate data sources into a unified reporting interface.