Use cases

Learn about GlassFlow use-cases and real-world examples.

Streaming data pipelines

Data pipeline applications involve the movement and transformation of data from various sources to destinations like databases, data lakes, or analytics platforms. These pipelines are essential for data integration, ETL (extract, transform, load) processes, and ensuring data is available where and when needed. Many modern distributed applications require real-time data processing capabilities, which traditional batch processing pipelines struggle to provide.

GlassFlow operates in a continuous streaming mode instead of being periodically triggered. It can read records from sources that continuously produce data and move them with low latency to their destination.

How GlassFlow helps:

• GlassFlow efficiently transforms data streams with custom logic to fit the requirements of downstream applications or cloud storage solutions.

• Developers do not need to deal with infrastructure and data scientists should not need to know Kubernetes. GlassFlow provides a common platform suited to each role's specific needs. Data scientists can plan their data models and immediately run them on real-time data streams.

• With the CLI tool, GlassFlow integrates smoothly into existing CI/CD workflows, making it easier to deploy and update data pipelines alongside application code.

• GlassFlow streamlines the process of capturing, transforming, and loading event-based data into data warehouses. Data is loaded into the data warehouse continuously in the right format, at the right time.

Example pipeline diagram for event-based data loading to a data warehouse

Event-driven applications

Event-driven applications respond to actions triggered by users, systems, or sensors, requiring real-time processing to react promptly. They need to be designed to respond dynamically to events or messages rather than relying on traditional request-response interactions.

How GlassFlow helps:

• GlassFlow excels in the real-time ingestion and processing of events, ensuring that applications can immediately react to new information.

• GlassFlow's architecture is designed to manage out-of-order events and late-arriving data efficiently.

• GlassFlow provides seamless integration with a variety of data sources and sinks, such as NATs for message distribution and Change Data Capture (CDC) services like Debezium by directly connecting to your primary database databases like PostgreSQL and MySQL for storage.

• GlassFlow complements the microservice architecture, where applications are decomposed into smaller, independently scalable services that communicate through events. GlassFlow processes those events and maintains asynchronous communication for microservices.

Example pipeline diagram for event-driven message exchange

Streaming data for ML applications

Machine Learning (ML) applications increasingly rely on streaming data for real-time analytics, predictions, and decision-making. Common examples of such workloads include machine learning for e-commerce websites, real-time bidding, and mobile gaming.

How GlassFlow helps:

• GlassFlow ensures that ML models have access to the most current data to make accurate predictions based on the latest information. ML models can be continuously trained and updated with new data so that models remain effective as data patterns change over time.

• GlassFlow's serverless architecture allows for the scalable processing of high-volume data streams by running parallel processes. This ensures that ML applications can handle large datasets without degradation in performance.

• GlassFlow enables ML engineers to effortlessly construct pipelines for processing real-time events, extracting vector embeddings via transformation functions, and continuously updating vector databases.

Example pipeline diagram for real-time feeding a vector database

Other real-world scenarios

Fleet management	In the context of fleet management, GlassFlow can be utilized to improve operational efficiency and customer satisfaction. By integrating real-time data sources, such as vehicle fuel level detectors and GPS locations, GlassFlow can process this data to detect vehicles with low fuel. It can then identify nearby gas stations and automatically send notifications to users that they get a trip for free if they put in 50 EUR of fuel.
Meta searches for travel	For travel platforms like Trivago, GlassFlow can streamline the process of aggregating and processing data from various hotel APIs. By efficiently handling real-time data streams, GlassFlow can score and rank hotel options based on user preferences and other criteria. This enables platforms to offer personalized and timely recommendations, improving the user's search experience and potentially increasing booking conversions.
Arrival times in logistics	GlassFlow is particularly useful in the logistics sector, where timely and accurate information is crucial. For a food chain company tracking the movement of trucks from farms, GlassFlow can process GPS data, and temperature readings, and integrate with Google Maps data to predict arrival times accurately. This allows partners to plan their operations effectively and guarantees the integrity of the cooling chain by providing real-time alerts if any disruptions are detected.
Advertising and bidding platforms	In the advertising industry, especially for platforms offering real-time bidding, GlassFlow can play a critical role in processing bids, combining data from various partners, and applying proprietary scoring algorithms to determine the best ad placements. By handling high-volume data streams efficiently, GlassFlow ensures that ads are displayed in the most relevant spots in real-time, maximizing visibility and click-through rates for advertisers while enhancing the user experience for the audience.