Create a pipeline

This page explains how to create a new data pipeline in GlassFlow.

What is a Pipeline in GlassFlow?

A pipeline in GlassFlow defines how you need to process your event data. It forms the basis of everything you do on GlassFlow.

A pipeline broadly consists of 3 parts:

1. Transformation Function - Python function that describes your business logic on what you want to do with the events data

2. Data Source - How to bring data into GlassFlow

3. Data Sink - Where the data should go from GlassFlow after transformation

A pipeline within GlassFlow always lives within a space. A space is a way to logically organize your pipelines. As soon as you create a pipeline in GlassFlow, it is automatically deployed inside the GlassFlow cloud and is ready instantly.

Creating a Pipeline

There are several ways to create a pipeline. No matter which way you choose, they all live in your GlassFlow account and are deployed and ready.

Create the pipeline via the WebApp

GlassFlow comes with a web application that you can use to create your pipelines. It provides a visual interface and gives additional information after the pipelines are created. The web application is available at GlassFlow WebApp. If you do not already have an account, you can simply sign up for a free account and start using GlassFlow.

Create the pipeline via python

GlassFlow's Python SDK can be used to create pipelines, completely in Python. Our glassflow-examples GitHub repo has several examples and a getting-started guide showcasing how to use python to create glassflow pipelines.

Note: You will need to first signup via our WebApp to create an account and get your Personal Access Token before using the Python SDK