GlassFlow on Docker

The GitHub repository is shipped with a pre-configured docker-compose file that sets up all necessary components.

Prerequisites

Before you begin, ensure you have the following installed on your system:

• Docker 
• Docker Compose 
• Git 

Quick Installation

1. Clone the repository:

git clone https://github.com/glassflow/clickhouse-etl.git
cd clickhouse-etl

2. Start the services using Docker Compose:

docker compose up

3. Follow the logs in real-time for all containers:

docker compose logs -f

Verify the Installation

1. Ensure that your containers are running correctly. To view the status of your containers, run the following command:

docker compose ps

The output should look like this:

NAME                     IMAGE                                       COMMAND                  SERVICE   CREATED              STATUS          PORTS                                                                                                                          
clickhouse-etl-api-1     ghcr.io/glassflow/glassflow-etl-be:stable   "/sbin/tini -- /usr/…"   api       About a minute ago   Up 20 seconds                                                                                                                                  
clickhouse-etl-nats-1    nats:alpine                                 "docker-entrypoint.s…"   nats      About a minute ago   Up 20 seconds   6222/tcp, 0.0.0.0:4222->4222/tcp, 8222/tcp                                                                                     
clickhouse-etl-nginx-1   nginx:1.27-alpine                           "/docker-entrypoint.…"   nginx     About a minute ago   Up 20 seconds   80/tcp, 0.0.0.0:8080-8081->8080-8081/tcp                                                                                       
clickhouse-etl-ui-1      ghcr.io/glassflow/glassflow-etl-fe:stable   "docker-entrypoint.s…"   ui        About a minute ago   Up 20 seconds   8080/tcp                                                                                                                       

For details about the Docker components that are running, see the Dcoker Components page.

2. Verify that the web interface is accessible at http://localhost:8080. In case you do not see the web interface, please ensure that the containers are in running state and no other services are using the port 8080. If needed, you can modify the ports in docker-compose.yaml file.

Creating a Pipeline

Access the web interface at http://localhost:8080 to configure the pipeline. For detailed instructions on setting up your first pipeline, see the Usage Guide.

Stopping the Services

To stop all services:

docker compose down

To stop services and remove volumes:

docker compose down -v

Troubleshooting

Common Issues

1. Port Conflicts

   • Ensure ports 8080 and 4222 are not in use by other applications
   • You can modify the ports in docker-compose.yaml if needed

2. Container Startup Issues

   • Check container logs: docker compose logs [service-name]
     • docker compose logs app for the backend app
     • docker compose logs ui for the UI
   • Ensure all required ports are accessible
   • Verify Docker has enough resources allocated

3. UI Not Accessible

   • Verify all containers are running: docker compose ps
   • Check Nginx logs: docker compose logs nginx
   • Ensure you’re accessing the correct URL: http://localhost:8080

Getting Help

If you encounter any issues during installation: