Docs

Getting Started

Your comprehensive guide to setting up and using dxflow for distributed computing and workflow orchestration

Getting Started with dxflow

Welcome to dxflow - a powerful and flexible workflow engine that allows you to build and execute complex workflows with ease. Whether you're running data science pipelines, orchestrating container workflows, or managing high-performance computing jobs, dxflow provides a unified interface across all your environments.

New to dxflow? This guide will take you from zero to productive in just a few minutes. We'll cover everything you need to know to start orchestrating workflows across your infrastructure.

What is dxflow?

dxflow is a powerful workflow engine that provides a unified interface for managing and orchestrating data & compute workflows across different computing environments. Built with a modular architecture, dxflow seamlessly integrates Docker Compose workflows, shell operations, object storage, network bridging, and proxy management.

In short: dxflow turns any machine you can access into a first-class citizen of your computational fleet, with one installer, a stable URL, and a unified interface for jobs, logs, and data.

Key Features

Universal Deployment

Deploy on any infrastructure: Linux, macOS, Windows - cloud VMs, GPU nodes, on-premise clusters, or your laptop

Unified Interface

Consistent CLI, REST API, and modern Nuxt.js web UI across all environments

Advanced Orchestration

Native Docker Compose workflows, shell management, WebSocket tunneling, and proxy capabilities

Secure & Flexible

JWT authentication, RSA key-pairs, flexible licensing, and self-update capabilities

How It Works

dxflow operates as a lightweight engine that transforms any compute resource into a unified platform:

Simple 4-Layer Architecture:

  • Your Infrastructure - Any compute: cloud, on-premise, or laptop
  • dxflow Engine - Lightweight daemon providing unified access
  • Native Schedulers - Works with existing Docker, Kubernetes, Slurm, etc.
  • Your Applications - Run your actual workloads unchanged

Core Benefits

Same interface everywhere - CLI, web UI, and APIs work identically across Linux, macOS, and Windows

Use Cases

dxflow is designed for anyone who needs to manage compute workflows across different environments:

Common Use Cases:

  • Data Science: Multi-GPU ML training and data processing pipelines
  • Research Computing: HPC simulations and bioinformatics workflows
  • DevOps: Distributed testing and container orchestration
  • Edge Computing: IoT processing and edge-to-cloud workflows

What You'll Learn

This getting started guide is organized to get you up and running quickly:

Installation

Quick Setup: Install dxflow with automated scripts

Quick Start

Hands-On: Get your first workflow running in minutes

Quick Navigation

For Advanced Setup

  1. CLI Reference - Command-line interface guide
  2. API Documentation - Programmatic integration

Why Choose dxflow?

dxflow transforms any accessible machine into a first-class computational resource with unified interfaces and enterprise-grade security.

Support & Community

Need help? Our community and documentation are here to support you at every step of your dxflow journey.

Getting Help:

  • Documentation: Comprehensive guides for all features
  • Issue Tracking: Report bugs and request features
  • Direct Support: Contact our team for enterprise needs

Quick Links:

Ready to Begin?

Choose your starting point based on your familiarity with dxflow:

Ready to Install

Start with installation to get dxflow on your system

Quick Demo

Skip to quick start if you have dxflow installed

Let's Get Started! 🎉 Your journey into distributed computing with dxflow begins now. Follow the guides in order or jump to what interests you most.

dxflow Engine

A powerful workflow engine for Docker, Slurm and more, providing both CLI and API interfaces for seamless integration

Installation

dxflow is a cross-platform tool, which means it can run on any operating system, such as Linux, macOS, and Windows. It is designed to be easy to install and use, with a simple command-line interface (CLI) and a web-based user interface (UI).