Docs

Getting Started

Your comprehensive guide to setting up and using dxflow for distributed computing and workflow orchestration

Getting Started with dxflow

Welcome to dxflow - the comprehensive distributed computing engine that transforms any accessible machine into a first-class member of your computational fleet. Whether you're running data science pipelines, orchestrating container workflows, or managing high-performance computing jobs, dxflow provides a unified interface across all your environments.

New to dxflow? This guide will take you from zero to productive in just a few minutes. We'll cover everything you need to know to start orchestrating workflows across your infrastructure.

What is dxflow?

dxflow is a comprehensive distributed computing engine that provides a unified interface for managing and orchestrating data & compute workflows across different computing environments.

In short: dxflow turns any machine you can access into a first-class citizen of your computational fleet, with one installer, a stable URL, and a unified interface for jobs, logs, and data.

Key Features

Universal Deployment

Deploy on any infrastructure: cloud VMs, GPU nodes, on-premise clusters, or even your laptop

Unified Interface

Consistent CLI, REST API, and web UI across all environments

Container Orchestration

Native Docker Compose integration with real-time monitoring

Secure by Design

RSA key-pair authentication with fine-grained access control

How It Works

dxflow operates as a lightweight engine that transforms any compute resource into a unified platform:

Simple 4-Layer Architecture:

  • Your Infrastructure - Any compute: cloud, on-premise, or laptop
  • dxflow Engine - Lightweight daemon providing unified access
  • Native Schedulers - Works with existing Docker, Kubernetes, Slurm, etc.
  • Your Applications - Run your actual workloads unchanged

Core Benefits

Same interface everywhere - CLI, web UI, and APIs work identically across all your infrastructure

Use Cases

dxflow is designed for anyone who needs to manage compute workflows across different environments:

Common Use Cases:

  • Data Science: Multi-GPU ML training and data processing pipelines
  • Research Computing: HPC simulations and bioinformatics workflows
  • DevOps: Distributed testing and container orchestration
  • Edge Computing: IoT processing and edge-to-cloud workflows

What You'll Learn

This getting started guide is organized to get you up and running quickly:

Installation

Quick Setup: Install dxflow with automated scripts

Quick Start

Hands-On: Get your first workflow running in minutes

Quick Navigation

For Advanced Setup

  1. CLI Reference - Command-line interface guide
  2. API Documentation - Programmatic integration

Why Choose dxflow?

dxflow transforms any accessible machine into a first-class computational resource with unified interfaces and enterprise-grade security.

Support & Community

Need help? Our community and documentation are here to support you at every step of your dxflow journey.

Getting Help:

  • Documentation: Comprehensive guides for all features
  • Issue Tracking: Report bugs and request features
  • Direct Support: Contact our team for enterprise needs

Quick Links:

Ready to Begin?

Choose your starting point based on your familiarity with dxflow:

Ready to Install

Start with installation to get dxflow on your system

Quick Demo

Skip to quick start if you have dxflow installed

Let's Get Started! 🎉 Your journey into distributed computing with dxflow begins now. Follow the guides in order or jump to what interests you most.

dxflow Engine

A powerful workflow engine for Docker, Slurm and more, providing both CLI and API interfaces for seamless integration

Installation

dxflow is a cross-platform tool, which means it can run on any operating system, such as Linux, macOS, and Windows. It is designed to be easy to install and use, with a simple command-line interface (CLI) and a web-based user interface (UI).