Docs
Advanced Topics

Remoting

Control distributed dxflow infrastructure from a single CLI using secure authentication and profile-based management

dxflow provides comprehensive remote management capabilities that allow you to control distributed server infrastructure from a single local CLI, enabling centralized operations across multiple environments.

Two-Instance Architecture: Remoting requires two dxflow instances - one running as a server (with API endpoints) and another as a client (CLI) that connects to control the remote server.

Overview

The dxflow CLI provides comprehensive remote management capabilities for distributed dxflow server instances. The remote architecture enables centralized control plane operations across distributed environments without requiring direct server access.

Key Benefits:

  • Multi-Server Control: Manage multiple dxflow servers from single CLI
  • No Direct Access: Control servers behind firewalls without SSH
  • Multi-Cloud Support: Uniform management across AWS, GCP, Azure
  • Profile Isolation: Separate dev/staging/production environments

How It Works

Remote Architecture

dxflow remoting requires two separate dxflow instances communicating over HTTP/HTTPS:

dxflow Client

Local CLI Instance

  • Sends CLI commands
  • Manages profile configs
  • Handles authentication
  • Converts commands to API calls

dxflow Server

Remote Target Instance

  • Runs REST API server
  • Executes workflow engine
  • Manages resources
  • Processes client requests

Remote dxflow Server

The target machine runs dxflow as a service:

# Start dxflow server on remote machine
dxflow boot up --daemon --port 8080

# Server exposes API endpoints
# http://your-server:8080/api/engine/ping
# http://your-server:8080/api/workflow/*
# http://your-server:8080/api/shell/*

The local CLI establishes secure connections to remote servers, handles authentication automatically, and provides transparent access to all remote functionality.

Authentication Process

Authentication Flow

The remoting system uses a sophisticated authentication process:

  1. Connection Type Check: TCP/IP connections require authentication for remote server access
  2. Token Discovery: CLI searches for existing connection-token in configuration
  3. Two-Stage Validation:
    • Quick Inspect: Local JWT validation (subject + expiration check)
    • Deep Inspect: Remote server validation via /api/auth/inspect/
  4. Challenge-Response Authentication: If token invalid/missing:
    • CLI requests challenge from /api/auth/challenge/ (receives identity + nonce)
    • CLI signs nonce with private key using RSA-PKCS1v15 + SHA256
    • CLI sends signature to /api/auth/verify/ (receives new JWT token)
  5. Token Persistence: New tokens automatically stored in profile configuration
  6. Automatic Recovery: On 401 responses, tokens are automatically reset and regenerated

Authentication Methods

MethodProcessUse Case
Private KeyRSA challenge-response authentication with automatic token lifecycleProduction environments
Direct TokenManual JWT token extraction from remote server instanceQuick setup/testing

Getting Started

Step 1: Configure Server Connection

Set the target remote server address:

# HTTPS for production (recommended)
dxflow config set connection-address "https://your-server.com"

# HTTP for development/testing
dxflow config set connection-address "http://192.168.1.100:8080"

Step 2: Set Up Authentication

Configure RSA private key for secure authentication:

# Use server's private key for authentication
dxflow config set connection-key "/path/to/server-private-key.pem"

Step 3: Verify Connection

Test the remote connection:

# Verify connectivity and authentication
dxflow engine ping
Success: Once authenticated, all dxflow commands automatically target the remote server.

Profile Management

Multiple Environment Support

dxflow uses configuration profiles to manage multiple remote servers independently:

Environment Isolation

Separate Configurations

  • Each profile maintains its own connection settings
  • Independent authentication credentials per profile
  • No cross-contamination between environments

Profile Switching

Flexible Operations

  • Switch between profiles with simple flags
  • Run commands on different environments simultaneously
  • Profile-specific command history and settings

Setting Up Profiles

Development Environment Setup

# Configure development server
dxflow --config-profile dev config set connection-address "http://dev-server:8080"
dxflow --config-profile dev config set connection-key "/keys/dev-key.pem"

# Test development connection
dxflow -C dev engine ping

Authentication Process

RSA Challenge-Response Authentication

dxflow uses the same secure authentication system for remote connections as described in the Authentication documentation:

Automatic Authentication (Recommended)

# Configure private key for automatic authentication
dxflow config set connection-key "/path/to/server-private-key.pem"

How it works:

  1. CLI requests authentication challenge from remote server
  2. CLI signs challenge using your private key (RSA-PKCS1v15 + SHA256)
  3. Server verifies signature and issues JWT token
  4. Token stored automatically and refreshed as needed

Common Use Cases

Develop locally while testing on remote infrastructure:

# Deploy to development environment
dxflow -C dev workflow compose deploy ./my-workflow.yaml
dxflow -C dev workflow logs my-workflow --follow

# Promote through environments
dxflow -C staging workflow compose deploy ./my-workflow.yaml
dxflow -C prod workflow compose deploy ./my-workflow.yaml

Configuration Storage

Profile Configuration Files

Each profile stores its configuration in a dedicated file:

ProfileConfiguration File
Default~/.dxflow/default_config.yaml
Custom~/.dxflow/{profile-name}_config.yaml

Configuration Structure

# Example: ~/.dxflow/production_config.yaml
connection-address: "https://prod.company.com"  # Base URL (no /api suffix)
connection-key: "/secure/keys/prod-key.pem"     # RSA private key path
connection-token: "eyJhbGc..."                  # Auto-generated JWT token

Environment Variable Overrides

Override configuration temporarily using environment variables:

Environment VariableConfiguration Key
CONNECTION_ADDRESSconnection-address
CONNECTION_KEYconnection-key
CONNECTION_TOKENconnection-token

Troubleshooting

Cannot connect to remote server

# 1. Verify server address configuration
dxflow config get connection-address

# 2. Test basic HTTP connectivity
curl -k "$(dxflow config get connection-address)/api/engine/healthcheck/"

# 3. Check DNS resolution and firewall rules
ping your-server.com
telnet your-server.com 443

Connection timeouts or network errors

  • Verify server is running and accessible
  • Check corporate firewall and proxy settings
  • Ensure proper HTTPS certificates are configured
  • Try HTTP instead of HTTPS for testing

Security Best Practices

  • Key Protection: Store private keys with chmod 600 permissions and separate keys per environment
  • HTTPS Only: Always use HTTPS for production environments
  • Profile Isolation: Create separate profiles for each team member and environment
  • Regular Audits: Monitor authentication logs and rotate keys periodically

Common Commands

Connection Management

# View current configuration
dxflow config
dxflow config get connection-address

# Update connection settings
dxflow config set connection-address "https://new-server.com"
dxflow config set connection-key "/path/to/new-key.pem"

# Test connection
dxflow engine ping
dxflow engine healthcheck

Profile Operations

# Use specific profile
dxflow --config-profile aws engine stat
dxflow -C gcp workflow list

# Profile-specific configuration
dxflow -C dev config set connection-address "https://dev-server.com"

# Force token refresh for profile
dxflow -C prod config unset connection-token
dxflow -C prod engine ping
Remote Management Ready: With proper configuration, all dxflow commands work seamlessly with remote servers, providing the same functionality as local installations while maintaining security and performance.

Tunneling

Create secure tunnels and proxy connections for exposing local services through WebSocket bridges

FAQs

Frequently Asked Questions about dxflow, covering common queries and troubleshooting tips to help users navigate