config

Configuration for worker processes including restart policies, timeouts, and queue sizes.

Restart Policy

RestartPolicy

 RestartPolicy (value, names=None, module=None, qualname=None, type=None,
                start=1, boundary=None)

Policy for restarting worker processes after failures.

The restart policy determines when worker processes should be automatically restarted after failures:

NEVER: Never restart workers (useful for debugging or when you want full manual control)
ON_CANCELLATION: Restart only when a job is cancelled (default behavior)
ALWAYS: Always restart on any failure
BACKOFF: Restart with exponential backoff delays between attempts

# Example usage
RestartPolicy.ON_CANCELLATION

<RestartPolicy.ON_CANCELLATION: 'on_cancellation'>

Worker Configuration

source

WorkerConfig

 WorkerConfig (request_queue_size:int=0, result_queue_size:int=100,
               response_queue_size:int=10, restart_policy:__main__.Restart
               Policy=<RestartPolicy.ON_CANCELLATION: 'on_cancellation'>,
               max_restart_attempts:int=3,
               restart_backoff_base_seconds:float=1.0,
               restart_backoff_max_seconds:float=60.0, max_workers:int=1,
               worker_start_timeout_seconds:float=30.0,
               reload_timeout_seconds:float=30.0,
               unload_timeout_seconds:float=10.0,
               shutdown_timeout_seconds:float=5.0,
               result_monitor_poll_interval_seconds:float=0.5)

Configuration for worker process behavior.

The WorkerConfig class provides comprehensive configuration options for worker processes:

Queue Sizes: - Control the buffer size for communication between parent and worker processes - Set to 0 for unlimited request queue - Larger result queue (100) accommodates streaming results

Restart Behavior: - Configure when and how workers restart after failures - Set maximum retry attempts to prevent infinite restart loops - Control backoff timing for gradual retry delays

Timeouts: - Prevent operations from hanging indefinitely - Separate timeouts for different operations (start, reload, unload, shutdown)

Monitoring: - Configure how frequently the result monitor checks for new results

The __post_init__ method validates the configuration to catch errors early.

# Create default config
config = WorkerConfig()
config

WorkerConfig(request_queue_size=0, result_queue_size=100, response_queue_size=10, restart_policy=<RestartPolicy.ON_CANCELLATION: 'on_cancellation'>, max_restart_attempts=3, restart_backoff_base_seconds=1.0, restart_backoff_max_seconds=60.0, max_workers=1, worker_start_timeout_seconds=30.0, reload_timeout_seconds=30.0, unload_timeout_seconds=10.0, shutdown_timeout_seconds=5.0, result_monitor_poll_interval_seconds=0.5)

# Create config with custom settings
config = WorkerConfig(
    restart_policy=RestartPolicy.BACKOFF,
    max_restart_attempts=5,
    result_queue_size=200
)
config

WorkerConfig(request_queue_size=0, result_queue_size=200, response_queue_size=10, restart_policy=<RestartPolicy.BACKOFF: 'backoff'>, max_restart_attempts=5, restart_backoff_base_seconds=1.0, restart_backoff_max_seconds=60.0, max_workers=1, worker_start_timeout_seconds=30.0, reload_timeout_seconds=30.0, unload_timeout_seconds=10.0, shutdown_timeout_seconds=5.0, result_monitor_poll_interval_seconds=0.5)

Validation Tests

The WorkerConfig validates configuration values and raises errors for invalid settings:

# Test validation: max_workers > 1 raises NotImplementedError
try:
    config = WorkerConfig(max_workers=2)
except NotImplementedError as e:
    print(f"✓ Caught expected error: {e}")

✓ Caught expected error: Worker pools (max_workers > 1) are not yet implemented. Current value: 2

# Test validation: max_workers < 1 raises ValueError
try:
    config = WorkerConfig(max_workers=0)
except ValueError as e:
    print(f"✓ Caught expected error: {e}")

✓ Caught expected error: max_workers must be >= 1, got 0

# Test validation: negative restart_backoff_base_seconds raises ValueError
try:
    config = WorkerConfig(restart_backoff_base_seconds=-1.0)
except ValueError as e:
    print(f"✓ Caught expected error: {e}")

✓ Caught expected error: restart_backoff_base_seconds must be > 0, got -1.0