feat: implement OmegaConf-based configuration management with uv

Configuration changes:
- Replace pydantic-settings with OmegaConf for flexible config management
- Support YAML config files via CONFIG_FILE environment variable
- Support environment variables with STEWARD__SECTION__KEY format
- Add config_schema.yaml as default configuration schema
- Create pydantic models for each configuration section for type safety
- Maintain backward compatibility via properties on Settings class
- Add CONFIGURATION.md with comprehensive setup and usage guide
- Update pyproject.toml to include config_schema.yaml in package data

Build/deployment changes:
- Replace pip with uv in Dockerfile for faster dependency installation
- Create docker-compose.dev.yml for local development (build .)
- Keep docker-compose.yml for production (uses ghcr.io/djw4/steward:latest)
- Update .env.example with new STEWARD__* variable format

This setup is designed for Kubernetes deployment:
- Non-sensitive config goes in ConfigMap (config.yaml)
- Secrets go in Secret resources (environment variables)
- Single unified configuration system for all environments
This commit is contained in:
Daniel Wagner 2026-07-26 12:34:23 +10:00
parent 669b57a317
commit d5eac108f8
9 changed files with 657 additions and 58 deletions

View File

@ -2,31 +2,41 @@
# Never commit .env to version control.
# Telegram bot token from @BotFather
TELEGRAM_BOT_TOKEN=
STEWARD__TELEGRAM__BOT_TOKEN=
# Comma-separated Telegram user IDs allowed to talk to Steward.
# Leave empty to allow everyone (not recommended for production).
TELEGRAM_ALLOWED_USER_IDS=
# List of user IDs allowed to interact via DM (JSON format, empty = all users)
STEWARD__TELEGRAM__ALLOWED_USER_IDS='[1234567890]'
# OpenAI (or compatible) credentials
OPENAI_API_KEY=
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4o
# List of group/channel IDs where bot operates (JSON format, empty = all groups)
STEWARD__TELEGRAM__GROUP_IDS='[-1005308306472]'
# Optional: override the default system prompt
# OPENAI_SYSTEM_PROMPT=
# OpenAI API key
STEWARD__OPENAI__API_KEY=
# Daily analysis target (optional)
# ANALYSIS_TARGET_URL=https://your-internal-api/endpoint
# ANALYSIS_TARGET_API_KEY=
# ANALYSIS_CRON_HOUR=8
# ANALYSIS_CRON_MINUTE=0
# OpenAI API base URL (can use any OpenAI-compatible provider)
STEWARD__OPENAI__BASE_URL=https://api.openai.com/v1
# Thread memory store path (JSON file for persisted thread summaries)
# THREAD_MEMORY_PATH=thread_memory.json
# Model to use
STEWARD__OPENAI__MODEL=gpt-4o
# MCP / OpenAPI tool server (open-webui/openapi-servers compatible)
# Point to any OpenAPI-spec tool server to enable LLM tool calling.
# The service fetches /openapi.json from MCP_SERVER_URL to discover tools.
# MCP_SERVER_URL=http://localhost:8000
# MCP_SERVER_API_KEY=
# Optional: Custom system prompt for the LLM
# STEWARD__OPENAI__SYSTEM_PROMPT="You are Steward..."
# Optional: Target URL for API analysis
STEWARD__ANALYSIS__TARGET_URL=
# Optional: API key for analysis target
STEWARD__ANALYSIS__TARGET_API_KEY=
# Cron schedule for automatic analysis
STEWARD__ANALYSIS__CRON_HOUR=8
STEWARD__ANALYSIS__CRON_MINUTE=0
# Path to thread memory storage (use absolute path in containers)
STEWARD__MEMORY__THREAD_MEMORY_PATH=/data/thread_memory.json
# Optional: MCP/OpenAPI tool server URL
STEWARD__TOOLS__MCP_SERVER_URL=
# Optional: API key for MCP server
STEWARD__TOOLS__MCP_SERVER_API_KEY=

251
CONFIGURATION.md Normal file
View File

@ -0,0 +1,251 @@
# Configuration Guide
Steward uses a flexible configuration system that supports multiple sources:
1. **YAML Config File** (via `CONFIG_FILE` env var)
2. **Environment Variables** (via `STEWARD__SECTION__KEY` format)
3. **Default Values** (embedded in the schema)
Configuration is loaded in order of precedence, with environment variables overriding config file values, which override defaults.
## Quick Start
### Minimal Setup (Environment Variables Only)
Set these required environment variables:
```bash
export TELEGRAM_BOT_TOKEN="your-token-here"
export STEWARD__OPENAI__API_KEY="your-api-key-here"
```
Then run:
```bash
steward
```
### With YAML Config File
Create a config file (e.g., `config.yaml`):
```yaml
telegram:
allowed_user_ids:
- 1234567890
group_ids:
- -1005308306472
openai:
model: "gpt-4-turbo"
base_url: "https://api.openai.com/v1"
analysis:
cron_hour: 9
cron_minute: 30
```
Then run with:
```bash
CONFIG_FILE=config.yaml STEWARD__TELEGRAM__BOT_TOKEN="..." STEWARD__OPENAI__API_KEY="..." steward
```
### Docker / Kubernetes Setup
**In docker-compose.yml:**
```yaml
services:
steward:
image: ghcr.io/djw4/steward:latest
environment:
CONFIG_FILE: /etc/steward/config.yaml
STEWARD__TELEGRAM__BOT_TOKEN: "${TELEGRAM_BOT_TOKEN}"
STEWARD__OPENAI__API_KEY: "${OPENAI_API_KEY}"
volumes:
- ./config.yaml:/etc/steward/config.yaml:ro
- ./data:/data
```
**In Kubernetes:**
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: steward-config
data:
config.yaml: |
telegram:
allowed_user_ids:
- 1234567890
group_ids:
- -1005308306472
openai:
model: "gpt-4o"
---
apiVersion: v1
kind: Secret
metadata:
name: steward-secrets
type: Opaque
stringData:
telegram-bot-token: "your-token"
openai-api-key: "your-key"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: steward
spec:
template:
spec:
containers:
- name: steward
image: ghcr.io/djw4/steward:latest
env:
- name: CONFIG_FILE
value: /etc/steward/config.yaml
- name: STEWARD__TELEGRAM__BOT_TOKEN
valueFrom:
secretKeyRef:
name: steward-secrets
key: telegram-bot-token
- name: STEWARD__OPENAI__API_KEY
valueFrom:
secretKeyRef:
name: steward-secrets
key: openai-api-key
volumeMounts:
- name: config
mountPath: /etc/steward
- name: data
mountPath: /data
volumes:
- name: config
configMap:
name: steward-config
- name: data
emptyDir: {}
```
## Configuration Reference
### Telegram Section
```yaml
telegram:
bot_token: "" # REQUIRED - set via STEWARD__TELEGRAM__BOT_TOKEN
allowed_user_ids: [] # DM access list (empty = all users)
group_ids: [] # Group/channel access list (empty = all groups)
```
**Environment Variables:**
- `STEWARD__TELEGRAM__BOT_TOKEN` - Telegram bot token (REQUIRED)
- `STEWARD__TELEGRAM__ALLOWED_USER_IDS` - JSON list: `[123,456,789]`
- `STEWARD__TELEGRAM__GROUP_IDS` - JSON list: `[-1005308306472]`
### OpenAI Section
```yaml
openai:
api_key: "" # REQUIRED - set via STEWARD__OPENAI__API_KEY
base_url: "https://api.openai.com/v1"
model: "gpt-4o"
system_prompt: "..." # Custom system prompt
```
**Environment Variables:**
- `STEWARD__OPENAI__API_KEY` - OpenAI API key (REQUIRED)
- `STEWARD__OPENAI__BASE_URL` - API endpoint URL
- `STEWARD__OPENAI__MODEL` - Model name
- `STEWARD__OPENAI__SYSTEM_PROMPT` - System prompt
### Analysis Section
```yaml
analysis:
target_url: "" # Optional: API endpoint for analysis
target_api_key: "" # Optional: set via env var
cron_hour: 8
cron_minute: 0
```
**Environment Variables:**
- `STEWARD__ANALYSIS__TARGET_URL`
- `STEWARD__ANALYSIS__TARGET_API_KEY`
- `STEWARD__ANALYSIS__CRON_HOUR`
- `STEWARD__ANALYSIS__CRON_MINUTE`
### Memory Section
```yaml
memory:
thread_memory_path: "thread_memory.json" # Path to memory store
```
**Environment Variables:**
- `STEWARD__MEMORY__THREAD_MEMORY_PATH`
- `THREAD_MEMORY_PATH` (legacy)
### Tools Section
```yaml
tools:
mcp_server_url: "" # Optional: MCP/OpenAPI tool server
mcp_server_api_key: "" # Optional: set via env var
```
**Environment Variables:**
- `STEWARD__TOOLS__MCP_SERVER_URL`
- `STEWARD__TOOLS__MCP_SERVER_API_KEY`
## Environment Variable Format
Environment variables follow the pattern: `STEWARD__SECTION__KEY=value`
Examples:
```bash
STEWARD__TELEGRAM__BOT_TOKEN="token123"
STEWARD__OPENAI__API_KEY="sk-..."
STEWARD__OPENAI__MODEL="gpt-4-turbo"
STEWARD__ANALYSIS__CRON_HOUR="9"
```
For list values, use JSON format:
```bash
STEWARD__TELEGRAM__ALLOWED_USER_IDS='[123456789, 987654321]'
STEWARD__TELEGRAM__GROUP_IDS='[-1005308306472]'
```
## Best Practices
### Security
- **Never** put secrets in YAML config files
- Always use environment variables for:
- `STEWARD__TELEGRAM__BOT_TOKEN`
- `STEWARD__OPENAI__API_KEY`
- `STEWARD__ANALYSIS__TARGET_API_KEY`
- `STEWARD__TOOLS__MCP_SERVER_API_KEY`
### Kubernetes
- Use `Secret` resources for sensitive data
- Use `ConfigMap` for non-sensitive configuration
- Mount both as environment variables for maximum flexibility
### Docker
- Use `.env` file for local development
- Use environment variables in production
- Mount ConfigMap volumes for complex configurations
## Backward Compatibility
The new configuration system is backward compatible with the old `pydantic-settings` approach:
- The `.env` file is no longer used by default (use `CONFIG_FILE` instead)
- Legacy environment variable names like `TELEGRAM_BOT_TOKEN` still work
- Properties on the `Settings` object work as before (e.g., `settings.telegram_bot_token`)
To migrate from `.env`:
1. Create a `config.yaml` with your configuration
2. Set `CONFIG_FILE` environment variable
3. Set secrets via `STEWARD__*` environment variables

View File

@ -2,10 +2,13 @@ FROM python:3.12-slim
WORKDIR /app
# Install uv
RUN pip install --no-cache-dir uv
# Install dependencies (separate layer for cache efficiency)
COPY pyproject.toml README.md ./
COPY steward/ steward/
RUN pip install --no-cache-dir .
RUN uv pip install --system --no-cache-dir .
# Create a non-root user (UID/GID 1000) and persistent data directory
RUN addgroup --gid 1000 steward \

69
config.example.yaml Normal file
View File

@ -0,0 +1,69 @@
# Example Steward Configuration (config.yaml)
#
# This file demonstrates how to configure Steward using a YAML config file.
#
# To use this config:
# 1. Copy this file to your deployment location (e.g., /etc/steward/config.yaml)
# 2. Set the CONFIG_FILE environment variable: CONFIG_FILE=/etc/steward/config.yaml
# 3. Secrets should still be provided via environment variables (see below)
#
# Configuration Precedence:
# 1. Environment variables (STEWARD__SECTION__KEY=value) - HIGHEST
# 2. CONFIG_FILE YAML file
# 3. Default values - LOWEST
#
# Secret Management:
# - Never put secrets (API keys, tokens) in this YAML file
# - Use environment variables instead:
# * STEWARD__TELEGRAM__BOT_TOKEN
# * STEWARD__OPENAI__API_KEY
# * STEWARD__ANALYSIS__TARGET_API_KEY
# * STEWARD__TOOLS__MCP_SERVER_API_KEY
telegram:
# Bot token is REQUIRED and should be set via STEWARD__TELEGRAM__BOT_TOKEN env var
bot_token: ""
# List of user IDs allowed to interact via DM (empty = all users allowed)
allowed_user_ids:
- 1234567890
- 9876543210
# List of group/channel IDs where bot operates (empty = all groups allowed)
group_ids:
- -1005308306472
openai:
# API key should be set via STEWARD__OPENAI__API_KEY env var
api_key: ""
# Base URL for OpenAI-compatible API
base_url: "https://api.openai.com/v1"
# Model to use
model: "gpt-4o"
# Custom system prompt (optional)
system_prompt: "You are Steward, a persistent, trustworthy AI-assisted personal operations platform. You reduce cognitive load by observing, remembering, planning, and proposing actions. You are conservative, transparent, and policy-aware. Always explain your reasoning."
analysis:
# Optional: URL for API analysis/proposals
target_url: ""
# Optional: API key for analysis target (set via STEWARD__ANALYSIS__TARGET_API_KEY)
target_api_key: ""
# Schedule for automatic analysis (cron-like)
cron_hour: 8
cron_minute: 0
memory:
# Path to persistent thread memory storage
thread_memory_path: "/data/thread_memory.json"
tools:
# Optional: MCP/OpenAPI tool server URL
mcp_server_url: ""
# Optional: API key for MCP server (set via STEWARD__TOOLS__MCP_SERVER_API_KEY)
mcp_server_api_key: ""

12
docker-compose.dev.yml Normal file
View File

@ -0,0 +1,12 @@
services:
steward:
build: .
user: "1000:1000" # matches the UID/GID created in the Dockerfile
env_file: .env # copy .env.example → .env and fill in your values
environment:
THREAD_MEMORY_PATH: /data/thread_memory.json
volumes:
# Bind-mount a local ./data directory for persistent storage.
# Create it before the first run: mkdir -p data
- ./data:/data
restart: unless-stopped

View File

@ -1,8 +1,6 @@
services:
steward:
# image: ghcr.io/djw4/steward:latest
# To build locally instead: uncomment the next line and comment out image above
build: .
image: ghcr.io/djw4/steward:latest
user: "1000:1000" # matches the UID/GID created in the Dockerfile
env_file: .env # copy .env.example → .env and fill in your values
environment:

View File

@ -16,6 +16,7 @@ dependencies = [
"apscheduler>=3.10",
"pydantic>=2.7",
"pydantic-settings>=2.3",
"omegaconf>=2.3",
]
[project.optional-dependencies]
@ -35,6 +36,9 @@ steward = "steward.main:main"
where = ["."]
include = ["steward*"]
[tool.setuptools.package-data]
steward = ["config_schema.yaml"]
[tool.ruff]
line-length = 100
target-version = "py312"

View File

@ -1,45 +1,246 @@
"""Configuration management for Steward."""
"""Configuration management for Steward.
from pydantic_settings import BaseSettings, SettingsConfigDict
Supports configuration via:
1. YAML config file (passed via CONFIG_FILE env var)
2. Environment variables (STEWARD__SECTION__KEY=value format)
3. Default schema values
Environment variables take precedence over config file values.
"""
import logging
import os
from pathlib import Path
from omegaconf import OmegaConf
from pydantic import BaseModel, Field
logger = logging.getLogger(__name__)
class Settings(BaseSettings):
"""Application settings loaded from environment variables or .env file."""
class TelegramConfig(BaseModel):
"""Telegram bot configuration."""
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
bot_token: str = Field(default="", description="Telegram bot token")
allowed_user_ids: list[int] = Field(default_factory=list, description="Allowed user IDs")
group_ids: list[int] = Field(default_factory=list, description="Allowed group/channel IDs")
# Telegram
telegram_bot_token: str = ""
telegram_allowed_user_ids: list[int] = []
telegram_group_ids: list[int] = []
# LLM (OpenAI-compatible)
openai_api_key: str = ""
openai_base_url: str = "https://api.openai.com/v1"
openai_model: str = "gpt-4o"
openai_system_prompt: str = (
"You are Steward, a persistent, trustworthy AI-assisted personal operations platform. "
"You reduce cognitive load by observing, remembering, planning, and proposing actions. "
"You are conservative, transparent, and policy-aware. "
"Always explain your reasoning."
class OpenAIConfig(BaseModel):
"""OpenAI/LLM configuration."""
api_key: str = Field(default="", description="OpenAI API key")
base_url: str = Field(default="https://api.openai.com/v1", description="API base URL")
model: str = Field(default="gpt-4o", description="Model to use")
system_prompt: str = Field(
default=(
"You are Steward, a persistent, trustworthy AI-assisted personal operations platform. "
"You reduce cognitive load by observing, remembering, planning, and proposing actions. "
"You are conservative, transparent, and policy-aware. "
"Always explain your reasoning."
),
description="System prompt for the LLM",
)
# Analysis / proposal worker
analysis_target_url: str = ""
analysis_target_api_key: str = ""
analysis_cron_hour: int = 8
analysis_cron_minute: int = 0
# Thread memory
thread_memory_path: str = "thread_memory.json"
class AnalysisConfig(BaseModel):
"""Analysis/proposal worker configuration."""
# MCP / OpenAPI tool server (open-webui/openapi-servers compatible)
# Set MCP_SERVER_URL to enable tool calling. The service fetches
# /openapi.json from this URL to discover available tools.
mcp_server_url: str = ""
mcp_server_api_key: str = ""
target_url: str = Field(default="", description="Target URL for analysis")
target_api_key: str = Field(default="", description="API key for target")
cron_hour: int = Field(default=8, description="Hour for cron schedule")
cron_minute: int = Field(default=0, description="Minute for cron schedule")
class MemoryConfig(BaseModel):
"""Memory/persistence configuration."""
thread_memory_path: str = Field(default="thread_memory.json", description="Thread memory path")
class ToolsConfig(BaseModel):
"""Tools/MCP configuration."""
mcp_server_url: str = Field(default="", description="MCP server URL")
mcp_server_api_key: str = Field(default="", description="MCP server API key")
class Settings(BaseModel):
"""Application settings with OmegaConf and pydantic integration."""
telegram: TelegramConfig = Field(default_factory=TelegramConfig)
openai: OpenAIConfig = Field(default_factory=OpenAIConfig)
analysis: AnalysisConfig = Field(default_factory=AnalysisConfig)
memory: MemoryConfig = Field(default_factory=MemoryConfig)
tools: ToolsConfig = Field(default_factory=ToolsConfig)
class Config:
"""Pydantic config."""
arbitrary_types_allowed = True
# Compatibility properties for existing code
@property
def telegram_bot_token(self) -> str:
"""Legacy property for backward compatibility."""
return self.telegram.bot_token
@property
def telegram_allowed_user_ids(self) -> list[int]:
"""Legacy property for backward compatibility."""
return self.telegram.allowed_user_ids
@property
def telegram_group_ids(self) -> list[int]:
"""Legacy property for backward compatibility."""
return self.telegram.group_ids
@property
def openai_api_key(self) -> str:
"""Legacy property for backward compatibility."""
return self.openai.api_key
@property
def openai_base_url(self) -> str:
"""Legacy property for backward compatibility."""
return self.openai.base_url
@property
def openai_model(self) -> str:
"""Legacy property for backward compatibility."""
return self.openai.model
@property
def openai_system_prompt(self) -> str:
"""Legacy property for backward compatibility."""
return self.openai.system_prompt
@property
def analysis_target_url(self) -> str:
"""Legacy property for backward compatibility."""
return self.analysis.target_url
@property
def analysis_target_api_key(self) -> str:
"""Legacy property for backward compatibility."""
return self.analysis.target_api_key
@property
def analysis_cron_hour(self) -> int:
"""Legacy property for backward compatibility."""
return self.analysis.cron_hour
@property
def analysis_cron_minute(self) -> int:
"""Legacy property for backward compatibility."""
return self.analysis.cron_minute
@property
def thread_memory_path(self) -> str:
"""Legacy property for backward compatibility."""
return self.memory.thread_memory_path
@property
def mcp_server_url(self) -> str:
"""Legacy property for backward compatibility."""
return self.tools.mcp_server_url
@property
def mcp_server_api_key(self) -> str:
"""Legacy property for backward compatibility."""
return self.tools.mcp_server_api_key
_settings_instance: Settings | None = None
def _load_config_from_file(config_path: str | Path) -> dict:
"""Load configuration from YAML file."""
config_path = Path(config_path)
if not config_path.exists():
logger.warning("Config file not found: %s", config_path)
return {}
logger.info("Loading config from %s", config_path)
cfg = OmegaConf.load(config_path)
return OmegaConf.to_container(cfg, resolve=True) or {}
def _load_config_from_env() -> dict:
"""Load configuration from environment variables (STEWARD__SECTION__KEY format)."""
cfg = {}
prefix = "STEWARD__"
for key, value in os.environ.items():
if not key.startswith(prefix):
continue
# Parse STEWARD__SECTION__KEY=value
parts = key[len(prefix) :].lower().split("__")
if len(parts) < 2:
continue
section = parts[0]
setting_key = "__".join(parts[1:])
if section not in cfg:
cfg[section] = {}
# Try to parse value as JSON first (for lists, etc.)
if isinstance(cfg[section], dict):
try:
import json
cfg[section][setting_key] = json.loads(value)
except (json.JSONDecodeError, ValueError):
cfg[section][setting_key] = value
return cfg
def get_settings() -> Settings:
"""Return application settings singleton."""
return Settings()
"""Get or create the settings singleton.
Configuration is loaded in order of precedence (highest to lowest):
1. Environment variables (STEWARD__SECTION__KEY=value)
2. YAML config file (path via CONFIG_FILE env var)
3. Default values from schema
Returns:
Settings: The application settings.
"""
global _settings_instance
if _settings_instance is not None:
return _settings_instance
# Start with schema defaults
schema_path = Path(__file__).parent / "config_schema.yaml"
base_cfg = OmegaConf.load(schema_path)
# Merge in config file if specified
config_file = os.environ.get("CONFIG_FILE")
if config_file:
file_cfg = OmegaConf.create(_load_config_from_file(config_file))
base_cfg = OmegaConf.merge(base_cfg, file_cfg)
# Merge in environment variables (takes precedence)
env_cfg = OmegaConf.create(_load_config_from_env())
if env_cfg:
base_cfg = OmegaConf.merge(base_cfg, env_cfg)
# Convert to dict and create Settings instance
config_dict = OmegaConf.to_container(base_cfg, resolve=True) or {}
_settings_instance = Settings(**config_dict)
logger.info("Configuration loaded successfully")
logger.debug("Active configuration: %s", OmegaConf.to_yaml(base_cfg))
return _settings_instance
def reload_settings() -> Settings:
"""Reload configuration from source (mainly for testing)."""
global _settings_instance
_settings_instance = None
return get_settings()

View File

@ -0,0 +1,51 @@
# Steward Configuration Schema
# This is the default configuration. Override values in your own config file or via environment variables.
# Environment variable format: STEWARD__<section>__<key>=value
# Example: STEWARD__TELEGRAM__BOT_TOKEN=your_token
telegram:
# Telegram bot token (REQUIRED - set via env or config)
bot_token: ""
# List of user IDs allowed to interact with the bot via DM
# Empty list means all users are allowed
allowed_user_ids: []
# List of group/channel IDs where the bot is allowed to operate
# Empty list means all groups are allowed
group_ids: []
openai:
# OpenAI API key (REQUIRED - set via env for security)
api_key: ""
# API base URL (can use any OpenAI-compatible provider)
base_url: "https://api.openai.com/v1"
# Model to use
model: "gpt-4o"
# System prompt for the bot
system_prompt: "You are Steward, a persistent, trustworthy AI-assisted personal operations platform. You reduce cognitive load by observing, remembering, planning, and proposing actions. You are conservative, transparent, and policy-aware. Always explain your reasoning."
analysis:
# Target URL for API analysis (optional - set to enable analysis feature)
target_url: ""
# API key for analysis target (optional)
target_api_key: ""
# Cron schedule for automatic analysis
cron_hour: 8
cron_minute: 0
memory:
# Path to thread memory storage file
thread_memory_path: "thread_memory.json"
tools:
# MCP/OpenAPI tool server URL (optional - set to enable tool calling)
mcp_server_url: ""
# API key for MCP server (optional)
mcp_server_api_key: ""