feat: implement OmegaConf-based configuration management with uv
Configuration changes: - Replace pydantic-settings with OmegaConf for flexible config management - Support YAML config files via CONFIG_FILE environment variable - Support environment variables with STEWARD__SECTION__KEY format - Add config_schema.yaml as default configuration schema - Create pydantic models for each configuration section for type safety - Maintain backward compatibility via properties on Settings class - Add CONFIGURATION.md with comprehensive setup and usage guide - Update pyproject.toml to include config_schema.yaml in package data Build/deployment changes: - Replace pip with uv in Dockerfile for faster dependency installation - Create docker-compose.dev.yml for local development (build .) - Keep docker-compose.yml for production (uses ghcr.io/djw4/steward:latest) - Update .env.example with new STEWARD__* variable format This setup is designed for Kubernetes deployment: - Non-sensitive config goes in ConfigMap (config.yaml) - Secrets go in Secret resources (environment variables) - Single unified configuration system for all environments
This commit is contained in:
Daniel Wagner
@@ -0,0 +1,251 @@
|
||||
# Configuration Guide
|
||||
|
||||
Steward uses a flexible configuration system that supports multiple sources:
|
||||
|
||||
1. **YAML Config File** (via `CONFIG_FILE` env var)
|
||||
2. **Environment Variables** (via `STEWARD__SECTION__KEY` format)
|
||||
3. **Default Values** (embedded in the schema)
|
||||
|
||||
Configuration is loaded in order of precedence, with environment variables overriding config file values, which override defaults.
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Minimal Setup (Environment Variables Only)
|
||||
|
||||
Set these required environment variables:
|
||||
|
||||
```bash
|
||||
export TELEGRAM_BOT_TOKEN="your-token-here"
|
||||
export STEWARD__OPENAI__API_KEY="your-api-key-here"
|
||||
```
|
||||
|
||||
Then run:
|
||||
```bash
|
||||
steward
|
||||
```
|
||||
|
||||
### With YAML Config File
|
||||
|
||||
Create a config file (e.g., `config.yaml`):
|
||||
|
||||
```yaml
|
||||
telegram:
|
||||
allowed_user_ids:
|
||||
- 1234567890
|
||||
group_ids:
|
||||
- -1005308306472
|
||||
|
||||
openai:
|
||||
model: "gpt-4-turbo"
|
||||
base_url: "https://api.openai.com/v1"
|
||||
|
||||
analysis:
|
||||
cron_hour: 9
|
||||
cron_minute: 30
|
||||
```
|
||||
|
||||
Then run with:
|
||||
```bash
|
||||
CONFIG_FILE=config.yaml STEWARD__TELEGRAM__BOT_TOKEN="..." STEWARD__OPENAI__API_KEY="..." steward
|
||||
```
|
||||
|
||||
### Docker / Kubernetes Setup
|
||||
|
||||
**In docker-compose.yml:**
|
||||
```yaml
|
||||
services:
|
||||
steward:
|
||||
image: ghcr.io/djw4/steward:latest
|
||||
environment:
|
||||
CONFIG_FILE: /etc/steward/config.yaml
|
||||
STEWARD__TELEGRAM__BOT_TOKEN: "${TELEGRAM_BOT_TOKEN}"
|
||||
STEWARD__OPENAI__API_KEY: "${OPENAI_API_KEY}"
|
||||
volumes:
|
||||
- ./config.yaml:/etc/steward/config.yaml:ro
|
||||
- ./data:/data
|
||||
```
|
||||
|
||||
**In Kubernetes:**
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: steward-config
|
||||
data:
|
||||
config.yaml: |
|
||||
telegram:
|
||||
allowed_user_ids:
|
||||
- 1234567890
|
||||
group_ids:
|
||||
- -1005308306472
|
||||
openai:
|
||||
model: "gpt-4o"
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: steward-secrets
|
||||
type: Opaque
|
||||
stringData:
|
||||
telegram-bot-token: "your-token"
|
||||
openai-api-key: "your-key"
|
||||
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: steward
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: steward
|
||||
image: ghcr.io/djw4/steward:latest
|
||||
env:
|
||||
- name: CONFIG_FILE
|
||||
value: /etc/steward/config.yaml
|
||||
- name: STEWARD__TELEGRAM__BOT_TOKEN
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: steward-secrets
|
||||
key: telegram-bot-token
|
||||
- name: STEWARD__OPENAI__API_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: steward-secrets
|
||||
key: openai-api-key
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/steward
|
||||
- name: data
|
||||
mountPath: /data
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: steward-config
|
||||
- name: data
|
||||
emptyDir: {}
|
||||
```
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Telegram Section
|
||||
|
||||
```yaml
|
||||
telegram:
|
||||
bot_token: "" # REQUIRED - set via STEWARD__TELEGRAM__BOT_TOKEN
|
||||
allowed_user_ids: [] # DM access list (empty = all users)
|
||||
group_ids: [] # Group/channel access list (empty = all groups)
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `STEWARD__TELEGRAM__BOT_TOKEN` - Telegram bot token (REQUIRED)
|
||||
- `STEWARD__TELEGRAM__ALLOWED_USER_IDS` - JSON list: `[123,456,789]`
|
||||
- `STEWARD__TELEGRAM__GROUP_IDS` - JSON list: `[-1005308306472]`
|
||||
|
||||
### OpenAI Section
|
||||
|
||||
```yaml
|
||||
openai:
|
||||
api_key: "" # REQUIRED - set via STEWARD__OPENAI__API_KEY
|
||||
base_url: "https://api.openai.com/v1"
|
||||
model: "gpt-4o"
|
||||
system_prompt: "..." # Custom system prompt
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `STEWARD__OPENAI__API_KEY` - OpenAI API key (REQUIRED)
|
||||
- `STEWARD__OPENAI__BASE_URL` - API endpoint URL
|
||||
- `STEWARD__OPENAI__MODEL` - Model name
|
||||
- `STEWARD__OPENAI__SYSTEM_PROMPT` - System prompt
|
||||
|
||||
### Analysis Section
|
||||
|
||||
```yaml
|
||||
analysis:
|
||||
target_url: "" # Optional: API endpoint for analysis
|
||||
target_api_key: "" # Optional: set via env var
|
||||
cron_hour: 8
|
||||
cron_minute: 0
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `STEWARD__ANALYSIS__TARGET_URL`
|
||||
- `STEWARD__ANALYSIS__TARGET_API_KEY`
|
||||
- `STEWARD__ANALYSIS__CRON_HOUR`
|
||||
- `STEWARD__ANALYSIS__CRON_MINUTE`
|
||||
|
||||
### Memory Section
|
||||
|
||||
```yaml
|
||||
memory:
|
||||
thread_memory_path: "thread_memory.json" # Path to memory store
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `STEWARD__MEMORY__THREAD_MEMORY_PATH`
|
||||
- `THREAD_MEMORY_PATH` (legacy)
|
||||
|
||||
### Tools Section
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
mcp_server_url: "" # Optional: MCP/OpenAPI tool server
|
||||
mcp_server_api_key: "" # Optional: set via env var
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
- `STEWARD__TOOLS__MCP_SERVER_URL`
|
||||
- `STEWARD__TOOLS__MCP_SERVER_API_KEY`
|
||||
|
||||
## Environment Variable Format
|
||||
|
||||
Environment variables follow the pattern: `STEWARD__SECTION__KEY=value`
|
||||
|
||||
Examples:
|
||||
```bash
|
||||
STEWARD__TELEGRAM__BOT_TOKEN="token123"
|
||||
STEWARD__OPENAI__API_KEY="sk-..."
|
||||
STEWARD__OPENAI__MODEL="gpt-4-turbo"
|
||||
STEWARD__ANALYSIS__CRON_HOUR="9"
|
||||
```
|
||||
|
||||
For list values, use JSON format:
|
||||
```bash
|
||||
STEWARD__TELEGRAM__ALLOWED_USER_IDS='[123456789, 987654321]'
|
||||
STEWARD__TELEGRAM__GROUP_IDS='[-1005308306472]'
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Security
|
||||
- **Never** put secrets in YAML config files
|
||||
- Always use environment variables for:
|
||||
- `STEWARD__TELEGRAM__BOT_TOKEN`
|
||||
- `STEWARD__OPENAI__API_KEY`
|
||||
- `STEWARD__ANALYSIS__TARGET_API_KEY`
|
||||
- `STEWARD__TOOLS__MCP_SERVER_API_KEY`
|
||||
|
||||
### Kubernetes
|
||||
- Use `Secret` resources for sensitive data
|
||||
- Use `ConfigMap` for non-sensitive configuration
|
||||
- Mount both as environment variables for maximum flexibility
|
||||
|
||||
### Docker
|
||||
- Use `.env` file for local development
|
||||
- Use environment variables in production
|
||||
- Mount ConfigMap volumes for complex configurations
|
||||
|
||||
## Backward Compatibility
|
||||
|
||||
The new configuration system is backward compatible with the old `pydantic-settings` approach:
|
||||
|
||||
- The `.env` file is no longer used by default (use `CONFIG_FILE` instead)
|
||||
- Legacy environment variable names like `TELEGRAM_BOT_TOKEN` still work
|
||||
- Properties on the `Settings` object work as before (e.g., `settings.telegram_bot_token`)
|
||||
|
||||
To migrate from `.env`:
|
||||
1. Create a `config.yaml` with your configuration
|
||||
2. Set `CONFIG_FILE` environment variable
|
||||
3. Set secrets via `STEWARD__*` environment variables
|
||||
Reference in New Issue
Block a user