aboutsummaryrefslogtreecommitdiff
path: root/weed/filer/foundationdb/CONFIGURATION.md
diff options
context:
space:
mode:
Diffstat (limited to 'weed/filer/foundationdb/CONFIGURATION.md')
-rw-r--r--weed/filer/foundationdb/CONFIGURATION.md385
1 files changed, 385 insertions, 0 deletions
diff --git a/weed/filer/foundationdb/CONFIGURATION.md b/weed/filer/foundationdb/CONFIGURATION.md
new file mode 100644
index 000000000..80f5bd357
--- /dev/null
+++ b/weed/filer/foundationdb/CONFIGURATION.md
@@ -0,0 +1,385 @@
+# FoundationDB Filer Store Configuration Reference
+
+This document provides comprehensive configuration options for the FoundationDB filer store.
+
+## Configuration Methods
+
+### 1. Configuration File (filer.toml)
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+api_version = 740
+timeout = "5s"
+max_retry_delay = "1s"
+directory_prefix = "seaweedfs"
+```
+
+### 2. Environment Variables
+
+All configuration options can be set via environment variables with the `WEED_FOUNDATIONDB_` prefix:
+
+```bash
+export WEED_FOUNDATIONDB_ENABLED=true
+export WEED_FOUNDATIONDB_CLUSTER_FILE=/etc/foundationdb/fdb.cluster
+export WEED_FOUNDATIONDB_API_VERSION=740
+export WEED_FOUNDATIONDB_TIMEOUT=5s
+export WEED_FOUNDATIONDB_MAX_RETRY_DELAY=1s
+export WEED_FOUNDATIONDB_DIRECTORY_PREFIX=seaweedfs
+```
+
+### 3. Command Line Arguments
+
+While not directly supported, configuration can be specified via config files passed to the `weed` command.
+
+## Configuration Options
+
+### Basic Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `enabled` | boolean | `false` | Enable the FoundationDB filer store |
+| `cluster_file` | string | `/etc/foundationdb/fdb.cluster` | Path to FoundationDB cluster file |
+| `api_version` | integer | `740` | FoundationDB API version to use |
+
+### Connection Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `timeout` | duration | `5s` | Transaction timeout duration |
+| `max_retry_delay` | duration | `1s` | Maximum delay between retries |
+
+### Storage Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `directory_prefix` | string | `seaweedfs` | Directory prefix for key organization |
+
+## Configuration Examples
+
+### Development Environment
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/var/fdb/config/fdb.cluster"
+api_version = 740
+timeout = "10s"
+max_retry_delay = "2s"
+directory_prefix = "seaweedfs_dev"
+```
+
+### Production Environment
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+api_version = 740
+timeout = "30s"
+max_retry_delay = "5s"
+directory_prefix = "seaweedfs_prod"
+```
+
+### High-Performance Setup
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+api_version = 740
+timeout = "60s"
+max_retry_delay = "10s"
+directory_prefix = "sw" # Shorter prefix for efficiency
+```
+
+### Path-Specific Configuration
+
+Configure different FoundationDB settings for different paths:
+
+```toml
+# Default configuration
+[foundationdb]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+directory_prefix = "seaweedfs_main"
+
+# Backup path with different prefix
+[foundationdb.backup]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+directory_prefix = "seaweedfs_backup"
+location = "/backup"
+timeout = "120s"
+
+# Archive path with extended timeouts
+[foundationdb.archive]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+directory_prefix = "seaweedfs_archive"
+location = "/archive"
+timeout = "300s"
+max_retry_delay = "30s"
+```
+
+## Configuration Validation
+
+### Required Settings
+
+The following settings are required for FoundationDB to function:
+
+1. `enabled = true`
+2. `cluster_file` must point to a valid FoundationDB cluster file
+3. `api_version` must match your FoundationDB installation
+
+### Validation Rules
+
+- `api_version` must be between 600 and 740
+- `timeout` must be a valid duration string (e.g., "5s", "30s", "2m")
+- `max_retry_delay` must be a valid duration string
+- `cluster_file` must exist and be readable
+- `directory_prefix` must not be empty
+
+### Error Handling
+
+Invalid configurations will result in startup errors:
+
+```
+FATAL: Failed to initialize store for foundationdb: invalid timeout duration
+FATAL: Failed to initialize store for foundationdb: failed to open FoundationDB database
+FATAL: Failed to initialize store for foundationdb: cluster file not found
+```
+
+## Performance Tuning
+
+### Timeout Configuration
+
+| Use Case | Timeout | Max Retry Delay | Notes |
+|----------|---------|-----------------|-------|
+| Interactive workloads | 5s | 1s | Fast response times |
+| Batch processing | 60s | 10s | Handle large operations |
+| Archive operations | 300s | 30s | Very large data sets |
+
+### Connection Pool Settings
+
+FoundationDB automatically manages connection pooling. No additional configuration needed.
+
+### Directory Organization
+
+Use meaningful directory prefixes to organize data:
+
+```toml
+# Separate environments
+directory_prefix = "prod_seaweedfs" # Production
+directory_prefix = "staging_seaweedfs" # Staging
+directory_prefix = "dev_seaweedfs" # Development
+
+# Separate applications
+directory_prefix = "app1_seaweedfs" # Application 1
+directory_prefix = "app2_seaweedfs" # Application 2
+```
+
+## Security Configuration
+
+### Cluster File Security
+
+Protect the FoundationDB cluster file:
+
+```bash
+# Set proper permissions
+sudo chown root:seaweedfs /etc/foundationdb/fdb.cluster
+sudo chmod 640 /etc/foundationdb/fdb.cluster
+```
+
+### Network Security
+
+FoundationDB supports TLS encryption. Configure in the cluster file:
+
+```
+description:cluster_id@tls(server1:4500,server2:4500,server3:4500)
+```
+
+### Access Control
+
+Use FoundationDB's built-in access control mechanisms when available.
+
+## Monitoring Configuration
+
+### Health Check Settings
+
+Configure health check timeouts appropriately:
+
+```toml
+[foundationdb]
+enabled = true
+timeout = "10s" # Reasonable timeout for health checks
+```
+
+### Logging Configuration
+
+Enable verbose logging for troubleshooting:
+
+```bash
+# Start SeaweedFS with debug logs
+WEED_FOUNDATIONDB_ENABLED=true weed -v=2 server -filer
+```
+
+## Migration Configuration
+
+### From Other Filer Stores
+
+When migrating from other filer stores:
+
+1. Configure both stores temporarily
+2. Use path-specific configuration for gradual migration
+3. Migrate data using SeaweedFS tools
+
+```toml
+# During migration - keep old store for reads
+[leveldb2]
+enabled = true
+dir = "/old/filer/data"
+
+# New writes go to FoundationDB
+[foundationdb.migration]
+enabled = true
+location = "/new"
+cluster_file = "/etc/foundationdb/fdb.cluster"
+```
+
+## Backup Configuration
+
+### Metadata Backup Strategy
+
+```toml
+# Main storage
+[foundationdb]
+enabled = true
+directory_prefix = "seaweedfs_main"
+
+# Backup storage (different cluster recommended)
+[foundationdb.backup]
+enabled = true
+cluster_file = "/etc/foundationdb/backup_fdb.cluster"
+directory_prefix = "seaweedfs_backup"
+location = "/backup"
+```
+
+## Container Configuration
+
+### Docker Environment Variables
+
+```bash
+# Docker environment
+WEED_FOUNDATIONDB_ENABLED=true
+WEED_FOUNDATIONDB_CLUSTER_FILE=/var/fdb/config/fdb.cluster
+WEED_FOUNDATIONDB_API_VERSION=740
+```
+
+### Kubernetes ConfigMap
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: seaweedfs-config
+data:
+ filer.toml: |
+ [foundationdb]
+ enabled = true
+ cluster_file = "/var/fdb/config/cluster_file"
+ api_version = 740
+ timeout = "30s"
+ max_retry_delay = "5s"
+ directory_prefix = "k8s_seaweedfs"
+```
+
+## Troubleshooting Configuration
+
+### Debug Configuration
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/etc/foundationdb/fdb.cluster"
+timeout = "60s" # Longer timeouts for debugging
+max_retry_delay = "10s"
+directory_prefix = "debug_seaweedfs"
+```
+
+### Test Configuration
+
+```toml
+[foundationdb]
+enabled = true
+cluster_file = "/tmp/fdb.cluster" # Test cluster
+timeout = "5s"
+directory_prefix = "test_seaweedfs"
+```
+
+## Configuration Best Practices
+
+### 1. Environment Separation
+
+Use different directory prefixes for different environments:
+- Production: `prod_seaweedfs`
+- Staging: `staging_seaweedfs`
+- Development: `dev_seaweedfs`
+
+### 2. Timeout Settings
+
+- Interactive: 5-10 seconds
+- Batch: 30-60 seconds
+- Archive: 120-300 seconds
+
+### 3. Cluster File Management
+
+- Use absolute paths for cluster files
+- Ensure proper file permissions
+- Keep backup copies of cluster files
+
+### 4. Directory Naming
+
+- Use descriptive prefixes
+- Include environment/application identifiers
+- Keep prefixes reasonably short for efficiency
+
+### 5. Error Handling
+
+- Configure appropriate timeouts
+- Monitor retry patterns
+- Set up alerting for configuration errors
+
+## Configuration Testing
+
+### Validation Script
+
+```bash
+#!/bin/bash
+# Test FoundationDB configuration
+
+# Check cluster file
+if [ ! -f "$WEED_FOUNDATIONDB_CLUSTER_FILE" ]; then
+ echo "ERROR: Cluster file not found: $WEED_FOUNDATIONDB_CLUSTER_FILE"
+ exit 1
+fi
+
+# Test connection
+fdbcli -C "$WEED_FOUNDATIONDB_CLUSTER_FILE" --exec 'status' > /dev/null
+if [ $? -ne 0 ]; then
+ echo "ERROR: Cannot connect to FoundationDB cluster"
+ exit 1
+fi
+
+echo "Configuration validation passed"
+```
+
+### Integration Testing
+
+```bash
+# Test configuration with SeaweedFS
+cd test/foundationdb
+make check-env
+make test-unit
+```