diff options
Diffstat (limited to 'seaweedfs-rdma-sidecar/DOCKER-TESTING.md')
| -rw-r--r-- | seaweedfs-rdma-sidecar/DOCKER-TESTING.md | 290 |
1 files changed, 290 insertions, 0 deletions
diff --git a/seaweedfs-rdma-sidecar/DOCKER-TESTING.md b/seaweedfs-rdma-sidecar/DOCKER-TESTING.md new file mode 100644 index 000000000..88ea1971d --- /dev/null +++ b/seaweedfs-rdma-sidecar/DOCKER-TESTING.md @@ -0,0 +1,290 @@ +# ๐ณ Docker Integration Testing Guide + +This guide provides comprehensive Docker-based integration testing for the SeaweedFS RDMA sidecar system. + +## ๐๏ธ Architecture + +The Docker Compose setup includes: + +``` +โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ +โ SeaweedFS Master โ โ SeaweedFS Volume โ โ Rust RDMA โ +โ :9333 โโโโโบโ :8080 โ โ Engine โ +โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ + โ โ + โผ โผ +โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ +โ Go RDMA Sidecar โโโโโบโ Unix Socket โโโโโบโ Integration โ +โ :8081 โ โ /tmp/rdma.sock โ โ Test Suite โ +โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ +``` + +## ๐ Quick Start + +### 1. Start All Services + +```bash +# Using the helper script (recommended) +./tests/docker-test-helper.sh start + +# Or using docker-compose directly +docker-compose up -d +``` + +### 2. Run Integration Tests + +```bash +# Run the complete test suite +./tests/docker-test-helper.sh test + +# Or run tests manually +docker-compose run --rm integration-tests +``` + +### 3. Interactive Testing + +```bash +# Open a shell in the test container +./tests/docker-test-helper.sh shell + +# Inside the container, you can run: +./test-rdma ping +./test-rdma capabilities +./test-rdma read --volume 1 --needle 12345 --size 1024 +curl http://rdma-sidecar:8081/health +curl http://rdma-sidecar:8081/stats +``` + +## ๐ Test Helper Commands + +The `docker-test-helper.sh` script provides convenient commands: + +```bash +# Service Management +./tests/docker-test-helper.sh start # Start all services +./tests/docker-test-helper.sh stop # Stop all services +./tests/docker-test-helper.sh clean # Stop and clean volumes + +# Testing +./tests/docker-test-helper.sh test # Run integration tests +./tests/docker-test-helper.sh shell # Interactive testing shell + +# Monitoring +./tests/docker-test-helper.sh status # Check service health +./tests/docker-test-helper.sh logs # Show all logs +./tests/docker-test-helper.sh logs rdma-engine # Show specific service logs +``` + +## ๐งช Test Coverage + +The integration test suite covers: + +### โ
Core Components +- **SeaweedFS Master**: Cluster leadership and status +- **SeaweedFS Volume Server**: Volume operations and health +- **Rust RDMA Engine**: Socket communication and operations +- **Go RDMA Sidecar**: HTTP API and RDMA integration + +### โ
Integration Points +- **IPC Communication**: Unix socket + MessagePack protocol +- **RDMA Operations**: Ping, capabilities, read operations +- **HTTP API**: All sidecar endpoints and error handling +- **Fallback Logic**: RDMA โ HTTP fallback behavior + +### โ
Performance Testing +- **Direct RDMA Benchmarks**: Engine-level performance +- **Sidecar Benchmarks**: End-to-end performance +- **Latency Measurements**: Operation timing validation +- **Throughput Testing**: Operations per second + +## ๐ง Service Details + +### SeaweedFS Master +- **Port**: 9333 +- **Health Check**: `/cluster/status` +- **Data**: Persistent volume `master-data` + +### SeaweedFS Volume Server +- **Port**: 8080 +- **Health Check**: `/status` +- **Data**: Persistent volume `volume-data` +- **Depends on**: SeaweedFS Master + +### Rust RDMA Engine +- **Socket**: `/tmp/rdma-engine.sock` +- **Mode**: Mock RDMA (development) +- **Health Check**: Socket existence +- **Privileged**: Yes (for RDMA access) + +### Go RDMA Sidecar +- **Port**: 8081 +- **Health Check**: `/health` +- **API Endpoints**: `/stats`, `/read`, `/benchmark` +- **Depends on**: RDMA Engine, Volume Server + +### Test Client +- **Purpose**: Integration testing and interactive debugging +- **Tools**: curl, jq, test-rdma binary +- **Environment**: All service URLs configured + +## ๐ Expected Test Results + +### โ
Successful Output Example + +``` +=============================================== +๐ SEAWEEDFS RDMA INTEGRATION TEST SUITE +=============================================== + +๐ต Waiting for SeaweedFS Master to be ready... +โ
SeaweedFS Master is ready +โ
SeaweedFS Master is leader and ready + +๐ต Waiting for SeaweedFS Volume Server to be ready... +โ
SeaweedFS Volume Server is ready +Volume Server Version: 3.60 + +๐ต Checking RDMA engine socket... +โ
RDMA engine socket exists +๐ต Testing RDMA engine ping... +โ
RDMA engine ping successful + +๐ต Waiting for RDMA Sidecar to be ready... +โ
RDMA Sidecar is ready +โ
RDMA Sidecar is healthy +RDMA Status: true + +๐ต Testing needle read via sidecar... +โ
Sidecar needle read successful +โ ๏ธ HTTP fallback used. Duration: 2.48ms + +๐ต Running sidecar performance benchmark... +โ
Sidecar benchmark completed +Benchmark Results: + RDMA Operations: 5 + HTTP Operations: 0 + Average Latency: 2.479ms + Operations/sec: 403.2 + +=============================================== +๐ ALL INTEGRATION TESTS COMPLETED! +=============================================== +``` + +## ๐ Troubleshooting + +### Service Not Starting + +```bash +# Check service logs +./tests/docker-test-helper.sh logs [service-name] + +# Check container status +docker-compose ps + +# Restart specific service +docker-compose restart [service-name] +``` + +### RDMA Engine Issues + +```bash +# Check socket permissions +docker-compose exec rdma-engine ls -la /tmp/rdma/rdma-engine.sock + +# Check RDMA engine logs +./tests/docker-test-helper.sh logs rdma-engine + +# Test socket directly +docker-compose exec test-client ./test-rdma ping +``` + +### Sidecar Connection Issues + +```bash +# Test sidecar health directly +curl http://localhost:8081/health + +# Check sidecar logs +./tests/docker-test-helper.sh logs rdma-sidecar + +# Verify environment variables +docker-compose exec rdma-sidecar env | grep RDMA +``` + +### Volume Server Issues + +```bash +# Check SeaweedFS status +curl http://localhost:9333/cluster/status +curl http://localhost:8080/status + +# Check volume server logs +./tests/docker-test-helper.sh logs seaweedfs-volume +``` + +## ๐ Manual Testing Examples + +### Test RDMA Engine Directly + +```bash +# Enter test container +./tests/docker-test-helper.sh shell + +# Test RDMA operations +./test-rdma ping --socket /tmp/rdma-engine.sock +./test-rdma capabilities --socket /tmp/rdma-engine.sock +./test-rdma read --socket /tmp/rdma-engine.sock --volume 1 --needle 12345 +./test-rdma bench --socket /tmp/rdma-engine.sock --iterations 10 +``` + +### Test Sidecar HTTP API + +```bash +# Health and status +curl http://rdma-sidecar:8081/health | jq '.' +curl http://rdma-sidecar:8081/stats | jq '.' + +# Needle operations +curl "http://rdma-sidecar:8081/read?volume=1&needle=12345&size=1024" | jq '.' + +# Benchmarking +curl "http://rdma-sidecar:8081/benchmark?iterations=5&size=2048" | jq '.benchmark_results' +``` + +### Test SeaweedFS Integration + +```bash +# Check cluster status +curl http://seaweedfs-master:9333/cluster/status | jq '.' + +# Check volume status +curl http://seaweedfs-volume:8080/status | jq '.' + +# List volumes +curl http://seaweedfs-master:9333/vol/status | jq '.' +``` + +## ๐ Production Deployment + +This Docker setup can be adapted for production by: + +1. **Replacing Mock RDMA**: Switch to `real-ucx` feature in Rust +2. **RDMA Hardware**: Add RDMA device mappings and capabilities +3. **Security**: Remove privileged mode, add proper user/group mapping +4. **Scaling**: Use Docker Swarm or Kubernetes for orchestration +5. **Monitoring**: Add Prometheus metrics and Grafana dashboards +6. **Persistence**: Configure proper volume management + +## ๐ Additional Resources + +- [Main README](README.md) - Complete project overview +- [Docker Compose Reference](https://docs.docker.com/compose/) +- [SeaweedFS Documentation](https://github.com/seaweedfs/seaweedfs/wiki) +- [UCX Documentation](https://github.com/openucx/ucx) + +--- + +**๐ณ Happy Docker Testing!** + +For issues or questions, please check the logs first and refer to the troubleshooting section above. |