diff options
Diffstat (limited to 'test/s3/retention/README.md')
| -rw-r--r-- | test/s3/retention/README.md | 264 |
1 files changed, 264 insertions, 0 deletions
diff --git a/test/s3/retention/README.md b/test/s3/retention/README.md new file mode 100644 index 000000000..7d92646e6 --- /dev/null +++ b/test/s3/retention/README.md @@ -0,0 +1,264 @@ +# SeaweedFS S3 Object Retention Tests + +This directory contains comprehensive tests for SeaweedFS S3 Object Retention functionality, including Object Lock, Legal Hold, and WORM (Write Once Read Many) capabilities. + +## Overview + +The test suite validates AWS S3-compatible object retention features including: + +- **Object Retention**: GOVERNANCE and COMPLIANCE modes with retain-until-date +- **Legal Hold**: Independent protection that can be applied/removed +- **Object Lock Configuration**: Bucket-level default retention policies +- **WORM Integration**: Compatibility with legacy WORM functionality +- **Version-specific Retention**: Different retention policies per object version +- **Enforcement**: Protection against deletion and overwriting + +## Test Files + +- `s3_retention_test.go` - Core retention functionality tests +- `s3_worm_integration_test.go` - WORM integration and advanced scenarios +- `test_config.json` - Test configuration (endpoints, credentials) +- `Makefile` - Comprehensive test automation +- `go.mod` - Go module dependencies + +## Prerequisites + +- Go 1.21 or later +- SeaweedFS binary built (`make build-weed`) +- AWS SDK Go v2 +- Testify testing framework + +## Quick Start + +### 1. Build and Start Server +```bash +# Build SeaweedFS and start test server +make start-server +``` + +### 2. Run Tests +```bash +# Run core retention tests +make test-retention-quick + +# Run all retention tests +make test-retention + +# Run WORM integration tests +make test-retention-worm + +# Run all tests with managed server +make test-with-server +``` + +### 3. Cleanup +```bash +make clean +``` + +## Test Categories + +### Core Retention Tests +- `TestBasicRetentionWorkflow` - Basic GOVERNANCE mode retention +- `TestRetentionModeCompliance` - COMPLIANCE mode (immutable) +- `TestLegalHoldWorkflow` - Legal hold on/off functionality +- `TestObjectLockConfiguration` - Bucket object lock settings + +### Advanced Tests +- `TestRetentionWithVersions` - Version-specific retention policies +- `TestRetentionAndLegalHoldCombination` - Multiple protection types +- `TestExpiredRetention` - Post-expiration behavior +- `TestRetentionErrorCases` - Error handling and edge cases + +### WORM Integration Tests +- `TestWORMRetentionIntegration` - New retention + legacy WORM +- `TestWORMLegacyCompatibility` - Backward compatibility +- `TestRetentionOverwriteProtection` - Prevent overwrites +- `TestRetentionBulkOperations` - Bulk delete with retention +- `TestRetentionWithMultipartUpload` - Multipart upload retention +- `TestRetentionExtendedAttributes` - Extended attribute storage +- `TestRetentionBucketDefaults` - Default retention application +- `TestRetentionConcurrentOperations` - Concurrent operation safety + +## Individual Test Targets + +Run specific test categories: + +```bash +# Basic functionality +make test-basic-retention +make test-compliance-retention +make test-legal-hold + +# Advanced features +make test-retention-versions +make test-retention-combination +make test-expired-retention + +# WORM integration +make test-worm-integration +make test-worm-legacy +make test-retention-bulk +``` + +## Configuration + +### Server Configuration +The tests use these default settings: +- S3 Port: 8333 +- Test timeout: 15 minutes +- Volume directory: `./test-volume-data` + +### Test Configuration (`test_config.json`) +```json +{ + "endpoint": "http://localhost:8333", + "access_key": "some_access_key1", + "secret_key": "some_secret_key1", + "region": "us-east-1", + "bucket_prefix": "test-retention-", + "use_ssl": false, + "skip_verify_ssl": true +} +``` + +## Expected Behavior + +### GOVERNANCE Mode +- Objects protected until retain-until-date +- Can be bypassed with `x-amz-bypass-governance-retention` header +- Supports time extension (not reduction) + +### COMPLIANCE Mode +- Objects immutably protected until retain-until-date +- Cannot be bypassed or shortened +- Strictest protection level + +### Legal Hold +- Independent ON/OFF protection +- Can coexist with retention policies +- Must be explicitly removed to allow deletion + +### Version Support +- Each object version can have individual retention +- Applies to both versioned and non-versioned buckets +- Version-specific retention retrieval + +## Development + +### Running in Development Mode +```bash +# Start server for development +make dev-start + +# Run quick test +make dev-test +``` + +### Code Quality +```bash +# Format code +make fmt + +# Run linter +make lint + +# Generate coverage report +make coverage +``` + +### Performance Testing +```bash +# Run benchmarks +make benchmark-retention +``` + +## Troubleshooting + +### Server Won't Start +```bash +# Check if port is in use +netstat -tlnp | grep 8333 + +# View server logs +make logs + +# Force cleanup +make clean +``` + +### Test Failures +```bash +# Run with verbose output +go test -v -timeout=15m . + +# Run specific test +go test -v -run TestBasicRetentionWorkflow . + +# Check server health +make health-check +``` + +### Dependencies +```bash +# Install/update dependencies +make install-deps + +# Check dependency status +make check-deps +``` + +## Integration with SeaweedFS + +These tests validate the retention implementation in: +- `weed/s3api/s3api_object_retention.go` - Core retention logic +- `weed/s3api/s3api_object_handlers_retention.go` - HTTP handlers +- `weed/s3api/s3_constants/extend_key.go` - Extended attribute keys +- `weed/s3api/s3err/s3api_errors.go` - Error definitions +- `weed/s3api/s3api_object_handlers_delete.go` - Deletion enforcement +- `weed/s3api/s3api_object_handlers_put.go` - Upload enforcement + +## AWS CLI Compatibility + +The retention implementation supports standard AWS CLI commands: + +```bash +# Set object retention +aws s3api put-object-retention \ + --bucket mybucket \ + --key myobject \ + --retention Mode=GOVERNANCE,RetainUntilDate=2024-12-31T23:59:59Z + +# Get object retention +aws s3api get-object-retention \ + --bucket mybucket \ + --key myobject + +# Set legal hold +aws s3api put-object-legal-hold \ + --bucket mybucket \ + --key myobject \ + --legal-hold Status=ON + +# Configure bucket object lock +aws s3api put-object-lock-configuration \ + --bucket mybucket \ + --object-lock-configuration ObjectLockEnabled=Enabled,Rule='{DefaultRetention={Mode=GOVERNANCE,Days=30}}' +``` + +## Contributing + +When adding new retention tests: + +1. Follow existing test patterns +2. Use descriptive test names +3. Include both positive and negative test cases +4. Test error conditions +5. Update this README with new test descriptions +6. Add appropriate Makefile targets for new test categories + +## References + +- [AWS S3 Object Lock Documentation](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) +- [AWS S3 API Reference - Object Retention](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html) +- [SeaweedFS S3 API Documentation](https://github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API)
\ No newline at end of file |
