Test object lock and retention (#6997)

* fix GetObjectLockConfigurationHandler

* cache and use bucket object lock config

* subscribe to bucket configuration changes

* increase bucket config cache TTL

* refactor

* Update weed/s3api/s3api_server.go

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

* avoid duplidated work

* rename variable

* Update s3api_object_handlers_put.go

* fix routing

* admin ui and api handler are consistent now

* use fields instead of xml

* fix test

* address comments

* Update weed/s3api/s3api_object_handlers_put.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/retention/s3_retention_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/object_lock_utils.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* change error style

* errorf

* read entry once

* add s3 tests for object lock and retention

* use marker

* install s3 tests

* Update s3tests.yml

* Update s3tests.yml

* Update s3tests.conf

* Update s3tests.conf

* address test errors

* address test errors

With these fixes, the s3-tests should now:
✅ Return InvalidBucketState (409 Conflict) for object lock operations on invalid buckets
✅ Return MalformedXML for invalid retention configurations
✅ Include VersionId in response headers when available
✅ Return proper HTTP status codes (403 Forbidden for retention mode changes)
✅ Handle all object lock validation errors consistently

* fixes

With these comprehensive fixes, the s3-tests should now:
✅ Return InvalidBucketState (409 Conflict) for object lock operations on invalid buckets
✅ Return InvalidRetentionPeriod for invalid retention periods
✅ Return MalformedXML for malformed retention configurations
✅ Include VersionId in response headers when available
✅ Return proper HTTP status codes for all error conditions
✅ Handle all object lock validation errors consistently
The workflow should now pass significantly more object lock tests, bringing SeaweedFS's S3 object lock implementation much closer to AWS S3 compatibility standards.

* fixes

With these final fixes, the s3-tests should now:
✅ Return MalformedXML for ObjectLockEnabled: 'Disabled'
✅ Return MalformedXML when both Days and Years are specified in retention configuration
✅ Return InvalidBucketState (409 Conflict) when trying to suspend versioning on buckets with object lock enabled
✅ Handle all object lock validation errors consistently with proper error codes

* constants and fixes

✅ Return InvalidRetentionPeriod for invalid retention values (0 days, negative years)
✅ Return ObjectLockConfigurationNotFoundError when object lock configuration doesn't exist
✅ Handle all object lock validation errors consistently with proper error codes

* fixes

✅ Return MalformedXML when both Days and Years are specified in the same retention configuration
✅ Return 400 (Bad Request) with InvalidRequest when object lock operations are attempted on buckets without object lock enabled
✅ Handle all object lock validation errors consistently with proper error codes

* fixes

✅ Return 409 (Conflict) with InvalidBucketState for bucket-level object lock configuration operations on buckets without object lock enabled
✅ Allow increasing retention periods and overriding retention with same/later dates
✅ Only block decreasing retention periods without proper bypass permissions
✅ Handle all object lock validation errors consistently with proper error codes

* fixes

✅ Include VersionId in multipart upload completion responses when versioning is enabled
✅ Block retention mode changes (GOVERNANCE ↔ COMPLIANCE) without bypass permissions
✅ Handle all object lock validation errors consistently with proper error codes
✅ Pass the remaining object lock tests

* fix tests

* fixes

* pass tests

* fix tests

* fixes

* add error mapping

* Update s3tests.conf

* fix test_object_lock_put_obj_lock_invalid_days

* fixes

* fix many issues

* fix test_object_lock_delete_multipart_object_with_legal_hold_on

* fix tests

* refactor

* fix test_object_lock_delete_object_with_retention_and_marker

* fix tests

* fix tests

* fix tests

* fix test itself

* fix tests

* fix test

* Update weed/s3api/s3api_object_retention.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* reduce logs

* address comments

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

1 files changed, 36 insertions, 23 deletions

diff --git a/weed/s3api/s3api_object_retention_test.go b/weed/s3api/s3api_object_retention_test.go
index 0caa50b42..638d37d14 100644
--- a/weed/s3api/s3api_object_retention_test.go
+++ b/weed/s3api/s3api_object_retention_test.go
@@ -498,8 +498,9 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode: "GOVERNANCE",
-						Days: 30,
+						Mode:    "GOVERNANCE",
+						Days:    30,
+						DaysSet: true,
 					},
 				},
 			},
@@ -511,8 +512,9 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode:  "COMPLIANCE",
-						Years: 1,
+						Mode:     "COMPLIANCE",
+						Years:    1,
+						YearsSet: true,
 					},
 				},
 			},
@@ -545,9 +547,11 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode:  "GOVERNANCE",
-						Days:  30,
-						Years: 1,
+						Mode:     "GOVERNANCE",
+						Days:     30,
+						Years:    1,
+						DaysSet:  true,
+						YearsSet: true,
 					},
 				},
 			},
@@ -573,8 +577,9 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode: "INVALID_MODE",
-						Days: 30,
+						Mode:    "INVALID_MODE",
+						Days:    30,
+						DaysSet: true,
 					},
 				},
 			},
@@ -587,8 +592,9 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode: "GOVERNANCE",
-						Days: 50000,
+						Mode:    "GOVERNANCE",
+						Days:    50000,
+						DaysSet: true,
 					},
 				},
 			},
@@ -601,8 +607,9 @@ func TestValidateObjectLockConfiguration(t *testing.T) {
 				ObjectLockEnabled: "Enabled",
 				Rule: &ObjectLockRule{
 					DefaultRetention: &DefaultRetention{
-						Mode:  "GOVERNANCE",
-						Years: 200,
+						Mode:     "GOVERNANCE",
+						Years:    200,
+						YearsSet: true,
 					},
 				},
 			},
@@ -651,23 +658,26 @@ func TestValidateDefaultRetention(t *testing.T) {
 		{
 			name: "Valid retention with days",
 			retention: &DefaultRetention{
-				Mode: "GOVERNANCE",
-				Days: 30,
+				Mode:    "GOVERNANCE",
+				Days:    30,
+				DaysSet: true,
 			},
 			expectError: false,
 		},
 		{
 			name: "Valid retention with years",
 			retention: &DefaultRetention{
-				Mode:  "COMPLIANCE",
-				Years: 1,
+				Mode:     "COMPLIANCE",
+				Years:    1,
+				YearsSet: true,
 			},
 			expectError: false,
 		},
 		{
 			name: "Missing mode",
 			retention: &DefaultRetention{
-				Days: 30,
+				Days:    30,
+				DaysSet: true,
 			},
 			expectError: true,
 			errorMsg:    "default retention must specify Mode",
@@ -675,8 +685,9 @@ func TestValidateDefaultRetention(t *testing.T) {
 		{
 			name: "Invalid mode",
 			retention: &DefaultRetention{
-				Mode: "INVALID",
-				Days: 30,
+				Mode:    "INVALID",
+				Days:    30,
+				DaysSet: true,
 			},
 			expectError: true,
 			errorMsg:    "invalid default retention mode",
@@ -684,9 +695,11 @@ func TestValidateDefaultRetention(t *testing.T) {
 		{
 			name: "Both days and years specified",
 			retention: &DefaultRetention{
-				Mode:  "GOVERNANCE",
-				Days:  30,
-				Years: 1,
+				Mode:     "GOVERNANCE",
+				Days:     30,
+				Years:    1,
+				DaysSet:  true,
+				YearsSet: true,
 			},
 			expectError: true,
 			errorMsg:    "default retention cannot specify both Days and Years",