aboutsummaryrefslogtreecommitdiff
path: root/weed/s3api/s3_constants
AgeCommit message (Collapse)AuthorFilesLines
2025-07-18use fields instead of xmlchrislu1-1/+5
2025-07-16Object locking need to persist the tags and set the headers (#6994)Chris Lu1-1/+4
* fix object locking read and write No logic to include object lock metadata in HEAD/GET response headers No logic to extract object lock metadata from PUT request headers * add tests for object locking * Update weed/s3api/s3api_object_handlers_put.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor * add unit tests * sync versions * Update s3_worm_integration_test.go * fix legal hold values * lint * fix tests * racing condition when enable versioning * fix tests * validate put object lock header * allow check lock permissions for PUT * default to OFF legal hold * only set object lock headers for objects that are actually from object lock-enabled buckets fix --- FAIL: TestAddObjectLockHeadersToResponse/Handle_entry_with_no_object_lock_metadata (0.00s) * address comments * fix tests * purge * fix * refactoring * address comment * address comment * Update weed/s3api/s3api_object_handlers_put.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers_put.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * avoid nil * ensure locked objects cannot be overwritten --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-15S3 Object Lock: ensure x-amz-bucket-object-lock-enabled header (#6990)Chris Lu2-0/+7
* ensure x-amz-bucket-object-lock-enabled header * fix tests * combine 2 metadata changes into one * address comments * Update s3api_bucket_handlers.go * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/object_lock_reproduce_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/object_lock_validation_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/s3_bucket_object_lock_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update test/s3/retention/s3_bucket_object_lock_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_bucket_handlers.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * package name --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-13admin ui adds object lock permissionschrislu1-9/+15
2025-07-13Add policy engine (#6970)Chris Lu3-9/+12
2025-07-12implement PubObjectRetention and WORM (#6969)Chris Lu1-0/+21
* implement PubObjectRetention and WORM * Update s3_worm_integration_test.go * avoid previous buckets * Update s3-versioning-tests.yml * address comments * address comments * rename to ExtObjectLockModeKey * only checkObjectLockPermissions if versioningEnabled * address comments * comments * Revert "comments" This reverts commit 6736434176f86c6e222b867777324b17c2de716f. * Update s3api_object_handlers_skip.go * Update s3api_object_retention_test.go * add version id to ObjectIdentifier * address comments * add comments * Add proper error logging for timestamp parsing failures * address comments * add version id to the error * Update weed/s3api/s3api_object_retention_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_retention.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * constants * fix comments * address comments * address comment * refactor out handleObjectLockAvailabilityCheck * errors.Is ErrBucketNotFound * better error checking * address comments --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-11More efficient copy object (#6665)Chris Lu1-0/+6
* it compiles * refactored * reduce to 4 concurrent chunk upload * CopyObjectPartHandler * copy a range of the chunk data, fix offset size in copied chunks * Update s3api_object_handlers_copy.go What the PR Accomplishes: CopyObjectHandler - Now copies entire objects by copying chunks individually instead of downloading/uploading the entire file CopyObjectPartHandler - Handles copying parts of objects for multipart uploads by copying only the relevant chunk portions Efficient Chunk Copying - Uses direct chunk-to-chunk copying with proper volume assignment and concurrent processing (limited to 4 concurrent operations) Range Support - Properly handles range-based copying for partial object copies * fix compilation * fix part destination * handling small objects * use mkFile * copy to existing file or part * add testing tools * adjust tests * fix chunk lookup * refactoring * fix TestObjectCopyRetainingMetadata * ensure bucket name not conflicting * fix conditional copying tests * remove debug messages * add custom s3 copy tests
2025-07-09S3: add object versioning (#6945)Chris Lu1-3/+10
* add object versioning * add missing file * Update weed/s3api/s3api_object_versioning.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_versioning.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_versioning.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * ListObjectVersionsResult is better to show multiple version entries * fix test * Update weed/s3api/s3api_object_handlers_put.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/s3api/s3api_object_versioning.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * multiple improvements * move PutBucketVersioningHandler into weed/s3api/s3api_bucket_handlers.go file * duplicated code for reading bucket config, versioningEnabled, etc. try to use functions * opportunity to cache bucket config * error handling if bucket is not found * in case bucket is not found * fix build * add object versioning tests * remove non-existent tests * add tests * add versioning tests * skip a new test * ensure .versions directory exists before saving info into it * fix creating version entry * logging on creating version directory * Update s3api_object_versioning_test.go * retry and wait for directory creation * revert add more logging * Update s3api_object_versioning.go * more debug messages * clean up logs, and touch directory correctly * log the .versions creation and then parent directory listing * use mkFile instead of touch touch is for update * clean up data * add versioning test in go * change location * if modified, latest version is moved to .versions directory, and create a new latest version Core versioning functionality: WORKING TestVersioningBasicWorkflow - PASS TestVersioningDeleteMarkers - PASS TestVersioningMultipleVersionsSameObject - PASS TestVersioningDeleteAndRecreate - PASS TestVersioningListWithPagination - PASS ❌ Some advanced features still failing: ETag calculation issues (using mtime instead of proper MD5) Specific version retrieval (EOF error) Version deletion (internal errors) Concurrent operations (race conditions) * calculate multi chunk md5 Test Results - All Passing: ✅ TestBucketListReturnDataVersioning - PASS ✅ TestVersioningCreateObjectsInOrder - PASS ✅ TestVersioningBasicWorkflow - PASS ✅ TestVersioningMultipleVersionsSameObject - PASS ✅ TestVersioningDeleteMarkers - PASS * dedupe * fix TestVersioningErrorCases * fix eof error of reading old versions * get specific version also check current version * enable integration tests for versioning * trigger action to work for now * Fix GitHub Actions S3 versioning tests workflow - Fix syntax error (incorrect indentation) - Update directory paths from weed/s3api/versioning_tests/ to test/s3/versioning/ - Add push trigger for add-object-versioning branch to enable CI during development - Update artifact paths to match correct directory structure * Improve CI robustness for S3 versioning tests Makefile improvements: - Increase server startup timeout from 30s to 90s for CI environments - Add progressive timeout reporting (logs at 30s, full logs at 90s) - Better error handling with server logs on failure - Add server PID tracking for debugging - Improved test failure reporting GitHub Actions workflow improvements: - Increase job timeouts to account for CI environment delays - Add system information logging (memory, disk space) - Add detailed failure reporting with server logs - Add process and network diagnostics on failure - Better error messaging and log collection These changes should resolve the 'Server failed to start within 30 seconds' issue that was causing the CI tests to fail. * adjust testing volume size * Update Makefile * Update Makefile * Update Makefile * Update Makefile * Update s3-versioning-tests.yml * Update s3api_object_versioning.go * Update Makefile * do not clean up * log received version id * more logs * printout response * print out list version response * use tmp files when put versioned object * change to versions folder layout * Delete weed-test.log * test with mixed versioned and unversioned objects * remove versionDirCache * remove unused functions * remove unused function * remove fallback checking * minor --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-01-17fix S3 per-user-directory Policy (#6443)Tom Crasset1-1/+12
* fix S3 per-user-directory Policy * Delete docker/config.json * add tests * remove logs * undo modifications of weed/shell/command_volume_balance.go * remove modifications of docker-compose * fix failing test --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-05-17added s3 iam DeleteBucket permission management (#5599)Riccardo Bertossa2-8/+9
2024-04-14[s3] fix s3 test_multipart_get_part (#5476)Konstantin Lebedev1-2/+5
* try fix s3 test_multipart_get_part * add passed s3 tests * fix SeaweedFSUploadId * rm spaces * convert part request to range * add passed s3 tests of multipart
2024-04-07[s3] add s3 pass test_multipart_upload (#5474)Konstantin Lebedev1-0/+1
add s3 pass test_multipart_upload
2024-03-07[filer.backup] add param uploader_part_size for S3sink (#5352)Konstantin Lebedev1-0/+1
* fix: install cronie * chore: refactor configure S3Sink * chore: refactor cinfig * add filer-backup compose file * fix: X-Amz-Meta-Mtime and resolve with comments * fix: attr mtime * fix: MaxUploadPartst is reduced to the maximum allowable * fix: env and force set max MaxUploadParts * fix: env WEED_SINK_S3_UPLOADER_PART_SIZE_MB
2023-09-25[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration ↵Konstantin Lebedev1-0/+6
(#4859) * move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface https://github.com/seaweedfs/seaweedfs/issues/4519 * fix: test bucket acl default and adjust the variable names * fix: s3 api config test --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-21[iam] Replace action read/write to readAcp/writeAcp for handlers with acl ↵Konstantin Lebedev2-6/+8
(#4858) Replace action read/write to readAcp/writeAcp for handlers with acl query https://github.com/seaweedfs/seaweedfs/issues/4519 Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-04-11s3 fix get fake dir object key (#4390)Konstantin Lebedev1-0/+1
2023-03-02 fix chunk.ModifiedTsNs (#4264)zemul1-1/+1
* fix * fix mtime s > ns --------- Co-authored-by: zemul <zhouzemiao@ihuman.com>
2022-10-12add acl helper functionalities (#3831)LHHDZ2-0/+20
2022-10-11split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unn… (#3824)LHHDZ1-1/+2
split `ExtAcpKey` to `ExtAmzOwnerKey` and `ExtAmzAclKey` to avoid unnecessary `json.Unmarshal()` call Signed-off-by: changlin.shi <changlin.shi@ly.com> Signed-off-by: changlin.shi <changlin.shi@ly.com>
2022-10-01add ownership rest apis (#3765)LHHDZ1-0/+1
2022-09-29s3: sync bucket info from filer (#3759)LHHDZ6-0/+113
2022-08-23s3: report http.StatusOK if the directory is explicitly createdchrislu1-0/+2
fix https://github.com/seaweedfs/seaweedfs/issues/3457
2022-06-29use const multipart uploads folderKonstantin Lebedev1-0/+1
avoid error bucket NotEmpty if multipart uploads folder exist
2022-06-20remove go.uber.org/atomic石昌林1-2/+2
2022-06-17some code optimizations石昌林1-0/+18
2022-05-30move s3 related constants from package http to s3_constantschrislu1-0/+66
2022-05-30use final destination to resolve fs configurationchrislu1-0/+2
related to https://github.com/chrislusf/seaweedfs/issues/3075
2020-12-22go fmtChris Lu1-1/+0
2020-12-07break import cycleChris Lu1-0/+10
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 04:54:52 +0000