aboutsummaryrefslogtreecommitdiff
path: root/weed/s3api/s3api_server.go
AgeCommit message (Collapse)AuthorFilesLines
2025-05-22WIPorigin/changing-to-zapchrislu1-3/+3
2024-10-04[s3] add {Get,Put,Delete}BucketTagging and PublicAccessBlock Handlers (#6088)Konstantin Lebedev1-0/+10
* add {Get,Put,Delete}BucketTagging Handlers * s3 add skip bucket PublicAccessBlock handlers --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-10-03[s3] add skip bucket encryption handlers (#6091)Konstantin Lebedev1-0/+5
s3 add skip bucket encryption handlers
2024-09-26fix: Prevent potential metadata change events from being lost. (#6066)steve.wei1-1/+3
2024-08-21also use `/healthz` for most consistent health checkchrislu1-1/+2
2024-07-16Added tls for http clients (#5766)vadimartynov1-5/+6
* Added global http client * Added Do func for global http client * Changed the code to use the global http client * Fix http client in volume uploader * Fixed pkg name * Fixed http util funcs * Fixed http client for bench_filer_upload * Fixed http client for stress_filer_upload * Fixed http client for filer_server_handlers_proxy * Fixed http client for command_fs_merge_volumes * Fixed http client for command_fs_merge_volumes and command_volume_fsck * Fixed http client for s3api_server * Added init global client for main funcs * Rename global_client to client * Changed: - fixed NewHttpClient; - added CheckIsHttpsClientEnabled func - updated security.toml in scaffold * Reduce the visibility of some functions in the util/http/client pkg * Added the loadSecurityConfig function * Use util.LoadSecurityConfiguration() in NewHttpClient func
2024-07-01refactor all methods strings to const (#5726)Konstantin Lebedev1-48/+48
2024-05-17added s3 iam DeleteBucket permission management (#5599)Riccardo Bertossa1-1/+1
2024-02-19refactor: put the auth outside (#5313)7y-91-1/+2
2024-02-19fix: only admin auth can delete S3 bucket (#5312)7y-91-1/+1
2023-12-20Set allowed origins in config (#5109)jerebear121-5/+33
* Add a way to use a JWT in an HTTP only cookie If a JWT is not included in the Authorization header or a query string, attempt to get a JWT from an HTTP only cookie. * Added a way to specify allowed origins header from config * Removed unecessary log * Check list of domains from config or command flag * Handle default wildcard and change name of config value to cors
2023-11-13s3 api add not implemented response for PutBucketVersioningKonstantin Lebedev1-0/+1
2023-11-13s3 api add default response for GetBucketVersioningKonstantin Lebedev1-0/+3
2023-10-19fixchrislu1-1/+1
2023-10-18[s3] do reload s3 static config (#4923)Konstantin Lebedev1-0/+11
* do reload s3 config * print error on reload s3 config * print success msg * Update weed/s3api/s3api_server.go --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-25[s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration ↵Konstantin Lebedev1-3/+0
(#4859) * move s3account.AccountManager into to iam.S3ApiConfiguration and switch to Interface https://github.com/seaweedfs/seaweedfs/issues/4519 * fix: test bucket acl default and adjust the variable names * fix: s3 api config test --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2023-09-21[iam] Replace action read/write to readAcp/writeAcp for handlers with acl ↵Konstantin Lebedev1-4/+4
(#4858) Replace action read/write to readAcp/writeAcp for handlers with acl query https://github.com/seaweedfs/seaweedfs/issues/4519 Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2023-05-16Use filerGroup for s3 buckets collection prefix (#4465)SmsS41-0/+1
* Use filerGroup for s3 buckets collection prefix * Fix templates * Remove flags * Remove s3CollectionPrefix
2022-10-10change s3_account.go package to avoid cycle dependency (#3813)LHHDZ1-2/+3
2022-10-01add ownership rest apis (#3765)LHHDZ1-0/+8
2022-09-29s3: sync bucket info from filer (#3759)LHHDZ1-1/+3
2022-09-28s3: add account (#3753)LHHDZ1-0/+2
associate `Account` and `Identity` by accountId
2022-09-01avoid DATA RACE on S3Options.localFilerSocket (#3571)Konstantin Lebedev1-3/+3
* avoid DATA RACE on S3Options.localFilerSocket https://github.com/seaweedfs/seaweedfs/issues/3552 * copy localSocket
2022-08-22fix:Handle preflight cors requests (#3496)famosss1-2/+4
2022-08-22Handle preflight cors requests (#3481)famosss1-0/+7
2022-08-04filer prefer volume server in same data center (#3405)Konstantin Lebedev1-0/+1
* initial prefer same data center https://github.com/seaweedfs/seaweedfs/issues/3404 * GetDataCenter * prefer same data center for ReplicationSource * GetDataCenterId * remove glog
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-7/+7
2022-06-17add some unit tests and some code optimizes石昌林1-40/+41
2022-06-15add s3 circuit breaker support for 'simultaneous request count' and ↵石昌林1-41/+42
'simultaneous request bytes' limitations configure s3 circuit breaker by 'command_s3_circuitbreaker.go': usage eg: # Configure the number of simultaneous global (current s3api node) requests s3.circuit.breaker -global -type count -actions Write -values 1000 -apply # Configure the number of simultaneous requests for bucket x read and write s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply # Configure the total bytes of simultaneous requests for bucket write s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply # Disable circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -enable false -apply # Delete circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -delete -apply
2022-05-15s3: add grpc server to accept configuration changeschrislu1-0/+2
2022-05-04skip unix socket mode for windowschrislu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/3013
2022-03-30allowDeleteBucketNotEmptyKonstantin Lebedev1-8/+9
2022-03-07s3 and filer transport using unix domain socket instead of tcpchrislu1-0/+18
2022-02-04weed/s3api: rearrange s3 methods handlers to ensure correct methods requestingzerospiel1-33/+55
Otherwise current calls for some methods (i.e. GetObjectAcl) ends up with wrong method selection (i.e. GetObject). Added generic comment rule of traversing methods
2022-02-03weed/s3api: added new bucket handlers for more compatibility with AWS S3zerospiel1-19/+25
Protocol Otherwise any requests to the underlying handlers results in calls to ListObjects (v1) that may intensively load gateway and volume servers. Added the following handlers with default responses: - GetBucketLocation - GetBucketRequestPayment Added the following handlers with NotFound and NotImplemented responses: - PutBucketAcl - GetBucketPolicy - PutBucketPolicy - DeleteBucketPolicy - GetBucketCors - PutBucketCors - DeleteBucketCors
2022-01-02Merge branch 'master' into metadata_follow_with_client_idChris Lu1-0/+12
2021-12-30FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 ClientSebastian Kurfuerst1-4/+17
- one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
2021-12-30add client id for all metadata listening clientschrislu1-4/+7
2021-12-07audit log configKonstantin Lebedev1-1/+0
2021-12-07audit logKonstantin Lebedev1-1/+0
2021-11-02s3: adjust permission for HEAD bucket operationChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/2417#issuecomment-958391856
2021-11-02s3: skip permission checking for creating bucket if the bucket already existsChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/2417 Rclone was trying to create the bucket even though the bucket already exists.
2021-10-28fix DeleteBucketLifecycleConfigurationKonstantin Lebedev1-4/+7
2021-10-11AclHandlersKonstantin Lebedev1-4/+18
2021-09-19s3: avoid overwriting object with ACL/LegalHold/Retension/LockConfiguration ↵Chris Lu1-0/+9
requests
2021-09-12change server address from string to a typeChris Lu1-2/+2
2021-08-10merge master, resolve conflictsBl1tz231-1/+2
2021-08-10Add liveness\readiness probe for s3 api handler on /status pathBl1tz231-2/+6
2021-06-10refactorChris Lu1-1/+2
2021-02-18use backticks instead of double quotes to avoid escaped additionally in regexbingoohuang1-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-21 09:26:46 +0000