aboutsummaryrefslogtreecommitdiff
path: root/weed/s3api/s3api_server.go
AgeCommit message (Collapse)AuthorFilesLines
2023-05-16Use filerGroup for s3 buckets collection prefix (#4465)SmsS41-0/+1
* Use filerGroup for s3 buckets collection prefix * Fix templates * Remove flags * Remove s3CollectionPrefix
2022-10-10change s3_account.go package to avoid cycle dependency (#3813)LHHDZ1-2/+3
2022-10-01add ownership rest apis (#3765)LHHDZ1-0/+8
2022-09-29s3: sync bucket info from filer (#3759)LHHDZ1-1/+3
2022-09-28s3: add account (#3753)LHHDZ1-0/+2
associate `Account` and `Identity` by accountId
2022-09-01avoid DATA RACE on S3Options.localFilerSocket (#3571)Konstantin Lebedev1-3/+3
* avoid DATA RACE on S3Options.localFilerSocket https://github.com/seaweedfs/seaweedfs/issues/3552 * copy localSocket
2022-08-22fix:Handle preflight cors requests (#3496)famosss1-2/+4
2022-08-22Handle preflight cors requests (#3481)famosss1-0/+7
2022-08-04filer prefer volume server in same data center (#3405)Konstantin Lebedev1-0/+1
* initial prefer same data center https://github.com/seaweedfs/seaweedfs/issues/3404 * GetDataCenter * prefer same data center for ReplicationSource * GetDataCenterId * remove glog
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-7/+7
2022-06-17add some unit tests and some code optimizes石昌林1-40/+41
2022-06-15add s3 circuit breaker support for 'simultaneous request count' and ↵石昌林1-41/+42
'simultaneous request bytes' limitations configure s3 circuit breaker by 'command_s3_circuitbreaker.go': usage eg: # Configure the number of simultaneous global (current s3api node) requests s3.circuit.breaker -global -type count -actions Write -values 1000 -apply # Configure the number of simultaneous requests for bucket x read and write s3.circuit.breaker -buckets -type count -actions Read,Write -values 1000 -apply # Configure the total bytes of simultaneous requests for bucket write s3.circuit.breaker -buckets -type bytes -actions Write -values 100MiB -apply # Disable circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -enable false -apply # Delete circuit breaker config of bucket 'x' s3.circuit.breaker -buckets x -delete -apply
2022-05-15s3: add grpc server to accept configuration changeschrislu1-0/+2
2022-05-04skip unix socket mode for windowschrislu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/3013
2022-03-30allowDeleteBucketNotEmptyKonstantin Lebedev1-8/+9
2022-03-07s3 and filer transport using unix domain socket instead of tcpchrislu1-0/+18
2022-02-04weed/s3api: rearrange s3 methods handlers to ensure correct methods requestingzerospiel1-33/+55
Otherwise current calls for some methods (i.e. GetObjectAcl) ends up with wrong method selection (i.e. GetObject). Added generic comment rule of traversing methods
2022-02-03weed/s3api: added new bucket handlers for more compatibility with AWS S3zerospiel1-19/+25
Protocol Otherwise any requests to the underlying handlers results in calls to ListObjects (v1) that may intensively load gateway and volume servers. Added the following handlers with default responses: - GetBucketLocation - GetBucketRequestPayment Added the following handlers with NotFound and NotImplemented responses: - PutBucketAcl - GetBucketPolicy - PutBucketPolicy - DeleteBucketPolicy - GetBucketCors - PutBucketCors - DeleteBucketCors
2022-01-02Merge branch 'master' into metadata_follow_with_client_idChris Lu1-0/+12
2021-12-30FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 ClientSebastian Kurfuerst1-4/+17
- one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
2021-12-30add client id for all metadata listening clientschrislu1-4/+7
2021-12-07audit log configKonstantin Lebedev1-1/+0
2021-12-07audit logKonstantin Lebedev1-1/+0
2021-11-02s3: adjust permission for HEAD bucket operationChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/2417#issuecomment-958391856
2021-11-02s3: skip permission checking for creating bucket if the bucket already existsChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/2417 Rclone was trying to create the bucket even though the bucket already exists.
2021-10-28fix DeleteBucketLifecycleConfigurationKonstantin Lebedev1-4/+7
2021-10-11AclHandlersKonstantin Lebedev1-4/+18
2021-09-19s3: avoid overwriting object with ACL/LegalHold/Retension/LockConfiguration ↵Chris Lu1-0/+9
requests
2021-09-12change server address from string to a typeChris Lu1-2/+2
2021-08-10merge master, resolve conflictsBl1tz231-1/+2
2021-08-10Add liveness\readiness probe for s3 api handler on /status pathBl1tz231-2/+6
2021-06-10refactorChris Lu1-1/+2
2021-02-18use backticks instead of double quotes to avoid escaped additionally in regexbingoohuang1-1/+1
2020-12-27s3: add option for "alllowEmptyFolder"Chris Lu1-0/+1
2020-12-25s3: support config action Admin:bucketChris Lu1-1/+1
2020-12-07break import cycleChris Lu1-0/+1
2020-12-07s3: subscribe to s3.configure changesChris Lu1-0/+4
2020-11-03load S3 config from filerKonstantin Lebedev1-1/+1
https://github.com/chrislusf/seaweedfs/issues/1500
2020-10-21multiplate DomainNames through commaKonstantin Lebedev1-4/+8
2020-10-08s3: only admin can list all bucketsChris Lu1-1/+1
2020-10-07s3: Added support for "List" action in weed s3 -config=... in the config file.Chris Lu1-8/+8
fix https://github.com/chrislusf/seaweedfs/issues/1511
2020-10-02s3: support object taggingChris Lu1-0/+7
* GetObjectTagging * PutObjectTagging * DeleteObjectTagging
2020-09-21refactorChris Lu1-3/+1
2020-09-20refactoringChris Lu1-20/+20
2020-09-19s3: add support for PostPolicyChris Lu1-0/+3
fix https://github.com/chrislusf/seaweedfs/issues/1426
2020-09-18fix compilationChris Lu1-2/+2
2020-09-18s3: collect metricsChris Lu1-19/+19
2020-08-24s3: list bucket permission change from admin to readChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/1430
2020-07-28s3: use bucket in the domainChris Lu1-1/+6
fix https://github.com/chrislusf/seaweedfs/issues/1405
2020-02-09add v2 supportChris Lu1-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 22:45:08 +0000