aboutsummaryrefslogtreecommitdiff
path: root/weed/server/filer_server.go
AgeCommit message (Collapse)AuthorFilesLines
2022-08-10ensure memory is alignedchrislu1-3/+3
fix https://github.com/seaweedfs/seaweedfs/issues/3427
2022-08-05Add download speed limit support (#3408)LHHDZ1-0/+1
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-37/+37
2022-07-28remove old codechrislu1-4/+0
2022-07-23metadata subscription uses client epochchrislu1-2/+2
2022-06-15Add filer command line parameter to let Filer UI show/hide directory delete ↵yulai.li1-1/+2
button
2022-06-06filer: remove replication, collection, disk_type info from entry metadatachrislu1-2/+0
these metadata can change and are not used
2022-05-31fix wrong logicchrislu1-1/+1
2022-05-30fresh filer store bootstrap from the oldest peerchrislu1-3/+9
2022-05-30refactorchrislu1-1/+3
2022-05-03Merge branch 'new_master' into ydbKonstantin Lebedev1-1/+2
# Conflicts: # go.mod # go.sum
2022-05-01filer: add filer groupchrislu1-1/+2
2022-05-02ydb-go-sdk move to v3Konstantin Lebedev1-0/+1
2022-03-28diffa1-1/+1
2022-03-26refactor: change masters from a slice to a mapchrislu1-1/+1
2022-03-17arangodb adapterelee1-0/+1
2022-03-06Set default leveldb2 enabledKonstantin Lebedev1-2/+2
avoid Filer store is enabled for both leveldb2 and mysql
2022-01-02Merge branch 'master' into metadata_follow_with_client_idChris Lu1-1/+12
2022-01-01Merge pull request #2543 from skurfuerst/seaweedfs-158Chris Lu1-1/+12
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
2021-12-30FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 ClientSebastian Kurfuerst1-1/+12
- one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
2021-12-30add client id for all metadata listening clientschrislu1-0/+5
2021-12-26use streaming mode for long poll grpc callschrislu1-1/+1
streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed.
2021-12-22filer server: allow upload file to specific dataNodebanjiaojuhao1-0/+1
2021-12-05upgrade protoc to 3.17.3chrislu1-0/+2
$brew install protobuf $ protoc --version libprotoc 3.17.3 $ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26 $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
2021-11-06filer: deprecate "-peers" optionChris Lu1-1/+0
2021-11-06auto updated filer peer listChris Lu1-2/+2
2021-11-05Revert "master: rename grpc function KeepConnected() to ↵Chris Lu1-1/+1
SubscribeVolumeLocationUpdates()" This reverts commit af71ae11aa29350a60ed7d3b9a16276a06ba9dcc.
2021-11-03master: rename grpc function KeepConnected() to SubscribeVolumeLocationUpdates()Chris Lu1-1/+1
2021-10-04add redis3Chris Lu1-0/+1
2021-09-12change server address from string to a typeChris Lu1-14/+6
2021-09-07handle ipv6 addressesChris Lu1-1/+1
2021-07-28add ReadRemote(), add read remote setup when filer startsChris Lu1-0/+2
2021-07-01go fmtChris Lu1-1/+1
2021-05-23filer: Support sqlite as filer meta storeChris Lu1-0/+1
2021-05-22s3: deprecating filer.options.buckets_fsyncChris Lu1-1/+1
2021-04-01dedicated upload processorChris Lu1-1/+1
avoid thundering effect of overloading volume servers
2021-03-30filer, volume: add concurrent upload size limit to avoid OOMChris Lu1-19/+24
add some back pressure when writes are slow
2021-03-02support IPv6Chris Lu1-1/+1
2021-01-19filer store: add postgres2Chris Lu1-0/+1
2021-01-19filer store: add mysql2Chris Lu1-0/+1
2021-01-12go fmtChris Lu1-12/+12
2021-01-12filer: add leveldb3Chris Lu1-0/+1
support dedicated leveldb instance for each bucket
2021-01-10filer: change to saveToFilerLimit from cacheToFilerLimitChris Lu1-12/+12
short circuit saving small files to volume server
2020-12-23hook upChris Lu1-0/+1
2020-12-22filer: add -defaultStoreDir so that filer.toml can be skippedChris Lu1-0/+2
fix https://github.com/chrislusf/seaweedfs/issues/1659
2020-12-04filer: add option to cache small files to filer storeChris Lu1-0/+1
2020-11-27filer: readonly handle static resourcesChris Lu1-0/+1
fix https://github.com/chrislusf/seaweedfs/issues/1641
2020-11-16add commentChris Lu1-0/+1
2020-11-16deprecating "filer.options.buckets_fsync" in filer.tomlChris Lu1-0/+1
2020-11-15filer: load filer conf when startingChris Lu1-0/+2
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-26 18:34:16 +0000