aboutsummaryrefslogtreecommitdiff
path: root/weed/command/scaffold
AgeCommit message (Collapse)AuthorFilesLines
2022-05-15s3: add grpc server to accept configuration changeschrislu1-0/+5
2022-05-03add options to scaffoldKonstantin Lebedev1-5/+7
2022-05-03Merge branch 'new_master' into ydbKonstantin Lebedev1-0/+1
# Conflicts: # go.mod # go.sum
2022-05-02ydb-go-sdk move to v3Konstantin Lebedev1-0/+11
2022-04-30s3 backend support customizing storage classchrislu1-0/+1
2022-04-01change user and pass to username and passworda1-2/+2
2022-03-17prefix search, bucket implementedelee1-3/+7
2022-03-17arangodb adapterelee1-0/+6
2022-03-16Add mTLS support for both master and volume http server.Berck Nash1-0/+6
2022-02-15FilerStore: add redis_luabanjiaojuhao1-0/+34
2022-01-18add gocql timeout settingguol-fnst1-0/+2
2022-01-12master: remove hard coded filer settings in master.tomlchrislu1-3/+0
fix https://github.com/chrislusf/seaweedfs/issues/2529
2021-12-31BUGFIX: security.toml contained wrong keysSebastian Kurfuerst1-2/+2
2021-12-30FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 ClientSebastian Kurfuerst1-4/+26
- one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
2021-12-24BUGFIX: add access.ui setting to scaffolded security.tomlSebastian Kurfürst1-0/+7
... The property is read here: https://github.com/chrislusf/seaweedfs/blob/b70cb3e0b2a75543f0410d4c09f42aa95fcf2ee2/weed/server/volume_server.go#L69
2021-11-29redis3 supports sentinelChris Lu1-0/+8
2021-11-29add s3.clean.uploads -timeAgo=24hChris Lu1-0/+1
2021-11-25fix redis2 sentinel config examplelimd1-1/+1
2021-11-25Merge remote-tracking branch 'origin/master'limd1-3/+3
2021-11-25support redis sentinellimd1-0/+8
2021-11-11filer: fix mysql2 SQL templateChris Lu1-1/+1
2021-11-11filer: fix mysql command to upsertChris Lu1-2/+2
2021-10-10removing tikv to resolve "go mod tidy" problemChris Lu1-8/+0
tikv is causing "go mod tidy" problem. Need to resolve this before adding tikv back. go mod tidy go: finding module for package github.com/coreos/etcd/clientv3/balancer/picker go: finding module for package cloud.google.com/go/kms/apiv1 go: finding module for package github.com/coreos/etcd/clientv3/balancer/resolver/endpoint go: finding module for package google.golang.org/grpc/naming go: finding module for package github.com/coreos/etcd/clientv3/credentials go: finding module for package github.com/coreos/etcd/clientv3/balancer go: finding module for package github.com/d4l3k/messagediff go: found github.com/coreos/etcd/clientv3/balancer in github.com/coreos/etcd v3.3.26+incompatible go: found github.com/coreos/etcd/clientv3/balancer/picker in github.com/coreos/etcd v3.3.26+incompatible go: found github.com/coreos/etcd/clientv3/balancer/resolver/endpoint in github.com/coreos/etcd v3.3.26+incompatible go: found github.com/coreos/etcd/clientv3/credentials in github.com/coreos/etcd v3.3.26+incompatible go: found cloud.google.com/go/kms/apiv1 in cloud.google.com/go/kms v1.0.0 go: found github.com/d4l3k/messagediff in github.com/d4l3k/messagediff v1.2.1 go: finding module for package google.golang.org/grpc/naming github.com/chrislusf/seaweedfs/weed/filer/tikv imports github.com/tikv/client-go/v2/tikv imports go.etcd.io/etcd/clientv3 tested by go.etcd.io/etcd/clientv3.test imports github.com/coreos/etcd/integration imports github.com/coreos/etcd/proxy/grpcproxy imports google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
2021-10-10removing etcd sequencerChris Lu1-4/+1
causing go mod tidy problem. If anyone wants this, please help to resolve this first. github.com/chrislusf/seaweedfs/weed/sequence imports go.etcd.io/etcd/client tested by go.etcd.io/etcd/client.test imports github.com/coreos/etcd/integration imports github.com/coreos/etcd/proxy/grpcproxy imports google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.41.0), but does not contain package google.golang.org/grpc/naming
2021-10-04add redis3Chris Lu1-0/+22
2021-09-08mysql table DDL: minor, not necessary, but a little more strictChris Lu1-2/+2
the directory is already hashed into dirhash. So this change is not strictly required.
2021-08-26Add deleterange_concurrency to filer configuration fileyulai.li1-0/+2
2021-08-26Update filer.toml scaffold to support tikv storeyulai.li1-0/+6
2021-08-01add -force option following #2228Chris Lu1-1/+1
2021-08-01add default quietFor valueChris Lu1-1/+1
2021-08-01shell: add `volume.deleteEmpty` commandChris Lu1-0/+1
2021-08-01remove buckets folder optionChris Lu1-2/+0
the related code still works for old deployments
2021-07-23snowflake sequencer need an unique idChris Lu1-0/+2
fix https://github.com/chrislusf/seaweedfs/issues/2213
2021-07-05extract embed toml example to separate filesbingoohuang7-0/+510
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-23 16:30:29 +0000