aboutsummaryrefslogtreecommitdiff
path: root/weed/command
AgeCommit message (Collapse)AuthorFilesLines
2022-02-10initial setupchrislu3-0/+137
2022-02-07filer read empty file may cause OOM in some caseschrislu1-2/+1
fix https://github.com/chrislusf/seaweedfs/issues/2641
2022-02-04change to util.WriteFilechrislu1-2/+2
2022-01-30edge case: old entry was not replicated to remote storagechrislu1-1/+1
2022-01-20Merge pull request #2602 from kmlebedev/master_metricsChris Lu1-0/+4
master metricsHttpPort
2022-01-19url should be always using forward slashchrislu1-1/+1
2022-01-19master metricsHttpPortKonstantin Lebedev1-0/+4
2022-01-18add gocql timeout settingguol-fnst1-0/+2
2022-01-17workingchrislu2-0/+9
2022-01-15add filer to iam optionchrislu1-0/+1
2022-01-13weed server: optionally start IAM servicechrislu1-4/+13
related to https://github.com/chrislusf/seaweedfs/issues/2560
2022-01-13add more help messageschrislu2-2/+2
2022-01-12master: remove hard coded filer settings in master.tomlchrislu1-3/+0
fix https://github.com/chrislusf/seaweedfs/issues/2529
2022-01-11filer.copy: don't crash when volume creation failsKyle Sanderson1-1/+2
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1d58247] goroutine 7482 [running]: github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks.func1(0x2) /go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:488 +0x2a7 created by github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks /go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:455 +0x225 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x1d58247] goroutine 7480 [running]: github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks.func1(0x0) /go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:488 +0x2a7 created by github.com/chrislusf/seaweedfs/weed/command.(*FileCopyWorker).uploadFileInChunks /go/src/github.com/chrislusf/seaweedfs/weed/command/filer_copy.go:455 +0x225
2022-01-11Merge branch 'master' of https://github.com/chrislusf/seaweedfschrislu1-48/+51
2022-01-11Update mount_std.gochrislu1-0/+1
2022-01-11Merge pull request #2575 from Radtoo/fix_paths2Chris Lu1-48/+51
Fix paths2
2022-01-10fix loggingchrislu1-1/+4
2022-01-08Using positional arguments rather than option flag to enable better shell usageRadtoo1-43/+43
2022-01-08Now works with a single file tooRadtoo1-28/+31
Parsing removed from doFixOneVolume Needle init removed from runFix
2022-01-07support fixing a collection of volumes, or volumes under one directorychrislu1-9/+47
2022-01-02Merge branch 'master' into metadata_follow_with_client_idChris Lu1-4/+26
2022-01-01Merge pull request #2543 from skurfuerst/seaweedfs-158Chris Lu1-4/+26
FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 Client
2021-12-31BUGFIX: security.toml contained wrong keysSebastian Kurfuerst1-2/+2
2021-12-30FEATURE: add JWT to HTTP endpoints of Filer and use them in S3 ClientSebastian Kurfuerst1-4/+26
- one JWT for reading and one for writing, analogous to how the JWT between Master and Volume Server works - I did not implement IP `whiteList` parameter on the filer Additionally, because http_util.DownloadFile now sets the JWT, the `download` command should now work when `jwt.signing.read` is configured. By looking at the code, I think this case did not work before. ## Docs to be adjusted after a release Page `Amazon-S3-API`: ``` # Authentication with Filer You can use mTLS for the gRPC connection between S3-API-Proxy and the filer, as explained in [Security-Configuration](Security-Configuration) - controlled by the `grpc.*` configuration in `security.toml`. Starting with version XX, it is also possible to authenticate the HTTP operations between the S3-API-Proxy and the Filer (especially uploading new files). This is configured by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. With both configurations (gRPC and JWT), it is possible to have Filer and S3 communicate in fully authenticated fashion; so Filer will reject any unauthenticated communication. ``` Page `Security Overview`: ``` The following items are not covered, yet: - master server http REST services Starting with version XX, the Filer HTTP REST services can be secured with a JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. ... Before version XX: "weed filer -disableHttp", disable http operations, only gRPC operations are allowed. This works with "weed mount" by FUSE. It does **not work** with the [S3 Gateway](Amazon S3 API), as this does HTTP calls to the Filer. Starting with version XX: secured by JWT, by setting `filer_jwt.signing.key` and `filer_jwt.signing.read.key` in `security.toml`. **This now works with the [S3 Gateway](Amazon S3 API).** ... # Securing Filer HTTP with JWT To enable JWT-based access control for the Filer, 1. generate `security.toml` file by `weed scaffold -config=security` 2. set `filer_jwt.signing.key` to a secret string - and optionally filer_jwt.signing.read.key` as well to a secret string 3. copy the same `security.toml` file to the filers and all S3 proxies. If `filer_jwt.signing.key` is configured: When sending upload/update/delete HTTP operations to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.key`. If `filer_jwt.signing.read.key` is configured: When sending GET or HEAD requests to a filer server, the request header `Authorization` should be the JWT string (`Authorization: Bearer [JwtToken]`). The operation is authorized after the filer validates the JWT with `filer_jwt.signing.read.key`. The S3 API Gateway reads the above JWT keys and sends authenticated HTTP requests to the filer. ``` Page `Security Configuration`: ``` (update scaffold file) ... [filer_jwt.signing] key = "blahblahblahblah" [filer_jwt.signing.read] key = "blahblahblahblah" ``` Resolves: #158
2021-12-30add client id for all metadata listening clientschrislu8-13/+24
2021-12-29dynamically adjust connection timeoutchrislu1-1/+1
better fix for https://github.com/chrislusf/seaweedfs/issues/2541
2021-12-29s3: increase timeout limitchrislu1-1/+1
https://github.com/chrislusf/seaweedfs/issues/2541
2021-12-28mount: remove limits on number of parallel requestschrislu1-3/+3
2021-12-28fuse: change to direct io modechrislu1-1/+1
before and after: chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 8192+0 records in 8192+0 records out 1073741824 bytes transferred in 4.534068 secs (236816430 bytes/sec) dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 0.01s user 3.86s system 84% cpu 4.561 total chrislu$ time dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 8192+0 records in 8192+0 records out 1073741824 bytes transferred in 3.824072 secs (280784948 bytes/sec) dd if=/dev/random of=/Users/chrislu/tmp/mm/testfile bs=131072 count=8192 0.01s user 3.22s system 83% cpu 3.857 total
2021-12-26use streaming mode for long poll grpc callschrislu14-25/+25
streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed.
2021-12-24BUGFIX: add access.ui setting to scaffolded security.tomlSebastian Kurfürst1-0/+7
... The property is read here: https://github.com/chrislusf/seaweedfs/blob/b70cb3e0b2a75543f0410d4c09f42aa95fcf2ee2/weed/server/volume_server.go#L69
2021-12-17s3: bind to a specific IPchrislu3-1/+5
fix https://github.com/chrislusf/seaweedfs/issues/2516
2021-12-14gateway to remote object store: skip replicating multipart upload part fileschrislu1-0/+3
fix https://github.com/chrislusf/seaweedfs/issues/2509
2021-12-13add more help messagechrislu1-1/+6
2021-12-10audit log SignatureVersionkmlebedev1-1/+3
2021-12-09non blocking audit logKonstantin Lebedev1-0/+1
2021-12-07Merge pull request #2498 from kmlebedev/s3_audit_logChris Lu3-0/+9
2021-12-07audit log configKonstantin Lebedev3-1/+9
2021-12-07audit logKonstantin Lebedev1-0/+1
2021-12-05avoid creating the same bucket with a different randomized namechrislu1-15/+19
related to https://github.com/chrislusf/seaweedfs/issues/2492
2021-12-05adjust randomized bucket namechrislu1-1/+1
2021-11-29redis3 supports sentinelChris Lu1-0/+8
2021-11-29add s3.clean.uploads -timeAgo=24hChris Lu1-0/+1
2021-11-28adjust error logChris Lu1-1/+3
2021-11-27add logs for writing to remote fileChris Lu1-3/+5
2021-11-25fix redis2 sentinel config examplelimd1-1/+1
2021-11-25Merge remote-tracking branch 'origin/master'limd3-16/+22
2021-11-25support redis sentinellimd1-0/+8
2021-11-22add retries when writing to remote s3Chris Lu2-13/+19
fix https://github.com/chrislusf/seaweedfs/issues/2465
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 07:15:52 +0000