aboutsummaryrefslogtreecommitdiff
path: root/weed/command/volume.go
AgeCommit message (Collapse)AuthorFilesLines
10 daysRemove default concurrent upload/download limits for best performance (#7712)Chris Lu1-2/+2
Change all concurrentUploadLimitMB and concurrentDownloadLimitMB defaults from fixed values (64, 128, 256 MB) to 0 (unlimited). This removes artificial throttling that can limit throughput on high-performance systems, especially on all-flash setups with many cores. Files changed: - volume.go: concurrentUploadLimitMB 256->0, concurrentDownloadLimitMB 256->0 - server.go: filer/volume/s3 concurrent limits 64/128->0 - s3.go: concurrentUploadLimitMB 128->0 - filer.go: concurrentUploadLimitMB 128->0, s3.concurrentUploadLimitMB 128->0 Users can still set explicit limits if needed for resource management.
2025-12-05Separate vacuum speed from replication speed (#7632)msementsov1-0/+3
2025-12-02Support separate volume server ID independent of RPC bind address (#7609)Chris Lu1-1/+6
* pb: add id field to Heartbeat message for stable volume server identification This adds an 'id' field to the Heartbeat protobuf message that allows volume servers to identify themselves independently of their IP:port address. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * storage: add Id field to Store struct Add Id field to Store struct and include it in CollectHeartbeat(). The Id field provides a stable volume server identity independent of IP:port. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * topology: support id-based DataNode identification Update GetOrCreateDataNode to accept an id parameter for stable node identification. When id is provided, the DataNode can maintain its identity even when its IP address changes (e.g., in Kubernetes pod reschedules). For backward compatibility: - If id is provided, use it as the node ID - If id is empty, fall back to ip:port Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * volume: add -id flag for stable volume server identity Add -id command line flag to volume server that allows specifying a stable identifier independent of the IP address. This is useful for Kubernetes deployments with hostPath volumes where pods can be rescheduled to different nodes while the persisted data remains on the original node. Usage: weed volume -id=node-1 -ip=10.0.0.1 ... If -id is not specified, it defaults to ip:port for backward compatibility. Fixes https://github.com/seaweedfs/seaweedfs/issues/7487 * server: add -volume.id flag to weed server command Support the -volume.id flag in the all-in-one 'weed server' command, consistent with the standalone 'weed volume' command. Usage: weed server -volume.id=node-1 ... Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * topology: add test for id-based DataNode identification Test the key scenarios: 1. Create DataNode with explicit id 2. Same id with different IP returns same DataNode (K8s reschedule) 3. IP/PublicUrl are updated when node reconnects with new address 4. Different id creates new DataNode 5. Empty id falls back to ip:port (backward compatibility) Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * pb: add address field to DataNodeInfo for proper node addressing Previously, DataNodeInfo.Id was used as the node address, which worked when Id was always ip:port. Now that Id can be an explicit string, we need a separate Address field for connection purposes. Changes: - Add 'address' field to DataNodeInfo protobuf message - Update ToDataNodeInfo() to populate the address field - Update NewServerAddressFromDataNode() to use Address (with Id fallback) - Fix LookupEcVolume to use dn.Url() instead of dn.Id() Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: trim whitespace from volume server id and fix test - Trim whitespace from -id flag to treat ' ' as empty - Fix store_load_balancing_test.go to include id parameter in NewStore call Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * refactor: extract GetVolumeServerId to util package Move the volume server ID determination logic to a shared utility function to avoid code duplication between volume.go and rack.go. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: improve transition logic for legacy nodes - Use exact ip:port match instead of net.SplitHostPort heuristic - Update GrpcPort and PublicUrl during transition for consistency - Remove unused net import Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: add id normalization and address change logging - Normalize id parameter at function boundary (trim whitespace) - Log when DataNode IP:Port changes (helps debug K8s pod rescheduling) Ref: https://github.com/seaweedfs/seaweedfs/issues/7487
2025-11-10fix help messageschrislu1-1/+1
2025-11-04Nit: use `time.Duration`s instead of constants in seconds. (#7438)Lisandro Pin1-1/+1
Nit: use `time.Durations` instead of constants in seconds. Makes for slightly more readable code.
2025-10-31Adjust cli option (#7418)Chris Lu1-2/+9
* adjust "weed benchmark" CLI to use readOnly/writeOnly * consistently use "-master" CLI option * If both -readOnly and -writeOnly are specified, the current logic silently allows it with -writeOnly taking precedence. This is confusing and could lead to unexpected behavior.
2025-07-07* Fix undefined http serve behaiver (#6943)zuzuviewer1-11/+8
2025-07-02[volume] refactor and add metrics for flight upload and download data limit ↵Konstantin Lebedev1-4/+7
condition (#6920) * refactor concurrentDownloadLimit * fix loop * fix cmdServer * fix: resolve conversation pr 6920 * Changes logging function (#6919) * updated logging methods for stores * updated logging methods for stores * updated logging methods for filer * updated logging methods for uploader and http_util * updated logging methods for weed server --------- Co-authored-by: akosov <a.kosov@kryptonite.ru> * Improve lock ring (#6921) * fix flaky lock ring test * add more tests * fix: build * fix: rm import util/version * fix: serverOptions * refactoring --------- Co-authored-by: Aleksey Kosov <rusyak777@list.ru> Co-authored-by: akosov <a.kosov@kryptonite.ru> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com>
2025-06-03change version directorychrislu1-2/+3
2024-12-02[security] reload whiteList on http seerver (#6302)Konstantin Lebedev1-0/+1
* reload whiteList * white_list add to scaffold
2024-07-16Added loadSecurityConfigOnce (#5792)vadimartynov1-1/+1
2024-07-12-metricsIp cmd flag (#5773)vadimartynov1-1/+11
* Added/Updated: - Added metrics ip options for all servers; - Fixed a bug with the selection of the binIp or ip parameter for the metrics handler; * Fixed cmd flags
2023-10-09 fix: skipping checking active volumes with the same number of files at the ↵Konstantin Lebedev1-1/+2
moment (#4893) * fix: skipping checking active volumes with the same number of files at the moment https://github.com/seaweedfs/seaweedfs/issues/4140 * refactor with comments https://github.com/seaweedfs/seaweedfs/issues/4140 * add TestShouldSkipVolume --------- Co-authored-by: Konstantin Lebedev <9497591+kmlebedev@users.noreply.github.co>
2022-11-24[metrics] Add the ability to control bind ip (#4012)Jiffs Maverick1-1/+1
2022-11-14Lazy loading (#3958)Guo Lei1-0/+3
* types packages is imported more than onece * lazy-loading * fix bugs * fix bugs * fix unit tests * fix test error * rename function * unload ldb after initial startup * Don't load ldb when starting volume server if ldbtimeout is set. * remove uncessary unloadldb * Update weed/command/server.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * Update weed/command/volume.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: guol-fnst <goul-fnst@fujitsu.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-10-12volume server:set the default value of "hasSlowRead" to true (#3710)famosss1-1/+1
* simplify a bit * feat: volume: add "readBufSize" option to customize read optimization * refactor : redbufSIze -> readBufferSize * simplify a bit * simplify a bit * volume server:set the default value of "hasSlowRead" to true
2022-09-28[volume] Add new volumes to HUP(reload) signal (#3755)Konstantin Lebedev1-0/+2
Add new volumes to HUP(reload) signal
2022-09-17volume server: rename readBufferSize to readBufferSizeMBchrislu1-3/+3
2022-09-16volume: add "readBufSize" option to customize read optimization (#3702)famosss1-0/+3
* simplify a bit * feat: volume: add "readBufSize" option to customize read optimization * refactor : redbufSIze -> readBufferSize * simplify a bit * simplify a bit
2022-09-15mark "hasSlowRead" as experimentalchrislu1-1/+1
2022-09-15volume: add "hasSlowRead" option to customize read optimizationchrislu1-0/+3
2022-09-08purge tcp implementationchrislu1-26/+0
2022-08-26make CodeQL happychrislu1-3/+3
2022-08-07refactor and fix strings.Splitchrislu1-3/+1
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-11/+11
2022-05-21add inflight upload data wait timeoutliubaojiang1-1/+4
2022-05-17just exit in case of duplicated volume directories were loadedguol-fnst1-1/+1
2022-03-16Add mTLS support for both master and volume http server.Berck Nash1-3/+11
2022-03-11default bind to one ip addresschrislu1-1/+4
fix https://github.com/chrislusf/seaweedfs/issues/1937
2022-02-27minorchrislu1-1/+1
2022-02-14StopTimeout 30 secKonstantin Lebedev1-1/+1
2022-02-14reduce to default http server KillTimeout and StopTimeoutKonstantin Lebedev1-2/+2
2022-01-13add more help messageschrislu1-1/+1
2021-09-20use default 10000 for grpc portChris Lu1-1/+4
2021-09-12change server address from string to a typeChris Lu1-7/+7
2021-09-12custom grpc port: volume serverChris Lu1-1/+3
2021-09-07better IP v6 supportChris Lu1-5/+5
2021-08-08volume: support concurrent download data size limitChris Lu1-26/+29
2021-07-13volume: change all writes to fsync during graceful stoppingChris Lu1-0/+1
fix https://github.com/chrislusf/seaweedfs/issues/2193
2021-07-03volume: default readMode to proxyChris Lu1-1/+1
2021-07-01go fmtChris Lu1-1/+1
2021-06-30add proxy mode to read non-local volumeszhangsong1-1/+1
2021-06-30add proxy mode to read non-local volumeszhangsong1-3/+3
2021-04-27minFreeSpace refactoredbingoohuang1-18/+8
2021-04-26minFreeSpace argument allows size like 10GiBbingoohuang1-14/+16
2021-04-02go fmtChris Lu1-1/+1
2021-03-30filer, volume: add concurrent upload size limit to avoid OOMChris Lu1-25/+28
add some back pressure when writes are slow
2021-03-24GOMAXPROCS use defaults to the value of runtime.NumCPUKonstantin Lebedev1-3/+0
https://github.com/chrislusf/seaweedfs/issues/1942
2021-03-23more help messageChris Lu1-1/+1
fix https://github.com/chrislusf/seaweedfs/issues/1937
2021-03-12default to empty host to bind to tcp4 and tcp6Chris Lu1-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 22:34:58 +0000