aboutsummaryrefslogtreecommitdiff
path: root/weed/server/volume_server.go
AgeCommit message (Collapse)AuthorFilesLines
2025-12-05Separate vacuum speed from replication speed (#7632)msementsov1-10/+13
2025-12-02Support separate volume server ID independent of RPC bind address (#7609)Chris Lu1-2/+2
* pb: add id field to Heartbeat message for stable volume server identification This adds an 'id' field to the Heartbeat protobuf message that allows volume servers to identify themselves independently of their IP:port address. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * storage: add Id field to Store struct Add Id field to Store struct and include it in CollectHeartbeat(). The Id field provides a stable volume server identity independent of IP:port. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * topology: support id-based DataNode identification Update GetOrCreateDataNode to accept an id parameter for stable node identification. When id is provided, the DataNode can maintain its identity even when its IP address changes (e.g., in Kubernetes pod reschedules). For backward compatibility: - If id is provided, use it as the node ID - If id is empty, fall back to ip:port Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * volume: add -id flag for stable volume server identity Add -id command line flag to volume server that allows specifying a stable identifier independent of the IP address. This is useful for Kubernetes deployments with hostPath volumes where pods can be rescheduled to different nodes while the persisted data remains on the original node. Usage: weed volume -id=node-1 -ip=10.0.0.1 ... If -id is not specified, it defaults to ip:port for backward compatibility. Fixes https://github.com/seaweedfs/seaweedfs/issues/7487 * server: add -volume.id flag to weed server command Support the -volume.id flag in the all-in-one 'weed server' command, consistent with the standalone 'weed volume' command. Usage: weed server -volume.id=node-1 ... Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * topology: add test for id-based DataNode identification Test the key scenarios: 1. Create DataNode with explicit id 2. Same id with different IP returns same DataNode (K8s reschedule) 3. IP/PublicUrl are updated when node reconnects with new address 4. Different id creates new DataNode 5. Empty id falls back to ip:port (backward compatibility) Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * pb: add address field to DataNodeInfo for proper node addressing Previously, DataNodeInfo.Id was used as the node address, which worked when Id was always ip:port. Now that Id can be an explicit string, we need a separate Address field for connection purposes. Changes: - Add 'address' field to DataNodeInfo protobuf message - Update ToDataNodeInfo() to populate the address field - Update NewServerAddressFromDataNode() to use Address (with Id fallback) - Fix LookupEcVolume to use dn.Url() instead of dn.Id() Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: trim whitespace from volume server id and fix test - Trim whitespace from -id flag to treat ' ' as empty - Fix store_load_balancing_test.go to include id parameter in NewStore call Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * refactor: extract GetVolumeServerId to util package Move the volume server ID determination logic to a shared utility function to avoid code duplication between volume.go and rack.go. Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: improve transition logic for legacy nodes - Use exact ip:port match instead of net.SplitHostPort heuristic - Update GrpcPort and PublicUrl during transition for consistency - Remove unused net import Ref: https://github.com/seaweedfs/seaweedfs/issues/7487 * fix: add id normalization and address change logging - Normalize id parameter at function boundary (trim whitespace) - Log when DataNode IP:Port changes (helps debug K8s pod rescheduling) Ref: https://github.com/seaweedfs/seaweedfs/issues/7487
2025-11-04Nit: use `time.Duration`s instead of constants in seconds. (#7438)Lisandro Pin1-3/+3
Nit: use `time.Durations` instead of constants in seconds. Makes for slightly more readable code.
2025-09-08fix: pass inflightDownloadDataTimeout to volumeServer (#7206)Konstantin Lebedev1-0/+1
2025-07-02[volume] refactor and add metrics for flight upload and download data limit ↵Konstantin Lebedev1-0/+5
condition (#6920) * refactor concurrentDownloadLimit * fix loop * fix cmdServer * fix: resolve conversation pr 6920 * Changes logging function (#6919) * updated logging methods for stores * updated logging methods for stores * updated logging methods for filer * updated logging methods for uploader and http_util * updated logging methods for weed server --------- Co-authored-by: akosov <a.kosov@kryptonite.ru> * Improve lock ring (#6921) * fix flaky lock ring test * add more tests * fix: build * fix: rm import util/version * fix: serverOptions * refactoring --------- Co-authored-by: Aleksey Kosov <rusyak777@list.ru> Co-authored-by: akosov <a.kosov@kryptonite.ru> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com>
2025-05-21Added middleware for processing request_id grpc and http requests (#6805)Aleksey Kosov1-5/+5
2024-12-02[security] reload whiteList on http seerver (#6302)Konstantin Lebedev1-0/+12
* reload whiteList * white_list add to scaffold
2022-11-14Lazy loading (#3958)Guo Lei1-1/+4
* types packages is imported more than onece * lazy-loading * fix bugs * fix bugs * fix unit tests * fix test error * rename function * unload ldb after initial startup * Don't load ldb when starting volume server if ldbtimeout is set. * remove uncessary unloadldb * Update weed/command/server.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> * Update weed/command/volume.go Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: guol-fnst <goul-fnst@fujitsu.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2022-09-28[volume] Add new volumes to HUP(reload) signal (#3755)Konstantin Lebedev1-0/+5
Add new volumes to HUP(reload) signal
2022-09-17volume server: rename readBufferSize to readBufferSizeMBchrislu1-3/+3
2022-09-16volume: add "readBufSize" option to customize read optimization (#3702)famosss1-0/+3
* simplify a bit * feat: volume: add "readBufSize" option to customize read optimization * refactor : redbufSIze -> readBufferSize * simplify a bit * simplify a bit
2022-09-15volume: add "hasSlowRead" option to customize read optimizationchrislu1-0/+3
2022-08-26make CodeQL happychrislu1-1/+1
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-8/+8
2022-05-21add inflight upload data wait timeoutliubaojiang1-0/+4
2022-05-20exclude replication from the concurrentUploadLimitMBliubaojiang1-0/+2
2022-05-17just exit in case of duplicated volume directories were loadedguol-fnst1-3/+0
2022-05-16avoid duplicated volume directoryguol-fnst1-2/+6
2022-03-23remove dead codechrislu1-1/+0
2022-03-23volume: fail fast if too many concurrent requests, to avoid dead lock due to ↵chrislu1-1/+0
replication. fix https://github.com/chrislusf/seaweedfs/issues/2755
2022-02-16healthz check to avoid drain pod with last replicasKonstantin Lebedev1-0/+1
2021-12-05upgrade protoc to 3.17.3chrislu1-0/+2
$brew install protobuf $ protoc --version libprotoc 3.17.3 $ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26 $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
2021-09-12change server address from string to a typeChris Lu1-5/+6
2021-09-07handle ipv6 addressesChris Lu1-2/+1
2021-08-08volume: support concurrent download data size limitChris Lu1-16/+22
2021-08-08renameChris Lu1-15/+15
2021-07-13volume: change all writes to fsync during graceful stoppingChris Lu1-0/+5
fix https://github.com/chrislusf/seaweedfs/issues/2193
2021-07-02volume: address "unaligned 64-bit atomic operation"Chris Lu1-4/+4
fix https://github.com/chrislusf/seaweedfs/issues/2177
2021-07-01go fmtChris Lu1-2/+2
2021-06-30add proxy mode to read non-local volumeszhangsong1-3/+3
2021-04-27minFreeSpace refactoredbingoohuang1-1/+1
2021-04-26minFreeSpace argument allows size like 10GiBbingoohuang1-2/+2
2021-03-30filer, volume: add concurrent upload size limit to avoid OOMChris Lu1-0/+8
add some back pressure when writes are slow
2021-02-16this can compile now!!!Chris Lu1-1/+2
2021-02-09Merge branch 'master' into support_ssd_volumeChris Lu1-2/+2
2021-02-07renaming NeedleMapType to NeedleMapKindbingoohuang1-2/+2
2020-12-13disk type configurable for each folderChris Lu1-4/+2
2020-12-13refactorChris Lu1-2/+2
2020-12-13rename from volumeType to diskTypeChris Lu1-4/+4
2020-12-13volume server set volume type and heartbeat to the masterChris Lu1-2/+4
2020-11-26pass in dir.idx parameter to DiskLocationChris Lu1-1/+2
2020-09-24adjust for metrics portChris Lu1-3/+1
2020-09-24add start metrics serverKonstantin Lebedev1-0/+2
2020-09-20adjust loggingChris Lu1-3/+1
2020-09-19volume: get metrics configuration from masterChris Lu1-3/+4
fix https://github.com/chrislusf/seaweedfs/issues/1354
2020-09-16simplify metrics configuration logicChris Lu1-4/+1
2020-09-16volume: load configuration from master at the startChris Lu1-0/+3
fix https://github.com/chrislusf/seaweedfs/issues/1469
2020-09-13shell: add volumeServer.leave commandChris Lu1-2/+4
2020-08-26stop send heartbeat before stop volume serverKonstantin Lebedev1-0/+2
2020-08-25volume server: remove whitelist for status checkingChris Lu1-1/+1
Volume liveness/readiness not work if jwt enabled, I try use param -whiteList but jwt breaks
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 22:41:45 +0000