aboutsummaryrefslogtreecommitdiff
path: root/weed/shell
AgeCommit message (Collapse)AuthorFilesLines
2025-06-16[shell] volume copy add param noLock (#6871)Konstantin Lebedev1-1/+4
2025-06-15Support filtering source disk type in volume.tier.upload (#6868)NyaMisty2-4/+14
2025-06-15Fix wrong error handling in volume.tier.upload when stream == nil but ↵NyaMisty1-5/+9
copyErr != nil (#6867)
2025-06-15Correctly sort in volume.list to ensure output consistency (#6866)NyaMisty1-2/+29
2025-05-28Add context with request (#6824)Aleksey Kosov3-3/+3
2025-05-22added context to filer_client method calls (#6808)Aleksey Kosov23-27/+39
Co-authored-by: akosov <a.kosov@kryptonite.ru>
2025-05-13Nit: unify the default `--maxParallelization` value for `weed shell` ↵Lisandro Pin3-2/+7
commands supporting this option (#6788)
2025-05-12Move `shell.ErrorWaitGroup` into a common file, to cleanly reuse across ↵Lisandro Pin2-56/+66
`weed shell` commands. (#6780) Move `shell.ErrorWaitGroup` into a dedicated common file, to cleanly reuse across `weed shell` commands.
2025-05-09Improve safety for weed shell's `ec.encode`. (#6773)Lisandro Pin2-35/+63
Improve safety for weed shells `ec.encode`. The current process for `ec.encode` is: 1. EC shards for a volume are generated and added to a single server 2. The original volume is deleted 3. EC shards get re-balanced across the entire topology It is then possible to lose data between #2 and #3, if the underlying volume storage/server/rack/DC happens to fail, for whatever reason. As a fix, this MR reworks `ec.encode` so: * Newly created EC shards are spread across all locations for the source volume. * Source volumes are deleted only after EC shards are converted and balanced.
2025-05-08Improve parallelization for `ec.encode` (#6769)Lisandro Pin1-34/+61
Improve parallelization for `ec.encode`. Instead of processing one volume at at time, perform all EC conversion steps (mark readonly -> generate EC shards -> delete volume -> remount) in parallel for all of them. This should substantially improve performance when EC encoding entire collections.
2025-04-27fix: volume.list volume info output not in order (#6737)NyaMisty1-1/+1
2025-03-31chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0 (#6679)dependabot[bot]4-8/+9
* chore(deps): bump gocloud.dev from 0.40.0 to 0.41.0 Bumps [gocloud.dev](https://github.com/google/go-cloud) from 0.40.0 to 0.41.0. - [Release notes](https://github.com/google/go-cloud/releases) - [Commits](https://github.com/google/go-cloud/compare/v0.40.0...v0.41.0) --- updated-dependencies: - dependency-name: gocloud.dev dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * fix error * fix printing errors * Update go.mod --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com>
2025-03-13add more help messagechrislu1-0/+1
fix https://github.com/seaweedfs/seaweedfs/issues/6625
2025-02-28`ec.encode`: Fix resolution of target collections. (#6585)Lisandro Pin3-13/+9
* Don't ignore empty (`""`) collection names when computing collections for a given volume ID. * `ec.encode`: Fix resolution of target collections. When no `volumeId` parameter is provided, compute volumes based on the provided collection name, even if it's empty (`""`). This restores behavior to before recent EC rebalancing rework. See also https://github.com/seaweedfs/seaweedfs/blob/ec30a504bae6cad75f859964e14c60d39cc43709/weed/shell/command_ec_encode.go#L99 .
2025-02-28Fix calculation of node's free EC shard slots. (#6584)Lisandro Pin2-1/+97
2025-02-23fix: error info size bug in command_fs_merge_volumes.go (#6567)Changrui Chen1-1/+1
2025-02-10`ec.encode`: Explictly mount EC shards after volume conversion. (#6528)Lisandro Pin1-2/+12
This guarantees EC shards are immediately available after encoding, even if not affected by subsequent re-balancing.
2025-02-07Nit: remove missing newlines on `weed shell` commands output. (#6524)Lisandro Pin1-1/+1
Nit: remove missing newlines on `weed` commands output.
2025-02-06Remove warning on EC balancing if no replica placement settings are found. ↵Lisandro Pin1-3/+0
(#6516) Effectively undoes c9399a68; with ff8bd862, a replica placement type `000` will no longer break shards re-balancing. Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2025-02-04Nit: fix missing newline on EC balancing warnings regarding replica settings ↵Lisandro Pin1-1/+1
(#6509) Nit: fix missing newline on EC balancing warnings regarding replica settings. See 79136812.
2025-01-30Improve EC shards balancing logic regarding replica placement settings. (#6491)Lisandro Pin2-6/+6
The replica placement type specifies numebr of _replicas_ on the same/different rack; that means we can have one EC shard copy on each, even if the replica setting is zero. This PR reworks replica placement parsing for EC rebalancing, so we check allow (replica placement + 1) when selecting racks and nodes to balance EC shards into.
2025-01-29`ec.balance`: Allow EC balancing without collections. (#6488)Lisandro Pin1-4/+4
2025-01-29`ec.encode`: Display a warning on EC balancing if no replica placement ↵Lisandro Pin1-10/+18
settings are found. (#6487)
2025-01-20Add message queue agent (#6463)Chris Lu2-2/+4
* scaffold message queue agent * adjust proto, add mq_agent * add agent client implementation * remove unused function * agent publish server implementation * adding agent
2025-01-20Add JWT authentication to fs.mergeVolumes command (#6461)Hadi Zamani1-1/+12
Add jwt authentication to fs.mergeVolumes command
2025-01-17`ec.encode`: Fix bug causing source volumes not being deleted after EC ↵Lisandro Pin1-1/+18
conversion. (#6447) This logic was originally part of `spreadEcShards()`, which got removed during the unification effort with `ec.balance` (https://github.com/seaweedfs/seaweedfs/pull/6344), accidentally breaking functionality in the process. The commit restores the deletion code for EC'd volumes - with parallelization support.
2025-01-12fix force arg dropped during volume balance command (#6432)ftong20201-3/+7
2025-01-08[weed] change -n to -force (#6421)dsd1-15/+15
2025-01-02Update command_fs_merge_volumes.go (#6406)Brad Murray1-4/+4
2024-12-31worm grace period and retention time support (#6404)Guang Jiong Lou1-13/+17
Signed-off-by: lou <alex1988@outlook.com>
2024-12-20skip error while executing volume.fix.replication (#6382)dsd1-5/+11
2024-12-19"golang.org/x/exp/slices" => "slices" and go fmtchrislu9-12/+10
2024-12-19Fix volume replica parallelization within `ec.encode`. (#6377)Lisandro Pin1-5/+3
See 826edd5d.
2024-12-18Allow configuring the maximum number of concurrent tasks for EC ↵Lisandro Pin3-49/+52
parallelization. (#6376) Follow-up to b0210df0.
2024-12-18Parallelize volume replica operations within `ec.encode`. (#6374)Lisandro Pin2-8/+21
2024-12-17Rework `shell.EcBalance()`'s waitgroup code into a standalone type. (#6373)Lisandro Pin1-54/+61
Rework `shell.EcBalance()`'s waitgroup with errors code into a standalone type. We'll re-use this for other EC jobs - for example, volume creation. Also fixes potential concurrency issues when collecting error results.
2024-12-15Parallelize EC shards balancing within racks (#6354)Lisandro Pin1-5/+5
Parallelize EC shards balancing within racks.
2024-12-13Parallelize EC shards balancing across racks. (#6352)Lisandro Pin1-5/+6
2024-12-13Parallelize EC balancing for racks. (#6351)Lisandro Pin1-15/+15
2024-12-13[shell] only apply the balancing for writable volumes (#6346)Konstantin Lebedev2-13/+26
2024-12-12Begin implementing EC balancing parallelization support. (#6342)Lisandro Pin3-12/+58
* Begin implementing EC balancing parallelization support. Impacts both `ec.encode` and `ec.balance`, * Nit: improve type naming. * Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof. * Nit: unify naming for `ecBalancer` wait group methods with the rest of the module. * Fix concurrency bug. * Fix whitespace after Gitlab automerge. * Delete stray TODO.
2024-12-12Limit EC re-balancing for `ec.encode` to relevant collections when a volume ↵Lisandro Pin3-5/+76
ID argument is provided. (#6347) Limit EC re-balancing for `ec.encode` to relevant collections when a volume ID is provided.
2024-12-12Delete legacy balancing code for `ec.encode`. (#6344)Lisandro Pin2-146/+0
2024-12-11[shell] volume.list show only writable volumes (#6338)Konstantin Lebedev1-6/+13
* show only writable volumes * fix import
2024-12-11volume.list avoid output empty data center and rack and disk info (#6341)Konstantin Lebedev1-15/+46
2024-12-10Unify the re-balancing logic for `ec.encode` with `ec.balance`. (#6339)Lisandro Pin6-136/+156
Among others, this enables recent changes related to topology aware re-balancing at EC encoding time.
2024-12-10[shell] use constant for hdd of type (#6337)Konstantin Lebedev3-4/+6
use constant for hdd of type
2024-12-06Remove average constraints when selecting nodes/racks to balance EC shards ↵Lisandro Pin3-97/+12
into. (#6325)
2024-12-05Share common parameters for EC re-balancing functions under a single struct. ↵Lisandro Pin3-187/+189
(#6319) TODO cleanup for https://github.com/seaweedfs/seaweedfs/discussions/6179.
2024-12-04Account for replication placement settings when balancing EC shards within ↵Lisandro Pin3-35/+57
the same rack. (#6317) * Account for replication placement settings when balancing EC shards within racks. * Update help contents for `ec.balance`. * Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-21 23:14:43 +0000