aboutsummaryrefslogtreecommitdiff
path: root/weed/shell/command_ec_balance.go
AgeCommit message (Collapse)AuthorFilesLines
10 daysec: add -diskType flag to EC commands for SSD support (#7607)Chris Lu1-2/+10
* ec: add diskType parameter to core EC functions Add diskType parameter to: - ecBalancer struct - collectEcVolumeServersByDc() - collectEcNodesForDC() - collectEcNodes() - EcBalance() This allows EC operations to target specific disk types (hdd, ssd, etc.) instead of being hardcoded to HardDriveType only. For backward compatibility, all callers currently pass types.HardDriveType as the default value. Subsequent commits will add -diskType flags to the individual EC commands. * ec: update helper functions to use configurable diskType Update the following functions to accept/use diskType parameter: - findEcVolumeShards() - addEcVolumeShards() - deleteEcVolumeShards() - moveMountedShardToEcNode() - countShardsByRack() - pickNEcShardsToMoveFrom() All ecBalancer methods now use ecb.diskType instead of hardcoded types.HardDriveType. Non-ecBalancer callers (like volumeServer.evacuate and ec.rebuild) use types.HardDriveType as the default. Update all test files to pass diskType where needed. * ec: add -diskType flag to ec.balance and ec.encode commands Add -diskType flag to specify the target disk type for EC operations: - ec.balance -diskType=ssd - ec.encode -diskType=ssd The disk type can be 'hdd', 'ssd', or empty for default (hdd). This allows placing EC shards on SSD or other disk types instead of only HDD. Example usage: ec.balance -collection=mybucket -diskType=ssd -apply ec.encode -collection=mybucket -diskType=ssd -force * test: add integration tests for EC disk type support Add integration tests to verify the -diskType flag works correctly: - TestECDiskTypeSupport: Tests EC encode and balance with SSD disk type - TestECDiskTypeMixedCluster: Tests EC operations on a mixed HDD/SSD cluster The tests verify: - Volume servers can be configured with specific disk types - ec.encode accepts -diskType flag and encodes to the correct disk type - ec.balance accepts -diskType flag and balances on the correct disk type - Mixed disk type clusters work correctly with separate collections * ec: add -sourceDiskType to ec.encode and -diskType to ec.decode ec.encode: - Add -sourceDiskType flag to filter source volumes by disk type - This enables tier migration scenarios (e.g., SSD volumes → HDD EC shards) - -diskType specifies target disk type for EC shards ec.decode: - Add -diskType flag to specify source disk type where EC shards are stored - Update collectEcShardIds() and collectEcNodeShardBits() to accept diskType Examples: # Encode SSD volumes to HDD EC shards (tier migration) ec.encode -collection=mybucket -sourceDiskType=ssd -diskType=hdd # Decode EC shards from SSD ec.decode -collection=mybucket -diskType=ssd Integration tests updated to cover new flags. * ec: fix variable shadowing and add -diskType to ec.rebuild and volumeServer.evacuate Address code review comments: 1. Fix variable shadowing in collectEcVolumeServersByDc(): - Rename loop variable 'diskType' to 'diskTypeKey' and 'diskTypeStr' to avoid shadowing the function parameter 2. Fix hardcoded HardDriveType in ecBalancer methods: - balanceEcRack(): use ecb.diskType instead of types.HardDriveType - collectVolumeIdToEcNodes(): use ecb.diskType 3. Add -diskType flag to ec.rebuild command: - Add diskType field to ecRebuilder struct - Pass diskType to collectEcNodes() and addEcVolumeShards() 4. Add -diskType flag to volumeServer.evacuate command: - Add diskType field to commandVolumeServerEvacuate struct - Pass diskType to collectEcVolumeServersByDc() and moveMountedShardToEcNode() * test: add diskType field to ecBalancer in TestPickEcNodeToBalanceShardsInto Address nitpick comment: ensure test ecBalancer struct has diskType field set for consistency with other tests. * ec: filter disk selection by disk type in pickBestDiskOnNode When evacuating or rebalancing EC shards, pickBestDiskOnNode now filters disks by the target disk type. This ensures: 1. EC shards from SSD disks are moved to SSD disks on destination nodes 2. EC shards from HDD disks are moved to HDD disks on destination nodes 3. No cross-disk-type shard movement occurs This maintains the storage tier isolation when moving EC shards between nodes during evacuation or rebalancing operations. * ec: allow disk type fallback during evacuation Update pickBestDiskOnNode to accept a strictDiskType parameter: - strictDiskType=true (balancing): Only use disks of matching type. This maintains storage tier isolation during normal rebalancing. - strictDiskType=false (evacuation): Prefer same disk type, but fall back to other disk types if no matching disk is available. This ensures evacuation can complete even when same-type capacity is insufficient. Priority order for evacuation: 1. Same disk type with lowest shard count (preferred) 2. Different disk type with lowest shard count (fallback) * test: use defer for lock/unlock to prevent lock leaks Use defer to ensure locks are always released, even on early returns or test failures. This prevents lock leaks that could cause subsequent tests to hang or fail. Changes: - Return early if lock acquisition fails - Immediately defer unlock after successful lock - Remove redundant explicit unlock calls at end of tests - Fix unused variable warning (err -> encodeErr/locErr) * ec: dynamically discover disk types from topology for evacuation Disk types are free-form tags (e.g., 'ssd', 'nvme', 'archive') that come from the topology, not a hardcoded set. Only 'hdd' (or empty) is the default disk type. Use collectVolumeDiskTypes() to discover all disk types present in the cluster topology instead of hardcoding [HardDriveType, SsdType]. * test: add evacuation fallback and cross-rack EC placement tests Add two new integration tests: 1. TestEvacuationFallbackBehavior: - Tests that when same disk type has no capacity, shards fall back to other disk types during evacuation - Creates cluster with 1 SSD + 2 HDD servers (limited SSD capacity) - Verifies pickBestDiskOnNode behavior with strictDiskType=false 2. TestCrossRackECPlacement: - Tests EC shard distribution across different racks - Creates cluster with 4 servers in 4 different racks - Verifies shards are spread across multiple racks - Tests that ec.balance respects rack placement Helper functions added: - startLimitedSsdCluster: 1 SSD + 2 HDD servers - startMultiRackCluster: 4 servers in 4 racks - countShardsPerRack: counts EC shards per rack from disk * test: fix collection mismatch in TestCrossRackECPlacement The EC commands were using collection 'rack_test' but uploaded test data uses collection 'test' (default). This caused ec.encode/ec.balance to not find the uploaded volume. Fix: Change EC commands to use '-collection test' to match the uploaded data. Addresses review comment from PR #7607. * test: close log files in MultiDiskCluster.Stop() to prevent FD leaks Track log files in MultiDiskCluster.logFiles and close them in Stop() to prevent file descriptor accumulation in long-running or many-test scenarios. Addresses review comment about logging resources cleanup. * test: improve EC integration tests with proper assertions - Add assertNoFlagError helper to detect flag parsing regressions - Update diskType subtests to fail on flag errors (ec.encode, ec.balance, ec.decode) - Update verify_disktype_flag_parsing to check help output contains diskType - Remove verify_fallback_disk_selection (was documentation-only, not executable) - Add assertion to verify_cross_rack_distribution for minimum 2 racks - Consolidate uploadTestDataWithDiskType to accept collection parameter - Remove duplicate uploadTestDataWithDiskTypeMixed function * test: extract captureCommandOutput helper and fix error handling - Add captureCommandOutput helper to reduce code duplication in diskType tests - Create commandRunner interface to match shell command Do method - Update ec_encode_with_ssd_disktype, ec_balance_with_ssd_disktype, ec_encode_with_source_disktype, ec_decode_with_disktype to use helper - Fix filepath.Glob error handling in countShardsPerRack instead of ignoring it * test: add flag validation to ec_balance_targets_correct_disk_type Add assertNoFlagError calls after ec.balance commands to ensure -diskType flag is properly recognized for both SSD and HDD disk types. * test: add proper assertions for EC command results - ec_encode_with_ssd_disktype: check for expected volume-related errors - ec_balance_with_ssd_disktype: require success with require.NoError - ec_encode_with_source_disktype: check for expected no-volume errors - ec_decode_with_disktype: check for expected no-ec-volume errors - upload_to_ssd_and_hdd: use require.NoError for setup validation Tests now properly fail on unexpected errors rather than just logging. * test: fix missing unlock in ec_encode_with_disk_awareness Add defer unlock pattern to ensure lock is always released, matching the pattern used in other subtests. * test: improve helper robustness - Make assertNoFlagError case-insensitive for pattern matching - Use defer in captureCommandOutput to restore stdout/stderr and close pipe ends to avoid FD leaks even if cmd.Do panics
2025-11-09Unify the parameter to disable dry-run on weed shell commands to `-apply` ↵Lisandro Pin1-3/+8
(instead of `-force`). (#7450) * Unify the parameter to disable dry-run on weed shell commands to --apply (instead of --force). * lint * refactor * Execution Order Corrected * handle deprecated force flag * fix help messages * Refactoring]: Using flag.FlagSet.Visit() * consistent with other commands * Checks for both flags * fix toml files --------- Co-authored-by: chrislu <chris.lu@gmail.com>
2025-05-13Nit: unify the default `--maxParallelization` value for `weed shell` ↵Lisandro Pin1-1/+1
commands supporting this option (#6788)
2024-12-18Allow configuring the maximum number of concurrent tasks for EC ↵Lisandro Pin1-2/+2
parallelization. (#6376) Follow-up to b0210df0.
2024-12-12Begin implementing EC balancing parallelization support. (#6342)Lisandro Pin1-2/+2
* Begin implementing EC balancing parallelization support. Impacts both `ec.encode` and `ec.balance`, * Nit: improve type naming. * Make the goroutine workgroup handler for `EcBalance()` a bit smarter/error-proof. * Nit: unify naming for `ecBalancer` wait group methods with the rest of the module. * Fix concurrency bug. * Fix whitespace after Gitlab automerge. * Delete stray TODO.
2024-12-10Unify the re-balancing logic for `ec.encode` with `ec.balance`. (#6339)Lisandro Pin1-67/+1
Among others, this enables recent changes related to topology aware re-balancing at EC encoding time.
2024-12-06Remove average constraints when selecting nodes/racks to balance EC shards ↵Lisandro Pin1-2/+2
into. (#6325)
2024-12-04Account for replication placement settings when balancing EC shards within ↵Lisandro Pin1-1/+1
the same rack. (#6317) * Account for replication placement settings when balancing EC shards within racks. * Update help contents for `ec.balance`. * Add a few more representative test cases for `pickEcNodeToBalanceShardsInto()`.
2024-12-04Account for replication placement settings when balancing EC shards across ↵Lisandro Pin1-2/+2
racks. (#6316)
2024-12-02Resolve replica placement for EC volumes from master server defaults. (#6303)Lisandro Pin1-2/+9
2024-11-04Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go`… (#6195)Lisandro Pin1-427/+5
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function. This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'. * s/Balance()/EcBalance()/g
2024-09-29refactorchrislu1-1/+1
2024-09-28add IsResourceHeavy() to command interfacechrislu1-0/+4
2024-07-15Add warning with hints to why a EC shard cant be moved (#5790)Dan1-2/+6
2023-09-25fix(ec): volumes created by foreign collection due to bug in ec balance (#4864)Tobias Gurtzick1-6/+10
* fix(ec): ignore 0 byte data files refers to parts of #4861 Signed-off-by: Tobias Gurtzick <magic@wizardtales.com> * fix(ec): ignore volumes not from the current collection during balance fixes #4861 Signed-off-by: Tobias Gurtzick <magic@wizardtales.com> --------- Signed-off-by: Tobias Gurtzick <magic@wizardtales.com>
2023-09-25Revert "Revert "Merge branch 'master' of ↵chrislu1-4/+4
https://github.com/seaweedfs/seaweedfs"" This reverts commit 8cb42c39
2023-09-18Revert "Merge branch 'master' of https://github.com/seaweedfs/seaweedfs"chrislu1-4/+4
This reverts commit 2e5aa06026750c99ea283181974d2ccfe5eb0468, reversing changes made to 4d414f54a224142f3f4d934f4af3b5dceb6fec6b.
2023-09-18Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 (#4850)dependabot[bot]1-4/+4
* Bump github.com/rclone/rclone from 1.63.1 to 1.64.0 Bumps [github.com/rclone/rclone](https://github.com/rclone/rclone) from 1.63.1 to 1.64.0. - [Release notes](https://github.com/rclone/rclone/releases) - [Changelog](https://github.com/rclone/rclone/blob/master/RELEASE.md) - [Commits](https://github.com/rclone/rclone/compare/v1.63.1...v1.64.0) --- updated-dependencies: - dependency-name: github.com/rclone/rclone dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * API changes * go mod --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com>
2022-09-14refactor(shell): `Decending` -> `Descending` (#3675)Ryan Russell1-1/+1
Signed-off-by: Ryan Russell <git@ryanrussell.org> Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu1-4/+4
2022-05-31help message when in simulation modechrislu1-0/+1
2022-04-18enhancement: replace sort.Slice with slices.SortFunc to reduce reflectionjustin1-8/+7
2021-12-10add lock messageschrislu1-1/+1
2021-09-13shell: do not need to lock to see volume -hChris Lu1-4/+4
2021-09-12change server address from string to a typeChris Lu1-2/+3
2021-02-16ec balance among nodes with HDDChris Lu1-1/+3
2021-02-16avoid possible nil disk infoChris Lu1-20/+28
2021-02-16this can compile now!!!Chris Lu1-6/+11
2020-10-21fix typoChris Lu1-1/+1
2020-09-14shell: add volumeServer.evacuate commandChris Lu1-1/+1
2020-04-23master: add cluster wide lock/unlock operation in weed shellChris Lu1-0/+4
fix https://github.com/chrislusf/seaweedfs/issues/1286
2020-02-25remove ctx if possibleChris Lu1-5/+2
2020-02-25avoid reusing context objectChris Lu1-24/+22
fix https://github.com/chrislusf/seaweedfs/issues/1182
2019-11-12passing in averageShardsPerEcNode instead of expectedTotalEcShardsChris Lu1-2/+1
addressing https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-12add sortEcNodesByFreeslotsDecending and sortEcNodesByFreeslotsAscendingChris Lu1-2/+2
addressing https://github.com/chrislusf/seaweedfs/issues/1111
2019-11-10fix rack_not_found caseChris Lu1-0/+4
fix rack_not_found in https://github.com/chrislusf/seaweedfs/issues/1111
2019-06-10go fmtChris Lu1-1/+1
2019-06-10ec shard balancingChris Lu1-121/+334
2019-06-07ec.balance: collect dc rack infoChris Lu1-9/+13
2019-06-05fix ec related bugsChris Lu1-0/+25
2019-06-05periodic scripts exeuction from leader masterChris Lu1-8/+8
2019-06-05adjust help messageChris Lu1-1/+1
2019-06-04correct count for free ec slots, avoid removing generated shardsChris Lu1-0/+1
2019-06-04purge duplicated ec shardsChris Lu1-2/+4
2019-06-03able to purge extra ec shard copiesChris Lu1-23/+60
2019-06-03fix multiple bugsChris Lu1-22/+3
2019-06-01adjust help messageChris Lu1-1/+1
2019-06-01ec.balance: use specific collection namesChris Lu1-7/+3
2019-06-01ec shard delete also check ec volumes, in addition to volumesChris Lu1-7/+20
2019-05-30collection list normal and ec volumesChris Lu1-1/+1
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-20 22:32:12 +0000