diff options
| author | Lisandro Pin <lisandro.pin@proton.ch> | 2025-05-09 18:01:32 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-05-09 09:01:32 -0700 |
| commit | 848d1f7c34f4332d1bef0fe05679ee95dcb4f75b (patch) | |
| tree | f82808ec3d1a9bfcb550050d523c7562a0a43c22 /weed/shell/command_ec_common.go | |
| parent | 2ae5b480a66fb21abbf6e27ff13af9b05742077a (diff) | |
| download | seaweedfs-848d1f7c34f4332d1bef0fe05679ee95dcb4f75b.tar.xz seaweedfs-848d1f7c34f4332d1bef0fe05679ee95dcb4f75b.zip | |
Improve safety for weed shell's `ec.encode`. (#6773)
Improve safety for weed shells `ec.encode`.
The current process for `ec.encode` is:
1. EC shards for a volume are generated and added to a single server
2. The original volume is deleted
3. EC shards get re-balanced across the entire topology
It is then possible to lose data between #2 and #3, if the underlying volume storage/server/rack/DC
happens to fail, for whatever reason. As a fix, this MR reworks `ec.encode` so:
* Newly created EC shards are spread across all locations for the source volume.
* Source volumes are deleted only after EC shards are converted and balanced.
Diffstat (limited to 'weed/shell/command_ec_common.go')
| -rw-r--r-- | weed/shell/command_ec_common.go | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/weed/shell/command_ec_common.go b/weed/shell/command_ec_common.go index c49ab7611..209d8a733 100644 --- a/weed/shell/command_ec_common.go +++ b/weed/shell/command_ec_common.go @@ -134,6 +134,14 @@ func NewErrorWaitGroup(maxConcurrency int) *ErrorWaitGroup { } } +func (ewg *ErrorWaitGroup) Reset() { + close(ewg.wgSem) + + ewg.wg = &sync.WaitGroup{} + ewg.wgSem = make(chan bool, ewg.maxConcurrency) + ewg.errors = nil +} + func (ewg *ErrorWaitGroup) Add(f ErrorWaitGroupTask) { if ewg.maxConcurrency <= 1 { // Keep run order deterministic when parallelization is off |