diff options
| author | Chris Lu <chrislusf@users.noreply.github.com> | 2025-12-14 17:06:13 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-12-14 17:06:13 -0800 |
| commit | 7ed75784241ad8d7635113e3e173959f0e8446ae (patch) | |
| tree | 11c7eeb84c6aaed2ba328b0a46ed760a93fc277c /weed/storage/erasure_coding/ec_decoder.go | |
| parent | 8bdc4390a04604af79f91c7dce94e3b2b58442f7 (diff) | |
| download | seaweedfs-origin/master.tar.xz seaweedfs-origin/master.zip | |
fix(ec.decode): purge EC shards when volume is empty (#7749)HEADorigin/masterorigin/HEADmaster
* fix(ec.decode): purge EC shards when volume is empty
When an EC volume has no live entries (all deleted), ec.decode should not generate an empty normal volume. Instead, treat decode as a no-op and allow shard purge to proceed cleanly.\n\nFixes: #7748
* chore: address PR review comments
* test: cover live EC index + avoid magic string
* chore: harden empty-EC handling
- Make shard cleanup best-effort (collect errors)\n- Remove unreachable EOF handling in HasLiveNeedles\n- Add empty ecx test case\n- Share no-live-entries substring between server/client\n
* perf: parallelize EC shard unmount/delete across locations
* refactor: combine unmount+delete into single goroutine per location
* refactor: use errors.Join for multi-error aggregation
* refactor: use existing ErrorWaitGroup for parallel execution
* fix: capture loop variables + clarify SuperBlockSize safety
Diffstat (limited to 'weed/storage/erasure_coding/ec_decoder.go')
| -rw-r--r-- | weed/storage/erasure_coding/ec_decoder.go | 22 |
1 files changed, 22 insertions, 0 deletions
diff --git a/weed/storage/erasure_coding/ec_decoder.go b/weed/storage/erasure_coding/ec_decoder.go index a1d929f6c..429dd7ac4 100644 --- a/weed/storage/erasure_coding/ec_decoder.go +++ b/weed/storage/erasure_coding/ec_decoder.go @@ -14,6 +14,23 @@ import ( "github.com/seaweedfs/seaweedfs/weed/util" ) +// EcNoLiveEntriesSubstring is used for server/client coordination when ec.decode determines that +// decoding should be a no-op (all entries are deleted). +const EcNoLiveEntriesSubstring = "has no live entries" + +// HasLiveNeedles returns whether the EC index (.ecx) contains at least one live (non-deleted) entry. +// This is used by ec.decode to avoid generating an empty normal volume when all entries were deleted. +func HasLiveNeedles(indexBaseFileName string) (hasLive bool, err error) { + err = iterateEcxFile(indexBaseFileName, func(_ types.NeedleId, _ types.Offset, size types.Size) error { + if !size.IsDeleted() { + hasLive = true + return io.EOF // stop early + } + return nil + }) + return +} + // write .idx file from .ecx and .ecj files func WriteIdxFileFromEcIndex(baseFileName string) (err error) { @@ -52,6 +69,11 @@ func FindDatFileSize(dataBaseFileName, indexBaseFileName string) (datSize int64, return 0, fmt.Errorf("read ec volume %s version: %v", dataBaseFileName, err) } + // Safety: ensure datSize is at least SuperBlockSize. While the caller typically + // checks HasLiveNeedles first, this protects against direct calls to FindDatFileSize + // when all needles are deleted (see issue #7748). + datSize = int64(super_block.SuperBlockSize) + err = iterateEcxFile(indexBaseFileName, func(key types.NeedleId, offset types.Offset, size types.Size) error { if size.IsDeleted() { |
