aboutsummaryrefslogtreecommitdiff
path: root/weed/storage/needle_map
AgeCommit message (Collapse)AuthorFilesLines
2025-07-16convert error fromating to %w everywhere (#6995)Chris Lu1-1/+1
2025-06-06Fix dumb typo in 08556257 (#6844)Lisandro Pin1-1/+1
2025-06-05New `needle_map.CompactMap()` implementation for reduced memory usage (#6842)Lisandro Pin7-439/+1298
* Rework `needle_map.CompactMap()` to maximize memory efficiency. * Use a memory-efficient structure for `CompactMap` needle value entries. This slightly complicates the code, but makes a **massive** difference in memory efficiency - preliminary results show a ~30% reduction in heap usage, with no measurable performance impact otherwise. * Clean up type for `CompactMap` chunk IDs. * Add a small comment description for `CompactMap()`. * Add the old version of `CompactMap()` for comparison purposes.
2025-06-02Merge branch 'master' of https://github.com/seaweedfs/seaweedfs3.89Chris Lu2-15/+39
2025-06-023.89Chris Lu1-6/+10
2025-06-02fix insert beyond look back window (#6838)Chris Lu2-12/+39
2025-06-02Minor fix for the `CompactMap()` performance test. (#6836)Lisandro Pin1-2/+2
Per-entry memory usage is based on `TotalAllocs`, which is incorrect - that value is a cummulative of heap usage, which doesn't decrease when objects are freeed. `Allocs` is instead an accurate represeentation of actual memory usage at the time metrics are reported.
2025-05-28Further improve memory usage of `needle_map.CompactMap()`. (#6825)Lisandro Pin2-36/+40
2025-05-23Rewrite `needle_map.CompactMap()` for more efficient memory usage (#6813)Lisandro Pin2-88/+83
2024-01-25Revert "Fix weed fix"chrislu1-28/+2
This reverts commit 6c1ce18541731b01cca7c13f6245cbaad3bca7a5.
2023-11-26go fmtchrislu2-5/+5
2023-11-20Fix weed fixSmsS41-2/+28
2022-10-30[volume.chek.disk] sync of deletions the fix (#3923)Konstantin Lebedev1-1/+5
* sync of deletions the fix * avoid return if only partiallyDeletedNeedles * refactor sync deletions
2022-10-28[voluche.chek.disk] needles older than the cutoff time are not missing yet ↵Konstantin Lebedev1-8/+39
(#3922) needles older than the cutoff time are not missing yet https://github.com/seaweedfs/seaweedfs/issues/3919
2022-10-13vacuum metrics and force sync dst files (#3832)Konstantin Lebedev1-1/+4
2022-10-09fix: compact_map get error mismatching cokie (#3748)binbinshi3-6/+49
* fix: compact_map get error * fix: CompactSection delete lock and move test to compact_map Co-authored-by: shibinbin <shibinbin@megvii.com>
2022-09-14go fmtchrislu2-5/+5
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu8-12/+12
2022-07-20rename parameter and reuse functionsguol-fnst1-1/+1
rename milestone to watermark
2022-06-23test compact map with snowflake sequencerchrislu1-0/+15
2022-06-16reduce upfront memory usage for low density volumechrislu1-1/+1
2021-09-01go fmtChris Lu1-0/+1
2021-06-28look back when adding to sorted valuesChris Lu2-5/+25
look back when adding to sorted values, before adding it to overflow
2021-06-28volume: large_volume version has bug when using in memory indexChris Lu2-0/+34
fix https://github.com/chrislusf/seaweedfs/issues/2162
2021-03-22shell: add volume.check.disk to fix inconsistency for replicated volumesChris Lu1-1/+8
fix https://github.com/chrislusf/seaweedfs/issues/1923
2020-09-12do not idx file formatChris Lu1-5/+2
revert c9ab8d05fa9b425352ce978b5c5b5b0d71d787ad
2020-09-10fixes for reading deleted fidChris Lu1-2/+5
2020-08-29go fmtChris Lu2-3/+3
2020-08-19fix testsChris Lu1-3/+3
2020-08-18support read option readDeleted=trueChris Lu1-6/+3
2020-08-18refactoringChris Lu2-3/+3
2020-08-18refactoringChris Lu1-2/+2
2020-08-18refactoring to typed SizeChris Lu6-25/+23
Go is amazing with refactoring!
2020-07-11refactoringChris Lu1-1/+1
2020-03-09volume: fix memory leak during compactionChris Lu2-0/+27
fix https://github.com/chrislusf/seaweedfs/issues/1222
2020-01-08volume: fix compactionChris Lu1-1/+4
2019-12-24go fmtChris Lu1-2/+2
2019-12-24testingChris Lu1-1/+8
2019-12-24change btree map to in memory level dbChris Lu2-53/+112
2019-06-27volume: fix bug with 8000GB version if using in memory indexChris Lu1-2/+3
fix https://github.com/chrislusf/seaweedfs/issues/994
2019-05-28working with reading remote intervalsChris Lu1-11/+11
2019-05-18generate ec01~ec14, generate ecx file with sorted needle valuesChris Lu5-21/+47
2019-04-19fix needle map entry sizeChris Lu1-2/+2
2019-04-18refactoringChris Lu6-0/+622
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 20:36:17 +0000