| Age | Commit message (Collapse) | Author | Files | Lines | |
|---|---|---|---|---|---|
| 2025-12-11 | filer.sync: fix checkpoint not being saved properly (#7719) | Chris Lu | 1 | -3/+49 | |
| * filer.sync: fix race condition on first checkpoint save Initialize lastWriteTime to time.Now() instead of zero time to prevent the first checkpoint save from being triggered immediately when the first event arrives. This gives async jobs time to complete and update the watermark before the checkpoint is saved. Previously, the zero time caused lastWriteTime.Add(3s).Before(now) to be true on the first event, triggering an immediate checkpoint save attempt. But since jobs are processed asynchronously, the watermark was still 0 (initial value), causing the save to be skipped due to the 'if offsetTsNs == 0 { return nil }' check. Fixes #7717 * filer.sync: save checkpoint on graceful shutdown Add graceful shutdown handling to save the final checkpoint when filer.sync is terminated. Previously, any sync progress within the last 3-second checkpoint interval would be lost on shutdown. Changes: - Add syncState struct to track current processor and offset save info - Add atomic pointers syncStateA2B and syncStateB2A for both directions - Register grace.OnInterrupt hook to save checkpoints on shutdown - Modify doSubscribeFilerMetaChanges to update sync state atomically This ensures that when filer.sync is restarted, it resumes from the correct position instead of potentially replaying old events. Fixes #7717 | |||||
| 2024-11-21 | filer sync: source path and exclude path support dir suffix (#6268) | Numblgw | 1 | -8/+13 | |
| filer sync: source path and exclude paht support dir suffix Co-authored-by: liguowei <liguowei@xinye.com> | |||||
| 2024-11-14 | feat(filer.backup): add ignore errors option (#6235) | Max Denushev | 1 | -4/+4 | |
| * feat(filer.backup): add ignore errors option * feat(filer.backup): fix 404 error wrap * feat(filer.backup): fix wrapping function * feat(filer.backup): fix wrapping errors in genProcessFunction * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go --------- Co-authored-by: Max Denushev <denushev@tochka.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> | |||||
| 2024-07-16 | minor | chrislu | 1 | -3/+3 | |
| 2024-07-16 | Added loadSecurityConfigOnce (#5792) | vadimartynov | 1 | -1/+1 | |
| 2024-07-12 | ensure metadata follow a specific folder | chrislu | 1 | -1/+6 | |
| fix https://github.com/seaweedfs/seaweedfs/issues/5774 | |||||
| 2024-03-20 | fix: sync without dir /buckets/some/.uploads/hash_hash (#5402) | Konstantin Lebedev | 1 | -1/+1 | |
| 2024-01-18 | fix | chrislu | 1 | -1/+0 | |
| 2024-01-18 | refactor | chrislu | 1 | -1/+3 | |
| 2024-01-18 | fix: skip s3 .uploads | XIAOYQ | 1 | -1/+4 | |
| 2024-01-12 | fix: doDeleteFiles deletes files (#5198) | Konstantin Lebedev | 1 | -1/+1 | |
| 2024-01-12 | Fix filer sync set offset (#5197) | Konstantin Lebedev | 1 | -10/+12 | |
| * fix: compose 2mount with sync * fix: DATA RACE https://github.com/seaweedfs/seaweedfs/issues/5194 https://github.com/seaweedfs/seaweedfs/issues/5195 | |||||
| 2024-01-06 | chore: filer sync add doDeleteFiles option for create only mode (#5166) | Konstantin Lebedev | 1 | -2/+8 | |
| 2023-11-17 | fix: add doDeleteFile option for filer backup | Konstantin Lebedev | 1 | -8/+11 | |
| 2023-11-13 | filer backup add option for exclude file names that match the regexp to sync ↵ | Konstantin Lebedev | 1 | -2/+6 | |
| on filer | |||||
| 2023-03-21 | refactoring | chrislu | 1 | -2/+14 | |
| 2023-01-20 | grpc connection to filer add sw-client-id header | chrislu | 1 | -2/+2 | |
| 2022-12-17 | more detailed logs | chrislu | 1 | -3/+15 | |
| 2022-11-24 | [metrics] Add the ability to control bind ip (#4012) | Jiffs Maverick | 1 | -1/+3 | |
| 2022-09-14 | go fmt | chrislu | 1 | -1/+1 | |
| 2022-09-14 | refactor: `DefaultConcurrencyLimit` var rename (#3658) | Ryan Russell | 1 | -4/+4 | |
| 2022-09-02 | new 'concurrency' parameter for filer.sync (#3579) | bernardx | 1 | -6/+16 | |
| Co-authored-by: XIAOYQ <xiaoyq@eudic.net> | |||||
| 2022-08-15 | fix(filer.sync): offset may be set to 0 (#3451) | qzh | 1 | -0/+3 | |
| * fix(filer.sync): initializing the offset is related to the path * fix(filer.sync): the offset maybe to be set to 0. Co-authored-by: zhihao.qu <zhihao.qu@ly.com> | |||||
| 2022-08-15 | fix(filer.sync): initializing the offset is related to the path (#3450) | qzh | 1 | -4/+4 | |
| Co-authored-by: zhihao.qu <zhihao.qu@ly.com> | |||||
| 2022-08-07 | refactor and fix strings.Split | chrislu | 1 | -2/+2 | |
| 2022-08-07 | filer.sync: parallelize the filer.sync | chrislu | 1 | -4/+9 | |
| 2022-08-07 | filer.sync: fix when excluded paths is empty | chrislu | 1 | -2/+2 | |
| 2022-07-29 | move to https://github.com/seaweedfs/seaweedfs | chrislu | 1 | -11/+11 | |
| 2022-07-27 | exclude directories to sync on filer | Konstantin Lebedev | 1 | -10/+48 | |
| 2022-07-23 | metadata subscription uses client epoch | chrislu | 1 | -4/+7 | |
| 2022-06-27 | fix reading time | chrislu | 1 | -2/+2 | |
| 2022-06-15 | fix(filer.sync): modify clientName format : from -> to | zhihao.qu | 1 | -1/+1 | |
| 2022-06-15 | feat(filer.sync): add metricsServer in filer.sync. | zhihao.qu | 1 | -1/+10 | |
| Metrics include: (1) the offset of the filer.sync (2) the last send timestamp of the filer subscription | |||||
| 2022-06-14 | feat(filer.sync): add offset to path. | zhihao.qu | 1 | -2/+12 | |
| 2022-06-09 | feat(filer.sync): add fromTsMs. Extract signature from ↵ | zhihao.qu | 1 | -13/+53 | |
| doSubscribeFilerMetaChanges | |||||
| 2022-06-02 | fix filer.sync missing source srv uploaded files to target when target down | creeew | 1 | -1/+1 | |
| 2022-05-30 | subscribe metadata between a range | chrislu | 1 | -1/+1 | |
| 2022-04-14 | Fix filer.backup deletes files in backup folder in incremental mode | chrislu | 1 | -1/+4 | |
| fix https://github.com/chrislusf/seaweedfs/issues/2919 | |||||
| 2022-02-25 | refactoring | chrislu | 1 | -3/+3 | |
| 2022-01-10 | fix logging | chrislu | 1 | -1/+4 | |
| 2021-12-30 | add client id for all metadata listening clients | chrislu | 1 | -4/+6 | |
| 2021-12-26 | use streaming mode for long poll grpc calls | chrislu | 1 | -2/+2 | |
| streaming mode would create separate grpc connections for each call. this is to ensure the long poll connections are properly closed. | |||||
| 2021-09-12 | change server address from string to a type | Chris Lu | 1 | -8/+10 | |
| 2021-08-31 | line wrap | Chris Lu | 1 | -1/+2 | |
| 2021-08-31 | add additionalPathPrefixes | Chris Lu | 1 | -2/+1 | |
| 2021-08-08 | go fmt | Chris Lu | 1 | -2/+2 | |
| 2021-08-08 | adjust help message | Chris Lu | 1 | -2/+2 | |
| 2021-08-04 | refactor client subscribe metadata | Chris Lu | 1 | -43/+6 | |
| 2021-08-03 | add TLS grpc support for filer meta clients | Chris Lu | 1 | -0/+1 | |
| 2021-06-24 | adjust logs | Chris Lu | 1 | -1/+1 | |
 |
index : seaweedfs/.git | |
| Unnamed repository; edit this file 'description' to name the repository. | root |
| aboutsummaryrefslogtreecommitdiff |