aboutsummaryrefslogtreecommitdiff
path: root/weed/filer/abstract_sql
AgeCommit message (Collapse)AuthorFilesLines
2025-11-25Add error list each entry func (#7485)tam-i131-3/+9
* added error return in type ListEachEntryFunc * return error if errClose * fix fmt.Errorf * fix return errClose * use %w fmt.Errorf * added entry in messege error * add callbackErr in ListDirectoryEntries * fix error * add log * clear err when the scanner stops on io.EOF, so returning err doesn’t surface EOF as a failure. * more info in error * add ctx to logs, error handling * fix return eachEntryFunc * fix * fix log * fix return * fix foundationdb test s * fix eachEntryFunc * fix return resEachEntryFuncErr * Update weed/filer/filer.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/filer/elastic/v7/elastic_store.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/filer/hbase/hbase_store.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/filer/foundationdb/foundationdb_store.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/filer/ydb/ydb_store.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fix * add scanErr --------- Co-authored-by: Roman Tamarov <r.tamarov@kryptonite.ru> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-08-03Admin: refactoring active topology (#7073)Chris Lu1-2/+3
* refactoring * add ec shard size * address comments * passing task id There seems to be a disconnect between the pending tasks created in ActiveTopology and the TaskDetectionResult returned by this function. A taskID is generated locally and used to create pending tasks via AddPendingECShardTask, but this taskID is not stored in the TaskDetectionResult or passed along in any way. This makes it impossible for the worker that eventually executes the task to know which pending task in ActiveTopology it corresponds to. Without the correct taskID, the worker cannot call AssignTask or CompleteTask on the master, breaking the entire task lifecycle and capacity management feature. A potential solution is to add a TaskID field to TaskDetectionResult and worker_pb.TaskParams, ensuring the ID is propagated from detection to execution. * 1 source multiple destinations * task supports multi source and destination * ec needs to clean up previous shards * use erasure coding constants * getPlanningCapacityUnsafe getEffectiveAvailableCapacityUnsafe should return StorageSlotChange for calculation * use CanAccommodate to calculate * remove dead code * address comments * fix Mutex Copying in Protobuf Structs * use constants * fix estimatedSize The calculation for estimatedSize only considers source.EstimatedSize and dest.StorageChange, but omits dest.EstimatedSize. The TaskDestination struct has an EstimatedSize field, which seems to be ignored here. This could lead to an incorrect estimation of the total size of data involved in tasks on a disk. The loop should probably also include estimatedSize += dest.EstimatedSize. * at.assignTaskToDisk(task) * refactoring * Update weed/admin/topology/internal.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * fail fast * fix compilation * Update weed/worker/tasks/erasure_coding/detection.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * indexes for volume and shard locations * dedup with ToVolumeSlots * return an additional boolean to indicate success, or an error * Update abstract_sql_store.go * fix * Update weed/worker/tasks/erasure_coding/detection.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/admin/topology/task_management.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * faster findVolumeDisk * Update weed/worker/tasks/erasure_coding/detection.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update weed/admin/topology/storage_slot_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * refactor * simplify * remove unused GetDiskStorageImpact function * refactor * add comments * Update weed/admin/topology/storage_impact.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update weed/admin/topology/storage_slot_test.go Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> * Update storage_impact.go * AddPendingTask The unified AddPendingTask function now serves as the single entry point for all task creation, successfully consolidating the previously separate functions while maintaining full functionality and improving code organization. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-16convert error fromating to %w everywhere (#6995)Chris Lu1-4/+4
2025-06-24Changes logging function (#6919)Aleksey Kosov2-5/+5
* updated logging methods for stores * updated logging methods for stores * updated logging methods for filer * updated logging methods for uploader and http_util * updated logging methods for weed server --------- Co-authored-by: akosov <a.kosov@kryptonite.ru>
2024-06-25check bucket name in sql backendchrislu1-0/+6
fix https://github.com/seaweedfs/seaweedfs/issues/5710
2023-12-09falls back to update only if error contains msg "duplicate entry"Konstantin Lebedev1-16/+9
https://github.com/seaweedfs/seaweedfs/issues/5062
2022-11-15refactor filer_pb.Entry and filer.Entry to use GetChunks()chrislu1-1/+1
for later locking on reading chunks
2022-07-29move to https://github.com/seaweedfs/seaweedfschrislu2-7/+7
2022-07-21explicit bucket aware declarationchrislu1-0/+2
2022-05-01refactor use const CountEntryChunksForGzipKonstantin Lebedev1-1/+1
2022-05-01ydb kv interfaceKonstantin Lebedev1-4/+4
2021-07-22filer: fix mysql, postgres batch delete errorChris Lu1-17/+9
2021-06-15avoid crashes Galera ClusterKonstantin Lebedev1-11/+17
https://github.com/chrislusf/seaweedfs/issues/2125
2021-06-10remove all bucket metadataChris Lu1-0/+3
fix https://github.com/chrislusf/seaweedfs/issues/2118
2021-04-22clean up .uploads directory in mysql and postgres tablesChris Lu1-1/+3
fix https://github.com/chrislusf/seaweedfs/issues/1957 When no uploads are running, you can run this SQL to clean up. delete from <bucket_name> where directory like '/.uploads/%'
2021-04-02ensure tables are createdChris Lu1-0/+4
fix https://github.com/chrislusf/seaweedfs/issues/1957
2021-03-25refactorChris Lu1-9/+9
2021-03-13filer: mysql2, postgres2 trigger actions on bucket creation and deletionChris Lu1-0/+19
fix https://github.com/chrislusf/seaweedfs/issues/1877
2021-02-15filer: postgres2 mysql2 avoid repeatedly creating tablesChris Lu1-1/+1
2021-01-19filer store: add postgres2Chris Lu1-2/+2
2021-01-19filer store: add mysql2Chris Lu1-1/+5
2021-01-19fix bucket creationChris Lu1-9/+24
2021-01-19fix errorChris Lu1-1/+1
2021-01-19restructuring sql storesChris Lu2-32/+89
2021-01-19refactor: prepare for bucket specific tablesChris Lu2-18/+77
2021-01-19add more input paramsChris Lu2-12/+12
2021-01-15add stream list directory entriesChris Lu1-12/+11
2021-01-14go fmtChris Lu1-1/+1
2021-01-14change limit to int64 in case of overflowChris Lu1-3/+3
2021-01-14implement sql changesChris Lu1-11/+16
2020-12-22go fmtChris Lu1-1/+1
2020-12-13filer store: sql update if any insert error happensChris Lu2-3/+5
fix https://github.com/chrislusf/seaweedfs/issues/1673
2020-12-13sql put kv: avoid unnecessary updateChris Lu1-6/+7
2020-10-16mysql, postgres, cassandra: change kv key to base64 encodingChris Lu1-2/+3
The exisitng key-value operation for stores using mysql, postgres, and maybe cassandra are already broken. The kv is used to store hardlink, filer store signature and replication progress. So users using hardlink and also uses mysql, postgres, or cassandra will have broken hard links. Users using filer.sync will need to re-sync the files.
2020-09-12filer: fix postgres prefixed directory listing problemChris Lu1-6/+8
fix https://github.com/chrislusf/seaweedfs/issues/1465
2020-09-03filer: compress stored metadataChris Lu1-2/+6
2020-09-02Update abstract_sql_store_kv.goChris Lu1-1/+2
2020-09-02sql kv upsertChris Lu1-9/+5
2020-09-02fix sql insert with duplicated primary keyChris Lu1-6/+4
2020-09-01fix error messageChris Lu1-1/+1
2020-09-01mysql/postgres: support kv operationsChris Lu1-3/+75
2020-09-01mysql/postgres: properly report entry not foundChris Lu1-2/+5
2020-09-01mysql or postgres: log find errorChris Lu1-0/+1
2020-09-01filer store adds kv supportChris Lu1-0/+18
can compile now, need to implement those unimplemented
2020-09-01rename filer2 to filerChris Lu1-0/+206
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-25 20:04:47 +0000