aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2024-11-18java 3.80chrislu8-15/+15
2024-11-18adds more info on skipped volumeschrislu1-0/+14
2024-11-18Introduce logic to resolve volume replica placement within EC rebalancing. ↵Lisandro Pin5-58/+141
(#6254) * Rename `command_ec_encode_test.go` to `command_ec_common_test.go`. All tests defined in this file are now for `command_ec_common.go`. * Minor code cleanups. - Fix broken `ec.balance` test. - Rework integer ceiling division to not use floats, which can introduce precision errors. * Introduce logic to resolve volume replica placement within EC rebalancing. This will be used to make rebalancing logic topology-aware. * Give shell.EcNode.dc a dedicated DataCenterId type.
2024-11-183.803.80chrislu2-3/+3
2024-11-18chore(deps): bump github.com/linxGnu/grocksdb from 1.9.5 to 1.9.7 (#6249)dependabot[bot]2-3/+3
Bumps [github.com/linxGnu/grocksdb](https://github.com/linxGnu/grocksdb) from 1.9.5 to 1.9.7. - [Release notes](https://github.com/linxGnu/grocksdb/releases) - [Commits](https://github.com/linxGnu/grocksdb/compare/v1.9.5...v1.9.7) --- updated-dependencies: - dependency-name: github.com/linxGnu/grocksdb dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18chore(deps): bump modernc.org/sqlite from 1.33.1 to 1.34.1 (#6251)dependabot[bot]2-3/+3
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.33.1 to 1.34.1. - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.1...v1.34.1) --- updated-dependencies: - dependency-name: modernc.org/sqlite dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18chore(deps): bump github.com/aws/aws-sdk-go-v2/config from 1.28.0 to 1.28.4 ↵dependabot[bot]2-18/+18
(#6252) chore(deps): bump github.com/aws/aws-sdk-go-v2/config Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.28.0 to 1.28.4. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.28.0...config/v1.28.4) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/config dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18chore(deps): bump go.etcd.io/etcd/client/v3 from 3.5.16 to 3.5.17 (#6253)dependabot[bot]2-9/+9
Bumps [go.etcd.io/etcd/client/v3](https://github.com/etcd-io/etcd) from 3.5.16 to 3.5.17. - [Release notes](https://github.com/etcd-io/etcd/releases) - [Commits](https://github.com/etcd-io/etcd/compare/v3.5.16...v3.5.17) --- updated-dependencies: - dependency-name: go.etcd.io/etcd/client/v3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-17Adjust threshold config (#6246)zemul1-0/+1
* fix:mount deadlock * fix --------- Co-authored-by: zemul <zhouzemiao@ihuman.com>
2024-11-15Fix/parse upload filename (#6241)Kevin Bulteel4-134/+123
* fix: parse filename in PUT + refactor * fix: master iu public url with http * fix: better parsing and handle disposition header * fix: take mime type from file extension if not set
2024-11-15Add a loop to wait for master and filer services in post-install-bucket-hook ↵Omar Hamadeh1-2/+22
(#6240) Add a loop to wait for master and filer services before creating buckets
2024-11-14fix java.lang.IllegalArgumentException: Comparison method violates its ↵fo402251-9/+12
general contract! (#6239)
2024-11-14feat(filer.backup): add ignore errors option (#6235)Max Denushev3-19/+52
* feat(filer.backup): add ignore errors option * feat(filer.backup): fix 404 error wrap * feat(filer.backup): fix wrapping function * feat(filer.backup): fix wrapping errors in genProcessFunction * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go * Update weed/command/filer_backup.go --------- Co-authored-by: Max Denushev <denushev@tochka.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-11-12added backward compatible CRC check and extra explanatory comments (#6233)Eugeniy E. Mikhailov2-3/+10
2024-11-11chore(deps): bump github.com/golang-jwt/jwt/v4 from 4.5.0 to 4.5.1 (#6231)dependabot[bot]2-3/+3
Bumps [github.com/golang-jwt/jwt/v4](https://github.com/golang-jwt/jwt) from 4.5.0 to 4.5.1. - [Release notes](https://github.com/golang-jwt/jwt/releases) - [Changelog](https://github.com/golang-jwt/jwt/blob/main/VERSION_HISTORY.md) - [Commits](https://github.com/golang-jwt/jwt/compare/v4.5.0...v4.5.1) --- updated-dependencies: - dependency-name: github.com/golang-jwt/jwt/v4 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.66.2 to ↵dependabot[bot]2-21/+21
1.66.3 (#6225) chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.66.2 to 1.66.3. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.66.2...service/s3/v1.66.3) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11chore(deps): bump github.com/aws/aws-sdk-go-v2 from 1.32.3 to 1.32.4 (#6227)dependabot[bot]2-3/+3
Bumps [github.com/aws/aws-sdk-go-v2](https://github.com/aws/aws-sdk-go-v2) from 1.32.3 to 1.32.4. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/v1.32.3...v1.32.4) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 from 3.89.2 to ↵dependabot[bot]2-3/+3
3.91.0 (#6228) chore(deps): bump github.com/ydb-platform/ydb-go-sdk/v3 Bumps [github.com/ydb-platform/ydb-go-sdk/v3](https://github.com/ydb-platform/ydb-go-sdk) from 3.89.2 to 3.91.0. - [Release notes](https://github.com/ydb-platform/ydb-go-sdk/releases) - [Changelog](https://github.com/ydb-platform/ydb-go-sdk/blob/master/CHANGELOG.md) - [Commits](https://github.com/ydb-platform/ydb-go-sdk/compare/v3.89.2...v3.91.0) --- updated-dependencies: - dependency-name: github.com/ydb-platform/ydb-go-sdk/v3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-11chore(deps): bump golang.org/x/tools from 0.26.0 to 0.27.0 (#6229)dependabot[bot]2-29/+24
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.26.0 to 0.27.0. - [Release notes](https://github.com/golang/tools/releases) - [Commits](https://github.com/golang/tools/compare/v0.26.0...v0.27.0) --- updated-dependencies: - dependency-name: golang.org/x/tools dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-09delete aborted ec shards from both source and target servers (#6221)Chris Lu1-1/+4
fix https://github.com/seaweedfs/seaweedfs/issues/6205#issuecomment-2465004586
2024-11-08[helm] fix s3 servicemonitor label matching (#6218)Nathan Dorsey1-2/+2
2024-11-07remove the direct_io flag, as it is not well-supported on macOSchrislu3-0/+11
2024-11-07adjust logschrislu1-1/+1
2024-11-06add debug infochrislu1-0/+1
2024-11-05[shell] add admin noLock for balance (#6209)Konstantin Lebedev2-2/+12
add admin noLock for balance
2024-11-05remove dup code for section.reader access (#6169)Bruce1-6/+2
2024-11-05fix testschrislu1-4/+4
2024-11-05remove case when read request is out side of the file sizechrislu1-5/+4
2024-11-05a bit optimizationchrislu1-0/+2
2024-11-05fix FUSE mount on macchrislu1-1/+4
2024-11-04Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go`… (#6195)Lisandro Pin2-428/+428
* Refactor `ec.balance` logic into a `weeed/shell/command_ec_common.go` standalone function. This is a prerequisite to unify the balance logic for `ec.balance` and `ec.encode'. * s/Balance()/EcBalance()/g
2024-11-04merge current message queue code changes (#6201)Chris Lu33-264/+1106
* listing files to convert to parquet * write parquet files * save logs into parquet files * pass by value * compact logs into parquet format * can skip existing files * refactor * refactor * fix compilation * when no partition found * refactor * add untested parquet file read * rename package * refactor * rename files * remove unused * add merged log read func * parquet wants to know the file size * rewind by time * pass in stop ts * add stop ts * adjust log * minor * adjust log * skip .parquet files when reading message logs * skip non message files * Update subscriber_record.go * send messages * skip message data with only ts * skip non log files * update parquet-go package * ensure a valid record type * add new field to a record type * Update read_parquet_to_log.go * fix parquet file name generation * separating reading parquet and logs * add key field * add skipped logs * use in memory cache * refactor * refactor * refactor * refactor, and change compact log * refactor * rename * refactor * fix format * prefix v to version directory
2024-11-04chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 from 1.61.2 to ↵dependabot[bot]2-27/+27
1.66.2 (#6197) chore(deps): bump github.com/aws/aws-sdk-go-v2/service/s3 Bumps [github.com/aws/aws-sdk-go-v2/service/s3](https://github.com/aws/aws-sdk-go-v2) from 1.61.2 to 1.66.2. - [Release notes](https://github.com/aws/aws-sdk-go-v2/releases) - [Commits](https://github.com/aws/aws-sdk-go-v2/compare/service/s3/v1.61.2...service/s3/v1.66.2) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go-v2/service/s3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-11-04chore(deps): bump github.com/gocql/gocql from 1.6.0 to 1.7.0 (#6196)dependabot[bot]2-3/+3
Bumps [github.com/gocql/gocql](https://github.com/gocql/gocql) from 1.6.0 to 1.7.0. - [Release notes](https://github.com/gocql/gocql/releases) - [Changelog](https://github.com/apache/cassandra-gocql-driver/blob/trunk/CHANGELOG.md) - [Commits](https://github.com/gocql/gocql/compare/v1.6.0...v1.7.0) --- updated-dependencies: - dependency-name: github.com/gocql/gocql dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04chore(deps): bump gocloud.dev/pubsub/natspubsub from 0.39.0 to 0.40.0 (#6198)dependabot[bot]2-3/+3
Bumps [gocloud.dev/pubsub/natspubsub](https://github.com/google/go-cloud) from 0.39.0 to 0.40.0. - [Release notes](https://github.com/google/go-cloud/releases) - [Commits](https://github.com/google/go-cloud/compare/v0.39.0...v0.40.0) --- updated-dependencies: - dependency-name: gocloud.dev/pubsub/natspubsub dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04chore(deps): bump golang.org/x/tools from 0.25.0 to 0.26.0 (#6199)dependabot[bot]2-3/+3
Bumps [golang.org/x/tools](https://github.com/golang/tools) from 0.25.0 to 0.26.0. - [Release notes](https://github.com/golang/tools/releases) - [Commits](https://github.com/golang/tools/compare/v0.25.0...v0.26.0) --- updated-dependencies: - dependency-name: golang.org/x/tools dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04chore(deps): bump cloud.google.com/go/pubsub from 1.43.0 to 1.45.1 (#6200)dependabot[bot]2-35/+31
Bumps [cloud.google.com/go/pubsub](https://github.com/googleapis/google-cloud-go) from 1.43.0 to 1.45.1. - [Release notes](https://github.com/googleapis/google-cloud-go/releases) - [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.43.0...pubsub/v1.45.1) --- updated-dependencies: - dependency-name: cloud.google.com/go/pubsub dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-03fix deadlock hang when broadcast to clients (#6184)wyang2-5/+21
fix deadlock when broadcast to clients when master thransfer leader, the old master will disconnect with all filers and volumeServers, if the cluster is a big , the broadcast messages may be more big than the max of the channel len 100, then if the KeepConnect was not listen on the channel in disconnect, it will deadlock. and the whole cluster will not serve!
2024-11-03fix mount: cannot locate OSXFUSEchrislu2-1/+5
addressing https://github.com/seaweedfs/seaweedfs/discussions/6193
2024-11-03[shell] volume.balance collect volume servers by dc rack node (#6191)Konstantin Lebedev4-8/+18
* chore: balance by rack * fix: rm check lock * fix: selected racks * fix: selected nodes * fix: containts * fix: one collectVolumeServersByDcRackNode * fix: revert lock and add lock * fix: panic test * revert noLock
2024-11-01Update filer_multipart.go (#6188)han1-0/+1
fix the #6177 bug
2024-10-31Fix 6181/6182 (#6183)Bruce4-14/+50
* set larger buf size for LogBuffer * jump to next day when no more entry found * Update weed/filer/filer_notify_read.go --------- Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com>
2024-10-31fix format (#6185)wyang1-1/+1
unitest weed/shell fail
2024-10-30fix parameterchrislu1-2/+2
2024-10-30pass in bytes buffer to avoid possible race conditionchrislu1-2/+2
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2024-10-30adding more debug messagechrislu1-0/+4
related to https://github.com/seaweedfs/seaweedfs/issues/6180
2024-10-283.793.79chrislu2-3/+3
2024-10-28backward compatible vif loadingchrislu3-727/+881
2024-10-28chore(deps): bump github.com/redis/go-redis/v9 from 9.6.2 to 9.7.0 (#6174)dependabot[bot]2-3/+3
Bumps [github.com/redis/go-redis/v9](https://github.com/redis/go-redis) from 9.6.2 to 9.7.0. - [Release notes](https://github.com/redis/go-redis/releases) - [Changelog](https://github.com/redis/go-redis/blob/master/CHANGELOG.md) - [Commits](https://github.com/redis/go-redis/compare/v9.6.2...v9.7.0) --- updated-dependencies: - dependency-name: github.com/redis/go-redis/v9 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28chore(deps): bump modernc.org/sqlite from 1.33.0 to 1.33.1 (#6173)dependabot[bot]2-3/+15
Bumps [modernc.org/sqlite](https://gitlab.com/cznic/sqlite) from 1.33.0 to 1.33.1. - [Commits](https://gitlab.com/cznic/sqlite/compare/v1.33.0...v1.33.1) --- updated-dependencies: - dependency-name: modernc.org/sqlite dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>