aboutsummaryrefslogtreecommitdiff
path: root/weed/mq/schema
AgeCommit message (Collapse)AuthorFilesLines
2025-07-16convert error fromating to %w everywhere (#6995)Chris Lu1-1/+1
2025-06-23chore(deps): bump github.com/parquet-go/parquet-go from 0.24.0 to 0.25.1 (#6851)dependabot[bot]1-10/+31
* chore(deps): bump github.com/parquet-go/parquet-go from 0.24.0 to 0.25.1 Bumps [github.com/parquet-go/parquet-go](https://github.com/parquet-go/parquet-go) from 0.24.0 to 0.25.1. - [Release notes](https://github.com/parquet-go/parquet-go/releases) - [Changelog](https://github.com/parquet-go/parquet-go/blob/main/CHANGELOG.md) - [Commits](https://github.com/parquet-go/parquet-go/compare/v0.24.0...v0.25.1) --- updated-dependencies: - dependency-name: github.com/parquet-go/parquet-go dependency-version: 0.25.1 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * adjust to updated API Fixed Reader Construction: Updated to use parquet.OpenFile() instead of passing io.Reader directly to NewReader() Fixed EOF Handling: Changed the order of operations to process rows before checking for EOF Added Zero Row Count Check: Added explicit check for rowCount == 0 as an additional termination condition --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Lu <chrislusf@users.noreply.github.com> Co-authored-by: chrislu <chris.lu@gmail.com>
2025-03-04adjust logschrislu1-3/+3
2025-01-20Add message queue agent (#6463)Chris Lu1-2/+7
* scaffold message queue agent * adjust proto, add mq_agent * add agent client implementation * remove unused function * agent publish server implementation * adding agent
2024-11-04merge current message queue code changes (#6201)Chris Lu1-0/+7
* listing files to convert to parquet * write parquet files * save logs into parquet files * pass by value * compact logs into parquet format * can skip existing files * refactor * refactor * fix compilation * when no partition found * refactor * add untested parquet file read * rename package * refactor * rename files * remove unused * add merged log read func * parquet wants to know the file size * rewind by time * pass in stop ts * add stop ts * adjust log * minor * adjust log * skip .parquet files when reading message logs * skip non message files * Update subscriber_record.go * send messages * skip message data with only ts * skip non log files * update parquet-go package * ensure a valid record type * add new field to a record type * Update read_parquet_to_log.go * fix parquet file name generation * separating reading parquet and logs * add key field * add skipped logs * use in memory cache * refactor * refactor * refactor * refactor, and change compact log * refactor * rename * refactor * fix format * prefix v to version directory
2024-10-03add helper functionschrislu1-0/+27
2024-05-20go fmtchrislu10-55/+54
2024-05-05fix testchrislu1-5/+5
2024-05-03workingchrislu1-1/+1
2024-05-02renamechrislu5-13/+13
2024-05-02rename field typeschrislu7-21/+21
2024-05-02consistent with protobuf data typeschrislu6-28/+28
2024-05-02rename to WithField()chrislu4-36/+36
2024-04-28Update write_parquet_test.gochrislu1-1/+1
2024-04-28record value APIchrislu2-34/+37
2024-04-28record type begin and endchrislu4-41/+55
2024-04-28improve apichrislu2-25/+7
2024-04-28rename apichrislu3-22/+10
2024-04-28Delete schema_builder_test.gochrislu1-17/+0
2024-04-28struct to schema correctlychrislu3-20/+152
2024-04-28compile testchrislu1-5/+5
2024-04-28recursive struct to schemachrislu2-8/+63
2024-04-27renamechrislu4-31/+31
2024-04-27renamechrislu2-22/+22
2024-04-25a little bit more efficientchrislu3-12/+14
2024-04-25minorchrislu1-1/+1
2024-04-25use parquet levels to construct rowchrislu1-28/+31
2024-04-25toRow conversion with levels infochrislu1-20/+18
2024-04-25file renamechrislu2-0/+0
2024-04-25printout test valueschrislu1-4/+8
2024-04-25add visitorchrislu2-0/+155
2024-04-24add field types but without valuechrislu1-1/+4
2024-04-24record type is also optionalchrislu1-0/+1
2024-04-24delete the generated filechrislu1-0/+4
2024-04-24add some noteschrislu1-0/+3
2024-04-24adjust testchrislu1-6/+10
2024-04-24works with single level of listchrislu4-60/+71
2024-04-24fix orderingchrislu1-1/+1
2024-04-22has a bugchrislu6-19/+317
2024-04-18all record fields are optional by defaultchrislu1-1/+4
2024-04-18add testchrislu2-9/+25
2024-04-18support list typechrislu1-10/+16
2024-04-18add schema builderchrislu2-0/+99
2024-04-17convert to parquet schema and valuechrislu3-15/+160
2024-04-12fix testchrislu1-13/+0
2024-04-12Create schema.gochrislu1-0/+32
2024-04-12add schema.protochrislu1-0/+98