aboutsummaryrefslogtreecommitdiff
path: root/weed/mq/client/sub_client/subscribe.go
AgeCommit message (Collapse)AuthorFilesLines
2025-03-09Accumulated changes for message queue (#6600)Chris Lu1-6/+6
* rename * set agent address * refactor * add agent sub * pub messages * grpc new client * can publish records via agent * send init message with session id * fmt * check cancelled request while waiting * use sessionId * handle possible nil stream * subscriber process messages * separate debug port * use atomic int64 * less logs * minor * skip io.EOF * rename * remove unused * use saved offsets * do not reuse session, since always session id is new after restart remove last active ts from SessionEntry * simplify printing * purge unused * just proxy the subscription, skipping the session step * adjust offset types * subscribe offset type and possible value * start after the known tsns * avoid wrongly set startPosition * move * remove * refactor * typo * fix * fix changed path
2025-01-20Add message queue agent (#6463)Chris Lu1-1/+17
* scaffold message queue agent * adjust proto, add mq_agent * add agent client implementation * remove unused function * agent publish server implementation * adding agent
2024-05-27balance subscriberschrislu1-0/+7
need to ensure the offsets are persisted
2024-05-23coordinator receives unassignment ackchrislu1-0/+7
2024-05-21client side stop partition subscribing if unassignedchrislu1-3/+13
2024-05-21refactorchrislu1-30/+35
ToDo 1. handle unassignment on client side 2. get notified of completed unassignment 3. in consumer_group.go, balance and send unassignment
2024-05-20remove ProcessorConfigurationchrislu1-1/+1
2024-05-20go fmtchrislu1-1/+0
2024-05-20track offsetchrislu1-1/+1
2024-05-16consumer acks received messageschrislu1-1/+6
2024-05-14subscriber receives partitions and dispatch to processorschrislu1-0/+77
2023-12-31clean up dead codechrislu1-42/+0
2023-12-28subscriber keep connected to the balancerchrislu1-0/+5
2023-12-11Merge accumulated changes related to message queue (#5098)Chris Lu1-1/+16
* balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * balance partitions on brokers * prepare topic partition first and then publish, move partition * purge unused APIs * clean up * adjust logs * add BalanceTopics() grpc API * configure topic * configure topic command * refactor * repair missing partitions * sequence of operations to ensure ordering * proto to close publishers and consumers * rename file * topic partition versioned by unixTimeNs * create local topic partition * close publishers * randomize the client name * wait until no publishers * logs * close stop publisher channel * send last ack * comments * comment * comments * support list of brokers * add cli options * Update .gitignore * logs * return io.eof directly * refactor * optionally create topic * refactoring * detect consumer disconnection * sub client wait for more messages * subscribe by time stamp * rename * rename to sub_balancer * rename * adjust comments * rename * fix compilation * rename * rename * SubscriberToSubCoordinator * sticky rebalance * go fmt * add tests * tracking topic=>broker * merge * comment
2023-10-02add subscriber coordinatorchrislu1-0/+5
2023-10-01refactorchrislu1-58/+14
2023-09-30adjust mq.protochrislu1-14/+14
2023-09-04api for subchrislu1-11/+12
2023-09-01can pub and subchrislu1-19/+62
2023-08-28connect and publishchrislu1-0/+28
generated by cgit v1.2.3 (git 2.25.1) at 2025-12-26 10:00:33 +0000