private HashMap<String/* brokerName */, Set<MessageQueue>> buildProcessQueueTableByBrokerName() { HashMap<String, Set<MessageQueue>> result = new HashMap<String, Set<MessageQueue>>(); for (MessageQueue mq : this.processQueueTable.keySet()) { Set<MessageQueue> mqs = result.get(mq.getBrokerName()); if (null == mqs) { mqs = new HashSet<MessageQueue>(); result.put(mq.getBrokerName(), mqs); } mqs.add(mq); } return result; }
@Override public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) { Set<String> brokerSet = (Set<String>) arg; List<MessageQueue> filterMqs = new ArrayList<MessageQueue>(); for (MessageQueue queue : mqs) { if (brokerSet.contains(queue.getBrokerName())) { filterMqs.add(queue); } } int index = sendWhichQueue.getAndIncrement(); int pos = Math.abs(index) % filterMqs.size(); if (pos < 0) { pos = 0; } return filterMqs.get(pos); } }, traceBrokerSet, callback);
private List<MessageQueueForC> convertOffsetTable2OffsetList(Map<MessageQueue, Long> table) { List<MessageQueueForC> list = new ArrayList<>(); for (Entry<MessageQueue, Long> entry : table.entrySet()) { MessageQueue mq = entry.getKey(); MessageQueueForC tmp = new MessageQueueForC(mq.getTopic(), mq.getBrokerName(), mq.getQueueId(), entry.getValue()); list.add(tmp); } return list; }
private MessageQueue getOpQueue(MessageQueue messageQueue) { MessageQueue opQueue = opQueueMap.get(messageQueue); if (opQueue == null) { opQueue = new MessageQueue(TransactionalMessageUtil.buildOpTopic(), messageQueue.getBrokerName(), messageQueue.getQueueId()); opQueueMap.put(messageQueue, opQueue); } return opQueue; }
public MessageQueue selectOneMessageQueue(final String lastBrokerName) { if (lastBrokerName == null) { return selectOneMessageQueue(); } else { int index = this.sendWhichQueue.getAndIncrement(); for (int i = 0; i < this.messageQueueList.size(); i++) { int pos = Math.abs(index++) % this.messageQueueList.size(); if (pos < 0) pos = 0; MessageQueue mq = this.messageQueueList.get(pos); if (!mq.getBrokerName().equals(lastBrokerName)) { return mq; } } return selectOneMessageQueue(); } }
@Override public MessageQueue select(List<MessageQueue> list, Message message, Object o) { for (MessageQueue mq : list) { if (mq.getQueueId() == queueId && mq.getBrokerName().equals(broker2Name)) { return mq; } } return list.get(8); } });
@Override public MessageQueue select(List<MessageQueue> list, Message message, Object o) { for (MessageQueue mq : list) { if (mq.getQueueId() == queueId && mq.getBrokerName().equals(broker1Name)) { return mq; } } return list.get(0); } });
@Override public MessageQueue select(List<MessageQueue> list, Message message, Object o) { for (MessageQueue mq : list) { if (mq.getQueueId() == queueId && mq.getBrokerName().equals(broker2Name)) { return mq; } } return list.get(8); } });
@Override public MessageQueue select(List<MessageQueue> list, Message message, Object o) { for (MessageQueue mq : list) { if (mq.getQueueId() == queueId && mq.getBrokerName().equals(broker1Name)) { return mq; } } return list.get(0); } });
private void writeOp(Message message, MessageQueue mq) { MessageQueue opQueue; if (opQueueMap.containsKey(mq)) { opQueue = opQueueMap.get(mq); } else { opQueue = getOpQueueByHalf(mq); MessageQueue oldQueue = opQueueMap.putIfAbsent(mq, opQueue); if (oldQueue != null) { opQueue = oldQueue; } } if (opQueue == null) { opQueue = new MessageQueue(TransactionalMessageUtil.buildOpTopic(), mq.getBrokerName(), mq.getQueueId()); } putMessage(makeOpMessageInner(message, opQueue)); }
public long minOffset(MessageQueue mq) throws MQClientException { String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); if (null == brokerAddr) { this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); } if (brokerAddr != null) { try { return this.mQClientFactory.getMQClientAPIImpl().getMinOffset(brokerAddr, mq.getTopic(), mq.getQueueId(), timeoutMillis); } catch (Exception e) { throw new MQClientException("Invoke Broker[" + brokerAddr + "] exception", e); } } throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null); }
public long maxOffset(MessageQueue mq) throws MQClientException { String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); if (null == brokerAddr) { this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); } if (brokerAddr != null) { try { return this.mQClientFactory.getMQClientAPIImpl().getMaxOffset(brokerAddr, mq.getTopic(), mq.getQueueId(), timeoutMillis); } catch (Exception e) { throw new MQClientException("Invoke Broker[" + brokerAddr + "] exception", e); } } throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null); }
public long earliestMsgStoreTime(MessageQueue mq) throws MQClientException { String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); if (null == brokerAddr) { this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); } if (brokerAddr != null) { try { return this.mQClientFactory.getMQClientAPIImpl().getEarliestMsgStoretime(brokerAddr, mq.getTopic(), mq.getQueueId(), timeoutMillis); } catch (Exception e) { throw new MQClientException("Invoke Broker[" + brokerAddr + "] exception", e); } } throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null); }
public long searchOffset(MessageQueue mq, long timestamp) throws MQClientException { String brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); if (null == brokerAddr) { this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); brokerAddr = this.mQClientFactory.findBrokerAddressInPublish(mq.getBrokerName()); } if (brokerAddr != null) { try { return this.mQClientFactory.getMQClientAPIImpl().searchOffset(brokerAddr, mq.getTopic(), mq.getQueueId(), timestamp, timeoutMillis); } catch (Exception e) { throw new MQClientException("Invoke Broker[" + brokerAddr + "] exception", e); } } throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null); }
private Set<String> tryGetMessageQueueBrokerSet(DefaultMQProducerImpl producer, String topic) { Set<String> brokerSet = new HashSet<String>(); TopicPublishInfo topicPublishInfo = producer.getTopicPublishInfoTable().get(topic); if (null == topicPublishInfo || !topicPublishInfo.ok()) { producer.getTopicPublishInfoTable().putIfAbsent(topic, new TopicPublishInfo()); producer.getmQClientFactory().updateTopicRouteInfoFromNameServer(topic); topicPublishInfo = producer.getTopicPublishInfoTable().get(topic); } if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) { for (MessageQueue queue : topicPublishInfo.getMessageQueueList()) { brokerSet.add(queue.getBrokerName()); } } return brokerSet; } }
public boolean consumed(final MessageExt msg, final String group) throws RemotingException, MQClientException, InterruptedException, MQBrokerException { ConsumeStats cstats = this.examineConsumeStats(group); ClusterInfo ci = this.examineBrokerClusterInfo(); Iterator<Entry<MessageQueue, OffsetWrapper>> it = cstats.getOffsetTable().entrySet().iterator(); while (it.hasNext()) { Entry<MessageQueue, OffsetWrapper> next = it.next(); MessageQueue mq = next.getKey(); if (mq.getTopic().equals(msg.getTopic()) && mq.getQueueId() == msg.getQueueId()) { BrokerData brokerData = ci.getBrokerAddrTable().get(mq.getBrokerName()); if (brokerData != null) { String addr = brokerData.getBrokerAddrs().get(MixAll.MASTER_ID); if (RemotingUtil.socketAddress2String(msg.getStoreHost()).equals(addr)) { if (next.getValue().getConsumerOffset() > msg.getQueueOffset()) { return true; } } } } } return false; }
public boolean sendMessageBack(final MessageExt msg, final ConsumeConcurrentlyContext context) { int delayLevel = context.getDelayLevelWhenNextConsume(); try { this.defaultMQPushConsumerImpl.sendMessageBack(msg, delayLevel, context.getMessageQueue().getBrokerName()); return true; } catch (Exception e) { log.error("sendMessageBack exception, group: " + this.consumerGroup + " msg: " + msg.toString(), e); } return false; }
private long fetchConsumeOffsetFromBroker(MessageQueue mq) throws RemotingException, MQBrokerException, InterruptedException, MQClientException { FindBrokerResult findBrokerResult = this.mQClientFactory.findBrokerAddressInAdmin(mq.getBrokerName()); if (null == findBrokerResult) { this.mQClientFactory.updateTopicRouteInfoFromNameServer(mq.getTopic()); findBrokerResult = this.mQClientFactory.findBrokerAddressInAdmin(mq.getBrokerName()); } if (findBrokerResult != null) { QueryConsumerOffsetRequestHeader requestHeader = new QueryConsumerOffsetRequestHeader(); requestHeader.setTopic(mq.getTopic()); requestHeader.setConsumerGroup(this.groupName); requestHeader.setQueueId(mq.getQueueId()); return this.mQClientFactory.getMQClientAPIImpl().queryConsumerOffset( findBrokerResult.getBrokerAddr(), requestHeader, 1000 * 5); } else { throw new MQClientException("The broker[" + mq.getBrokerName() + "] not exist", null); } } }
private MessageQueue getOpQueueByHalf(MessageQueue halfMQ) { MessageQueue opQueue = new MessageQueue(); opQueue.setTopic(TransactionalMessageUtil.buildOpTopic()); opQueue.setBrokerName(halfMQ.getBrokerName()); opQueue.setQueueId(halfMQ.getQueueId()); return opQueue; }
public void unlock(final MessageQueue mq, final boolean oneway) { FindBrokerResult findBrokerResult = this.mQClientFactory.findBrokerAddressInSubscribe(mq.getBrokerName(), MixAll.MASTER_ID, true); if (findBrokerResult != null) { UnlockBatchRequestBody requestBody = new UnlockBatchRequestBody(); requestBody.setConsumerGroup(this.consumerGroup); requestBody.setClientId(this.mQClientFactory.getClientId()); requestBody.getMqSet().add(mq); try { this.mQClientFactory.getMQClientAPIImpl().unlockBatchMQ(findBrokerResult.getBrokerAddr(), requestBody, 1000, oneway); log.warn("unlock messageQueue. group:{}, clientId:{}, mq:{}", this.consumerGroup, this.mQClientFactory.getClientId(), mq); } catch (Exception e) { log.error("unlockBatchMQ exception, " + mq, e); } } }