Kafka - 如何收看新消息

Question

I have PHP application that consumes Kafka messages. The problem is how to know that there are new messages in Kafka? First solution is to create consumer in PHP then run it in a loop to check new messages. Something like this

<?php

namespace MyAppBundle\Command;

use MyAppBundle\EventSourcing\EventSerializer\JSONEventSerializer;
use MyAppBundle\Service\EventProjectorService;
use MyAppBundle\Service\KafkaService;
use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Process\Exception\RuntimeException;

class EventCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this
            ->setName('events:fetch');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        /** @var KafkaService $kafkaService */
        $kafkaService = $this->getContainer()->get('store_locator.kafka_service');
        /** @var EventProjectorService $eventProjector */
        $eventProjector = $this->getContainer()->get('store_locator.event_projector');

        while(1){
          $messages = $kafkaService->fetchEvents();

          foreach ($messages as $message) {
              $eventProjector->aggregate($message);
          }
        }
        $output->writeln("Finish");
    }
}

But I don't like it... Is there any other way?

If no better way, how to keep it running? For instance when something fail.

Answer 1

As far as I know there are no better ways than to loop infinitely and keep checking for new messages. A common approach is to have the task kill itself after a set amount of time or number of iterations and then use something like supervisord to detect the death and resurrect the consumer in order to prevent it eating up all your resources.