Apache Ingestion Flume Kafka Bundle: kafka.common.LeaderNotAvailableException

[Merlijn Sebrechts]

Hi all


I'm happy to say that I found the bug in the Charm. It had to do with the
default /etc/hosts file in lxc containers, and kafka not restarting after a
config change. I'll be posting pull requests for these issues soon.
The  /etc/hosts file actually looks like a juju bug. The hostname of the
container is not in /etc/hosts, making the hostname unresolvable. I think
this should be changed in the default lxc template, but for now the Charm
can patch this.

The hadoop plugin framework seems really cool! However, I'm skeptical about
seperating the flume kafka source and flume hdfs sink into two seperate
services. Isn't this a but overkill and waste of bandwith?



Kind regards
Merlijn

2015-10-28 17:45 GMT+01:00 Merlijn Sebrechts <merlijn.sebrechts at gmail.com>:

> Hi all
>
>
> I'm trying out the Apache ingestion flume Kafka bundle
> <https://jujucharms.com/u/bigdata-dev/apache-ingestion-flume-kafka/3>,
> however, I'm having a strange problem. After connecting the flume kafka
> charm to kafka, kafka seems to break. When I consume existing topics with
> messages in them, I get no result. When I try to add messages to a topic, I
> get the  *kafka.common.LeaderNotAvailableException.*
>
> I see the bundle is still in bigdata-dev. Is this bundle supposed to work,
> or does it still have known bugs? This functionality is critical to our
> infrastructure, so I'd like to debug this problem and fix the Charm and the
> bundle. However, my knowledge of these technologies is still very small, so
> I don't really know where to start. Any thoughts and/or pointers?
>
>
>
> Kind regards
> Merlijn
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bigdata/attachments/20151028/03d5ed2b/attachment.html>