Apache Ingestion Flume Kafka Bundle: kafka.common.LeaderNotAvailableException

[Cory Johns]

Merlijn,

Excellent, thank you for debugging that.  As you noted, the charms and
bundle for the Kafka services are not yet considered 100% complete.  One
aspect that we're looking to change is to switch to using Gobblin' to get
Kafka data into HDFS, which I think should address your concern regarding
the kafka-flume charm, as that would go away.

Another change that we'd like to make is to switch over to using layers and
the new interface-hadoop-plugin interface layer (
https://github.com/juju-solutions/interface-hadoop-plugin).  That interface
layer is brand new, so reviews on it would be welcome, and consider it
untested as well.

On Wed, Oct 28, 2015 at 5:31 PM, Merlijn Sebrechts <
merlijn.sebrechts at gmail.com> wrote:

> Hi all
>
>
> I'm happy to say that I found the bug in the Charm. It had to do with the
> default /etc/hosts file in lxc containers, and kafka not restarting after a
> config change. I'll be posting pull requests for these issues soon.
> The  /etc/hosts file actually looks like a juju bug. The hostname of the
> container is not in /etc/hosts, making the hostname unresolvable. I think
> this should be changed in the default lxc template, but for now the Charm
> can patch this.
>
> The hadoop plugin framework seems really cool! However, I'm skeptical
> about seperating the flume kafka source and flume hdfs sink into two
> seperate services. Isn't this a but overkill and waste of bandwith?
>
>
>
> Kind regards
> Merlijn
>
> 2015-10-28 17:45 GMT+01:00 Merlijn Sebrechts <merlijn.sebrechts at gmail.com>
> :
>
>> Hi all
>>
>>
>> I'm trying out the Apache ingestion flume Kafka bundle
>> <https://jujucharms.com/u/bigdata-dev/apache-ingestion-flume-kafka/3>,
>> however, I'm having a strange problem. After connecting the flume kafka
>> charm to kafka, kafka seems to break. When I consume existing topics with
>> messages in them, I get no result. When I try to add messages to a topic, I
>> get the  *kafka.common.LeaderNotAvailableException.*
>>
>> I see the bundle is still in bigdata-dev. Is this bundle supposed to
>> work, or does it still have known bugs? This functionality is critical to
>> our infrastructure, so I'd like to debug this problem and fix the Charm and
>> the bundle. However, my knowledge of these technologies is still very
>> small, so I don't really know where to start. Any thoughts and/or pointers?
>>
>>
>>
>> Kind regards
>> Merlijn
>>
>>
>>
>>
>
> --
> Bigdata mailing list
> Bigdata at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/bigdata
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bigdata/attachments/20151029/a79a3800/attachment.html>