Anything providing Zookeeper?
Tom Barber
tom at analytical-labs.com
Tue May 31 15:16:43 UTC 2016
Which charms?
--------------
Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316
(Thanks to the Saiku community we reached our Kickstart
<http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
goal, but you can always help by sponsoring the project
<http://www.meteorite.bi/products/saiku/sponsorship>)
On 31 May 2016 at 16:16, Merlijn Sebrechts <merlijn.sebrechts at gmail.com>
wrote:
> Cool! I'm interested in these Charms. Are they online somewhere?
>
> 2016-05-31 16:59 GMT+02:00 Tom Barber <tom at analytical-labs.com>:
>
>> Just to clear up this thread.
>>
>> Thanks to Kos we tracked the the condition, where Drill was setting its
>> config before the whole quorum was available.
>>
>> For reference I ended up implementing something a bit like:
>>
>> https://github.com/juju-solutions/layer-apache-spark/blob/master/reactive/spark.py#L159
>>
>> Thanks guys, I now have a working basic Apache Drill cluster \o/
>>
>> Tom
>>
>> --------------
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>> goal, but you can always help by sponsoring the project
>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>
>> On 31 May 2016 at 13:34, Konstantinos Tsakalozos <
>> kos.tsakalozos at canonical.com> wrote:
>>
>>> Hm... I do not see anything wrong with this part of the code. Looking at
>>> [0] you should be getting all three Zookeeper units/conversations.
>>>
>>> Would it be possible to share the charm you are building so that I can
>>> test it locally or ping us on IRC channel #juju at freenode for a pair
>>> debugging session?
>>>
>>>
>>> [0]
>>> https://github.com/juju-solutions/interface-zookeeper/blob/master/requires.py#L40
>>>
>>> On Tue, May 31, 2016 at 3:19 PM, Tom Barber <tom at analytical-labs.com>
>>> wrote:
>>>
>>>> Okay, slightly different question:
>>>>
>>>> zookeeper/0 active idle 2.0-beta7 8
>>>> 2181/tcp,9998/tcp 10.106.143.126 Ready (3 zk units)
>>>> zookeeper/1 active idle 2.0-beta7 9
>>>> 2181/tcp,9998/tcp 10.106.143.234 Ready (3 zk units)
>>>> zookeeper/2 active idle 2.0-beta7 10
>>>> 2181/tcp,9998/tcp 10.106.143.133 Ready (3 zk units)
>>>>
>>>> I have 3 ZK units and I used the example here:
>>>> https://github.com/juju-solutions/interface-zookeeper for the client
>>>> config so my function looks like:
>>>>
>>>> @when('zookeeper.ready')
>>>> @when_not('charm.configured')
>>>> def configure(zookeeper):
>>>> zklist = ''
>>>> for zk_unit in zookeeper.zookeepers():
>>>> zklist += add_zookeeper(zk_unit['host'], zk_unit['port'])
>>>> zklist = zklist[:-1]
>>>>
>>>> But it only iterates once even though 3 are listed:
>>>>
>>>> a) did I do something wrong
>>>> b) is this due to some leader election thing and is actually correct?
>>>> c) another answer?
>>>>
>>>> Thanks
>>>>
>>>> Tom
>>>>
>>>> --------------
>>>>
>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>> Tel: +44(0)5603641316
>>>>
>>>> (Thanks to the Saiku community we reached our Kickstart
>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>> goal, but you can always help by sponsoring the project
>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>
>>>> On 31 May 2016 at 12:37, Tom Barber <tom at analytical-labs.com> wrote:
>>>>
>>>>> Fair enough, that resolves that then!
>>>>>
>>>>> Thanks Kos.
>>>>>
>>>>> Tom
>>>>>
>>>>> --------------
>>>>>
>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>> Tel: +44(0)5603641316
>>>>>
>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>> goal, but you can always help by sponsoring the project
>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>
>>>>> On 31 May 2016 at 12:25, Konstantinos Tsakalozos <
>>>>> kos.tsakalozos at canonical.com> wrote:
>>>>>
>>>>>> Ah, it is clear now, thanks.
>>>>>>
>>>>>> The Apache Hadoop charms use Zookeeper when in HA mode. The namenode
>>>>>> is the charm that relates to Zookeeper [0,1]. So to answer your question
>>>>>> the Hadoop charms use ZK but they do not carry ZK under the hood. ZK
>>>>>> is treated as separate service/charm.
>>>>>>
>>>>>>
>>>>>> [0] https://github.com/juju-solutions/layer-apache-hadoop-namenode
>>>>>> /blob/master/metadata.yaml
>>>>>> [1] https://github.com/juju-solutions/layer-apache-hadoop-namenode
>>>>>> /blob/master/reactive/namenode_failover.py
>>>>>>
>>>>>> On Tue, May 31, 2016 at 1:50 PM, Tom Barber <tom at analytical-labs.com>
>>>>>> wrote:
>>>>>>
>>>>>>> No worries Kos
>>>>>>>
>>>>>>> I'm rubbish at explaining stuff properly!
>>>>>>>
>>>>>>> My query was more along the lines of "I can see the ZK charms
>>>>>>> available, but surely the Hadoop charms use ZK under the hood somewhere, do
>>>>>>> any of them expose the ZK relation so you can hook into that", because it
>>>>>>> seems like overkill running a second ZK setup if one already exists inside
>>>>>>> an existing Juju Hadoop deployment.
>>>>>>>
>>>>>>> Tom
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --------------
>>>>>>>
>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>> Tel: +44(0)5603641316
>>>>>>>
>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>
>>>>>>> On 31 May 2016 at 11:47, Konstantinos Tsakalozos <
>>>>>>> kos.tsakalozos at canonical.com> wrote:
>>>>>>>
>>>>>>>> I am sorry Tom, I replied to your first email.
>>>>>>>>
>>>>>>>> Not sure I fully get your question. Not sure what "this" refers to
>>>>>>>> in "Is there a way to hook this".
>>>>>>>>
>>>>>>>> Charms interacting with a Zookeeper quorum need to use this
>>>>>>>> interface: http://interfaces.juju.solutions/interface/zookeeper/ .
>>>>>>>> Through the zookeeper interface you can be informed about zookeeper units
>>>>>>>> joining and departing the quorum.
>>>>>>>>
>>>>>>>> You can have a single Zookeeper service that you can scale to an
>>>>>>>> odd number of units. Then you can have any charm "requiring" the above
>>>>>>>> relation use the zookeeper service. Typically you would need to construct a
>>>>>>>> connection string or edit some configuration file within your charm in
>>>>>>>> order to reference the zookeeper units. Have a look at https://
>>>>>>>> github.com/juju-solutions/layer-apache
>>>>>>>> -spark/blob/master/lib/charms/layer/apache_spark.py#L175
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Konstantinos
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 31, 2016 at 1:28 PM, Konstantinos Tsakalozos <
>>>>>>>> kos.tsakalozos at canonical.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Tom,
>>>>>>>>>
>>>>>>>>> Have you looked at: https://jujucharms.com/apache
>>>>>>>>> -zookeeper/trusty/3
>>>>>>>>> The apache-zookeeper provides the zookeeper interface, see
>>>>>>>>> https://api.jujucharms.com/charmstore/v5/trusty/apache
>>>>>>>>> -zookeeper-3/archive/metadata.yaml
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Konstantinos
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 31, 2016 at 1:13 PM, Tom Barber <
>>>>>>>>> tom at analytical-labs.com> wrote:
>>>>>>>>>
>>>>>>>>>> Morning
>>>>>>>>>>
>>>>>>>>>> There is a zookeeper interface and a bucket load of Big Data
>>>>>>>>>> stuff which all must run ZK under the hood. Is there anything that provides
>>>>>>>>>> the relationit so my charm can hook into ZK? I can't see anything obvious
>>>>>>>>>> in the Charm Store.
>>>>>>>>>>
>>>>>>>>>> Tom
>>>>>>>>>> --------------
>>>>>>>>>>
>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>>>
>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Juju mailing list
>>>>>>>>>> Juju at lists.ubuntu.com
>>>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>> --
>> Juju mailing list
>> Juju at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160531/b58b9574/attachment.html>
More information about the Juju
mailing list