Apache Drill Charm

[Tom Barber]

Hi folks:

http://spicule.co.uk/2016/06/02/apache-drill-juju.html

I've not slept enough due to ill kids and travel, so it might be complete
nonsense, but here is a brain dump about some of Drills bits and pieces.

The charm works well enough for basic use and you can certainly connect it
to a bunch of stuff, but I need to add a load more config/relation/action
stuff to solve the automation aspect.

Anyway, fill ya boots.

Tom

--------------

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart
<http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
goal, but you can always help by sponsoring the project
<http://www.meteorite.bi/products/saiku/sponsorship>)

On 1 June 2016 at 13:47, Merlijn Sebrechts <merlijn.sebrechts at gmail.com>
wrote:

> Yeah, we should see if we can do the same with YARN...
>
> 2016-06-01 14:42 GMT+02:00 Marco Ceppi <marco.ceppi at canonical.com>:
>
>> +1 to using % logic to make it scale across any sized instance. Awesome
>> stuff!
>>
>> On Wed, Jun 1, 2016 at 6:28 AM Tom Barber <tom at analytical-labs.com>
>> wrote:
>>
>>> Okay latest "stable" build has RAM config options.
>>>
>>> Drill ships with defaults of 8GB and 3GB but I didn't want it to die on
>>> EC2 Large etc boxes that dont have that much. So I added a bit of logic,
>>> you can (I hope) add XXG and it will use that fixed amount, or you can, as
>>> it ships, tell it you want XX% MAX and XX% Heap and it will try and figure
>>> that out and stand you up a drill box.
>>>
>>>
>>> Tom
>>>
>>> --------------
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>> goal, but you can always help by sponsoring the project
>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>
>>> On 1 June 2016 at 00:50, Tom Barber <tom at analytical-labs.com> wrote:
>>>
>>>> Oh, also currently the RAM is clamped down real low
>>>> in /opt/drill/conf/drill-env.sh I will set it back to some sane defaults
>>>> tomorrow as soon as I put the RAM limits into the config options, just ran
>>>> out of time this evening!
>>>>
>>>> --------------
>>>>
>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>> Tel: +44(0)5603641316
>>>>
>>>> (Thanks to the Saiku community we reached our Kickstart
>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>> goal, but you can always help by sponsoring the project
>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>
>>>> On 31 May 2016 at 23:50, Tom Barber <tom at analytical-labs.com> wrote:
>>>>
>>>>> Here we are then, for Merlijn and anyone else interested in SQL
>>>>> interfaces to big data/NOSQL stuff.
>>>>>
>>>>> This is less than a days effort, so its patchy at best:
>>>>>
>>>>> https://jujucharms.com/u/spicule/drillbit
>>>>>
>>>>> For those of you who don't know Apache Drill, it will let you run SQL
>>>>> querys over, CSV/JSON data, MongoDB, HBase, Parquet files etc in a number
>>>>> of different locations. Basically its a great way for analysts who use
>>>>> "traditional" SQL tools to leverage data stored within NOSQL solutions.
>>>>>
>>>>> Getting something like this into the CS has been high on my list of
>>>>> priorities for Saiku Analytics as it suddenly offers up loads of new
>>>>> connection prospects(of course I can do this manually in the past, but this
>>>>> is what Juju is for, right?)
>>>>>
>>>>> You need to deploy a ZK node (or 3) and connect it to that and OpenJDK
>>>>> to run it. Currently its relations-lite, the only one in there is a MongoDB
>>>>> test relation that will connect Drill to your MongoDB cluster if you run
>>>>> one, but there will be actions and relations coming shortly for other
>>>>> stuff. Also its missing a fat load of config options, again, coming soon.
>>>>> You can set all of this stuff pretty simply though and there is a web
>>>>> console for queries/connections etc, on http://serverip:8047/
>>>>>
>>>>> Test it, let me know what you think or its its entirely broken. Adding
>>>>> this and a few other NOSQL SQL interfaces is key to people being able to
>>>>> sanely consume all this big data stuff that is great on Juju. Thats not a
>>>>> knock on the Zeppelin guys, but companies will continue to use SQL for a
>>>>> long time to come so we should service that requirement.
>>>>>
>>>>> Tom
>>>>>
>>>>> --------------
>>>>>
>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>> Tel: +44(0)5603641316
>>>>>
>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>> goal, but you can always help by sponsoring the project
>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>
>>>>
>>>>
>>> --
>>> Juju mailing list
>>> Juju at lists.ubuntu.com
>>> Modify settings or unsubscribe at:
>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>
>>
>> --
>> Juju mailing list
>> Juju at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/juju
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160603/2620e6bf/attachment.html>