Why is spark connected to zookeeper in the "Hadoop Spark" bundle?

[Konstantinos Tsakalozos]

Hi Merlijn,

You are right Zookeeper is used only for HA in the standalone mode of
Spark. In the initial setup of the hadoop-spark bundle Zookeeper is not
used. However, we decided to have Zookeeper there to accommodate the case
where you start with hadoop-spark bundle and you switch to the standalone
execution mode. If I remember correctly you can also override the execution
mode during job submission.

Thanks,
Konstantinos

On Tue, Jan 3, 2017 at 3:25 PM, Merlijn Sebrechts <
merlijn.sebrechts at gmail.com> wrote:

> Hi all
>
>
> According to the bigtop spark charm[1] zookeeper is used for standalone HA
> mode. If this is the case, then why does the hadoop-spark[2] bundle include
> a relationship between Spark and Zookeeper when spark_execution_mode
> is"yarn-client"?
>
> [1] https://jujucharms.com/spark/xenial
> [2] https://jujucharms.com/hadoop-spark/
>
>
>
> Kind regards
> Merlijn
>
> --
> Bigdata mailing list
> Bigdata at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/
> mailman/listinfo/bigdata
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bigdata/attachments/20170103/f37a677a/attachment.html>