problem when building a charm for giraph

Tue Nov 22 14:31:02 UTC 2016

Cool! The error that you are getting now says that the giraph charm has no
giraph relation [0].

Looking at the metadata.yaml, you are using the mahout interface to relate
to the hadoop client. So the relation call should look like:
cls.d.relate('giraph:mahout', 'client:mahout') . This will work for now
since both Mahout and Giraph just want to add jars on the hadoop-client
charm. However, on the long run we should refactor charms such as
hadoop-client, spark, pig, etc, so that they have an interface exactly for
that purpose (adding jars to the file system). For now you have two options
either keep using the mahout interface or use the "juju-info" [1] interface
that is present in all charms. Note, however, that if you do use
"juju-info" your charm would be a subordinate to any other charm even if
that does not make sense.

[0]
http://pythonhosted.org/amulet/amulet.html?highlight=relate#amulet.deployer.Deployment.relate
[1] https://jujucharms.com/docs/2.0/authors-implicit-relations

On Tue, Nov 22, 2016 at 4:02 PM, Panagiotis Liakos <p.liakos at di.uoa.gr>
wrote:

> Thanks Konstantinos, you were right. Now the script progresses a little
> further:
>
> $ python3 ./tests/01-giraph-test.py
> E
> ======================================================================
> ERROR: setUpClass (__main__.TestDeploy)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "./tests/01-giraph-test.py", line 42, in setUpClass
>     cls.d.relate('giraph:giraph', 'client:mahout')
>   File "/usr/lib/python3/dist-packages/amulet/deployer.py", line 431, in
> relate
>     raise ValueError('%s does not exist for %s' % (rel, srv))
> ValueError: giraph does not exist for giraph
>
> ----------------------------------------------------------------------
> Ran 0 tests in 2.285s
>
> FAILED (errors=1)
>
>
> Is this a namespace issue? Any ideas?
>
> Panagiotis
>
> 2016-11-22 14:16 GMT+02:00 Konstantinos Tsakalozos
> <kos.tsakalozos at canonical.com>:
> > Hi Panagiotis,
> >
> > The 404 error seems to be because the Giraph charm is not yet promulgated
> > (your charm currently lives in cs:~panagiotisl/giraph). Could you try the
> > same test only this time with a charm revision you have pushed under your
> > namespace in line
> > https://github.com/panagiotisl/bigtop/blob/master/bigtop-packages/src/
> charm/giraph/layer-giraph/tests/01-giraph-test.py#L29
> > ? You can also use a local path pointing to where your charm build output
> > is.
> >
> > Glad the multiple SLF4J bindings are not causing any problem.
> >
> > Thanks,
> > Konstantinos
> >
> > On Tue, Nov 22, 2016 at 1:15 PM, Panagiotis Liakos <p.liakos at di.uoa.gr>
> > wrote:
> >>
> >> Dear Merlijn and Konstantinos,
> >>
> >> Thanks a lot for the very valuable information. I will have another
> >> look at the documentation for writing tests and debugging and I'll try
> >> to add some logs in my test.
> >>
> >> When I execute my smoke-test (through ssh to giraph) I get the
> >> following (which is exactly what I expect):
> >> https://pastebin.ubuntu.com/23516317/
> >>
> >> When I execute the 01-giraph-test.py (and after I installed amulet) I
> >> get the following: https://pastebin.ubuntu.com/23516415/
> >>
> >> The URL mentioned in the HTTPError exception is valid for mahout but
> >> not for giraph so I guess I am requesting something that is not there
> >> (yet).
> >>
> >> Thanks again, I'll keep you updated.
> >>
> >> Panagiotis
> >>
> >>
> >> 2016-11-22 12:08 GMT+02:00 Konstantinos Tsakalozos
> >> <kos.tsakalozos at canonical.com>:
> >> > Hi Panagiotis,
> >> >
> >> > Merlijn is right, the output of the smoke-test script is available in
> >> > the
> >> > juju debug logs (juju debug-log). Here is what I got when running your
> >> > smoke-test: http://pastebin.ubuntu.com/23516167/ (multiple SLF4J
> >> > bindings)
> >> > It seems the slf4j-log4j12-1.7.5.jar jar from Giraph is slightly older
> >> > that
> >> > the slf4j-log4j12-1.7.10.jar from Hadoop.
> >> >
> >> > In order to add some output to the smoke-test action you could call
> >> > "action
> >> > set" and/or "action fail" as we do here
> >> >
> >> > https://github.com/panagiotisl/bigtop/blob/
> master/bigtop-packages/src/charm/hbase/layer-hbase/actions/smoke-test
> >> >
> >> > Thank you for your work,
> >> > Konstantinos
> >> >
> >> >
> >> > On Tue, Nov 22, 2016 at 11:06 AM, Merlijn Sebrechts
> >> > <merlijn.sebrechts at gmail.com> wrote:
> >> >>
> >> >> Hi Panagiotis Liakos
> >> >>
> >> >>
> >> >> Cool that you're charming Giraph! Feel free to let me know when you
> >> >> have a
> >> >> working prototype, some of my colleagues are interested in using
> >> >> Giraph.
> >> >>
> >> >> Now for your question; tests and actions are two different things.
> >> >>
> >> >> Actions are similar to hooks; they run on the charm unit itself. The
> >> >> mahout smoke-test is an action. You run them using `run-action`, you
> >> >> can
> >> >> debug them using `juju debug-hooks` and all output gets logged by
> juju
> >> >> (you
> >> >> can see the output in `juju debug-log`). More info on actions:
> >> >> https://jujucharms.com/docs/stable/actions
> >> >> Tests are scripts that run on your own machine that deploy and test
> >> >> charms. You run them using bundletester. More info on tests:
> >> >> https://jujucharms.com/docs/2.0/developer-testing
> >> >>
> >> >> So if you want to alter the `smoke-test` action; you should alter
> this
> >> >> file:
> >> >>
> >> >> https://github.com/apache/bigtop/blob/master/bigtop-
> packages/src/charm/mahout/layer-mahout/actions/smoke-test
> >> >>
> >> >> The actual tests of the Mahout Charm use the smoke-test action to
> >> >> verify
> >> >> the deployment. That is what you see at line 52:
> >> >>
> >> >> https://github.com/apache/bigtop/blob/master/bigtop-
> packages/src/charm/mahout/layer-mahout/tests/01-mahout-test.py#L52
> >> >> `self.mahout.run_action('smoke-test')` calls `juju run-action` in
> the
> >> >> background.
> >> >>
> >> >> Does that help you a bit?
> >> >>
> >> >>
> >> >>
> >> >> Kind regards
> >> >> Merlijn
> >> >>
> >> >> 2016-11-22 9:42 GMT+01:00 Panagiotis Liakos <p.liakos at di.uoa.gr>:
> >> >>>
> >> >>> Hi all,
> >> >>>
> >> >>> I am trying to build a charm for Apache Giraph. I have used the
> mahout
> >> >>> charm (
> >> >>>
> >> >>> https://github.com/apache/bigtop/tree/master/bigtop-
> packages/src/charm/mahout
> >> >>> ) and made the necessary changes to deploy giraph instead of mahout.
> >> >>>
> >> >>> Deployment seems to work fine and I am able to submit giraph jobs
> >> >>> through an ssh connection.
> >> >>>
> >> >>> Now I am trying to create a test similar to the smoke-test of the
> >> >>> mahout charm (
> >> >>>
> >> >>> https://github.com/apache/bigtop/blob/master/bigtop-
> packages/src/charm/mahout/layer-mahout/actions/smoke-test
> >> >>> )
> >> >>>
> >> >>> I have successfully run the script that I have included in my
> >> >>> smoke-test through ssh and I
> >> >>> would expect that the test would also execute without errors.
> >> >>> However, I am not at all familiar with the 'run-action' process of
> >> >>> juju and I have no idea how I should alter the following file:
> >> >>>
> >> >>>
> >> >>> https://github.com/apache/bigtop/blob/master/bigtop-
> packages/src/charm/mahout/layer-mahout/tests/01-mahout-test.py
> >> >>>
> >> >>> In particular, I am puzzled by the way the result status is set in
> >> >>> this example. Simply changing the references of mahout to giraph
> does
> >> >>> not seem to work. You can see my file here:
> >> >>>
> >> >>>
> >> >>> https://github.com/panagiotisl/bigtop/blob/
> master/bigtop-packages/src/charm/giraph/layer-giraph/
> tests/01-giraph-test.py
> >> >>>
> >> >>> When I execute show-action-output I receive very limited
> information:
> >> >>> message: exit status 1
> >> >>> status: failed
> >> >>> timing:
> >> >>>   completed: 2016-11-21 15:48:01 +0000 UTC
> >> >>>   enqueued: 2016-11-21 15:47:38 +0000 UTC
> >> >>>   started: 2016-11-21 15:47:40 +0000 UTC
> >> >>>
> >> >>> Is there a way I can view the full output of the 'smoke-test'
> >> >>> execution?
> >> >>>
> >> >>> Thank you,
> >> >>> Panagiotis Liakos
> >> >>>
> >> >>> --
> >> >>> Juju mailing list
> >> >>> Juju at lists.ubuntu.com
> >> >>> Modify settings or unsubscribe at:
> >> >>> https://lists.ubuntu.com/mailman/listinfo/juju
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Juju mailing list
> >> >> Juju at lists.ubuntu.com
> >> >> Modify settings or unsubscribe at:
> >> >> https://lists.ubuntu.com/mailman/listinfo/juju
> >> >>
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20161122/18c09881/attachment.html>