[CoLoCo] Job/batch management

[David Overcash]

Upstart has gotten a lot of attention recently because it allows just what
you're looking for...  running scripts on the condition of others completing
and whatnot.  This is exactly why Ubuntu has moved over to upstart and is
just starting to take advantage of it.  Also, it is completely backwards
compatible with cron scripts (afaik) so you should be able to migrate at
your own pace.

For more information check out:
http://upstart.ubuntu.com/

Zelut (on irc.freenode.net ) is fairly well-educated concerning upstart and
you could probably ask him for more direction on where to research and work
with upstart (besides the obvious sources such as google).

Cheers,
David

On Tue, Mar 11, 2008 at 6:35 PM, Mitch Mahan <mitch at kci.net> wrote:

>  You could probably jerry-rig Nagios to do this for you.
>
>
>
> Create a 'check' (in perl?) which runs your 'program' and you can use
> Nagios's built-in pager/up-time/historical recording for your needs.
>
>
>
> Just an idea,
>
> - Mitch
>
>
>  ------------------------------
>
> *From:* ubuntu-us-co-bounces at lists.ubuntu.com [mailto:
> ubuntu-us-co-bounces at lists.ubuntu.com] *On Behalf Of *David J W
> *Sent:* Sunday, March 09, 2008 5:12 PM
> *To:* ubuntu-us-co at lists.ubuntu.com
> *Subject:* [CoLoCo] Job/batch management
>
>
>
> Hello,
>     First off an introduction as this is my first post:
>
>       I work in Denver as a multi-hat developer: php/python/bash/c++
> developer, linux system admin, and a intermediate MySQL database
> administrator, and also my current company's R&D developer.  I've been
> working with *nix since 1996-97 when a c++ programming course I took was
> taught exclusively on v220 terminals connected to a mini-computer.
>
> I'm on week 2 or maybe 3 of doing research for consolidating my company's
> small and growing horde of cronjobs... its well over 150 script at this
> point.  The company CEO has directed me to start research and research I
> have done.
>
> Why crontab sucks for what I need:
>
>   Cron does a fantastic job at what it does, run something at X time.
>   The problem is, what if I need alt_X to run if X exists non-zero....
> alright so I could do:
>     x || alt_x
>
>    Now what if I need y to run after x if its successfull
>
>    (x && y ) || alt_x
>
>    Well thats not right?  x could perform correctly then have y fail,
> causing alt_x to run and possibly redo important operations/data.
>
>    (x || alt_x) && y  - except what if I don't want y to run if alt_x ends
> up running?
>
>
>    Now to make things fun.... lets say there is a chain of 4-5 scripts
> that must run conditionally.  I'm pretty decent with simple and/or logic,
> but I know this would confuse anyone else to read and decypher.  Making it
> so only I and maybe one or two disciples can maintain the abomination that
> is the company's backend underworld.
>
> Plan B (partially in effect)
>    Complex jobs are written as bash scripts.  This is a step up from the
> above, I can build a job with variables, capture output to temp files, and
> use mail or sendmail to make customized emails.  Whats lacking is
> resuability.  Sure I can write shell script libraries to reuse certain
> things, like a bash wrapper function to handle individual scripts.... but
> adding in context sensitive checks "has the word failure, critical, OMFG,
> etc" would be hard coded in.
>
> Plan C ( what I am aiming for)
>    So I'm looking for a cronjob equivalent scheduling system that can be
> configured by information stored in a database (MySQL, sqllite, whatever).
> This phantom program would have the following specs:
>
>     Run any process/job as either root or a specific user account
>     Record the job fitness ( maximum memory, time stats (rtime, utime,
> overall)
>     Historical recording ( save input to DB or serialized filestore )
>     Context sensitive output handling
>     Digest email capability ( instead of 120-140 emails a day, I'd be
> estatic with like 30)
>     SMP/pager capability
>
>    I've started prototyping the above in python, but I figure to get this
> to a stable enough product, it would take me 3-4 days of development time
> then a residual 5-15 hours for the next month or two in responsive
> debugging.  From my prototypes, I'd have a daemonized python script that
> would sleep every 60 seconds, check the configuration database for changes
> and execute jobs as needed.   Job execution would consist of checking the
> jobs credentials ( if run as X, fork, lock records to job handler PID,
> descend privileges to specified user, chdir to script path, wrap the job in
> a subprocess, respond to job events as needed ( stderr, ! zero return,
> excessive run time, etc ).
>
>     My question is, anyone see a better way, or a OSS project out there
> that could do the above for me... even a beta project written in python or
> c++ could be workable by me.  If you haven't noticed the varying levels of
> sarcasm, I'm somewhat stretched thin right now.
>
>     Thanks,
>             David
>
> --
> Ubuntu-us-co mailing list
> Ubuntu-us-co at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-us-co
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/ubuntu-us-co/attachments/20080311/40fc325c/attachment.htm