utils/fslock needs to DIAF

[Andrew Wilkins]

On Tue, Dec 1, 2015 at 8:57 AM David Cheney <david.cheney at canonical.com>
wrote:

> fcntl won't work in threaded go applications, it barely works in non
> threaded applications.
>

This is news to me. I've used it plenty in the past, in multi-threaded
programs. Please point me at relevant literature that explains where it
doesn't work.


> I'm not interested in "doesn't work on windows" arguments. Yes, we
> have to support windows, but that doesn't mean we have to be dragged
> down to it's lowest common denominator.
>

Agreed, if we're actually crippling anything.


> I think it's fine to develop a lock type that does the best available
> for each platform using conditional compilation.
>

Again, agreed, but only if there's something to be gained by doing this.
I'm still not convinced there is.

On Tue, Dec 1, 2015 at 11:47 AM, Andrew Wilkins
> <andrew.wilkins at canonical.com> wrote:
> > On Tue, Dec 1, 2015 at 8:03 AM David Cheney <david.cheney at canonical.com>
> > wrote:
> >>
> >> On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins
> >> <andrew.wilkins at canonical.com> wrote:
> >> > On Tue, Dec 1, 2015 at 7:57 AM David Cheney <
> david.cheney at canonical.com>
> >> > wrote:
> >> >>
> >> >> Doesn't look like there is windows support, and it uses fcntl (flock)
> >> >> under the hood, which is what we have now.
> >> >
> >> >
> >> > flock isn't the problematic thing Tim is talking about. utils/fslock
> >> > attempts to create a directory in a known location, and if it
> succeeds,
> >> > it
> >> > "holds the lock". Unlocking means removing the directory.
> >>
> >> The problem is if the process dies/exits/goes mental while holding the
> >> lock we get into this existential gridlock where we're not sure if the
> >> process _is_ alive, so we shouldn't remove the lock, or the process is
> >> dead, so we should remove the lock.
> >>
> >> abstract unix domain sockets do not have this problem on windows; kill
> >> the process, file is removed from the abstract namespace.
> >
> >
> > POSIX advisory file locks (flock) do not have this problem either. See:
> > man(2) fcntl, section "Advisory record locking". When the file
> descriptor is
> > closed, the lock is released; file descriptors are closed when the
> process
> > dies.
> >
> > We could build a mutex on top of a unix domain socket, but then we'll
> have
> > something completely separate for Windows. Shared named mutex? I'm not
> > convinced the overall solution would be any more robust, and I'm pretty
> sure
> > it's going to be more complicated. Happy to be proven wrong though.
> >
> >> >
> >> > We would have to contribute Windows support, but it's not hard, I've
> >> > done it
> >> > before.
> >> >
> >> >>
> >> >> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
> >> >> <casey.marshall at canonical.com> wrote:
> >> >> > How about github.com/camlistore/lock ?
> >> >> >
> >> >> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey
> >> >> > <tim.penhey at canonical.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi folks,
> >> >> >>
> >> >> >> The fslock was a mistake that I added to the codebase some time
> >> >> >> back.
> >> >> >> It
> >> >> >> provided an overly simplistic solution to a more complex problem.
> >> >> >>
> >> >> >> Really the filesystem shouldn't be used as a locking mechanism.
> >> >> >>
> >> >> >> Most of the code that exists for the fslock now is working around
> >> >> >> its
> >> >> >> deficiencies. Instead we should be looking for a better
> replacement.
> >> >> >>
> >> >> >> Some "features" that were added to fslock were added to work
> around
> >> >> >> the
> >> >> >> issue that the lock did not die with the process that created it,
> so
> >> >> >> some mechanism was needed to determine whether the lock should be
> >> >> >> broken
> >> >> >> or not.
> >> >> >>
> >> >> >> What we really need is a good OS agnostic abstraction that
> provides
> >> >> >> the
> >> >> >> ability to create a "named" lock, acquire the lock, release the
> >> >> >> lock,
> >> >> >> and make sure that the lock dies when the process dies, so another
> >> >> >> process that is waiting can acquire the lock. This way no
> >> >> >> "BreakLock"
> >> >> >> functionality is required, nor do we need to try and do think like
> >> >> >> remember which process owns the lock.
> >> >> >>
> >> >> >> So...
> >> >> >>
> >> >> >> We have three current operating systems we need to support:
> >> >> >>
> >> >> >> Linux - Ubuntu and CentOS
> >> >> >> MacOS - client only - bit the CLI uses a lock for the local cache
> >> >> >> Windows
> >> >> >>
> >> >> >> For Linux, and possibly MacOS, flock is a possibility, but can we
> do
> >> >> >> better? Is there something that is better suited?
> >> >> >>
> >> >> >> For Windows, while you can create global semaphores or mutex
> >> >> >> instances,
> >> >> >> I'm not sure of entities that die with the process. Can people
> >> >> >> recommend
> >> >> >> solutions?
> >> >> >>
> >> >> >> Cheers,
> >> >> >> Tim
> >> >> >>
> >> >> >> --
> >> >> >> Juju-dev mailing list
> >> >> >> Juju-dev at lists.ubuntu.com
> >> >> >> Modify settings or unsubscribe at:
> >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Juju-dev mailing list
> >> >> > Juju-dev at lists.ubuntu.com
> >> >> > Modify settings or unsubscribe at:
> >> >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
> >> >> >
> >> >>
> >> >> --
> >> >> Juju-dev mailing list
> >> >> Juju-dev at lists.ubuntu.com
> >> >> Modify settings or unsubscribe at:
> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20151201/7f8587fa/attachment-0001.html>