utils/fslock needs to DIAF

Tue Dec 1 16:58:48 UTC 2015

On 1 December 2015 at 16:43, Nate Finch <nate.finch at canonical.com> wrote:
> I think half the problem is that someone named the package fslock and not
> oslock, so we're stuck asking the wrong question.
>
> If the question is "How do I acquire an OS-level lock for a process?"  The
> answer on Windows is "Use a named mutex".

That's not the question we need to answer here, for the configstore case
anyway. In that case, we really do want to protect access to a shared data
store in the filesystem, so the term "fslock" is appropriate I think.

>  Dave seems to be saying that the
> only problem with unix domain sockets is if you try to think about them like
> files... which may be a result of us thinking too much about the solution
> and not the problem. (again, I'm not an expert, you guys can duke out what
> the best solution is)  I think using hackery with the filesystem locking on
> Windows is a mistake.

Is there something wrong with using the Windows file-exclusive-open
semantics to implement a lock associated with a entity in the file system?
Isn't that what it's for?

> On Tue, Dec 1, 2015 at 4:10 AM roger peppe <roger.peppe at canonical.com>
> wrote:
>>
>> So I'm with axw on this one - flock seems like it is a reasonable tool for
>> the job here. FWIW a Unix domain socket also suffers from the "won't
>> work across NFS" problem. Abstract unix domain sockets also
>> have the problem that they won't work with long file paths (what
>> were they thinking?)
>>
>> We have been using github.com/camlistore/lock and although it's not
>> totally ideal (see https://github.com/camlistore/lock/issues/10 )
>> it does the job. Note that it's non-blocking, so a blocking
>> layer above it is necessary, for example see the lockFile
>> function in
>> https://github.com/juju/persistent-cookiejar/blob/master/serialize.go
>>
>> The Windows named mutex thing does sound interesting because
>> it's a blocking API, which is actually what we need. On the other
>> hand, under Windows, files opened for writing are locked by default
>> anyway, so it might be easier just to leverage that property.
>> The camlistore lock code could use some improvement for the
>> Windows platform - we could either fork it, or bradfitz would
>> probably be amenable to a PR.
>>
>>   cheers,
>>     rog.
>>
>> On 1 December 2015 at 04:12, Nate Finch <nate.finch at canonical.com> wrote:
>> > I'm not a linux expert, but definitely a named mutex is exactly the
>> > correct
>> > thing to use for Windows.  Using something else for this purpose would
>> > be
>> > very surprising to a Windows dev and much more likely to be buggy.  I
>> > don't
>> > see any reason we need to use the exact same implementation on all OSes
>> > if
>> > there is something that does exactly the right thing without any
>> > finagling.
>> > FWIW, mutexes do die with the process:
>> >
>> > https://msdn.microsoft.com/en-us/library/windows/desktop/ms682411(v=vs.85).aspx
>> >
>> > On Mon, Nov 30, 2015 at 8:29 PM Andrew Wilkins
>> > <andrew.wilkins at canonical.com> wrote:
>> >>
>> >> On Tue, Dec 1, 2015 at 9:07 AM David Cheney
>> >> <david.cheney at canonical.com>
>> >> wrote:
>> >>>
>> >>> http://0pointer.de/blog/projects/locking.html
>> >>>
>> >>> In short, opening the same file twice and asserting a lock on it will
>> >>> succeed.
>> >>
>> >>
>> >> Thanks. The article is a bit exasperated. Yes, there are problems to be
>> >> aware of, but it doesn't make them unusable in all cases.
>> >>  - Juju agents don't get installed onto NFS file systems, so doesn't
>> >> matter for the agents.
>> >>  - We're in full control of the files we're locking, we're not locking
>> >> some file like /etc/passwd where some other random bit of code in the
>> >> process is going to open/close it and release the lock by accident.
>> >>  - We don't need byte-range locking.
>> >>
>> >> So only the original uncertainty remains: do we care about clients
>> >> running
>> >> their home directory on an NFS share, where the NFS *server* is too old
>> >> to
>> >> support flock?
>> >>
>> >> Maybe a named mutex on Windows and a domain socket on *NIX is the way
>> >> to
>> >> go. I'm not dead set on flock; I just want something that is simple,
>> >> robust
>> >> and portable.
>> >>
>> >> Cheers,
>> >> Andrew
>> >>
>> >>> On Tue, Dec 1, 2015 at 12:04 PM, Andrew Wilkins
>> >>> <andrew.wilkins at canonical.com> wrote:
>> >>> > On Tue, Dec 1, 2015 at 8:57 AM David Cheney
>> >>> > <david.cheney at canonical.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> fcntl won't work in threaded go applications, it barely works in
>> >>> >> non
>> >>> >> threaded applications.
>> >>> >
>> >>> >
>> >>> > This is news to me. I've used it plenty in the past, in
>> >>> > multi-threaded
>> >>> > programs. Please point me at relevant literature that explains where
>> >>> > it
>> >>> > doesn't work.
>> >>> >
>> >>> >>
>> >>> >> I'm not interested in "doesn't work on windows" arguments. Yes, we
>> >>> >> have to support windows, but that doesn't mean we have to be
>> >>> >> dragged
>> >>> >> down to it's lowest common denominator.
>> >>> >
>> >>> >
>> >>> > Agreed, if we're actually crippling anything.
>> >>> >
>> >>> >>
>> >>> >> I think it's fine to develop a lock type that does the best
>> >>> >> available
>> >>> >> for each platform using conditional compilation.
>> >>> >
>> >>> >
>> >>> > Again, agreed, but only if there's something to be gained by doing
>> >>> > this. I'm
>> >>> > still not convinced there is.
>> >>> >
>> >>> >> On Tue, Dec 1, 2015 at 11:47 AM, Andrew Wilkins
>> >>> >> <andrew.wilkins at canonical.com> wrote:
>> >>> >> > On Tue, Dec 1, 2015 at 8:03 AM David Cheney
>> >>> >> > <david.cheney at canonical.com>
>> >>> >> > wrote:
>> >>> >> >>
>> >>> >> >> On Tue, Dec 1, 2015 at 11:00 AM, Andrew Wilkins
>> >>> >> >> <andrew.wilkins at canonical.com> wrote:
>> >>> >> >> > On Tue, Dec 1, 2015 at 7:57 AM David Cheney
>> >>> >> >> > <david.cheney at canonical.com>
>> >>> >> >> > wrote:
>> >>> >> >> >>
>> >>> >> >> >> Doesn't look like there is windows support, and it uses fcntl
>> >>> >> >> >> (flock)
>> >>> >> >> >> under the hood, which is what we have now.
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> > flock isn't the problematic thing Tim is talking about.
>> >>> >> >> > utils/fslock
>> >>> >> >> > attempts to create a directory in a known location, and if it
>> >>> >> >> > succeeds,
>> >>> >> >> > it
>> >>> >> >> > "holds the lock". Unlocking means removing the directory.
>> >>> >> >>
>> >>> >> >> The problem is if the process dies/exits/goes mental while
>> >>> >> >> holding
>> >>> >> >> the
>> >>> >> >> lock we get into this existential gridlock where we're not sure
>> >>> >> >> if
>> >>> >> >> the
>> >>> >> >> process _is_ alive, so we shouldn't remove the lock, or the
>> >>> >> >> process
>> >>> >> >> is
>> >>> >> >> dead, so we should remove the lock.
>> >>> >> >>
>> >>> >> >> abstract unix domain sockets do not have this problem on
>> >>> >> >> windows;
>> >>> >> >> kill
>> >>> >> >> the process, file is removed from the abstract namespace.
>> >>> >> >
>> >>> >> >
>> >>> >> > POSIX advisory file locks (flock) do not have this problem
>> >>> >> > either.
>> >>> >> > See:
>> >>> >> > man(2) fcntl, section "Advisory record locking". When the file
>> >>> >> > descriptor is
>> >>> >> > closed, the lock is released; file descriptors are closed when
>> >>> >> > the
>> >>> >> > process
>> >>> >> > dies.
>> >>> >> >
>> >>> >> > We could build a mutex on top of a unix domain socket, but then
>> >>> >> > we'll
>> >>> >> > have
>> >>> >> > something completely separate for Windows. Shared named mutex?
>> >>> >> > I'm
>> >>> >> > not
>> >>> >> > convinced the overall solution would be any more robust, and I'm
>> >>> >> > pretty
>> >>> >> > sure
>> >>> >> > it's going to be more complicated. Happy to be proven wrong
>> >>> >> > though.
>> >>> >> >
>> >>> >> >> >
>> >>> >> >> > We would have to contribute Windows support, but it's not
>> >>> >> >> > hard,
>> >>> >> >> > I've
>> >>> >> >> > done it
>> >>> >> >> > before.
>> >>> >> >> >
>> >>> >> >> >>
>> >>> >> >> >> On Tue, Dec 1, 2015 at 10:55 AM, Casey Marshall
>> >>> >> >> >> <casey.marshall at canonical.com> wrote:
>> >>> >> >> >> > How about github.com/camlistore/lock ?
>> >>> >> >> >> >
>> >>> >> >> >> > On Mon, Nov 30, 2015 at 5:43 PM, Tim Penhey
>> >>> >> >> >> > <tim.penhey at canonical.com>
>> >>> >> >> >> > wrote:
>> >>> >> >> >> >>
>> >>> >> >> >> >> Hi folks,
>> >>> >> >> >> >>
>> >>> >> >> >> >> The fslock was a mistake that I added to the codebase some
>> >>> >> >> >> >> time
>> >>> >> >> >> >> back.
>> >>> >> >> >> >> It
>> >>> >> >> >> >> provided an overly simplistic solution to a more complex
>> >>> >> >> >> >> problem.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Really the filesystem shouldn't be used as a locking
>> >>> >> >> >> >> mechanism.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Most of the code that exists for the fslock now is working
>> >>> >> >> >> >> around
>> >>> >> >> >> >> its
>> >>> >> >> >> >> deficiencies. Instead we should be looking for a better
>> >>> >> >> >> >> replacement.
>> >>> >> >> >> >>
>> >>> >> >> >> >> Some "features" that were added to fslock were added to
>> >>> >> >> >> >> work
>> >>> >> >> >> >> around
>> >>> >> >> >> >> the
>> >>> >> >> >> >> issue that the lock did not die with the process that
>> >>> >> >> >> >> created
>> >>> >> >> >> >> it,
>> >>> >> >> >> >> so
>> >>> >> >> >> >> some mechanism was needed to determine whether the lock
>> >>> >> >> >> >> should be
>> >>> >> >> >> >> broken
>> >>> >> >> >> >> or not.
>> >>> >> >> >> >>
>> >>> >> >> >> >> What we really need is a good OS agnostic abstraction that
>> >>> >> >> >> >> provides
>> >>> >> >> >> >> the
>> >>> >> >> >> >> ability to create a "named" lock, acquire the lock,
>> >>> >> >> >> >> release
>> >>> >> >> >> >> the
>> >>> >> >> >> >> lock,
>> >>> >> >> >> >> and make sure that the lock dies when the process dies, so
>> >>> >> >> >> >> another
>> >>> >> >> >> >> process that is waiting can acquire the lock. This way no
>> >>> >> >> >> >> "BreakLock"
>> >>> >> >> >> >> functionality is required, nor do we need to try and do
>> >>> >> >> >> >> think
>> >>> >> >> >> >> like
>> >>> >> >> >> >> remember which process owns the lock.
>> >>> >> >> >> >>
>> >>> >> >> >> >> So...
>> >>> >> >> >> >>
>> >>> >> >> >> >> We have three current operating systems we need to
>> >>> >> >> >> >> support:
>> >>> >> >> >> >>
>> >>> >> >> >> >> Linux - Ubuntu and CentOS
>> >>> >> >> >> >> MacOS - client only - bit the CLI uses a lock for the
>> >>> >> >> >> >> local
>> >>> >> >> >> >> cache
>> >>> >> >> >> >> Windows
>> >>> >> >> >> >>
>> >>> >> >> >> >> For Linux, and possibly MacOS, flock is a possibility, but
>> >>> >> >> >> >> can we
>> >>> >> >> >> >> do
>> >>> >> >> >> >> better? Is there something that is better suited?
>> >>> >> >> >> >>
>> >>> >> >> >> >> For Windows, while you can create global semaphores or
>> >>> >> >> >> >> mutex
>> >>> >> >> >> >> instances,
>> >>> >> >> >> >> I'm not sure of entities that die with the process. Can
>> >>> >> >> >> >> people
>> >>> >> >> >> >> recommend
>> >>> >> >> >> >> solutions?
>> >>> >> >> >> >>
>> >>> >> >> >> >> Cheers,
>> >>> >> >> >> >> Tim
>> >>> >> >> >> >>
>> >>> >> >> >> >> --
>> >>> >> >> >> >> Juju-dev mailing list
>> >>> >> >> >> >> Juju-dev at lists.ubuntu.com
>> >>> >> >> >> >> Modify settings or unsubscribe at:
>> >>> >> >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > --
>> >>> >> >> >> > Juju-dev mailing list
>> >>> >> >> >> > Juju-dev at lists.ubuntu.com
>> >>> >> >> >> > Modify settings or unsubscribe at:
>> >>> >> >> >> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >>> >> >> >> >
>> >>> >> >> >>
>> >>> >> >> >> --
>> >>> >> >> >> Juju-dev mailing list
>> >>> >> >> >> Juju-dev at lists.ubuntu.com
>> >>> >> >> >> Modify settings or unsubscribe at:
>> >>> >> >> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >>
>> >> --
>> >> Juju-dev mailing list
>> >> Juju-dev at lists.ubuntu.com
>> >> Modify settings or unsubscribe at:
>> >> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >
>> >
>> > --
>> > Juju-dev mailing list
>> > Juju-dev at lists.ubuntu.com
>> > Modify settings or unsubscribe at:
>> > https://lists.ubuntu.com/mailman/listinfo/juju-dev
>> >