[Bug 2151166] [NEW] Race in install -D fails to create parent directories

[Emil Renner Berthing]

Public bug reported:

Running multiple instances of install -D in parallel will race to create parent directories and encounter a fail. This is easily reproducible with make as below:
```
$ cat Makefile
seq = 01 02 03
source = hello.txt
prefix = a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z

all: $(seq:%=$(prefix)/%/hello.txt)

clean:
        rm -rf $(source) a

$(source):
        echo hello > $@

$(prefix)/%/hello.txt: $(source)
        install -D $< $@
$ make # it works serially
echo hello > hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
$ make clean
rm -rf hello.txt a
$ make -j # but not 3 install's in parallel
echo hello > hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02'
install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03'
make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt] Error 1
make: *** Waiting for unfinished jobs....
make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt] Error 1
```

I first encountered this bug when building the Linux kernel in a PPA,
where `make .. dtbs_install` will install multiple device trees to a
deep path in parallel (debian/<package name>/usr/lib/firmware/<kernel
version>/device-tree/<vendor>/<device tree>).

** Affects: rust-coreutils (Ubuntu)
     Importance: Undecided
         Status: New

** Description changed:

  Running multiple instances of install -D in parallel will race to create parent directories and encounter a fail. This is easily reproducible with make as below:
  ```
  $ cat Makefile
  seq = 01 02 03
  source = hello.txt
  prefix = a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z
  
  all: $(seq:%=$(prefix)/%/hello.txt)
  
  clean:
-         rm -rf $(source) a
+         rm -rf $(source) a
  
  $(source):
-         echo hello > $@
+         echo hello > $@
  
  $(prefix)/%/hello.txt: $(source)
-         install -D $< $@
+         install -D $< $@
  $ make # it works serially
  echo hello > hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
  $ make clean
  rm -rf hello.txt a
  $ make -j # but not 3 install's in parallel
  echo hello > hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
  install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02'
  install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03'
  make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt] Error 1
  make: *** Waiting for unfinished jobs....
  make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt] Error 1
  ```
  
  I first encountered this bug when building the Linux kernel in a PPA,
  where `make .. dtbs_install` will install multiple device trees to a
  deep path in parallel (debian/<package name>/usr/lib/firmware/<kernel
- version>/device-tree).
+ version>/device-tree/<vendor>/<device tree>).

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to rust-coreutils in Ubuntu.
https://bugs.launchpad.net/bugs/2151166

Title:
  Race in install -D fails to create parent directories

Status in rust-coreutils package in Ubuntu:
  New

Bug description:
  Running multiple instances of install -D in parallel will race to create parent directories and encounter a fail. This is easily reproducible with make as below:
  ```
  $ cat Makefile
  seq = 01 02 03
  source = hello.txt
  prefix = a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z

  all: $(seq:%=$(prefix)/%/hello.txt)

  clean:
          rm -rf $(source) a

  $(source):
          echo hello > $@

  $(prefix)/%/hello.txt: $(source)
          install -D $< $@
  $ make # it works serially
  echo hello > hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
  $ make clean
  rm -rf hello.txt a
  $ make -j # but not 3 install's in parallel
  echo hello > hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/01/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt
  install -D hello.txt a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt
  install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02'
  install: cannot create directory 'a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03'
  make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/02/hello.txt] Error 1
  make: *** Waiting for unfinished jobs....
  make: *** [Makefile:14: a/b/c/d/e/f/g/h/i/j/k/l/m/n/o/p/q/r/s/t/u/v/x/y/z/03/hello.txt] Error 1
  ```

  I first encountered this bug when building the Linux kernel in a PPA,
  where `make .. dtbs_install` will install multiple device trees to a
  deep path in parallel (debian/<package name>/usr/lib/firmware/<kernel
  version>/device-tree/<vendor>/<device tree>).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2151166/+subscriptions