[Bug 1253638] Re: dynamic linker does not use DT_RUNPATH for transitive dependencies

[Mario Sánchez Prada]

> I don't know if this is expected behavior, but it's certainly annoying
behavior. I'm writing an app that depends on libtcodxx.so, which in turn
depends on libtcod.so. I want to ship them both in the lib directory
next to my app. With RUNPATH as "$ORIGIN/lib", it finds the direct
dependency (libtcodxx.so) and then doesn't look in the same directory
for its dependency. To get it to search there for both, I have to set
RPATH to "$ORIGIN/lib", and I suspect that RPATH may not technically be
supposed to support the $ORIGIN syntax.

It looks like it's the expected behaviour after all. From "Shared Object
Dependencies" section in [1]:

"""
The set of directories specified by a given DT_RUNPATH entry is used to find only the immediate dependencies of the executable or shared object containing the DT_RUNPATH entry. That is, it is used only for those dependencies contained in the DT_NEEDED entries of the dynamic structure containing the DT_RUNPATH entry, itself. One object's DT_RUNPATH entry does not affect the search for any other object's dependencies.
"""

Another interesting bit in [1] is the "Dynamic Array Tags" table, where
it's stated that RUNPATH is optionally considered for both shared
objects and executables, while RPATH is ignored for shared objects. It
seems like that suggests the idea when using RUNPATH is that you'd have
both executables and shared objects specifying it, instead of setting it
just for the "top level" executable and letting that work transitively.

I've investigated this a bit and it seems that the move to using
--enable-new-dtags by default is something Debian changed explicitly a
while ago:

  binutils (2.27.51.20161116-2) unstable; urgency=medium
  
    * Stop building the mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el
      variants; can't continue with this work, because package uploads with
      these architectures are still rejected.
    * Add homepage attribute to the control file: Closes: #841432.
    * ld: enable new dtags by default for linux/gnu targets. Closes: #835859.
    * Fix PR ld/20827, using proposed patch. Closes: #844378.
  
   -- Matthias Klose <doko at debian.org>  Thu, 17 Nov 2016 11:56:55 +0100

While I understand this probably makes sense in a world where everyone
is aware of these differences and either upstream projects are built
considering RUNPATH (e.g. WebKitGTK+) or packagers explicitly workaround
this by passing --disable-new-dtags to projects that haven't been
adapted upstream yet (e.g. GNOME Shell), it still seems to me like this
has huge potential to break things if it just gets enabled in other
Debian-based without further consideration (and rebuilds!), so if you're
having this problem and you control your platform but not every single
package built on top of it, you might consider reverting the Debian
patch that closes #835859 for the time being.

That's precisely what we're considering to do in Endless for now, and I
thought I'd share it here in case this analysis was useful for someone
else, if anything as a sign of gratitude for this bug report in itself,
which was very useful for me to understand better what was going on when
we first encountered this problem ourselves (similar to what was
reported already in [2] and [3])

Hope this helps too!
Mario

[1] https://refspecs.linuxfoundation.org/elf/gabi4+/ch5.dynamic.html#shobj_dependencies
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844796#5
[3] https://bugzilla.gnome.org/show_bug.cgi?id=777519

** Bug watch added: Debian Bug tracker #844796
   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=844796

** Bug watch added: GNOME Bug Tracker #777519
   https://bugzilla.gnome.org/show_bug.cgi?id=777519

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to eglibc in Ubuntu.
https://bugs.launchpad.net/bugs/1253638

Title:
  dynamic linker does not use DT_RUNPATH for transitive dependencies

Status in eglibc package in Ubuntu:
  Confirmed

Bug description:
  $ lsb_release -rd
  Description:	Ubuntu 13.10
  Release:	13.10

  $ uname -a
  Linux mhassert 3.11.0-13-generic #20-Ubuntu SMP Wed Oct 23 07:38:26 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

  $gcc -dumpversion
  4.8

  $ ld -v
  GNU ld (GNU Binutils for Ubuntu) 2.23.52.20130913

  $ LC_ALL=C apt-cache policy libc-bin
  libc-bin:
    Installed: 2.17-93ubuntu4

  * What you expected to happen
  Binaries with DT_RPATH or DT_RUNPATH behaving identical in the absence of LD_LIBRARY_PATH

  * What happened instead
  DT_RUNPATH not searched for transitive dependencies.

  
  When running a binary that depends on custom libraries which in turn depend on custom libraries, hard-coded search paths in DT_RUNPATH behave differently from those in DT_RPATH.
  Paths in DT_RPATH are being considered for everything that is dynamically loaded, even dependencies of dependencies. Paths in DT_RUNPATH seem being considered only for direct dependencies of the binary.

  Searching the web I think that the one and only difference between
  DT_RPATH and DT_RUNPATH should be that DT_RPATH is considered _before_
  LD_LIBRARY_PATH and DT_RUNPATH _afterwards_. In the absence of
  LD_LIBRARY_PATH there should be no difference at all.

  I stumbled upon this problem when switching from "ld" to "gold" for
  the linker. The default for ld on Ubuntu 13.10 is "--disable-new-
  dtags" while the default for gold is "--enable-new-dtags". Therefore
  ld produces binaries with DT_RPATH and gold ones with DT_RUNPATH.

  In the attached minimal example
  - the binaries rpath and runpath both depend on libb but not directly on liba.
  - libb depends on liba.
  - liba and libb are linked without any hard-coded library paths.
  - rpath and runpath are linked with  hard-coded library paths for both liba and libb
  - rpath is linked with --disable-new-dtags (producing DT_RPATH)
  - rpath is linked with --enable-new-dtags (producing DT_RUNPATH)

  To test, please run make all and observe how "rpath" works while
  "runpath" fails to find liba at runtime.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/eglibc/+bug/1253638/+subscriptions