[Bug 2058094] [NEW] scotch ftbfs when using default stack protector flags

[Steve Langasek]

Public bug reported:

scotch ftbfs in Ubuntu (but not in Debian) with a buffer overflow error
in the testsuite.

./test_scotch_graph_map data/m4x4_b100000.grf
*** buffer overflow detected ***: terminated

I have examined the source and run the code under gdb and cannot confirm
that there is a buffer overflow here.

The code triggering the assertion is in
src/libscotch/bgraph_bipart_gg.c:

  flagtax = (byte *) (vexxtax + grafptr->s.baseval) - grafptr->s.baseval; /* Re-
use extended vertex array for flag array */
  memSet (flagtax + grafptr->s.baseval, ~0, grafptr->s.vertnbr * sizeof (byte));

There's a lot in this code that's ugly (a lot of deliberate setting of
pointers to addresses that are outside of allocated space), but as best
I'm able to tell this is not an out-of-bounds write.  However, with the
terrible pointer manipulation it's quite likely that this has confused
the compiler.

The only way I've found to work around this (without taking a scalpel to
the scotch code) is by downgrading to -D_FORTIFY_SOURCE=2.

This is less than ideal, so I'm opening a bug on scotch+gcc to track it.

** Affects: gcc-13 (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: scotch (Ubuntu)
     Importance: Undecided
         Status: New

** Also affects: gcc-13 (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to gcc-13 in Ubuntu.
https://bugs.launchpad.net/bugs/2058094

Title:
  scotch ftbfs when using default stack protector flags

Status in gcc-13 package in Ubuntu:
  New
Status in scotch package in Ubuntu:
  New

Bug description:
  scotch ftbfs in Ubuntu (but not in Debian) with a buffer overflow
  error in the testsuite.

  ./test_scotch_graph_map data/m4x4_b100000.grf
  *** buffer overflow detected ***: terminated

  I have examined the source and run the code under gdb and cannot
  confirm that there is a buffer overflow here.

  The code triggering the assertion is in
  src/libscotch/bgraph_bipart_gg.c:

    flagtax = (byte *) (vexxtax + grafptr->s.baseval) - grafptr->s.baseval; /* Re-
  use extended vertex array for flag array */
    memSet (flagtax + grafptr->s.baseval, ~0, grafptr->s.vertnbr * sizeof (byte));

  There's a lot in this code that's ugly (a lot of deliberate setting of
  pointers to addresses that are outside of allocated space), but as
  best I'm able to tell this is not an out-of-bounds write.  However,
  with the terrible pointer manipulation it's quite likely that this has
  confused the compiler.

  The only way I've found to work around this (without taking a scalpel
  to the scotch code) is by downgrading to -D_FORTIFY_SOURCE=2.

  This is less than ideal, so I'm opening a bug on scotch+gcc to track
  it.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/gcc-13/+bug/2058094/+subscriptions