[Bug 1250941] [NEW] optimization of a recursive function cause segmentation fault during execution (in very rare cases)

[Shkarnikov Sergey]

Public bug reported:

OS: Ubuntu 12.04.3 LTS
gcc version: 4.6.3-1ubuntu5

It's a modificated test case from gcc-4.6.3 testsuite (20000412-2.c):

int f(int a){
  int x = (( & a)[(((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a % 10))) % 10))) * 314160879)]);
	if (x==0)
		return 1;
  return f(x-1);
}

int main(int argc,char **argv)
{
  if (f (2) != 1)
    abort ();
  exit (0);
}

"((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a %
10))) % 10)))" is a predicate  and always equal to 0. So "x"  is
actually equal to "a".

As you can see this test should exit successfully, but if it was
compiled with optimization (O2 and higher, O1 works fine) segmentation
fault happens during execution.

** Affects: gcc-4.6 (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: optimizer

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to gcc-4.6 in Ubuntu.
https://bugs.launchpad.net/bugs/1250941

Title:
  optimization of a recursive function cause segmentation fault during
  execution (in very rare cases)

Status in “gcc-4.6” package in Ubuntu:
  New

Bug description:
  OS: Ubuntu 12.04.3 LTS
  gcc version: 4.6.3-1ubuntu5

  It's a modificated test case from gcc-4.6.3 testsuite (20000412-2.c):

  int f(int a){
    int x = (( & a)[(((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a % 10))) % 10))) * 314160879)]);
  	if (x==0)
  		return 1;
    return f(x-1);
  }

  int main(int argc,char **argv)
  {
    if (f (2) != 1)
      abort ();
    exit (0);
  }

  "((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a %
  10))) % 10)))" is a predicate  and always equal to 0. So "x"  is
  actually equal to "a".

  As you can see this test should exit successfully, but if it was
  compiled with optimization (O2 and higher, O1 works fine) segmentation
  fault happens during execution.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/gcc-4.6/+bug/1250941/+subscriptions