[Bug 1358835] Re: numa_node_of_cpu() returns warning when cpu_index > 79

Thu Feb 26 20:27:53 UTC 2015

** Description changed:

- == Comment: #0 - TSAI-YANG JEA <tjea at us.ibm.com> - 2014-08-12 14:52:38 ==
+ ========================================================================
+ SRU Justification
+ 1. Impact: numactl fails on systems with > 79 cpus
+ 2. Test case: see 'steps to reproduce' below in the description.
+ 3. Regression potential: this is a straight cherrypick of an upstream commit and should be safe.  It changes core code so if there were regressions we can't say they'd be contained in any way, but they should not affect other packages
+ ========================================================================
+ 
+ == Comment: #0 - TSAI-YANG JEA <tjea at us.ibm.com> - 2014-08-12 14:52:38
+ ==
+ 
  ---Problem Description---

- In a ppc64le P8 node with SMT 8 mode, numa_node_of_cpu (80) returns following warning:
- libnuma: Warning: /sys not mounted or invalid. Assuming one node: No such file or directory
+ 
+ In a ppc64le P8 node with SMT 8 mode, numa_node_of_cpu (80) returns
+ following warning:
+ 
+ libnuma: Warning: /sys not mounted or invalid. Assuming one node: No
+ such file or directory
+ 

  ---uname output---
- Linux c656f2n01 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:29:58 UTC 2014 ppc64le ppc64le ppc64le GNU/Linux
-  
- Machine Type = Tuletta P8 22L  
-  
+ 
+ Linux c656f2n01 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:29:58 UTC
+ 2014 ppc64le ppc64le ppc64le GNU/Linux
+ 
+ 
+ Machine Type = Tuletta P8 22L
+ 
+ 
  ---Debugger---
+ 
  A debugger is not configured
-  
+ 
+ 
  ---Steps to Reproduce---
-  Problem can be created with following test case on a P8 LE machine with SMT=8.
+ 
+  Problem can be created with following test case on a P8 LE machine with
+ SMT=8.
+ 

  #include <dlfcn.h>
+ 
  #include <stdio.h>

+ 
  int main(int argc, char *argv[])
+ 
  {
+ 
      int dlopen_flags = RTLD_NOW|RTLD_GLOBAL;
+ 
      int(*node_of_cpu)(int cpu);
+ 
      int node = 0;
+ 
      int cpu;

+ 
      if(argc != 2) {
+ 
          printf("Usage: %s <cpu index>\n", argv[0]);
+ 
          return -1;
+ 
      }

+ 
      cpu = atoi(argv[1]);
+ 
      printf("%d\n", cpu);

+ 
      void *handle = dlopen("libnuma.so.1", dlopen_flags);
+ 
      if(handle) {
-         node_of_cpu = (__typeof__(node_of_cpu))dlsym(handle, "numa_node_of_cpu");
+ 
+         node_of_cpu = (__typeof__(node_of_cpu))dlsym(handle,
+ "numa_node_of_cpu");
+ 
          if(node_of_cpu) {
+ 
              node = node_of_cpu(cpu);
+ 
              printf("cpu %d belongs to node %d\n", cpu, node);
+ 
          }
+ 
      }
+ 
      return 0;
+ 
  }
+ 

  To compile the test: gcc test.c -ldl -o test

+ 
  To run: ./test 80
+ 

  Other information:

+ 
  $ numactl -H
+ 
  available: 4 nodes (0-1,16-17)
- node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
+ 
+ node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
+ 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39
+ 
  node 0 size: 65115 MB
+ 
  node 0 free: 2955 MB
- node 1 cpus: 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
+ 
+ node 1 cpus: 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
+ 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
+ 
  node 1 size: 65473 MB
+ 
  node 1 free: 3313 MB
- node 16 cpus: 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119
+ 
+ node 16 cpus: 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98
+ 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116
+ 117 118 119
+ 
  node 16 size: 65457 MB
+ 
  node 16 free: 9310 MB
- node 17 cpus: 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159
+ 
+ node 17 cpus: 120 121 122 123 124 125 126 127 128 129 130 131 132 133
+ 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151
+ 152 153 154 155 156 157 158 159
+ 
  node 17 size: 65209 MB
+ 
  node 17 free: 10999 MB
+ 
  node distances:
- node   0   1  16  17 
-   0:  10  20  40  40 
-   1:  20  10  40  40 
-  16:  40  40  10  20 
-  17:  40  40  20  10 
+ 
+ node   0   1  16  17
+ 
+   0:  10  20  40  40
+ 
+   1:  20  10  40  40
+ 
+  16:  40  40  10  20
+ 
+  17:  40  40  20  10

-  
- Userspace tool common name: libnuma 
-  
- The userspace tool has the following bit modes: 64-bit 

- == Comment: #4 - Kamalesh Babulal <kamaleshb at in.ibm.com> - 2014-08-19 12:29:15 ==
+ 
+ Userspace tool common name: libnuma
+ 
+ 
+ The userspace tool has the following bit modes: 64-bit
+ 
+ 
+ == Comment: #4 - Kamalesh Babulal <kamaleshb at in.ibm.com> - 2014-08-19
+ 12:29:15 ==
+ 
  numa_node_of_cpu
+ 
  |
+ 
  |_numa_node_to_cpus_v2:
-                                                   Allocates and updates bitmask of /sys/devices/system/node/node%d/cpumap for node 0,1 and checks if the CPU is present/isset in the node bitmap of CPU0 and CPU1. As CPU 80 is present in node 16, it continues to allocate buffer for node 2 (basically till the configured nodes)
- and tries to read /sys/devices/system/node/node2/cpumap which does not exist, so it assumes that the /sys was not mounted or some thing wrong with map file and sets all the cpus in the node2 as present.
+ 
+                                                   Allocates and updates
+ bitmask of /sys/devices/system/node/node%d/cpumap for node 0,1 and
+ checks if the CPU is present/isset in the node bitmap of CPU0 and CPU1.
+ As CPU 80 is present in node 16, it continues to allocate buffer for
+ node 2 (basically till the configured nodes)
+ 
+ and tries to read /sys/devices/system/node/node2/cpumap which does not
+ exist, so it assumes that the /sys was not mounted or some thing wrong
+ with map file and sets all the cpus in the node2 as present.
+ 

  That's the reason why your seeing the warning and wrong information
  about node. We should be skipping the nodes which are not set in the
  numa_nodes_ptr

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to numactl in Ubuntu.
https://bugs.launchpad.net/bugs/1358835

Title:
  numa_node_of_cpu() returns warning  when cpu_index > 79

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/numactl/+bug/1358835/+subscriptions