Number CPU cores incorrectly detected (UpCloud.com VPS)


(ljpp) #1

Setting up a new server at UpCloud.com and found a bug in discourse-setup.

Found 8GB of memory and 1 physical CPU cores
setting UNICORN_WORKERS = 2

This is incorrect, as the server has 6 cores.

cat /proc/cpuinfo | awk '/^processor/{print $3}'
0
1
2
3
4
5

UpCloud.com experiences
(Matt Palmer) #2

Are they physical or logical cores? What does the cpu cores value in /proc/cpuinfo say? We only count physical cores for the purposes of determining how many unicorns to use. If the autodetected setting is incorrect, you can edit the UNICORN_WORKERS setting in app.yml and rebuild.


(ljpp) #3
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.996
cache size      : 25600 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1

Aha.

And yes, currently running with 8 unicors, as per Jeff’s suggestion:


(Matt Palmer) #4

There you go. The system is reporting that it only has one physical core. As it’s a VPS, you’ll want to tell the provider to fix their virtualization to more accurately report the underlying physical CPU configuration, if there are, in fact, multiple physical cores allocated to your VPS.


#5

Checking the CPU count from "cpu cores : " -line in /proc/cpuinfo is a wrong way to do it.

It reports just the amount of cpu-cores in one socket, therefore:

  1. it doesn’t take multiple sockets into account (can be detected from the ‘physical id’ -line)
  2. it doesn’t take multiple threads into account (can be detected from the ‘siblings’, if it differs from ‘cpu cores’)
    –> ok, someone might hate simultaneous multithreading (hyper-threading) and wants just “the real cores”

For example, a physical machine with four Intel Xeon E7-4820v4 -processors has the following configuration:
4 sockets, 10 cores per socket, 2 threads per core = 40 physical cores, 80 threads in total.
“cpu cores” -line in /proc/cpuinfo says 10, quite suboptimal detection…

CPU topology under virtualization could be presented in many different ways. 8 vCPUs could be anything from “1 socket, 1 CPU/socket, 8 threads/CPU” to “8 sockets, 1 CPU/socket, 1 thread/CPU” and the software should be able to detect all the different options - because usually you don’t get physical CPU:s pinned only for your VPS, they are all shared.


(Jay Pfaffman) #6

Sounds true. I wrote that code and had little way to test it. It worked when I tested on Digital Ocean $10, $20, and $40/month droplets. I suspected that @ljpp was right and that the code was broken and was relieved that @mpalmer blamed upcloud.

From your post I mostly understand how it is that it doesn’t work, but I can’t tell from what you say how to write a test that does.

It is clear to me that you’ve likely forgotten more about CPU architecture than I care to know. Can you either submit a PR that has a more reliable test or post code or outputs from whatever it is that would provide a better test so that I could update it?


(ljpp) #7

As a certified SW functional testing engineer, I represent the eyes of the customer (or end-user, as this is OSS). Customer/end-user seems something not working as expected, but does not really care why. I also have very limited back-end or coding skillz :slight_smile: . So I do my best to report issues, occasionally propose improvements.


#8

Quickly checking the installer, currently the test under linux is (line 138):

avail_cores=`cat /proc/cpuinfo | grep "cpu cores" | uniq | awk '{print $4}'`

Easy solution (just all cores, logical and physical):

avail_cores=`grep -c "processor" /proc/cpuinfo`

Physical cores only (take number of cores per socket, multiply by number of sockets):

avail_cores=$((`awk '/cpu cores/ {print $4;exit}' /proc/cpuinfo`*`sort /proc/cpuinfo | uniq | grep -c "physical id"`))

(ljpp) #9

Hey guys, if you are about the improve this, there is a different kind of issue with ScaleWay (or any other Atom/Avoton based server) - the number of unicorns gets a too high value for non-hyperthreading CPUs.


(Jeff Atwood) #10

Hyperthreading is a non issue, the correct value is 2 unicorns per real, physical (not fake-logical-hyperthreaded) core.


(ljpp) #11

Alright. But this results 2 unicorns per Atom/Avoton based non-hyperthreading CPU core, which most likely is too much. ScaleWay uses these and I would not be surprised if these will gain more popularity, as they are energy and physical space efficient boxes.


(Jeff Atwood) #12

Unlikely, Intel wants to sell Xeons and has demoted low-margin Atoms to near-nonexistence. Atom performance is also quite poor overall, even factoring in per-watt. Unless you want to rack a tablet or smartphone, in which case you’d literally be better off racking the iPhone 7 which is insanely fast per watt.


(ljpp) #13

UpCloud submitted a PR and it looks like @mpalmer accepted.


(Rafael dos Santos Silva) #14

This topic was automatically closed after 6 hours. New replies are no longer allowed.