Ruby compiled with Clang is 8% faster than with GCC 4.9 and 44% faster than with GCC 4.7.2!

(Bráulio Bhavamitra) #1

See Ruby compiled with Clang is 8% faster than with GCC 4.9 and 44% faster than with GCC 4.7.2! - Bráulio Bhavamitra

We are using it in production now!

(Sam Saffron) #2

How / what did you measure? Can you replicate in a Dockerfile?

(Bráulio Bhavamitra) #3


I used a very simple test for ruby:

time ruby -e "count = 0; while(count < 100000000); count = count + 1; end; puts count"  

and also have tested with apache bench:

ab -n 20 -c 5

The boost was pretty similiar with both tests.

(Sam Saffron) #4

Neither of those are properly representative, what does discourse bench say?

(Bráulio Bhavamitra) #5

sorry @sam, I’ve tested with another rails app and posted here to also contribute to discourse, as I got many tips from you :slight_smile:

(Sam Saffron) #6

I will run a bench, but 44% sounds too good to be true imo

(Bráulio Bhavamitra) #7

What GCC are you using? 4.7.2?

(Kane York) #8

Whichever GCC is provided in the Ubuntu/Debian package build-essential at the time that the base image was built.

You could probably add CC=clang to this line and run the benchmark:

(Erick Guan) #9

FYI: Benchmarking Ruby with GCC (4.4, 4.7, 4.8, 4.9) and Clang (3.2, 3.3, 3.4, 3.5)

(Jeff Atwood) #10

GCC 4.9 02 looks quite good there!

(Bráulio Bhavamitra) #11

@fantasticfears fantastic benchmarks! you really got into the details! it was really a surprise to see -O2 better than -O3. I missed only one thing: ruby compiled with clang used -O2 or -O3??

(Robin Ward) #12

I like the numbers, but isn’t benchmarking on EC2 a bad idea? The CPU is quite unpredictable on a shared resource.

(Sam Saffron) #13

FYI I updated our image to gcc 4.9, which will be available once its done pushing.

I need a new image anyway for nodejs uglify and latest ruby.

That said, all this needs to be replicated on bare metal, I have extreme trouble trusting anything measured on shared virtual hosts.

(Bráulio Bhavamitra) #14

just used GitHub - acangiano/ruby-benchmark-suite: A set of Ruby benchmarks for testing Ruby implementations. benchmarks for gcc 4.9 -O2 and clang 3.5 -O2. It was a technical drawn (267s for both). But the difference on some benchmarks was quite big. So, I think benchmark rails directly might be better. Anybody could recommend a rails benchmark suite?

(Sam Saffron) #15

FYI, we started running some more extensive tests and are not noticing any major improvement

(Bráulio Bhavamitra) #16

Nice tests @sam! On some benchmarks, -O3 seems much better.

Also, it is very frustating to see ruby 2.0+ slower than 1.8 and 1.9 on many tests! (regexp is one)
Anybody knows if these regressions were reported?

About the rails benchmarks, I’ve found, will try later.

(Jeff Atwood) #17

I think it’s safe to say that newer versions of compilers will be, on the whole, faster… but I don’t expect any magic bullets and any indication of magical massive speed improvements based on a new compiler version should be viewed with extreme skepticism.

(Alan Tan) #18

I decided to investigate and it doesn’t seem like Discourse was affected at all.