Ruby compiled with Clang is 8% faster than with GCC 4.9 and 44% faster than with GCC 4.7.2!


(Bráulio Bhavamitra) #1

See Ruby compiled with Clang is 8% faster than with GCC 4.9 and 44% faster than with GCC 4.7.2! - Bráulio Bhavamitra

We are using it in production now!


(Sam Saffron) #2

How / what did you measure? Can you replicate in a Dockerfile?


(Bráulio Bhavamitra) #3

@sam

I used a very simple test for ruby:

time ruby -e "count = 0; while(count < 100000000); count = count + 1; end; puts count"  

and also have tested with apache bench:

ab -n 20 -c 5 http://127.0.0.1:3000/

The boost was pretty similiar with both tests.


(Sam Saffron) #4

Neither of those are properly representative, what does discourse bench say?


(Bráulio Bhavamitra) #5

sorry @sam, I’ve tested with another rails app and posted here to also contribute to discourse, as I got many tips from you :slight_smile:


(Sam Saffron) #6

I will run a bench, but 44% sounds too good to be true imo


(Bráulio Bhavamitra) #7

What GCC are you using? 4.7.2?


(Kane York) #8

Whichever GCC is provided in the Ubuntu/Debian package build-essential at the time that the base image was built.

https://github.com/discourse/discourse_docker/blob/master/image/base_21/Dockerfile
https://github.com/discourse/discourse_docker/blob/master/image/discourse_21/Dockerfile

You could probably add CC=clang to this line and run the benchmark:

https://github.com/discourse/discourse_docker/blob/master/image/base_21/Dockerfile#L44


(Erick Guan) #9

FYI: Benchmarking Ruby with GCC (4.4, 4.7, 4.8, 4.9) and Clang (3.2, 3.3, 3.4, 3.5)


(Jeff Atwood) #10

GCC 4.9 02 looks quite good there!


(Bráulio Bhavamitra) #11

@fantasticfears fantastic benchmarks! you really got into the details! it was really a surprise to see -O2 better than -O3. I missed only one thing: ruby compiled with clang used -O2 or -O3??


(Robin Ward) #12

I like the numbers, but isn’t benchmarking on EC2 a bad idea? The CPU is quite unpredictable on a shared resource.


(Sam Saffron) #13

FYI I updated our image to gcc 4.9, which will be available once its done pushing.

I need a new image anyway for nodejs uglify and latest ruby.

That said, all this needs to be replicated on bare metal, I have extreme trouble trusting anything measured on shared virtual hosts.


(Bráulio Bhavamitra) #14

just used GitHub - acangiano/ruby-benchmark-suite: A set of Ruby benchmarks for testing Ruby implementations. benchmarks for gcc 4.9 -O2 and clang 3.5 -O2. It was a technical drawn (267s for both). But the difference on some benchmarks was quite big. So, I think benchmark rails directly might be better. Anybody could recommend a rails benchmark suite?


(Sam Saffron) #15

FYI, we started running some more extensive tests and are not noticing any major improvement

http://rubybench-rycco.rhcloud.com/benchmarks/binary-trees


(Bráulio Bhavamitra) #16

Nice tests @sam! On some benchmarks, -O3 seems much better.

Also, it is very frustating to see ruby 2.0+ slower than 1.8 and 1.9 on many tests! (regexp is one)
Anybody knows if these regressions were reported?

About the rails benchmarks, I’ve found https://github.com/rails-bench, will try later.


(Jeff Atwood) #17

I think it’s safe to say that newer versions of compilers will be, on the whole, faster… but I don’t expect any magic bullets and any indication of magical massive speed improvements based on a new compiler version should be viewed with extreme skepticism.


(Alan Tan) #18

http://www.guoxiang.me/posts/33-with-bare-metal-benchmarking-ruby-with-gcc-4-6-4-7-4-8-4-9-and-clang-3-3-3-4-3-5

I decided to investigate and it doesn’t seem like Discourse was affected at all.