Groovy vs. Ruby vs. Python Performance

Whilst I’ve been doing the Project Euler problems and writing them up here, I’ve had to focus on getting a scripting language, i.e. Groovy, to perform in an adequate manner to solve the problems in a “reasonable” time. This has meant that, in some cases, micro-tuning has been necessary and during this I’ve discovered that the “syntactic sugar” in Groovy is actually quite expensive in terms of CPU usage when running.

In Groovy, however, it’s quite possible, for example, to fall back to traditional iterative for-loops which run their body clause as inline code rather than closures being passed as code blocks in to a function. I wondered how some of the solutions might fare when ported over to Ruby and Python.

I selected the solutions for Problem #52 and converted them over to the other languages. I also tried the Ruby solution in JRuby in the same JVM as the Groovy script.

I tried to be faithful in the ports and although the solutions may not be (i) the most efficient algorithm or (ii) truly Rubyesque or Pythonic in style, I believe that they are fairly representative. You can see the scripts towards the bottom of the page.

Here are the timings in seconds. The platform was Debian Linux/Ubuntu 8.10 (Intrepid Ibex) x86 in a single core VMWare guest hosted on Vista running on a 2.53 GHz machine. I used the Sun 1.6.0_20 VM.

Run Timings

	Groovy 1.7.2		JRuby 1.5.0		Ruby 1.8.7		Python 2.5.2
	pe052_1	pe052_2	pe052_1	pe052_2	pe052_1	pe052_2	pe052_1	pe052_2
Elapsed	2.744	0.746	5.668	1.615	19.598	4.554	4.670	1.100
Real	3.736	1.721	6.672	2.209	19.676	4.568	4.712	1.117
User	2.812	0.788	5.016	1.376	16.817	4.052	4.680	1.100
Sys	0.836	0.892	1.168	0.796	2.712	0.480	0.020	0.012

The differences between the elapsed time i.e. the timing as measured by the script, and the real “wall clock” timing is interesting. The startup/shutdown time of Ruby & Python is nominal – worst case overhead was less than 0.08s, whilst both Groovy & JRuby were taking a whole additional second. This meant that Python was actually the fastest in real terms when the actual duration of the user code was short though if the overhead could be amortized over a longer run then Groovy came out ahead.

I also tested Ruby 1.9.0 (2008-06-20 rev 17492) as that’s meant to be a lot faster. Also, to be fair to Python, I used the 2.6.5 release and the "set" built-in type rather than the Set module. The timings were:

	Ruby 1.9.0		Python 2.6.5
	pe052_1	pe052_2	pe052_1	pe052_2
Elapsed	10.288	2.867	3.020	0.790
Real	10.306	2.889	3.079	0.861
User	10.265	2.252	3.020	0.792
Sys	0.016	0.016	0.012	0.020

Note also that the Python & Ruby 1.9.0 versions spent only a negligible time running in kernel mode compared to the other VMs.

Normalized Performance (Elapsed Time)

	Groovy 1.7.2	JRuby 1.5.0	Ruby 1.8.7	Ruby 1.9.0	Python 2.5.2	Python 2.6.5
pe052_1	1.00	0.48	0.14	0.27	0.59	0.91
pe052_2	1.00	0.46	0.16	0.26	0.68	0.94

Relative performance seems to be quite consistent for the two solutions, within 15% or so.

Obviously these just fall within the realm of micro-benchmarks and, as such, aren’t necessarily realistic of normal production work. They do, however, exercise several features that would be in common use; list iteration, string conversions, list/set construction, string comparisons, some basic maths functions and loop constructs with termination conditions. The results are therefore informative but the performance ratios found here will vary according to the mix of these functions.

Note that the Ruby 1.9 VM is about twice as fast as 1.8.7 but still lags behind JRuby and Python by quite a margin. Maybe Rubinius will recover some of that ground – the authors are expecting it to be something like 12x faster than Ruby 1.8 which would rate it at 1.68 in the table above.

Conclusion

Based on these few figures, it does appear that the Java VM is head and shoulders above the Ruby 1.8.7/1.9.0 offerings and that the Groovy implementation is pretty good when compared to these other languages so perhaps I shouldn’t moan so much about it!

Source Code

pe052_1.rb

#!/usr/bin/env ruby

require 'set'

ticks = Time.new

answer = 0

i = 1
while (answer == 0) do
  answer = ( (1..6).collect { | n | (i * n).to_s.split("").sort() }.to_set.size == 1 ) ? i : 0
  i += 1
end

ticks = Time.new - ticks

puts "Answer: #{answer}"
puts "Elapsed: #{ticks * 1000} ms"

pe052_2.rb

#!/usr/bin/env ruby

require 'set'

ticks = Time.new

answer = 0
i = 1

while (answer == 0) do

  s = i.to_s.split("").sort()

  (2..6).each do | j |
    break if s != (i * j).to_s.split("").sort()
    answer = i if j == 6
  end

  i += 1

end

ticks = Time.new - ticks

puts "Answer: #{answer}"
puts "Elapsed: #{ticks * 1000} ms"

pe052_1.py

#!/usr/bin/env python

from time import clock
from sets import Set      # deprecated in Python 2.6

ticks = clock()

answer = 0

i = 1
while answer == 0:
  if len(Set([ tuple(sorted(str(i * n))) for n in range(1,6+1) ])) == 1: answer = i
  i += 1

ticks = clock() - ticks

print "Answer: ", answer
print "Elapsed: %f ms" % (ticks * 1000)

pe052_2.py

#!/usr/bin/env python

from time import clock
from sets import Set

ticks = clock()

answer = 0

i = 1
while answer == 0:
  s = sorted(str(i))
  for j in xrange(2,6+1):
    if s != sorted(str(i * j)): break
    if j == 6: answer = i
  i += 1

ticks = clock() - ticks

print "Answer: ", answer
print "Elapsed: %f ms" % (ticks * 1000)