Discussion:
[Caml-list] Is OCaml fast?
(too old to reply)
Thanassis Tsiodras
2010-11-22 13:21:08 UTC
Permalink
I apologize beforehand if this is not the forum to ask.

I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.

However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)

Is it just hype, then? Or am I missing something?
--
What I gave, I have; what I spent, I had; what I kept, I lost. (Old Epitaph)

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gregory Bellier
2010-11-22 13:36:27 UTC
Permalink
If you want speed then you should learn assembly or choose C as a second
choice.
It depends on the kind of apps you'd like to write. Even though O'Caml is
fast, it's not the first criteria I have in mind which would be security :
no segfault, no need to handle horrible stuff like in C, ...
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html<http://www.cs.ubc.ca/%7Emurphyk/Software/Ocaml/why_ocaml.html>),
I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(
http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php
)
Is it just hype, then? Or am I missing something?
--
What I gave, I have; what I spent, I had; what I kept, I lost. (Old Epitaph)
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Lukasz Stafiniak
2010-11-22 13:39:45 UTC
Permalink
On Mon, Nov 22, 2010 at 2:35 PM, Gregory Bellier
Post by Gregory Bellier
If you want speed then you should learn assembly or choose C as a second
choice.
It depends on the kind of apps you'd like to write. Even though O'Caml is
no segfault, no need to handle horrible stuff like in C, ...
No! You should use ATS, just as the shootout suggests. (Portable)
assembly is not a language for humans ;-)

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Thomas Fischbacher
2010-11-22 13:42:41 UTC
Permalink
Post by Gregory Bellier
If you want speed then you should learn assembly or choose C as a second
choice.
Certainly not assembly. Modern microprocessors just see assembly as a
kind-of high level language which they interpret in a funny way, doing
all sorts of re-schedulings, register renamings, delayed branching,
speculative execution etc. Producing code that can take advantage of
this is best left to the compiler - unless one has to deal with
instruction set extensions for which there is no good compiler
support yet.
Post by Gregory Bellier
It depends on the kind of apps you'd like to write. Even though O'Caml
is fast, it's not the first criteria I have in mind which would be
security : no segfault, no need to handle horrible stuff like in C, ...
I'd say the OCaml native compiler is reasonably fast for pretty much all
applications that need the speed of compiled code. But I'd say the same
for GHC and SBCL, say. There are some good reasons to take a closer look
at OCaml, but these are related to other qualities of the language.

In my view, its greatest benefit is that it makes working with closures
simple while being fast and by far not as intimidating to (prospective)
PhD students as Scheme/Lisp (but that only because most of them have
been spoilt by imperative languages beforehand).
--
best regards,
Thomas Fischbacher
***@soton.ac.uk

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Sylvain Le Gall
2010-11-22 13:43:32 UTC
Permalink
Hello,
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Shootout benchmarks doesn't always allow to set some variables that
could greatly improve the speed of OCaml.
Post by Thanassis Tsiodras
Is it just hype, then? Or am I missing something?
You could write very fast application with OCaml -- even beating C code
in some case. But what OCaml is really helpful for, is that you can do
algorithmic optimizations that improve the speed. It means that OCaml
is terse enough to allow you to write complex algorithm without making
too much bugs.

All in all: development time + execution time is fast.

But if you spend 10x development time on the same C code, you will
obviously get something faster in C.

Regards,
Sylvain Le Gall

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Dario Teixeira
2010-11-22 13:55:10 UTC
Permalink
Hi,
Post by Thanassis Tsiodras
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html),
I saw that OCaml was praised for the speed of the executables it
generates - and was referred to, speed-wise, as "second to none",
except C and C++.
Yes, Ocaml is fast (more on that later), but you should consider speed
as just the icing in the cake. Besides all the cool language features,
what makes me recommend Ocaml to people who -- like you -- may be on
the fence, is that among the not so mainstream languages, Ocaml is
perhaps the one people are more likely to stick with for actual work
after they get past the learning stage. Remember that though Ocaml
favours the functional paradigm, it is flexible enough to support those
(few) situations where it actually makes sense to be imperative and/or
use an object oriented approach, and where a purely functional solution
is just intellectual masturbation.
Post by Thanassis Tsiodras
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Is it just hype, then? Or am I missing something?
There's lies, damn lies, and shootout statistics. Getting good performance
in Haskell is often something of a black art that baffles even gurus. In
contrast, Ocaml has very good "performance predictability". Moreover, it
is possible to get good performance from fairly idiomatic Ocaml programmes,
whereas in other languages you may need to write ugly code which looks like
nothing one would write for elegance and sanity.

Best regards,
Dario Teixeira





_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-22 18:27:50 UTC
Permalink
On Nov 22, 5:55 am, Dario Teixeira <***@yahoo.com> wrote:
-snip-
Post by Dario Teixeira
There's lies, damn lies, and shootout statistics.
-snip-

After all, facts are facts,
and although we may quote one to another with a chuckle
the words of the Wise Statesman, 'Lies--damned lies--and statistics,'
still there are some easy figures the simplest must understand,
and the astutest cannot wriggle out of.
Leonard Henry Courtney, 1895

:-)
Isaac Gouy
2010-11-23 02:20:12 UTC
Permalink
Dario Teixeira <darioteixeira <at> yahoo.com> writes:
-snip-
Post by Dario Teixeira
There's lies, damn lies, and shootout statistics.
-snip-

After all, facts are facts,
and although we may quote one to another with a chuckle
the words of the Wise Statesman, 'Lies--damned lies--and statistics,'
still there are some easy figures the simplest must understand,
and the astutest cannot wriggle out of.

Leonard Henry Courtney, 1895

:-)


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Fabrice Le Fessant
2010-11-23 09:15:44 UTC
Permalink
Maybe you should read "Tainted Truth: The Manipulation of Fact In
America" by Cynthia Crossen ?

--Fabrice
Post by Isaac Gouy
-snip-
Post by Dario Teixeira
There's lies, damn lies, and shootout statistics.
-snip-
After all, facts are facts,
and although we may quote one to another with a chuckle
the words of the Wise Statesman, 'Lies--damned lies--and statistics,'
still there are some easy figures the simplest must understand,
and the astutest cannot wriggle out of.
Leonard Henry Courtney, 1895
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 18:00:12 UTC
Permalink
Is that book the source for the quotation 'Lies--damned lies--and statistics'?
Post by Fabrice Le Fessant
Maybe you should read "Tainted Truth: The Manipulation of Fact In
America" by Cynthia Crossen ?
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Mark Diekhans
2010-11-23 19:54:26 UTC
Permalink
The source of this pharse is unknown, however Mark Twain is credited with
making it well known:

"There are three kinds of lies: lies, damned lies and statistics."
Post by Isaac Gouy
Is that book the source for the quotation 'Lies--damned lies--and statistics'?
Post by Fabrice Le Fessant
Maybe you should read "Tainted Truth: The Manipulation of Fact In
America" by Cynthia Crossen ?
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 20:05:20 UTC
Permalink
Post by Mark Diekhans
The source of this pharse is unknown, however Mark Twain is credited with
"There are three kinds of lies: lies, damned lies and statistics."
http://www.york.ac.uk/depts/maths/histstat/lies.htm

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
o***@first.in-berlin.de
2010-11-23 22:56:22 UTC
Permalink
Post by Dario Teixeira
Hi,
Post by Thanassis Tsiodras
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html),
I saw that OCaml was praised for the speed of the executables it
generates - and was referred to, speed-wise, as "second to none",
except C and C++.
Yes, Ocaml is fast (more on that later), but you should consider speed
as just the icing in the cake.
[...]

But if it were not performant enough, I (and many others too, I think)
would consider it being a nice toy language.

AFAIK in the past, functional langauges were not adapted, because they were
very unperformant - at least this is one reason.
Another reason might be, that the available functional languages in the past
were overloaded with parenthess ;)

Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-23 23:54:38 UTC
Permalink
Post by o***@first.in-berlin.de
AFAIK in the past, functional langauges were not adapted, because they were
very unperformant - at least this is one reason.
Another reason might be, that the available functional languages in the past
were overloaded with parenthess ;)
That was also true of early ML implementations. When I was first taught ML at university we used the Cambridge ML interpreter and you actually had to sit there and wait for it to solve the 8-queens problem. At the time, I thought ML was a complete joke and could see no use for it outside its very specific domain of theorem proving and actually really resented being taught it on a general CS course. I know better now though. ;-)

Later language implementations inherited many of these inefficiencies though. Many of the things that can make OCaml and Java slow were inherited from Lisp. They are, in effect, designs that still bear the burden of dynamic typing despite being statically typed.

Cheers,
Jon.


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Erik de Castro Lopo
2010-11-24 01:24:28 UTC
Permalink
Post by Jon Harrop
Many of the things that can make OCaml and Java slow were inherited
from Lisp. They are, in effect, designs that still bear the burden
of dynamic typing despite being statically typed.
I'm curious, what in your opinion are those things?

Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-25 11:17:51 UTC
Permalink
Boxing and generics with type erasure making it difficult or impossible to
reduce the density of references in the heap. Structs and reified generics‎
can give huge performance improvements. Hash tables and complex arithmetic
are some examples where I often see F# running 5-20× faster than idiomatic
OCaml/Java code.

Cheers,
Jon.
-----Original Message-----
Sent: 24 November 2010 01:24
Subject: Re: [Caml-list] Is OCaml fast?
Post by Jon Harrop
Many of the things that can make OCaml and Java slow were inherited
from Lisp. They are, in effect, designs that still bear the burden
of dynamic typing despite being statically typed.
I'm curious, what in your opinion are those things?
Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gerd Stolpmann
2010-11-22 14:04:42 UTC
Permalink
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Is it just hype, then? Or am I missing something?
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?

Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
***@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Sylvain Le Gall
2010-11-22 14:22:29 UTC
Permalink
Post by Gerd Stolpmann
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)
Is it just hype, then? Or am I missing something?
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
Maybe it is time to join forces and update the shootout for OCaml?

If some people are interested here, I can setup a repository on the forge
to update all this tests.

If I get 2 people interested, I will setup a VCS repository + project on
the forge, this afternoon. Drop me an email + preferred VCS (among svn,
git, darcs) and your account login on http://forge.ocamlcore.org. I will
take care, when ready, to made this code available in the shootout once
finished.

Regards,
Sylvain Le Gall

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
bluestorm
2010-11-22 14:37:00 UTC
Permalink
Post by Gerd Stolpmann
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
As Sylvain noticed, some (in not most) of the OCaml poor performances in the
shootout are actually not due to bad OCaml programs, but to arbitrary
restrictions in the shootout rules. For example, one of the bad-performing
benchmark for OCaml is the binary-tree benchmark, where it is nearly four
times slower than C, but on closer inspection you discover that this is due
to the arbitrary choice to forbid any change of the GC parameters. With
appropriate GC parameters, the very same OCaml program is exactly as fast as
C.

http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees

« Note: these programs are being measured with *the default initial heap
size* - the measurements may be very different with a larger initial heap
size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative" section) :
12.67 secs


Therefore, there is nothing that can be changed to the OCaml submission for
this benchmark to improve performances, except changing the default GC
parameters; while this might be a good idea in general, changing it only for
the sake of shootout-obsessed people is ridiculous.
Török Edwin
2010-11-22 15:01:57 UTC
Permalink
On Mon, 22 Nov 2010 15:36:30 +0100
On Mon, Nov 22, 2010 at 3:04 PM, Gerd Stolpmann
Post by Gerd Stolpmann
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml
was #1 at the shootout a few years ago, faster than C.) So maybe a
good opportunity to post better Ocaml solutions there?
As Sylvain noticed, some (in not most) of the OCaml poor performances
in the shootout are actually not due to bad OCaml programs, but to
arbitrary restrictions in the shootout rules. For example, one of the
bad-performing benchmark for OCaml is the binary-tree benchmark,
where it is nearly four times slower than C, but on closer inspection
you discover that this is due to the arbitrary choice to forbid any
change of the GC parameters. With appropriate GC parameters, the very
same OCaml program is exactly as fast as C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with *the default initial
heap size* - the measurements may be very different with a larger
initial heap size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
Therefore, there is nothing that can be changed to the OCaml
submission for this benchmark to improve performances, except
changing the default GC parameters; while this might be a good idea
in general, changing it only for the sake of shootout-obsessed people
is ridiculous.
Isn't it possible for the GC to realise its doing too many collections
and increase the minor heap size on its own?
Or isn't it possible to determine at compile time an approximation for
a good heap size, and use that?

Best regards,
--Edwin

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Goswin von Brederlow
2010-11-22 16:00:20 UTC
Permalink
Post by Török Edwin
On Mon, 22 Nov 2010 15:36:30 +0100
On Mon, Nov 22, 2010 at 3:04 PM, Gerd Stolpmann
Post by Gerd Stolpmann
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml
was #1 at the shootout a few years ago, faster than C.) So maybe a
good opportunity to post better Ocaml solutions there?
As Sylvain noticed, some (in not most) of the OCaml poor performances
in the shootout are actually not due to bad OCaml programs, but to
arbitrary restrictions in the shootout rules. For example, one of the
bad-performing benchmark for OCaml is the binary-tree benchmark,
where it is nearly four times slower than C, but on closer inspection
you discover that this is due to the arbitrary choice to forbid any
change of the GC parameters. With appropriate GC parameters, the very
same OCaml program is exactly as fast as C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with *the default initial
heap size* - the measurements may be very different with a larger
initial heap size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
Therefore, there is nothing that can be changed to the OCaml
submission for this benchmark to improve performances, except
changing the default GC parameters; while this might be a good idea
in general, changing it only for the sake of shootout-obsessed people
is ridiculous.
Isn't it possible for the GC to realise its doing too many collections
and increase the minor heap size on its own?
Maybe. The GC would have to see that with a larger minor heap it has
sufficient space so that most values die before the collection. It could
see that most values die after 1 major sweep and increasing the minor
heap would likely keep them out of the major heap alltogether.
Post by Török Edwin
Or isn't it possible to determine at compile time an approximation for
a good heap size, and use that?
Usualy that is dependent on the input.

MfG
Goswin

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gerd Stolpmann
2010-11-22 15:02:11 UTC
Permalink
On Mon, Nov 22, 2010 at 3:04 PM, Gerd Stolpmann
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
As Sylvain noticed, some (in not most) of the OCaml poor performances
in the shootout are actually not due to bad OCaml programs, but to
arbitrary restrictions in the shootout rules. For example, one of the
bad-performing benchmark for OCaml is the binary-tree benchmark, where
it is nearly four times slower than C, but on closer inspection you
discover that this is due to the arbitrary choice to forbid any change
of the GC parameters. With appropriate GC parameters, the very same
OCaml program is exactly as fast as C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with the default initial
heap size - the measurements may be very different with a larger
initial heap size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
Therefore, there is nothing that can be changed to the OCaml
submission for this benchmark to improve performances, except changing
the default GC parameters; while this might be a good idea in general,
changing it only for the sake of shootout-obsessed people is
ridiculous.
It's in deed an unfair comparison: In C they use the Apache runtime
which provides memory pools. This is something that does not extend to
most real world programs.

Because it's ridiculous anyway: Encode the tree in an array. Not really
idiomatic, but in C they also do not use the idiomatic memory management
(malloc/free).

Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
***@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Fabrice Le Fessant
2010-11-22 16:46:56 UTC
Permalink
Post by Török Edwin
Isn't it possible for the GC to realise its doing too many collections
and increase the minor heap size on its own?
Indeed, it could notice that a lot of data is being moved to the major
heap, and double its size in consequence, until a maximum limit is
reached.

The problem is that it is the kind of things that are application
dependent, and should be put in the program itself (the program would
have a trigger on each minor heap collection, and, depending on the
moved bytes, would increase the size of the minor heap). The problem
is that the Shootout does not allow that, so the winner is the
language whose runtime allocates the most memory at the beginnning...

--Fabrice

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Török Edwin
2010-11-22 18:33:45 UTC
Permalink
On Mon, 22 Nov 2010 17:46:49 +0100
Post by Fabrice Le Fessant
Post by Török Edwin
Isn't it possible for the GC to realise its doing too many
collections and increase the minor heap size on its own?
Indeed, it could notice that a lot of data is being moved to the major
heap, and double its size in consequence, until a maximum limit is
reached.
I did some benchmarks on 2 CPUs, and here are the wall clock times
for GC minor heap size:

"minorheap"," phenomII",,," core2",,,"cycle diff %"
," time",," cycles","time",," cycles",
32," 17.76","17.74",56.8," 48.60","48.86",58.48,2.96
64," 16.81","16.80",53.78,"44.79","44.79",53.75,-0.06
128," 15.41","15.38",49.26,"41.38","40.76",49.28,0.04
256," 14.39","14.35",45.98,"39.75","38.98",47.24,2.74
512," 12.97","13.15",41.79,"35.75","35.59",42.8,2.42
1024," 12.89","12.94",41.33,"33.57","33.13",40.02,-3.17
2048," 11.05","11.09",35.42,"29.26","29.12",35.03,-1.1
4096," 9.79"," 9.81",31.36,"26.21","25.96",31.3,-0.19
8192," 8.71"," 8.85",28.1," 23.36","23.34",28.02,-0.28
16384," 7.89"," 7.86",25.2," 21.41","21.54",25.77,2.26
32768," 7.55"," 7.55",24.16,"20.74","20.88",24.97,3.35

(minorheap is in KWords, time is in seconds, cycles is divided by 10^9)

Increasing minor heap beyond that yielded no improvement (number of
minor heap collections stayed the same).

phenomII has 64 Kb L1 data cache, 512Kb L2 cache, 6144Kb L3 cache
(shared), runs at 3.2Ghz. That would be 516k words if only 1 core used.

core2 has 32 Kb L1 data cache, 4MB L2 cache, runs at 1.2Ghz.
That would be 840k words if only 1 core used.

Both used exact same binaries on 64-bit Linux, ocaml 3.11.2.

Despite the difference in CPUs and heap sizes the number of CPU cycles
spent for a given size of the minor heap is about the same (within 3%
difference).

Not sure what the max should be for the minor heap increase, but based
on this benchmark increasing size of minor heap never slows down the
program. Even when size of minor heap exceeds what fits in the cache.
I guess there is another microbenchmark that can show the opposite
though.

Is there some real world benchmark for OCaml's GC that can be used
to get some better values for minor heap size based on CPU's L2/L3 size
(as suggested in the 'Optimizing garbage collection' thread)?
Post by Fabrice Le Fessant
The problem is that it is the kind of things that are application
dependent,
Yes, maybe optimal size of minor heap doesn't depend on CPU's cache
size but on the application, in which case a heuristic for
increasing/decreasing the heap size may be better?
Post by Fabrice Le Fessant
and should be put in the program itself (the program would
have a trigger on each minor heap collection, and, depending on the
moved bytes, would increase the size of the minor heap). The problem
is that the Shootout does not allow that, so the winner is the
language whose runtime allocates the most memory at the beginnning...
If tuning minor heap can double performance of the program, then the GC
should have some heuristics to tune it automatically. Sure applications
which are not happy with that tuning should tune it themselves.

Best regards,
--Edwin

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Pierre Etchemaïté
2010-11-27 21:11:40 UTC
Permalink
Post by Török Edwin
Not sure what the max should be for the minor heap increase, but based
on this benchmark increasing size of minor heap never slows down the
program. Even when size of minor heap exceeds what fits in the cache.
I guess there is another microbenchmark that can show the opposite
though.
Another consequence of increasing minor heap size is the latency
introduced by minor heap collections. That said, when does it become
the worse garbage collector latency? (... when heap compaction is
disabled?)
And of course, for many programs such latency matters a lot less than
thoughput...
Christophe Raffalli
2010-11-28 13:26:31 UTC
Permalink
Post by Pierre Etchemaïté
Post by Török Edwin
Not sure what the max should be for the minor heap increase, but based
on this benchmark increasing size of minor heap never slows down the
program. Even when size of minor heap exceeds what fits in the cache.
I guess there is another microbenchmark that can show the opposite
though.
Another consequence of increasing minor heap size is the latency
introduced by minor heap collections. That said, when does it become
the worse garbage collector latency? (... when heap compaction is
disabled?)
And of course, for many programs such latency matters a lot less than
thoughput...
In fact, an allocated block is visited at most twice by a minor GC and
if promoted (that if visited twice by the minor GC) at least once by the
major GC (when freed because
major GC is mark and sweep) ... So a heap allocated block is traversed
0, 1, 3 or more time by one of the GC ...

So increasing the minor heap size strictly decreases the number of time
a word is examined by one of the GC ...

I think that having two heaps is mainly (only ?) for one reason:
collecting some memory sooner (and therefore using less memory for dead
objects).
This may speed up the program because saving memory decreases cache
misses, but normaly it slows down the
program. But it is still better than one generation GCed more frequently ...

The latency is not directly related to minor collection ... If there was
only the major heap,
OCaml could still to a bit of major-gc cycle at each collection ... This
is a (good) design choice of OCaml
to have a small minor heap that can be collected in a short time by a
simple and fast "stop and copy"
algorithm.

The main problem of the bin-tree benchmark is when you know that an
object will live long enough to be promoted, you can not ask OCaml to
allocate it in the major heap directly. This is why increasing the minor
heap to have all object collected at the first GC works.

A syntax like Node#(x,d,y) to allocate a variant in the major heap
directly together with similar syntax for
tuples, records, ... could be quite useful in OCaml in general for known
long living objects ... An the is the Ancient modules for object that
will live even longer.

Cheers,
Christophe
Post by Pierre Etchemaïté
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Benedikt Meurer
2010-11-28 14:29:17 UTC
Permalink
Speaking of the OCaml GC in general, wouldn't it make sense to replace the current generational collector with a collector framework that requires less copying in the common case. For example, dividing the heap into two parts, "large object space" and "small object space", where LOS is managed by mark-sweep/mark-compact (could use the current major heap) and SOS is managed by a recent mark-copy/mark-region collector (Immix [1] comes to mind here). That way objects would no longer need to be copied between generations, and using Immix may also reduce cache misses and improve locality of small/medium sized objects (with proper evacuation heuristics). This should be doable with a moderate amount of work, and should require no incompatible changes, while opening the door for concurrent/parallel garbage collection (this would require incompatible changes then, replacing caml_young_ptr/caml_young_limit with TLS vars, etc.).

Benedikt

[1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.144.3640
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-22 19:11:10 UTC
Permalink
On Nov 22, 6:37 am, bluestorm <***@gmail.com> wrote:
-snip-
With appropriate GC parameters, the very same OCaml program is exactly as
fast as C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with *the default initial heap
size* - the measurements may be very different with a larger initial heap
size or GC tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
12.67 secs
And of course you know that because that GC tuned OCaml program is
shown on the benchmarks game website ;-)

http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&lang=ocaml&id=1#log
Isaac Gouy
2010-11-23 02:15:11 UTC
Permalink
bluestorm <bluestorm.dylc <at> gmail.com> writes:

-snip-
With appropriate GC parameters, the very same OCaml program is exactly as fast
as C.
http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees
« Note: these programs are being measured with the default initial heap size -
the measurements may be very different with a larger initial heap size or GC
tuning. »
C version : 12.11 secs
OCaml version : 47.22 secs
12.67 secs



And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website ;-)

http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&lang=ocaml&id=1

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Christophe TROESTLER
2010-11-23 10:37:42 UTC
Permalink
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website ;-)
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an “alternative”.

Best,
C.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-23 15:50:59 UTC
Permalink
And almost all of the "Haskell" solutions (reverse-complement,
spectral-norm, Mandelbrot, n-body, fannkuch-redux, k-nucleotide, regex-dna)
abuse GHC's FFI in order to work around Haskell.

The k-nucleotide benchmark in Haskell even uses malloc! Ketil Malde crafted
a much better solution but noted:

"This is considered cheating, since it is the easy and natural way to do
it." -
http://www.haskell.org/haskellwiki/Shootout/Knucleotide

Cheers,
Jon.
-----Original Message-----
Sent: 23 November 2010 10:38
Subject: Re: [Caml-list] Re: Is OCaml fast?
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website ;-)
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an "alternative".
Best,
C.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 18:07:29 UTC
Permalink
Post by Jon Harrop
Ketil Malde crafted
"This is considered cheating, since it is the easy and natural way to do
it." -
http://www.haskell.org/haskellwiki/Shootout/Knucleotide
Not even cheating - just an answer to a different question.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-23 21:35:04 UTC
Permalink
Yes, an answer to a better question.
-----Original Message-----
Sent: 23 November 2010 18:07
Subject: [Caml-list] Re: Is OCaml fast?
Post by Jon Harrop
Ketil Malde crafted
"This is considered cheating, since it is the easy and natural way
to do
Post by Jon Harrop
it." -
http://www.haskell.org/haskellwiki/Shootout/Knucleotide
Not even cheating - just an answer to a different question.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-23 16:08:42 UTC
Permalink
Note that the regex-dna solution for Haskell tweaks its GC parameters via
the -H command-line parameter:

http://shootout.alioth.debian.org/u64q/program.php?test=regexdna&lang=ghc&id
=2
-----Original Message-----
Sent: 23 November 2010 10:38
Subject: Re: [Caml-list] Re: Is OCaml fast?
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website ;-)
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an "alternative".
Best,
C.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 18:05:10 UTC
Permalink
Post by Jon Harrop
Note that the regex-dna solution for Haskell tweaks its GC parameters via
Note that there is no restriction on "tuning the GC" for regex-dna.

Note that there is no restriction on "tuning the GC" for any task except
binary-trees.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Török Edwin
2010-11-23 19:14:44 UTC
Permalink
On Tue, 23 Nov 2010 18:03:10 +0000 (UTC)
Post by Isaac Gouy
Post by Jon Harrop
Note that the regex-dna solution for Haskell tweaks its GC
Note that there is no restriction on "tuning the GC" for regex-dna.
Note that there is no restriction on "tuning the GC" for any task
except binary-trees.
Sounds good. Then Ocaml could still win if it performs well on the
other benchmarks.

Best regards,
--Edwin

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 20:18:11 UTC
Permalink
Török Edwin <edwintorok <at> gmail.com> writes:

-snip-
Post by Török Edwin
Sounds good. Then Ocaml could still win if it performs well on the
other benchmarks.
The main benchmarks game summary is median and quartiles so one measurement
doesn't have much influence.

There are several tasks which have worse performing OCaml programs than
binary-trees.


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 00:12:30 UTC
Permalink
Post by Isaac Gouy
Note that there is no restriction on "tuning the GC" for regex-dna.
Note that there is no restriction on "tuning the GC" for any task except
binary-trees.
Don't you think this is a pretty ludicrous restriction?
Tuning the GC / adjusting the size of pools or mallocs is essential
for optimizing for modern processors. I bet the C programs are doing
this, except that it won't obviously be called "tuning the GC"
although it amounts to precisely the same thing.
What you say would be interesting if you'd taken the trouble to look at the C
programs.


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 17:53:33 UTC
Permalink
Post by Christophe TROESTLER
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an “alternative”.
You seem to be suggesting that "tuning the GC" is considered "alternative" only
for OCaml programs.

You seem to be suggesting that "tuning the GC" is considered "alternative" for
every task.

Neither is true.

You seem to be suggesting some kind of equivalence between vector instructions
and "tuning the GC".
You haven't said why they should be considered equivalent.

Nor have you said why you think C should not be allowed to use memory pools.






_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gerd Stolpmann
2010-11-23 19:25:05 UTC
Permalink
Post by Isaac Gouy
Post by Christophe TROESTLER
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an “alternative”.
You seem to be suggesting that "tuning the GC" is considered "alternative" only
for OCaml programs.
You seem to be suggesting that "tuning the GC" is considered "alternative" for
every task.
Neither is true.
You seem to be suggesting some kind of equivalence between vector instructions
and "tuning the GC".
You haven't said why they should be considered equivalent.
Nor have you said why you think C should not be allowed to use memory pools.
Quite easy: because you are comparing results that cannot be compared.
The reader of this benchmark (binary trees) might have the impression
that C is generally that fast - however, this would be no longer true if
these binary trees were used as library in a bigger program where using
memory pools is inappropriate, e.g. because the data managed by the
binary trees has an unpredictable lifetime.

I do not say that it is complete nonsense to do this comparison, but
only that it is more specific than a reader would assume. The innocent
reader expects a comparison of binary tree performance, not of methods
of managing memory (and this is it finally). The true name of this test
should be "manage_many_small_memory_cells_where_pools_are_allowed". (It
would be actually interesting to compare various versions of this test
with different memory management methods.)

Gerd
Post by Isaac Gouy
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
***@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 20:30:24 UTC
Permalink
Gerd Stolpmann <info <at> gerd-stolpmann.de> writes:

-snip-
Post by Gerd Stolpmann
I do not say that it is complete nonsense to do this comparison, but
only that it is more specific than a reader would assume.
A reader's wrong assumptions are their own responsibility:

http://shootou.alioth.debian.org/flawed-benchmarks.php
Post by Gerd Stolpmann
The innocent reader expects a comparison of binary tree performance,
not of methods of managing memory (and this is it finally).
Perhaps rather than "innocent reader" you mean careless reader who didn't bother
to read what the programs should do?

http://shootout.alioth.debian.org/u32q/benchmark.php?test=binarytrees&lang=all#about
Post by Gerd Stolpmann
The true name of this test should be
"manage_many_small_memory_cells_where_pools_are_allowed".
"binary-trees benchmark : Allocate and deallocate many many binary trees"
Post by Gerd Stolpmann
(It would be actually interesting to compare various versions of this test
with different memory management methods.)
So do that comparison and publish the results.



_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Gerd Stolpmann
2010-11-23 20:56:07 UTC
Permalink
Post by Isaac Gouy
Post by Gerd Stolpmann
(It would be actually interesting to compare various versions of this test
with different memory management methods.)
So do that comparison and publish the results.
Please don't tell me what I am supposed to do. I'm not a troll like
others.

Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
***@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
------------------------------------------------------------

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 21:32:37 UTC
Permalink
Post by Gerd Stolpmann
Post by Isaac Gouy
Post by Gerd Stolpmann
(It would be actually interesting to compare various versions of this test
with different memory management methods.)
So do that comparison and publish the results.
Please don't tell me what I am supposed to do. I'm not a troll like
others.
The tone of my comment was not as harsh as you heard.




_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Goswin von Brederlow
2010-11-24 22:30:03 UTC
Permalink
Post by Gerd Stolpmann
Post by Isaac Gouy
Post by Christophe TROESTLER
Post by bluestorm
C version : 12.11 secs
OCaml version : 47.22 secs
OCaml version with GC parameters tuned ("interesting alternative"
section) : 12.67 secs
And of course you know because that GC tuned OCaml program is shown on the
benchmarks game website
http://shootout.alioth.debian.org/u32/program.php?test=binarytrees&
lang=ocaml&id=1
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an “alternative”.
You seem to be suggesting that "tuning the GC" is considered "alternative" only
for OCaml programs.
You seem to be suggesting that "tuning the GC" is considered "alternative" for
every task.
Neither is true.
You seem to be suggesting some kind of equivalence between vector instructions
and "tuning the GC".
You haven't said why they should be considered equivalent.
Nor have you said why you think C should not be allowed to use memory pools.
Quite easy: because you are comparing results that cannot be compared.
The reader of this benchmark (binary trees) might have the impression
that C is generally that fast - however, this would be no longer true if
these binary trees were used as library in a bigger program where using
memory pools is inappropriate, e.g. because the data managed by the
binary trees has an unpredictable lifetime.
I do not say that it is complete nonsense to do this comparison, but
only that it is more specific than a reader would assume. The innocent
reader expects a comparison of binary tree performance, not of methods
of managing memory (and this is it finally). The true name of this test
should be "manage_many_small_memory_cells_where_pools_are_allowed". (It
would be actually interesting to compare various versions of this test
with different memory management methods.)
Gerd
So write an ocaml programm that uses and array or bigarray to pool its
memory. That is the same as C does.

MfG
Goswin

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Christophe TROESTLER
2010-11-23 21:14:32 UTC
Permalink
Post by Christophe TROESTLER
Since you are here, please explain why C can use memory pools and vec
tor instructions but tuning the GC of OCaml ― although it is part of
the standard library ― is considered an “alternative”.
[...] You seem to be suggesting some kind of equivalence between vector
instructions and "tuning the GC". You haven't said why they should
be considered equivalent.
I did not say they are equivalent. It is a matter of what is allowed
(or not) and for what reason. The question is why is C allowed to use
an external library for managing its memory but — for this specific
benchmark which is precisely about managing memory — OCaml is not
authorized to make use of its very own library!
Note that there is no restriction on "tuning the GC" for regex-dna.
Note that there is no restriction on "tuning the GC" for any task except
binary-trees.
Note that you did not answer my question.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-23 21:55:44 UTC
Permalink
Christophe TROESTLER <Christophe.Troestler+ocaml <at> umh.ac.be> writes:

-snip-
Post by Christophe TROESTLER
The question is why is C allowed to use
an external library for managing its memory
I asked why you think C should not be allowed to use memory pools - you haven't
tried to answer that question.

If you think that C should not be allowed to use memory pools because they are
provided in an external library then I disagree - I think it's fine to show use
of popular external libraries.

Of course, a C program that just uses malloc is also shown.
Post by Christophe TROESTLER
but — for this specific
benchmark which is precisely about managing memory — OCaml is not
authorized to make use of its very own library!
OCaml is authorized to make use of its very own library.

Just like all the other programming language implementations, for binary-trees,
OCaml is not authorized to tune the GC.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
evil sloot
2010-11-23 23:21:33 UTC
Permalink
Hi,

Regarding alternative solutions to the shootout problems:
The thread ring problem can be easily implemented using Lwt
yielding (pun intended ;)) performance
comparable to the haskell solution

have fun,

Romain.





_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Rajchenbach-Teller
2010-11-24 06:54:32 UTC
Permalink
Submitted this a few years ago (just with streams, not Lwt), and the solution was rejected.
Post by evil sloot
Hi,
The thread ring problem can be easily implemented using Lwt
yielding (pun intended ;)) performance
comparable to the haskell solution
have fun,
Romain.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Oliver Bandel
2010-11-22 17:02:10 UTC
Permalink
Zitat von "Gerd Stolpmann" <***@gerd-stolpmann.de>:
[...]
Post by Gerd Stolpmann
(I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
[...]

Yes I also remember that.
I hope that the new OCaml compilers did not
make OCaml lessperformance by enhancing other features.

And I don't realy think so.

But were the old code-snippets emoved, or what was going on,
that OCaml degraded that much?

From my experience - normally using ocamlc for most of my OCaml
programs - it's even fast enough with this bytecode.
If it will be not fast enough, I have an option as a joker:
compiling to native code. :)

Doing this is rather psychologically, as I like to have a potential
for enhancement. :)

If you really always need the best performance, of course compiling
to native code will be best choice.
But at lest for my work I have not found out really bottlenecks,
and always was faster than doing things in Perl or Python or so.

Learning OCaml IMHO does make sense in any case. :)

But be aware one big disadvantage: if you have learned Ocaml,
this may absolutely kill your motivation in learning other languages
afterwards.So,if you need some other languages for your job, learn
them first, and learn OCaml after this, for your pleasure.

But you may never have fun at work then, until your boss allows you to
use OCaml.

And this is not a joke, I mean it as I wrote it!

Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Rajchenbach-Teller
2010-11-22 17:32:24 UTC
Permalink
I can confirm that old code-snippets were removed (and that both faster solutions and environment variable tweaks were rejected).
Post by Oliver Bandel
[...]
Post by Gerd Stolpmann
(I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
[...]
Yes I also remember that.
I hope that the new OCaml compilers did not
make OCaml lessperformance by enhancing other features.
And I don't realy think so.
But were the old code-snippets emoved, or what was going on,
that OCaml degraded that much?
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Oliver Bandel
2010-11-22 17:49:05 UTC
Permalink
..hmhhh..

..looks like they are biased...


... not that we are not ;)

..but... as the GC-stuff is available FROM WITHING the language,
in the standard-lib, this is nothing added on later.


And I think it should also be allowed to be used.

To reject environment variables, I can see as acceptable in this case,
but rejecting the GC-stuff does not make sense, because, as just
mentioned, it is avalable by the programmer from within the code.

What about compiling parameters?
I mean: in C you can use -O for optimization.
This should also be forbidden then.... Is it?

There are so much possibilities to influence the results,
that blocking Gc-module is idiotic, IMHO.


Ciao,
Oliver


P.S.:
I looked at one of the C-makefiles:

usr/bin/gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native
-fopenmp -D_FILE_OFFSET_BITS=64 -I/usr/include/apr-1.0 -lapr-1 -lgomp
binarytrees.gcc-7.c -o binarytrees.gcc-7.gcc_run
rm binarytrees.gcc-7.c


So, -O3 is allowed.
AFAIK with O3 and higher, inline does work.
__inline__ must be forbidden as well as -O3

Optimization should be switched off completely, if
OCaml's optimizations are also not allowed.
Post by David Rajchenbach-Teller
I can confirm that old code-snippets were removed (and that both
faster solutions and environment variable tweaks were rejected).
Post by Oliver Bandel
[...]
Post by Gerd Stolpmann
(I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
[...]
Yes I also remember that.
I hope that the new OCaml compilers did not
make OCaml lessperformance by enhancing other features.
And I don't realy think so.
But were the old code-snippets emoved, or what was going on,
that OCaml degraded that much?
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Rajchenbach-Teller
2010-11-22 17:54:25 UTC
Permalink
Note: I'm not saying that they are biased. It's quite possible they did the same thing for other languages, too, I didn't take the time to check.
...hmhhh..
...looks like they are biased...
.... not that we are not ;)
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jeff Schultz
2010-11-22 23:56:12 UTC
Permalink
...looks like they are biased...
.... not that we are not ;)
To reject environment variables, I can see as acceptable in this case,
but rejecting the GC-stuff does not make sense, because, as just mentioned,
it is avalable by the programmer from within the code.
I haven't looked at the "benchmark game" or its predecessor for many
years, but all this really says is that the programs it measures are
sufficiently unlike real applications that the OCaml developers
haven't been tempted to tune the GC defaults for them.

The conclusion to draw would be that the shootout results are a
poor, and probably misleading, evidence base for selecting a
programming language for execution time performance. Measured
programming language implementation performance gets faster or
slower over the years due to its own rule changes and language and
library implementation changes, but I doubt that this conclusion
changes much ;-)


The one thing the shootout programs are useful for is to get a feel
for a variety of different languages' expressivity for a common set
of "small" problems.


Jeff Schultz

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-22 19:28:01 UTC
Permalink
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".

During autumn 2004, on Brent Fulgham's website, new tasks were added
and old Doug Bagley tasks removed.

By 2005 only 2 tasks remained from that old Doug Bagley website.

By August 2008 none of those old Doug Bagley tasks were measured for
the current benchmarks game.
Post by David Rajchenbach-Teller
I can confirm that old code-snippets were removed (and that both faster solutions and environment variable tweaks were rejected).
Post by Oliver Bandel
[...]
Post by Gerd Stolpmann
(I remember Ocaml was #1
at theshootouta few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
[...]
Yes I also remember that.
I hope that the new OCaml compilers did not
make OCaml lessperformance by enhancing other features.
And I don't realy think so.
But were the old code-snippets emoved, or what was going on,
that OCaml degraded that much?
_______________________________________________
Caml-list mailing list. Subscription management:http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives:http://caml.inria.fr
Beginner's list:http://groups.yahoo.com/group/ocaml_beginners
Bug reports:http://caml.inria.fr/bin/caml-bugs
Eray Ozkural
2010-11-22 23:28:34 UTC
Permalink
It's always funny to look at benchmarks made by amateurs. We get it all the
time in parallel computing, when some sharp programmer decides to implement
a parallel algorithm in Java. :) I keep asking myself questions like do you
also program ciphers in Flash? Graph algorithms in PHP?

In my experience, ocaml is pretty fast, quite comparable to C++, for
computation-intensive tasks and of course excellent at traditional
functional programming tasks. Since I use both kinds of algorithms in my
research, ocaml is better than C++ for me (despite the great boost libs),
except for shared memory parallelism, which is still non-existent in ocaml.

Try implementing something that has a large time/memory complexity like a
data mining algorithm, and you'll see the difference from Haskell/Java/Blah.
Just run the algorithms in Weka, the famous data mining suite written in
Java, and compare them to any sensible implementation of the same
algorithms. Measure the difference, it's unbelievable how awful Java is at
any kind of serious computing! I think some conservative humans from the
future sent the designers of Java back in time to stop the singularity.

Cheers,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Isaac Gouy
2010-11-23 02:05:13 UTC
Permalink
Post by David Rajchenbach-Teller
I can confirm that old code-snippets were removed (and that both faster
solutions and environment
Post by David Rajchenbach-Teller
variable tweaks were rejected).
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".

During autumn 2004, on Brent Fulgham's website, new tasks were added
and old Doug Bagley tasks removed.

By 2005 only 2 tasks remained from that old Doug Bagley website.

By August 2008 none of those old Doug Bagley tasks were measured for
the current benchmarks game.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
o***@first.in-berlin.de
2010-11-23 23:27:53 UTC
Permalink
Post by David Rajchenbach-Teller
Post by David Rajchenbach-Teller
I can confirm that old code-snippets were removed (and that both faster
solutions and environment
Post by David Rajchenbach-Teller
variable tweaks were rejected).
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".
And what was wrong in his eyes?
Post by David Rajchenbach-Teller
During autumn 2004, on Brent Fulgham's website, new tasks were added
and old Doug Bagley tasks removed.
By 2005 only 2 tasks remained from that old Doug Bagley website.
By August 2008 none of those old Doug Bagley tasks were measured for
the current benchmarks game.
So, now the comparisions are perfect?
What problems were removed?


Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 00:23:33 UTC
Permalink
Post by o***@first.in-berlin.de
Post by Isaac Gouy
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".
And what was wrong in his eyes?
Find out for yourself:

http://web.archive.org/web/20010617014807/www.bagley.org/~doug/shootout/
Post by o***@first.in-berlin.de
So, now the comparisions are perfect?
Has anyone said so?
Post by o***@first.in-berlin.de
What problems were removed?
All of them.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Eray Ozkural
2010-11-24 01:36:35 UTC
Permalink
Hello,

I think that this benchmark is lacking in the algorithms department. Where
is a dynamic programming problem? A graph algorithm? Anything with
non-trivial time/space complexity? Anything a little more complex than
matrix product?

Also, it's not uncommon to disallow low-level optimizations such as writing
memory allocators and async file access when comparing implementations of an
algorithm, but such restrictions should be carried out uniformly. In such a
benchmark I would expect each entry to stick to their guns, i.e. use only
the standard libraries and programming styles for instance. Linking in
foreign libraries must most definitely be disallowed. So, if in Java, it's
necessary to call the garbage collector explicitly from time to time, and we
had to do that for a long time, so be it. Or again, if in Java, performance
will suffer unless you only use arrays of integral types, the implementer
may wish to implement as much as is possible with arrays, though I wonder if
it is not better to choose the most natural implementation style for the
particular language. In the case of Java, the claim was that object-oriented
was some kind of a programming-aid that can replace talented programmers :)
It's unfortunate of course that some kinds of optimizations always have to
be made by hand, for instance in functional languages many compilers do not
have deforestation.

Otherwise, of course, any implementation may include a compiler for the
fastest language and present a program in that language, which is not the
objective.

An alternative objective could be to compare the shortest and most
comprehensible, if possible line-to-line compatible implementation of a
given pseudocode in different languages. That would be extremely informative
for serious algorithm researchers! If a computer scientist isn't sure of the
performance of the primitives, he cannot make sure his implementation will
comply with the time-complexity of the given algorithm.

Best Regards,
Post by o***@first.in-berlin.de
Post by Isaac Gouy
Even back in 2001, Doug Bagley had noted all the things that were
wrong with the tasks on his "The Great Computer Language Shootout".
And what was wrong in his eyes?
http://web.archive.org/web/20010617014807/www.bagley.org/~doug/shootout/<http://web.archive.org/web/20010617014807/www.bagley.org/%7Edoug/shootout/>
Post by o***@first.in-berlin.de
So, now the comparisions are perfect?
Has anyone said so?
Post by o***@first.in-berlin.de
What problems were removed?
All of them.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Isaac Gouy
2010-11-24 02:14:07 UTC
Permalink
Hello, I think that this benchmark is lacking ...
http://shootout.alioth.debian.org/help.php#why


Please make the kind of comparison you think should be done and publish it.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jeff Meister
2010-11-24 04:39:29 UTC
Permalink
Everyone in this thread is capable of reading your site and has
probably already done so. We know what your rules are for
binary-trees; repeating them does not help. Richard's objection, which
you dismissed out of hand, was that your no-GC-tuning rule is silly in
the light of actual uses of garbage collected programming languages on
modern processors. It makes your results unrealistic, and an
unrealistic benchmark is misleading, or at best merely useless. You
are free to tersely reject our constructive criticism, but the only
meaningful consequence will be that OCaml users consider the shootout
untrustworthy and completely ignore its results... what good are the
"language comparisons" your project makes if the communities behind
those languages don't support your benchmarking methods?
Post by Isaac Gouy
Hello, I think that this benchmark is lacking ...
http://shootout.alioth.debian.org/help.php#why
Please make the kind of comparison you think should be done and publish it.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Andrew
2010-11-24 06:22:30 UTC
Permalink
::Jeff Meister
Everyone in this thread is capable of reading your site and has
probably already done so. We know what your rules are for
binary-trees; repeating them does not help. Richard's objection, which
you dismissed out of hand, was that your no-GC-tuning rule is silly in
the light of actual uses of garbage collected programming languages on
modern processors. It makes your results unrealistic, and an
unrealistic benchmark is misleading, or at best merely useless. You
are free to tersely reject our constructive criticism, but the only
meaningful consequence will be that OCaml users consider the shootout
untrustworthy and completely ignore its results... what good are the
"language comparisons" your project makes if the communities behind
those languages don't support your benchmarking methods?
+1. Seriously, Isaac, try to calm down, everything is fine. You might want
to read what others write, I have the feeling that many people were making
valid points, whereas you have mostly been turning down any objection by
pointing people to the same webpage and again.

Andrew.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 07:16:25 UTC
Permalink
Post by Andrew
+1. Seriously, Isaac, try to calm down, everything is fine. You might want
to read what others write, I have the feeling that many people were making
valid points, whereas you have mostly been turning down any objection by
pointing people to the same webpage and again.
Seriously, Andrew, I have been calm.

You might want to read what others write.

I have the feeling that a list of those "valid points" would be a better basis
for discussion than your feeling.

A quick count shows 8 URIs all different - so you are plainly wrong, I have not
been "pointing people to the same webpage and again".

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 06:50:49 UTC
Permalink
Post by Jeff Meister
We know what your rules are for
binary-trees; repeating them does not help.
When Christophe TROESTLER wrongly states - "OCaml is not authorized to make use
of its very own library!" - he shows that those rules are not known.
Post by Jeff Meister
Richard's objection, which you dismissed out of hand, was that your
no-GC-tuning rule is silly in the light of actual uses of garbage collected
programming languages on modern processors.
When said Richard opines about programs he apparently hasn't bothered to read, I
take that as a sign his opinions might not be based on anything solid.
Post by Jeff Meister
It makes your results unrealistic, and an
unrealistic benchmark is misleading, or at best merely useless.
You should think that benchmarks (not just these) are unrealistic - your
application is the ultimate benchmark.

Useless? Wouldn't that depend on the objectives? The post you replied to
linked-to a 3 line statement of objectives - did you read it?
Post by Jeff Meister
You are free to tersely reject our constructive criticism
Do you really think saying something is "ludicrous" or "silly" is constructive
criticism? :-)


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Christophe Troestler
2010-11-24 10:55:44 UTC
Permalink
Post by Isaac Gouy
We know what your rules are for binary-trees; repeating them does
not help.
When Christophe TROESTLER wrongly states - "OCaml is not authorized
to make use of its very own library!" - he shows that those rules
are not known.
Isolating a sentence gives a misleading idea of what I said. Must I
really repeat that I was asking WHY is C allowed to use an external
library to enhance its memory management¹ but OCaml cannot use its own
library to do that. I am not asking WHAT the rules are but a
JUSTIFICATION for them (which you have been incapable of providing so
far).

C.


¹ Apparently the "Please don't implement your own custom memory pool
or free list." does not count if it's implemented by others!

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Eray Ozkural
2010-11-24 11:33:12 UTC
Permalink
On Wed, Nov 24, 2010 at 12:24 PM, Christophe Troestler <
Post by Christophe Troestler
Post by Isaac Gouy
We know what your rules are for binary-trees; repeating them does
not help.
When Christophe TROESTLER wrongly states - "OCaml is not authorized
to make use of its very own library!" - he shows that those rules
are not known.
Isolating a sentence gives a misleading idea of what I said. Must I
really repeat that I was asking WHY is C allowed to use an external
library to enhance its memory management¹ but OCaml cannot use its own
library to do that. I am not asking WHAT the rules are but a
JUSTIFICATION for them (which you have been incapable of providing so
far).
I didn't get it, either.

Best,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
Isaac Gouy
2010-11-24 17:32:48 UTC
Permalink
Post by Christophe Troestler
Isolating a sentence gives a misleading idea of what I said.
gmane demands "Prune quoted stuff."
Post by Christophe Troestler
Must I
really repeat that I was asking WHY is C allowed to use an external
library to enhance its memory management¹ but OCaml cannot use its own
library to do that.
Do the OCaml binary-trees programs that are shown not use OCaml memory

management?

Do the C programs that are shown tune the external library memory management

they use?
Post by Christophe Troestler
I am not asking WHAT the rules are but a JUSTIFICATION for them (which you
have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for anything.
Post by Christophe Troestler
¹ Apparently the "Please don't implement your own custom memory pool
or free list." does not count if it's implemented by others!
Many C and C++ programmmers rolled their own custom free list implementations as

part of their program.

Library memory pool and free list implementations have to be more generally

useful (and probably less efficient) than custom solutions created specifically

for this tiny task.

"Please don't implement your own custom memory pool or free list." means exactly
what it says.




_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Christophe Raffalli
2010-11-24 17:57:33 UTC
Permalink
Hello,

Here is a test of gctweak.ml on the "now famous" binary-tree shootout bench ...
As you can see it is a 30% speed up which is not too bad, just adding
a file on the compilation command line !

I reattached the file, because I correct a few comments in it ... and a syntax error
that is only visible when not using camlp4 in ocaml-3.11.2 ( { get () with ... } is
valid with camlp4 and invalid without ???) ... Anyway, more work on Gctweak is still needed ...

***@d45-lama:~/Caml$ ocamlopt -o binary_tree gctweak.ml binary_tree.ml ; time ./binary_tree 20
stretch tree of depth 21 check: -1
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1

real 0m19.212s
user 0m18.960s
sys 0m0.180s
***@d45-lama:~/Caml$ ocamlopt -o binary_tree binary_tree.ml ; time ./binary_tree 20
stretch tree of depth 21 check: -1
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1

real 0m27.484s
user 0m27.270s
sys 0m0.110s

Here is the run with debug := 1 and you see that minor heap size is guessed at
524288, with almost no promoted word (model = 1 means no promoted word)

***@d45-lama:~/Caml$ ocamlopt -o binary_tree gctweak.ml binary_tree.ml ; time ./binary_tree 20
MHS DOUBLED <- 65536 (model 3.996155)
MHS DOUBLED <- 131072 (model 3.000397)
stretch tree of depth 21 check: -1
MHS DOUBLED <- 262144 (model 2.495375)
MHS DOUBLED <- 524288 (model 1.027698)
2097152 trees of depth 4 check: -2097152
524288 trees of depth 6 check: -524288
131072 trees of depth 8 check: -131072
32768 trees of depth 10 check: -32768
8192 trees of depth 12 check: -8192
2048 trees of depth 14 check: -2048
512 trees of depth 16 check: -512
128 trees of depth 18 check: -128
32 trees of depth 20 check: -32
long lived tree of depth 20 check: -1

real 0m19.342s
user 0m19.100s
sys 0m0.170s
--
Christophe Raffalli
Universite de Savoie
Batiment Le Chablais, bureau 21
73376 Le Bourget-du-Lac Cedex

tel: (33) 4 79 75 81 03
fax: (33) 4 79 75 87 42
mail: ***@univ-savoie.fr
www: http://www.lama.univ-savoie.fr/~RAFFALLI
---------------------------------------------
IMPORTANT: this mail is signed using PGP/MIME
At least Enigmail/Mozilla, mutt or evolution
can check this signature. The public key is
stored on www.keyserver.net
---------------------------------------------
Ed Keith
2010-11-24 19:08:07 UTC
Permalink
Post by Christophe Troestler
Post by Christophe Troestler
I am not asking WHAT the rules are but a JUSTIFICATION
for them (which you
Post by Christophe Troestler
have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for
anything.
Am I to interpret this to mean that the rules are purely arbitrary and capricious with no though behind them?

I hope I am misinterpreting this statement, but can think of no other interpretation

-EdK

Ed Keith
***@yahoo.com

Blog: edkeith.blogspot.com







_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 19:14:15 UTC
Permalink
Post by Ed Keith
Post by Christophe Troestler
Post by Christophe Troestler
I am not asking WHAT the rules are but a JUSTIFICATION
for them (which you
Post by Christophe Troestler
have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for
anything.
Am I to interpret this to mean that the rules are purely arbitrary and
capricious with no though behind them?

Tendentious.

A great deal of thought has been given to all aspects of the benchmarks game.
Post by Ed Keith
I hope I am misinterpreting this statement, but can think of no other interpretation
Poverty of imagination.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Rajchenbach-Teller
2010-11-24 19:17:42 UTC
Permalink
Hey, guys.
Time to stop this, please.

Thanks,
David
Post by Ed Keith
Post by Ed Keith
Post by Christophe Troestler
Post by Christophe Troestler
I am not asking WHAT the rules are but a JUSTIFICATION
for them (which you
Post by Christophe Troestler
have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for
anything.
Am I to interpret this to mean that the rules are purely arbitrary and
capricious with no though behind them?
Tendentious.
A great deal of thought has been given to all aspects of the benchmarks game.
Post by Ed Keith
I hope I am misinterpreting this statement, but can think of no other
interpretation
Poverty of imagination.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Oliver Bandel
2010-11-24 22:26:33 UTC
Permalink
halting problem discovered practically ;)
Post by David Rajchenbach-Teller
Hey, guys.
Time to stop this, please.
Thanks,
David
Post by Ed Keith
Post by Ed Keith
Post by Christophe Troestler
Post by Christophe Troestler
I am not asking WHAT the rules are but a JUSTIFICATION
for them (which you
Post by Christophe Troestler
have been incapable of providing so far).
I feel no need to provide a JUSTIFICATION to you for
anything.
Am I to interpret this to mean that the rules are purely arbitrary and
capricious with no though behind them?
Tendentious.
A great deal of thought has been given to all aspects of the
benchmarks game.
Post by Ed Keith
I hope I am misinterpreting this statement, but can think of no other
interpretation
Poverty of imagination.
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Stefan Monnier
2010-11-25 16:59:35 UTC
Permalink
Post by Jeff Meister
Richard's objection, which you dismissed out of hand, was that your
no-GC-tuning rule is silly in the light of actual uses of garbage
collected programming languages on modern processors. It makes your
results unrealistic, and an unrealistic benchmark is misleading, or at
best merely useless.
To the extent that this rule is the same for all languages and that most
languages on the shootout are also garbage collected, I think OCaml's
problem with this benchmark do point at a weakness of the current
GC code.

Of course, the shootout could be improved. E.g. maybe it could allow
extra submissions that break the rules, along with a description of
which rules were broken and how. Then there could be a "score according
to the rules", then a "score when all gloves are off", together with
some kind of "measure" of what was needed to go from one to the other.
This way people could maybe get a better feel for the languages's
performance and how (and how much) that performance can be affected.
Doesn't seem like an easy undertaking, tho.


Stefan

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-25 18:22:39 UTC
Permalink
Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

-snip-
Post by Stefan Monnier
Of course, the shootout could be improved.
Of course, anyone interested can do their own more extensive comparisons for a
couple of language implementations, for example -

http://blog.marketcetera.com/2007/03/08/java-and-the-computer-language-shootout/
Post by Stefan Monnier
E.g. maybe it could allow
extra submissions that break the rules, along with a description of
which rules were broken and how.
-snip-


Page search "alternative" on

http://shootout.alioth.debian.org/u32q/performance.php?test=binarytrees

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Jon Harrop
2010-11-25 22:12:10 UTC
Permalink
Post by Stefan Monnier
I think OCaml's
problem with this benchmark do point at a weakness of the current
GC code.
A difference but not necessarily a weakness. OCaml makes performance
transparent and gives you the ability to improve it with a high glass
ceiling. I prefer that to hiding the controls and using ad-hoc rules to
tweak things behind your back, rendering performance unpredictable and
making it hard to optimize your code.

Cheers,
Jon.


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Fabrice Le Fessant
2010-11-25 22:51:12 UTC
Permalink
I think OCaml's problem with this benchmark do point at a weakness of the current
GC code.
What makes you think that ?

I have contributed to some of the solutions that you can find there (and
some other ones were rejected because caching results is not allowed too
;-) ), and the GC was _NEVER_ a problem !

(except for binary-trees, but for absurd reasons)

The main problem was that other languages have bigger standard
libraries, whereas OCaml has a very small one (just what is needed to
compile the compiler, actually). In many problems, you could benefit
from using a very simple shared-memory library (in mandelbrot, the ocaml
multicore solution has to copy the image in a socket between processes,
whereas it could just be in a shared memory segment), and in general,
many solutions could benefit from specialised data structures that are
provided in other languages by their standard libraries, and from some
system calls that are currently not in the Unix library.

Anyway, I decided to stop trying to improve the solutions, and work on
improving the compiler and its libraries instead. It might benefit to
ocaml ranking in the shootout, but more importantely, it will benefit to
everybody in the community also.

--Fabrice
Isaac Gouy
2010-11-26 20:26:33 UTC
Permalink
Fabrice Le Fessant <Fabrice.Le_fessant <at> inria.fr> writes:

-snip-
Post by Fabrice Le Fessant
Anyway, I decided to stop trying to improve the solutions, and work on
improving the compiler and its libraries instead. It might benefit to
ocaml ranking in the shootout, but more importantely, it will benefit to
everybody in the community also.
¡bravo!

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Stefan Monnier
2010-11-27 18:56:13 UTC
Permalink
Post by Fabrice Le Fessant
I think OCaml's problem with this benchmark do point at a weakness
of the current GC code.
What makes you think that ?
I don't really understand the question: it was just stating the obvious.
OCaml's GC (including its default settings) is generally very good, but
like all GCs it has its weaknesses. This is fundamentally unavoidable.
There should be no shame admitting that this particular case hits one of
those weak spots.


Stefan

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
o***@first.in-berlin.de
2010-11-28 18:14:40 UTC
Permalink
On Thu, Nov 25, 2010 at 11:50:58PM +0100, Fabrice Le Fessant wrote:
[...]
Post by Fabrice Le Fessant
The main problem was that other languages have bigger standard
libraries, whereas OCaml has a very small one (just what is needed
to compile the compiler, actually). In many problems, you could
benefit from using a very simple shared-memory library (in
mandelbrot, the ocaml multicore solution has to copy the image in a
socket between processes, whereas it could just be in a shared
memory segment),
..so you work on a shared-mem module?!
Post by Fabrice Le Fessant
and in general, many solutions could benefit from
specialised data structures that are provided in other languages by
their standard libraries, and from some system calls that are
currently not in the Unix library.
[...]

During the last some releases a lot more unix syscalls were added
and that's fine of course).

Which calls are you missing there?

Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Christophe Raffalli
2010-11-27 16:00:57 UTC
Permalink
Hello,
Post by Stefan Monnier
To the extent that this rule is the same for all languages and that most
languages on the shootout are also garbage collected, I think OCaml's
problem with this benchmark do point at a weakness of the current
GC code.
This is untrue ... the bintree example, is just bad in OCaml because the
default
value of the minor heap size is the correct value for reactive programs
where you want fast minor GC slice, because they interrupt the program ...

Maybe the Gc library could provide reasonable default settings for both
cases
accessible via a simple function GC.is_reactive : bool -> unit ... And
the shootout
would allow to call such a function ?

Clearly, all benchmarks which just measure speed and space should use a
larger
minor heap size (except if reactive communication between processes is
important in the
benchmark).

More importantly, there is little hope to discover at runtime that a
program is
a reactive one ... In other words there is no hope for a perfect GC for
all situations.
So it is a good thing that OCaml allows tuning the GC.

Moreover, even when you automatically tune the GC (like changing the
number of generations in some modern GC) you do it based on past
observation of the behavior of your program ... and the behavior of the
program can change just after you changed some GC parameters ... So
begin able to help the GC, in a comprehensive way,
is good ...

Cheers,
Christophe
o***@first.in-berlin.de
2010-11-28 18:17:53 UTC
Permalink
Post by Sylvain Le Gall
Hello,
Post by Stefan Monnier
To the extent that this rule is the same for all languages and that most
languages on the shootout are also garbage collected, I think OCaml's
problem with this benchmark do point at a weakness of the current
GC code.
This is untrue ... the bintree example, is just bad in OCaml because the
default
value of the minor heap size is the correct value for reactive programs
where you want fast minor GC slice, because they interrupt the program ...
[...]

And if your program contains both kinds of functionality?
What possible solution would you recommend?


Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

David Rajchenbach-Teller
2010-11-24 06:55:55 UTC
Permalink
Maybe the solution is to get one of our numbers among the maintainers of the shootout. This would guarantee, if not objectivity, then at least informed choices wrt OCaml.
Post by Sylvain Le Gall
Hello,
I think that this benchmark is lacking in the algorithms department. Where is a dynamic programming problem? A graph algorithm? Anything with non-trivial time/space complexity? Anything a little more complex than matrix product?
Also, it's not uncommon to disallow low-level optimizations such as writing memory allocators and async file access when comparing implementations of an algorithm, but such restrictions should be carried out uniformly. In such a benchmark I would expect each entry to stick to their guns, i.e. use only the standard libraries and programming styles for instance. Linking in foreign libraries must most definitely be disallowed. So, if in Java, it's necessary to call the garbage collector explicitly from time to time, and we had to do that for a long time, so be it. Or again, if in Java, performance will suffer unless you only use arrays of integral types, the implementer may wish to implement as much as is possible with arrays, though I wonder if it is not better to choose the most natural implementation style for the particular language. In the case of Java, the claim was that object-oriented was some kind of a programming-aid that can replace talented programmers :) It's unfortunat
e of course that some kinds of optimizations always have to be made by hand, for instance in functional languages many compilers do not have deforestation.
Post by Sylvain Le Gall
Otherwise, of course, any implementation may include a compiler for the fastest language and present a program in that language, which is not the objective.
An alternative objective could be to compare the shortest and most comprehensible, if possible line-to-line compatible implementation of a given pseudocode in different languages. That would be extremely informative for serious algorithm researchers! If a computer scientist isn't sure of the performance of the primitives, he cannot make sure his implementation will comply with the time-complexity of the given algorithm.
Best Regards,
--
Eray Ozkural, PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
http://groups.yahoo.com/group/ai-philosophy
http://myspace.com/arizanesil http://myspace.com/malfunct
_______________________________________________
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 07:23:39 UTC
Permalink
Post by David Rajchenbach-Teller
Maybe the solution is to get one of our numbers among the maintainers of the
shootout. This would guarantee, if not objectivity, then at least informed
choices wrt OCaml.

Presumably you'd need all of the maintainers to be OCaml certified to guarantee
"objectivity" :-)

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-22 19:01:49 UTC
Permalink
On Nov 22, 6:04 am, Gerd Stolpmann <***@gerd-stolpmann.de> wrote:
-snip-
(I remember Ocaml was #1 at the shootout a few years ago, faster than C.)
-snip-

Is your memory infallible?

Even back in 2001, on Doug Bagley's website called "The Great Computer
Language Shootout" Ocaml was not #1-

http://web.archive.org/web/20010602051335/www.bagley.org/~doug/shootout/craps.shtml

Doug Bagley stopped updating his website in Fall 2001.


Even back in 2004, when Brent Fulgham started his website using the
same programs, OCaml was not #1

http://web.archive.org/web/20040717184911/shootout.alioth.debian.org/craps.php
Isaac Gouy
2010-11-23 02:10:14 UTC
Permalink
Gerd Stolpmann <info <at> gerd-stolpmann.de> writes:

-snip-
Post by Gerd Stolpmann
I think the shootout is not a good data source. There are definitely
some very poor Ocaml results there, so I'd guess the shootout got
recently more attention by enthusiasts of other languages, and the
current Ocaml programs there are not very good. (I remember Ocaml was #1
at the shootout a few years ago, faster than C.) So maybe a good
opportunity to post better Ocaml solutions there?
Even back in 2001, on Doug Bagley's website called "The Great Computer Language
Shootout" Ocaml was not #1-

http://web.archive.org/web/20010602051335/www.bagley.org/~doug/shootout/craps.shtml

Doug Bagley stopped updating his website in Fall 2001.


Even back in 2004, when Brent Fulgham started his website using the same
programs, OCaml was not #1

http://web.archive.org/web/20040717184911/shootout.alioth.debian.org/craps.php


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Fabrice Le Fessant
2010-11-22 16:40:02 UTC
Permalink
So maybe a good opportunity to post better Ocaml solutions there?
I spent some time improving OCaml solutions, and most of the time, my
solutions were refused: the organizers don't let you unroll loops, fix
GC parameters, etc. One strength of OCaml is that it runs fairly fast
immediatly, but you can make it run even faster with some tuning,
something that you cannot do with many other languages. Such tuning is
not allowed there, so, it is not a good place for OCaml hackers. If
the organizers had been consistent, they would have forbidden Haskell
programmers from adding strictness in their programs, but then, there
programs would have been 2x slower than OCaml programs.

Well, actually, things might improve in the future, if we manage to
improve the performance of the compiler itself, and have more
"official" libraries. For example, most parallel solutions in OCaml
currently use fork() and sockets for communication. Using an official
shared-memory library, we could directly improve the performances of
many OCaml solutions.

--Fabrice

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Philippe Strauss
2010-11-22 17:49:05 UTC
Permalink
answer to the original poster:

ocaml is surprisingly fast for such a high level language, i like this
insight/comparison:

http://youinfinitesnake.blogspot.com/2010/09/programming-language-wars-movie.html


(even if the benchmarks can be improved reading others posts of this
thread)

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Oliver Bandel
2010-11-22 18:33:15 UTC
Permalink
Hi,
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
[...]


I remember one argument on that topic, many years ago,
maybe rom Xavier leoy or other core developers, and it goes like that:

Benchmarks on such small problem solving is ONE comparision, what
makes sense, but it must be added by comparisions of "optimizations"
of the code in large applications.

The lower the level of the öanguage the harder it is to keep track of
possible optimizations on a very big program.

But the higher the language level is, the easier it is to
keep that in mind.

So: when optmizing small snippets of code this helps optimizing
performance in maybe critical sections, but to get an overview on all
interdependencies is close to unmanagable with lower leveled languages.



I have not seen a comparison of performance in this repect,
but I also would think higher level languages might be a better choice here.


Any comparisions on that scale of programming, which you know of?

Then please throw in a link here.

Thanks.

Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-22 19:34:23 UTC
Permalink
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while
reading an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw
that OCaml was praised for the speed of the executables it generates -
and was referred to, speed-wise, as "second to none", except C and
C++.
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are...)
Is it just hype, then? Or am I missing something?
Note the "Why Ocaml?" date - December 2002.
Isaac Gouy
2010-11-23 02:00:16 UTC
Permalink
Thanassis Tsiodras <ttsiodras <at> gmail.com> writes:

-snip-
Post by Thanassis Tsiodras
Is it just hype, then? Or am I missing something?
Note the "Why Ocaml?" date - December 2002.

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Allsopp
2010-11-24 10:55:43 UTC
Permalink
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
I am on the fence about whether to learn OCaml or not, and while reading
an article called "Why OCaml"
(http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw that
OCaml was praised for the speed of the executables it generates - and was
referred to, speed-wise, as "second to none", except C and
C++.
Do remember that speed isn't always everything - there are many applications where it's just not critical. Maintainability, readability, ease-of-use, availability of libraries are large concerns too.
Post by Thanassis Tsiodras
However, when I actually went to the Language Shootout page suggested in
the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-
fastest.php)
As the kinda large number of messages in this thread suggests, benchmarks can be very subjective :o)

The Wiktionary definition of "to benchmark" is "To measure the performance of (an item) relative to another similar item in an impartial scientific manner". IMHO the problem benchmarking "programming languages" (note - "programming languages", not "problems solved in programming languages") lies in the word "similar" in that definition. Reducing an entire programming language's strengths (or weaknesses!) to a single number is just not really realistic - the truth is more complex than one single-precision floating point number (or even an array of them) can describe. (NB. The shootout doesn't claim that the final ranking displayed is anything other than a score of how well the languages did at the various benchmarks given - but a graph like that is easy to interpret erroneously in that way)
Post by Thanassis Tsiodras
Is it just hype, then? Or am I missing something?
OCaml's big strengths (by no means unique to OCaml, incidentally) for me are its ML legacy - automatic memory (de-)allocation, type inference, static type-checking (i.e. no BizarreRuntimeTypeException's here...) and polymorphism - all things which allow me to write less (and clearer) code to solve a given problem. I also find some of the more exotic features useful - private types and polymorphic variants, for example. I'm forced to program in a few less desirable languages from time to time and sorely miss the basic features of ML. I'd also echo Sylvain's point: I too find the time spent testing and debugging OCaml is much, much lower than, say, equivalent C(++) or Java code.

Personally, I've found the best way to find out if you like a new programming language is to write a non-trivial program in it (maybe with the occasional assistance of skilled people in its community) and then see what happens when you go back to programming in something you used before.

And if speed really is worrying you, the big thing you may have noticed from the, ahem, slightly animated discussion that's followed is that there are plenty of people on this list who know how to help you identify what's causing the problem and then offer suggestions to tweak it. Personally, in six years of programming in OCaml for a variety of tasks I've never hit the brick wall where I felt the task I was carrying out would have been orders of magnitude faster in another language *and also easy to code in that language*!

HTH,


David
Isaac Gouy
2010-11-24 18:39:52 UTC
Permalink
David Allsopp <dra-news <at> metastack.com> writes:

-snip-
Post by David Allsopp
Reducing an entire programming language's strengths (or
weaknesses!) to a single number is just not really realistic - the truth is
more complex than one
Post by David Allsopp
single-precision floating point number (or even an array of them) can
describe. (NB. The shootout
Post by David Allsopp
doesn't claim that the final ranking displayed is anything other than a score
of how well the languages did
Post by David Allsopp
at the various benchmarks given - but a graph like that is easy to interpret
erroneously in that way)
-snip-


That summary page
Post by David Allsopp
Post by Thanassis Tsiodras
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-
fastest.php)
shows box plots (quartiles, outliers, median) of the normalized timing
measurements for each of the tasks, for each of the language implementations.

A graph like that shows some of those language implementations are very fast for
some benchmark programs and very slow for others.

To characterize that as "reducing an entire programming language's strengths (or
weaknesses!) to a single number" seems kind-of misleading.

Especially when the question that page answers is stated 3 times - "Which
programming language implementations have the fastest benchmark programs?"




_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
David Allsopp
2010-11-24 21:04:21 UTC
Permalink
Post by Isaac Gouy
-snip-
Post by David Allsopp
Reducing an entire programming language's strengths (or
weaknesses!) to a single number is just not really realistic - the
truth is more complex than one single-precision floating point
number (or even an array of them) can describe. (NB. The shootout
doesn't claim that the final ranking displayed is anything other than
a score of how well the languages did
at the various benchmarks given - but a graph like that is easy to
interpret erroneously in that way)
-snip-
That summary page
Post by David Allsopp
Post by Thanassis Tsiodras
(http://shootout.alioth.debian.org/u32/which-programming-languages-a
re-
fastest.php)
shows box plots (quartiles, outliers, median) of the normalized timing
measurements for each of the tasks, for each of the language
implementations.
A graph like that shows some of those language implementations are very
fast for some benchmark programs and very slow for others.
I'm not sure I disputed that anywhere, in fact I think if anything implicitly agreed with it...
Post by Isaac Gouy
To characterize that as "reducing an entire programming language's
strengths (or weaknesses!) to a single number" seems kind-of misleading.
I'm not clear how you extracted that from what I said which was that reducing a programming language to a single number (or three numbers, if you want bars) is not a good summary of the *total* strengths of that programming language. I then went on to stress that the combined results from the shootout don't claim to do that, they simply show an interpretation of the combined results from the different benchmarks.

My attempt at a point was that ranking programming languages (that's just "programming languages" not "execution speed of programming languages") is a largely futile activity, because it's just too subjective and hard to quantify in a rigorous manner, and importantly that the shootout doesn't try to do that. The relevance and value of that point to Thanassis' original question is for him to decide...
Post by Isaac Gouy
Especially when the question that page answers is stated 3 times - "Which
programming language implementations have the fastest benchmark programs?"
Which part of my statement "The shootout doesn't claim that the final ranking displayed is anything other than a score of how well the languages did at the various benchmarks given" (prefixed with the Latin abbreviation for "Note well") caused you to need to write this?


David

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-25 00:16:55 UTC
Permalink
Post by David Allsopp
Post by Isaac Gouy
Post by David Allsopp
(NB. The shootout
doesn't claim that the final ranking displayed is anything other than
a score of how well the languages did
at the various benchmarks given - but a graph like that is easy to
interpret erroneously in that way)
-snip-
Which part of my statement "The shootout doesn't claim that the final ranking
displayed is anything other
Post by David Allsopp
than a score of how well the languages did at the various benchmarks given"
(prefixed with the Latin
Post by David Allsopp
abbreviation for "Note well") caused you to need to write this?
The part you haven't included in your quotation ;-)






_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Cedric Cellier
2010-11-24 14:07:31 UTC
Permalink
I was in a similar position than yours two years ago, looking for a
higher level language than C to gain some expressive power but without
willing to sacrifice C's speed. I consulted a lot of benchmarks from
the shootout, and also many webpages similar to the one you refer to,
some praising a language some criticizing it.

The various contenders of the shootout seamed to fall in three categories :
languages that are fast compared to C, but does not reduce much the size
of the source code (C++, java, APS...), languages that allow concise
manipulations of non-trivial data structures (Lisps, MLs, Haskells), and
non-compiled languages that I never considered adopting.

I eliminated Haskell because GHC was only usable on x86 architecture at
that time, so I tried various Lisp at first. Then a friend pointed me to
Ocaml since, according to him, it's easier to cross the gap between C
and Ocaml than between C and anything else. I thus finally tried Ocaml
and, despite my initial apprehensions, I felt myself comfortable with
its syntax after am couple of days.

I am now using it whenever I have the choice to, after many years of C,
mixing with C or ASM when necessary (which is quite easy), because I
feel it's pleasant to use, and that I have still many things to learn
from it, without the bad feeling to waste CPU cycles along the way.

I like to think that if yon consider only speed and terseness of the
code, Ocaml is still amongst the top languages ; probably not the #1 as
stated by the (quite old) webpage you pointed to, but still excellent.


_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Vincent Aravantinos
2010-11-24 14:34:18 UTC
Permalink
Hi Thanassis,
Post by Thanassis Tsiodras
I apologize beforehand if this is not the forum to ask.
(...)
Is it just hype, then? Or am I missing something?
may we know, after all this intense discussion, what is your feeling?

Did this debate enlightened your views?
Did it make arise new questions?
Are you going to learn Ocaml?

At least you know now that Ocaml has a lively community :)
I just hope it did not disgust you...

Cheers,
V. Aravantinos
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Thanassis Tsiodras
2010-11-24 15:30:09 UTC
Permalink
On Wed, Nov 24, 2010 at 4:34 PM, Vincent Aravantinos
Post by Vincent Aravantinos
may we know, after all this intense discussion, what is your feeling?
Well... (ducks, wears helmet).

Dr Jon Harrop communicated with me directly (two days ago)... and when
I expressed my lack of faith after reading his "Rise and fall of
OCaml" article at
http://flyingfrogblog.blogspot.com/2010/08/rise-and-fall-of-ocaml.html
, he in fact counter-suggested that F# is now the horse to bet on -
so... what do you guys think?

Over the last couple of days, I've played a lot with ocaml (to be
exact, Linux/ocamlopt, since my interest in the speed of what I make
remains dominant) as well as F# (with Visual Studio 2008). To my
limited understanding, the differences between OCaml and F# are small
- and the benefits of direct access to the .NET ecosystem of libraries
seems to counter the ... uncertain status of OCaml libraries. I just
begun looking into all this, so I could be very wrong, of course - but
I am spoiled rotten with Python's libraries, so not having "batteries
included" in OCaml seemed like quite a problem... until I realized F#
completely covers this.

So, to conclude - what do you guys think about F# ?
Post by Vincent Aravantinos
Did this debate enlightened your views?
Did it make arise new questions?
Are you going to learn Ocaml?
At least you know now that Ocaml has a lively community :)
I just hope it did not disgust you...
Cheers,
V. Aravantinos
--
What I gave, I have; what I spent, I had; what I kept, I lost. (Old Epitaph)

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Oliver Bandel
2010-11-24 16:27:00 UTC
Permalink
Hi,


Zitat von "Thanassis Tsiodras" <***@gmail.com>:

[...]
Post by Thanassis Tsiodras
Over the last couple of days, I've played a lot with ocaml (to be
exact, Linux/ocamlopt, since my interest in the speed of what I make
remains dominant)
[...]

OK, then I asume, you will be one of the programmers who
pick out the right datastructures for his applications.

When looking at C because of speed, this is completely ok.

But I would estimate that 80% or 90% of the programmers
just don#t think enough on that topic.

Many people who work as programmers, just pick a language
and a library.... just that library that is the most mainstream
library and start working.

Also many C-programmers I have seen have just done
seemingly simple hacking and not thought about data structures enough.

For simple programs and little amount of input data that's often
ok, but in the history of a program will turn out as bad decision.

So: what are all these benchmarks on different data structures worth,
if C-programmers always stick to arrays, or, maybe, to linked lists?

When I see how Gnome (and I would guess other desktop environments
will handle it thje same) handles areas of the screen.... that it does
not use trees that are appropriate for handling two-dimensional
spatial data, and then experience slow behaviour, then all is said....


I mean: OK, a low performant language will definitley make things
slower than a higher performant one - when usd with the same algorithms.

But when one does not use intelligent programming, this all does not
help much.
Then you can use C and be slower than, say, Python (which btw also
offers some nice datastructures, like Sets and so on).


But maybe what I write here is nothing new to you.

I just wanted to have it mentioned, that benchmarks alone does not
guarantee fast running programs.

A langauge that supports you to easily implement the datastructures you
would like to use, is much better supporting you then in this respect,
because if it needs toomuch effort to implement such a datastructure,
lazyness of the programmer might be stronger than the will to use
beter data structures.... especially if you have to fight with
dangling pointers on the one hand, where you on the other hand will be
warned in advance, that you just didn't got the types right and the
program will not compile then... which yields in fixing errors at
compilation stage.


Ciao,
Oliver

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Vincent Aravantinos
2010-11-24 16:34:09 UTC
Permalink
Post by Thanassis Tsiodras
On Wed, Nov 24, 2010 at 4:34 PM, Vincent Aravantinos
Post by Vincent Aravantinos
may we know, after all this intense discussion, what is your feeling?
Well... (ducks, wears helmet).
Dr Jon Harrop communicated with me directly (two days ago)... and when
I expressed my lack of faith after reading his "Rise and fall of
OCaml" article at
http://flyingfrogblog.blogspot.com/2010/08/rise-and-fall-of-ocaml.html
, he in fact counter-suggested that F# is now the horse to bet on -
so... what do you guys think?
Over the last couple of days, I've played a lot with ocaml (to be
exact, Linux/ocamlopt, since my interest in the speed of what I make
remains dominant) as well as F# (with Visual Studio 2008). To my
limited understanding, the differences between OCaml and F# are small
- and the benefits of direct access to the .NET ecosystem of libraries
seems to counter the ... uncertain status of OCaml libraries. I just
begun looking into all this, so I could be very wrong, of course - but
I am spoiled rotten with Python's libraries, so not having "batteries
included" in OCaml seemed like quite a problem... until I realized F#
completely covers this.
So, to conclude - what do you guys think about F# ?
I personally do not know much of F#. Particularly because I do not
have a Windows machine :( We all know here that Jon is very fond of F#
after having been fond of Ocaml for a while.

The socalled "Ocaml mass exodus" mentionned in Jon's article seems to
me as as objective as the language shootout benchmarks ;) In
particular the plot I made myself of the posts to the mailing list is
definitely not as clear as the one presented in the article (you can
do it yourself from the figures at <http://caml.inria.fr/pub/ml-archives/caml-list/index.fr.html
Post by Thanassis Tsiodras
). Jon also mentions (in the comments) the figures from Google
Trends about Ocaml vs F# (<http://www.google.co.uk/trends?q=f
%23%2Cocaml>). But funnily enough, you can see from the excerpts
selected by Google that F# deals as much with the language than
with... ahem, other stuff. Furthermore the plot was already increasing
before the release of F#, so is this increase really significative? My
whole point here is not that Jon is wrong or right: maybe he is right,
I just say that the supposed exodus does not seem significative to me.
Anyhow I do not "feel" it. I would even say that I feel the contrary
(and I am not the only one: see Paolo's comment in Jon's article).

Then Jon suggests in his article that that this is due to the
"inability of Ocaml's GC to [deal with parallelism]". Again, all of us
here know that Jon has been quite frustratred with this for a while
(and not only him, cf some very recent thread <http://caml.inria.fr/pub/ml-archives/caml-list/2010/11/642bb03b01227d825e71b0a46ae7e73f.fr.html
Post by Thanassis Tsiodras
). However there are also plenty of guys who are not bothered by
this (see the very same thread). Choose your camp. On one hand, if you
go to F# you won't have to choose. On the other hand, this does not
make F# necessarily faster than Ocaml (as Jon himself proved it: <http://flyingfrogblog.blogspot.com/2010/08/pure-f-now-only-2-slower-than-ocaml.html
Post by Thanassis Tsiodras
).
Of course if you listen to Jon you will be convincend that you should
choose F#. Many people here will probably react by saying that you
should choose Ocaml. I am personnally neutral here: in the end, I
would say that both languages are great (about F# I should say "look
great" since I never tried it). So whatever is your final choice I
guess you won't be disappointed.

Cheers,
Vincent

PS: BTW about the complain in Jon's article about the lack of a native
REPL in ocaml, I think this recent post also answers it very partly: <http://caml.inria.fr/pub/ml-archives/caml-list/2010/11/aef35ed6ad039ce80c5c66175e80fcc5.fr.html
Post by Thanassis Tsiodras
. Probably nothing compared, though, to the F# REPL (trusting Jon on
that).
_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Isaac Gouy
2010-11-24 18:16:34 UTC
Permalink
Thanassis Tsiodras <ttsiodras <at> gmail.com> writes:
-snip-
Post by Thanassis Tsiodras
However, when I actually went to the Language Shootout page suggested
in the article, I found out that OCaml is not 2nd, it is 13th, behind
languages like Haskell and C#...
(http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php)


You've noted the rank of OCaml (2nd and then 13th)

Can the rank position of OCaml change without the performance of the fastest
programs or the OCaml programs changing?

Can rank position in some list indicate how much faster or slower OCaml may be
than the fastest language implementation?

Is the rank position of OCaml stated on that old Doug Bagley scorecard?

http://web.archive.org/web/20010602051335/www.bagley.org/~doug/shootout/craps.shtml

Is the rank position of OCaml stated on the current benchmarks game page you
linked-to?

_______________________________________________
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Continue reading on narkive:
Loading...