can a lazy language give fast code?

[D. Tweed]

On Tue, 30 Jul 2002, Andrew J Bromage wrote:

[snip]
> In the end, though, benchmarks ignore one of the most important rules
> of software performance: "throughput" (i.e. the amount of processing
> that your system can do just prior to being overloaded) is almost never
> the most important consideration.  Other considerations such as
> flexibility, robustness, responsiveness and scalability are almost
> always more important.

Mmm, such statements really assume that there's a sensible meaning to
`almost always' when applied to the set of all programmers, whereas I
think a much more realistic assumption is that `there's lots of people out
there, all with different priorities' and present things in way which 
lets people perform their own evaluations. (In cases where I've reason to
believe how I code I can simply, reliably and significantly affect
throughput I care very much about it.) The problem with language
benchmarks is not that they `over-rate' the importance of performance but
that they assume per se that choice of language is a single-variable
(execution speed) optimization problem; there's no attempt to measure the
other items in your list, most especially flexibility. (I'm assuming you
mean flexibility of the programmer rewriting, retargeting, refactoring and
re-engineering exisiting code.) Of course, I don't have any good ideas
about how to measure these, particularly flexibility, in a practicallty
implementable and accurate way :-)

> I've thought for a while that what we need is more benchmarks like
> pseudoknot: Real tasks which real people want to do.  Computing
> Ackermann's function is all well and good, but when's the last time you
> actually needed to compute it in a real program?

I suspect there probably are things that make Ackermann's function a bad
`test-case' (eg, computationally simple and regular => good cache
utilisation after optimzation which don't extrapolate?)  but, for the
purposes I'd want to use benchmarks for these deficiencies compared to
things like BLAS performance -- which are things that _some_ real people
do all day -- are probably don't affect the results all that much. Of more
concern to me is, when's the last time you actually got a well specified
computational problem and a reasonable amount of time to write a carefully
crafted program to solve it, (particularly when you had some reassurance
that the very specification of what to solve wouldn't change after the
first time you ran the code :-) )?

> Off the top of my head, some "real" tasks which could be benchmarked
> include:
> 
> 	- MPEG video compression.

Here you really want to measure `glue-code' overhead (both performance
wise and `human rewriting'-wise) of linking together core processing
elements either written in low-level code (MMX, etc) or available on DSP
chips, in a way which would allow shifting components and algorithms
about. (I.e., in my opinion it's an informative task to have
`benchmark-type' data about because it's complicated with many ways to
solve the problem, not because it's `real world'.)

___cheers,_dave_________________________________________________________
www.cs.bris.ac.uk/~tweed/  |  `It's no good going home to practise
email:tweed@cs.bris.ac.uk  |   a Special Outdoor Song which Has To Be
work tel:(0117) 954-5250   |   Sung In The Snow' -- Winnie the Pooh