[Haskell-cafe] Haskell-Cafe Digest, Vol 158, Issue 29

[Richard A. O'Keefe]

On 2/11/16 10:28 AM, Joachim Durchholz wrote:
We're still not really communicating, so it may be time to draw this
thread to a close soon.


> Am 01.11.2016 um 01:37 schrieb Richard A. O'Keefe:
>>
>>
>> On 1/11/16 9:54 AM, Joachim Durchholz wrote:
>>>
>>> And you need to control memory coherence, i.e. you need to define what
>>> data goes together with what processes.
>>
>> At this point I'm puzzled.  Languages like Occam, ZPL, and Co-Array
>> Fortran basically say NO! to memory coherence.
>
> Sure, but unrelated.

How is it unrelated?  If you don't need, don't want, and don't have
"memory coherence", then you don't have to control it.

> The hope with FPLs was that you do not need to explicitly specify it
> anymore, because the compiler can manage that.

The Alan Perlis quote applies:
    When someone says: "I want a programming language in which
    I need only say what I wish done", give him a lollipop.

In declarative languages, you give up explicit control over some things
in order to make other things easier to express.
A compiler has three ways to decide which are the "important" things:
  - by analysis
  - by being told (there's the famous anecdote about the original
    Fortran FREQUENCY statement being implemented backwards and nobody
    noticing)
  - by measuring (standard technology in C compilers for years now)

Nearly 20 years ago I was using a C compiler that would shuffle
functions around in your executable in order to reduce page faults.
That's the "measurement" approach.

>> If the data need to be available
>> in some other process, there is some sort of fairly explicit
>> communication.
>
> Which means that you do not have a simple function call anymore, but an
> extra API layer.

Here you have left me behind. WHAT was "a simple function call"?
WHAT is "an extra API layer", as opposed to annotations like the
distribution annotations in ZPL and HPFortran?

>> For what it's worth, the "Clean" programming language used to be
>> called "Concurrent Clean" because it was set up to run on a cluster
>> of Macintoshes.
>
> Clean is strict ;-)

That one line provoked this reply.  Clean isn't any stricter than
Haskell.  It does strictness inference, just like GHC.  It allows
strictness annotations, just like GHC (though not in Haskell 2010).

  Plus, the more I read about various
> forms of partitioning computations (not just NUMA but also IPC and
> networking), the more it seems that hardware moves towards making the
> barriers higher, not lower (the reason being that this helps making
> computations within the barrier more efficient).
>
> If that's a general trend, that's bad news for network, IPC, or NUMA
> transparency. Which is going to make programming for these harder, not
> easier :-(

On the one hand, agreed.  On the other hand, while programming in the
1960s still *is* getting harder, it's not clear that we cannot find an
approach that will be easier.

To my astonishment, quite a lot of the people using NZ's supercomputer
facility are programming in Python (basically glue code hooking up
existing applications), or Matlab (serious number-crunching, which
they say they've benchmarked against C), or even R (yes, R; NOT one
of the world's faster languages, BUT again we're mostly talking about
glue code hooking together existing building-blocks).  And this facility
has people who will help researchers with their supercomputer
programming for free.

The Science Data Processor for the Square Kilometre Array will
have a multi-level structure
  - compute elements contain multiple cores, probably GPUs, and
    probably FPGAs.  These are roughly the equivalent of laptops.
  - compute nodes are tightly integrated clusters containing
    multiple compute elements + interconnects + extra storage
  - compute islands are looser clusters containing multiple
    compute nodes and management stuff
  - the SDP will contain multiple compute islands + massive
    interconnects (3TB/sec data coming in)
  - there will be two SDPs, each sharing a multimegawatt
    power station with a signal processing system that talks
    directly to the telescopes.

Managing massive data flows and scheduling the computational tasks
is going to be seriously hard work.

One of the main tools for managing stuff like this is ISOLATION,
and LIMITING communication as much as possible,
which is the very opposite of the large flat shared memory model.