[Haskell-cafe] Re: Go Haskell! -> array libraries

[Claus Reinke]

>> Should mutable arrays have list-like APIs? All the usual operations,
>> just in-place and destructive where appropriate?
> 
> I don't know. To be honest, I don't think that the term "mutable  
> array" describes a single data structure. For instance, one of the  
> central questions which unveils a whole bunch of design possibilities  
> is: can mutable arrays be concatenated and how does that work if yes?

wrt the first part: within a pure functional language, mutable arrays
are something I (am forced to) choose when the implementation
is unable to see some essential optimization opportunities in the
way I am using immutable arrays; whenever I am forced to go
down that route, I'd at least like to have an equivalent set of
operations. The move from immutable arrays to mutable arrays
is painful enough as it is. So the suggested design criterion would
be: as uniform an API as possible, just that immutable arrays
cannot make some essential performance guarantees and some
operations sit a little uncomfortably in a librart where those
guarantees are important.

wrt the second part: I'm not sure whether you'll be able to choose
a single one of those design possibilities as the only one (perhaps
the concatenation is about simpler code, so virtual concatenation
would be sufficient and copying possible, but perhaps the goal
was locality, so copying would be needed, or perhaps a mixture
of both, and the limited disruption in locality is less expensive than
the cost of copying, ..), so you might want to expose several of 
them, with a separate way of selecting a default or I-don't-care 
choice (imilar to why you have several array libs at present, 
just that there should be a common API and documentation of
design alternatives/rationales;-).

--

Btw, it would really be nice if packages took a hint from academic
publishing: good papers are expected not only (a) to provide new
content, but also (b) to position that content wrt related work
and (c) to make explicit what the new contributions/goals are.

As forks have become more common on hackage, perhaps
.cabal files could be extended with two fields, pointing to related
packages (this is more specific than category, linking to packages
that might be used instead or in combination with the current 
package) and giving the rationale/high-lights for the package in 
text form. 

In the meantime, it would be great if all packages came with good 
old README files, which should be linked from the .cabal-based 
package descriptions on hackage, covering all those useful bits of 
information that are not yet covered in .cabal files, and giving an 
overview of what the package is about if the package does not 
have its own webspace (as is often the case). That would improve 
on the current situation, where sometimes all one has to go on is 
the package name and a one-liner description.

Random example: looking at hackage package categories, I'd
look for array libraries under 'data structures', but they are 
actually spread over 'data structures', 'data', and 'maths' (perhaps
others?). Once I've found one, say 'vector' or 'uvector', what
do the package descriptions tell me about the packages and
their relation, or their relative advantages? 

The descriptions are brief, the "home pages" are actually "home 
repositories", the haddocks seem to have details only, no overview, 
and from the references/authors/uvector README, one might
think these are actually the same library, sequential libs spun 
off from the same data-parallelism project; only that the 
haddock details suggest that these libraries are quite different.
How? Why? What other alternatives are there?

Note that I'm not attacking any specific packages, I would 
just encourage all package authors to try and see their packages
from the perspective of a hackage user: what information does
the user look for, what information does the package description
offer?

--

While we're on the topic: hackage has grown so much and so
rapidly, that it might be worthwhile to have a hackage "editor",
not in the sense of accepting/rejecting packages (not wanted
at present), nor in the sense of objective quality measurements
(that is in the process of being automated, according to Duncan),
but in the sense of trying to guarantee a subjectively useful overall 
presentation (category organisation, finding and putting side-by-side 
related packages despite different names/categories, suitability of 
package descriptions, getting package authors to talk, etc.). 

And no, I'm not volunteering!-) But I would imagine that someone 
might find this a useful way of contributing. Such a hackage editor 
would also be able to give the Haskell Community Report editor 
a hand by summarizing hackage activity/trends and highlighting 
interesting projects that the HCAR editor might want to solicit 
reports for. From my own time as HCAR editor, I recall finding
and chasing interesting projects as well as deciding how to 
structure the tools&libraries sections as difficult parts of the job.

To take another hint from paper publishing: once the volume
and variety of publications passes certain thresholds, survey
papers become very valuable contributions. We do have time
based surveys (biannual, weekly, even daily, if you count
unedited RSS feeds;-), but topical surveys would help a lot
as well (what are all the options, relative advantages and status 
of hackage packages for arrays, or databases, or web 
programming, or..). 

Such mini-surveys could appear in the Monad.Reader wiki
magazine, for instance, and be written by anyone looking for 
that information the first time ("my experiences in looking for 
database libraries", etc.), then updated on the wiki as the 
package authors respond and improve the experience.

Obviously, I can't speak for the current hackage maintainers
or HCAR and Monad.Reader editors, but I assume they are 
busy enough that they might welcome this kind of help.

Claus