From dominic at steinitz.org  Mon Jul  6 14:06:07 2020
From: dominic at steinitz.org (dominic at steinitz.org)
Date: Mon, 6 Jul 2020 15:06:07 +0100
Subject: [Numeric] HMatrix collaboration
In-Reply-To: <CAJGqWSM85QUaMsDSnK++7sVENo55AneBmD0MrDEZewv_j2fmXQ@mail.gmail.com>
References: <CAJGqWSMtQ2_-M5r-J-Bz_9QLzXyDM+xzHQMN-nMc_O8106z0gg@mail.gmail.com>
 <EA56150C-9D3E-489F-9158-ED4D69B667A9@steinitz.org>
 <CAJGqWSM85QUaMsDSnK++7sVENo55AneBmD0MrDEZewv_j2fmXQ@mail.gmail.com>
Message-ID: <A955DF0B-7ED6-4782-B8F4-93497BC81762@steinitz.org>

Hi Huw,

You are right. I have given you access to the hmatrix repo. We don’t currently have any modus operandi for the organisation in which this repo lives. I assume that we will all create branches for review before merging?

I’ve cc’d the somewhat unused mailing list. If that doesn’t work for e.g. Q&A then we can try something else.

Dominic Steinitz
dominic at steinitz.org
http://idontgetoutmuch.org
Twitter: @idontgetoutmuch

> On 6 Jul 2020, at 08:09, Huw Campbell <huw.campbell at gmail.com> wrote:
> 
> Hi Dominic,
> 
> Thanks very much. I still can't seem to push to the hmatrix repository, I think the permissions are separate to the haskell-numerics organisation.
> 
> Is there a mailing list or unofficial list of email addresses we can use for questions and change proposals? Some questions (like, why is CSR 1 indexed?) feel a bit strange for an issue tracker.
> 
> Regards,
> Huw
> 
> On Sat, Jul 4, 2020 at 5:56 PM <dominic at steinitz.org <mailto:dominic at steinitz.org>> wrote:
> Hi Huw
> 
> Great 😁
> 
> I am just about to go out but some thoughts
> 
> Roman Cheplyaka has updated hmatrix-sundials to use sparse matrices - I have an application for this solving some PDEs which I have yet to try.
> 
> I think Henning Thielemann has also done some work on sparse matrices / bindings to BLAS / LAPACK.
> 
> The collaborative effort to improve random number generation worked really well - hopefully we can repeat the experience in this important area.
> 
> Dominic Steinitz
> dominic at steinitz.org <mailto:dominic at steinitz.org>
> http://idontgetoutmuch.org <http://idontgetoutmuch.org/>
> Twitter: @idontgetoutmuch
> 
>> On 4 Jul 2020, at 05:05, Huw Campbell <huw.campbell at gmail.com <mailto:huw.campbell at gmail.com>> wrote:
>> 
>> Hi Dominic,
>> 
>> I've been thinking about putting some more work into hmatrix for a few
>> years now, ever since I used it effectively in my neural network
>> library Grenade.
>> 
>> I currently have a few PRs up to fix and speed up sparse matrix creation.
>> 
>> I would like to really tighten up its benchmark and test suites (bring
>> them into the packages themselves instead of hosting them in an
>> external package; remove the poorly conceived `Testable` type class;
>> using criterion a lot more; and possibly porting tests to hedgehog).
>> 
>> Other items on the cards are using im2col to speed up convolutions (I
>> have this working in Grenade already) and make them easier to use with
>> user defined padding and stride.
>> 
>> It's quite a bit to do, so I was wondering if I could become a collaborator?
>> 
>> In terms of Haskell projects I maintain, the main open source one
>> these days is optparse-applicative, which I've been incrementally
>> improving over the past 5 years.
>> 
>> Kind regards,
>> Huw
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/numeric/attachments/20200706/770b2c0b/attachment.html>

From huw.campbell at gmail.com  Tue Jul  7 04:01:58 2020
From: huw.campbell at gmail.com (Huw Campbell)
Date: Tue, 7 Jul 2020 14:01:58 +1000
Subject: [Numeric] HMatrix collaboration
In-Reply-To: <A955DF0B-7ED6-4782-B8F4-93497BC81762@steinitz.org>
References: <CAJGqWSMtQ2_-M5r-J-Bz_9QLzXyDM+xzHQMN-nMc_O8106z0gg@mail.gmail.com>
 <EA56150C-9D3E-489F-9158-ED4D69B667A9@steinitz.org>
 <CAJGqWSM85QUaMsDSnK++7sVENo55AneBmD0MrDEZewv_j2fmXQ@mail.gmail.com>
 <A955DF0B-7ED6-4782-B8F4-93497BC81762@steinitz.org>
Message-ID: <CAJGqWSN8Zck0LfWZ8Y2hUyMTbbG97byr6SQDnLguejstzarBiQ@mail.gmail.com>

Hi Dominic,

Cheers. Yes, I agree that PRs and branches are the way to go. I have 3 or 4
minor ones there already.

I've joined the mailing list; I'll post a few questions in the coming days
and we'll see how people respond.

Regards,
Huw

On Tue, Jul 7, 2020 at 12:06 AM <dominic at steinitz.org> wrote:

> Hi Huw,
>
> You are right. I have given you access to the hmatrix repo. We don’t
> currently have any modus operandi for the organisation in which this repo
> lives. I assume that we will all create branches for review before merging?
>
> I’ve cc’d the somewhat unused mailing list. If that doesn’t work for e.g.
> Q&A then we can try something else.
>
> Dominic Steinitz
> dominic at steinitz.org
> http://idontgetoutmuch.org
> Twitter: @idontgetoutmuch
>
> On 6 Jul 2020, at 08:09, Huw Campbell <huw.campbell at gmail.com> wrote:
>
> Hi Dominic,
>
> Thanks very much. I still can't seem to push to the hmatrix repository, I
> think the permissions are separate to the haskell-numerics organisation.
>
> Is there a mailing list or unofficial list of email addresses we can use
> for questions and change proposals? Some questions (like, why is CSR 1
> indexed?) feel a bit strange for an issue tracker.
>
> Regards,
> Huw
>
> On Sat, Jul 4, 2020 at 5:56 PM <dominic at steinitz.org> wrote:
>
>> Hi Huw
>>
>> Great 😁
>>
>> I am just about to go out but some thoughts
>>
>> Roman Cheplyaka has updated hmatrix-sundials to use sparse matrices - I
>> have an application for this solving some PDEs which I have yet to try.
>>
>> I think Henning Thielemann has also done some work on sparse matrices /
>> bindings to BLAS / LAPACK.
>>
>> The collaborative effort to improve random number generation worked
>> really well - hopefully we can repeat the experience in this important area.
>>
>> Dominic Steinitz
>> dominic at steinitz.org
>> http://idontgetoutmuch.org
>> Twitter: @idontgetoutmuch
>>
>> On 4 Jul 2020, at 05:05, Huw Campbell <huw.campbell at gmail.com> wrote:
>>
>> Hi Dominic,
>>
>> I've been thinking about putting some more work into hmatrix for a few
>> years now, ever since I used it effectively in my neural network
>> library Grenade.
>>
>> I currently have a few PRs up to fix and speed up sparse matrix creation.
>>
>> I would like to really tighten up its benchmark and test suites (bring
>> them into the packages themselves instead of hosting them in an
>> external package; remove the poorly conceived `Testable` type class;
>> using criterion a lot more; and possibly porting tests to hedgehog).
>>
>> Other items on the cards are using im2col to speed up convolutions (I
>> have this working in Grenade already) and make them easier to use with
>> user defined padding and stride.
>>
>> It's quite a bit to do, so I was wondering if I could become a
>> collaborator?
>>
>> In terms of Haskell projects I maintain, the main open source one
>> these days is optparse-applicative, which I've been incrementally
>> improving over the past 5 years.
>>
>> Kind regards,
>> Huw
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/numeric/attachments/20200707/a609646d/attachment.html>

From huw.campbell at gmail.com  Wed Jul  8 07:18:05 2020
From: huw.campbell at gmail.com (Huw Campbell)
Date: Wed, 8 Jul 2020 17:18:05 +1000
Subject: [Numeric] HMatrix priorities
Message-ID: <CAJGqWSNfP-00kJmZdT2sghE9O6JepUsZqce86nxB00XHKvFqVg@mail.gmail.com>

Hi all,

I've put a bit of time into hmatrix over the past week. My main
priorities at the moment are fixing bugs, segfaults, memory leaks, and
other infelicities. But after that's done I'm planning on tackling
some broader API and design issues.

I have a few questions:

- Why are CSR and CSC 1 indexed? The early literature uses 0 indexing
AFAICS, and it means that Ben Gamari's libsvd wrapper has to do extra
leg work reindexing before calling its C code.

- Why do we have our own mutable vector type? I would have thought the
version in
the vector package would be sufficient.

- Should we try to move more of the library towards the general matrix
`GMatrix`? At the moment there are not very many functions which
actually use it (and it has partial record labels), and one can't
project out back to any of the underlying matrix forms.

- Can we bring the test suites and benchmarks back into the libraries
proper? The test suite is currently lagging the major version, and on
master doesn't compile with GHC 8.6+ (I have a PR for that).

- CI looks like it's only testing one version of GHC, do we have a
policy on how many versions we support?

- What are all of your priorities? Dominic has mentioned a few use cases around
sparse matrices people have been working on.

We've also spoken about github etiquette. I agreed with Dominic that
branches and pull requests are probably the best approach at the
moment. I won't merge my own PRs without input from another maintainer
at this stage, so please do have a look.

Regards,
Huw

From lemming at henning-thielemann.de  Wed Jul  8 08:25:04 2020
From: lemming at henning-thielemann.de (Henning Thielemann)
Date: Wed, 8 Jul 2020 10:25:04 +0200 (CEST)
Subject: [Numeric] HMatrix priorities
In-Reply-To: <CAJGqWSNfP-00kJmZdT2sghE9O6JepUsZqce86nxB00XHKvFqVg@mail.gmail.com>
References: <CAJGqWSNfP-00kJmZdT2sghE9O6JepUsZqce86nxB00XHKvFqVg@mail.gmail.com>
Message-ID: <alpine.DEB.2.20.2007081020000.5397@sputnik>


Hi Huw,


On Wed, 8 Jul 2020, Huw Campbell wrote:

> I have a few questions:
>
> - Why are CSR and CSC 1 indexed? The early literature uses 0 indexing
> AFAICS, and it means that Ben Gamari's libsvd wrapper has to do extra
> leg work reindexing before calling its C code.

I may point you to my package 'lapack' and the underlying 'comfort-array' 
which addresses indexing in a very general manner:
    https://hackage.haskell.org/package/lapack

> - Should we try to move more of the library towards the general matrix
> `GMatrix`? At the moment there are not very many functions which
> actually use it (and it has partial record labels), and one can't
> project out back to any of the underlying matrix forms.

My lapack package has individual types for different matrix forms and thus 
better documents what is going on in your numeric code.

> - CI looks like it's only testing one version of GHC, do we have a
> policy on how many versions we support?

My lapack wrapper supports back to GHC-7.4.

> - What are all of your priorities? Dominic has mentioned a few use cases 
> around sparse matrices people have been working on.

I have heard many people asking for sparse matrices.

What do people think is the best low-level library to interface to? 
suitesparse?

From dominic at steinitz.org  Fri Jul 10 08:16:08 2020
From: dominic at steinitz.org (dominic at steinitz.org)
Date: Fri, 10 Jul 2020 09:16:08 +0100
Subject: [Numeric] Fwd:  HMatrix priorities
References: <07501BE6-2178-498D-A454-6C9D4043F3CE@blueyonder.co.uk>
Message-ID: <B9852AAC-B533-4C69-8BE5-622546F744D7@steinitz.org>

Hi Huw,

Thanks very much for the questions and working on hmatrix. Apologies for not having more informative answers.

Dominic Steinitz
dominic at steinitz.org
http://idontgetoutmuch.org
Twitter: @idontgetoutmuch

> On 8 Jul 2020, at 08:18, Huw Campbell <huw.campbell at gmail.com> wrote:
> 
> Hi all,
> 
> I've put a bit of time into hmatrix over the past week. My main
> priorities at the moment are fixing bugs, segfaults, memory leaks, and
> other infelicities. But after that's done I'm planning on tackling
> some broader API and design issues.
> 
> I have a few questions:
> 
> - Why are CSR and CSC 1 indexed? The early literature uses 0 indexing
> AFAICS, and it means that Ben Gamari's libsvd wrapper has to do extra
> leg work reindexing before calling its C code.

I didn’t even know it did this. I don’t think the author (Alberto Ruiz) is subscribed to this list. We could try asking him directly.

> - Why do we have our own mutable vector type? I would have thought the
> version in
> the vector package would be sufficient.

My guess would be that when the code was written mutable vectors were not available in vector.

> - Should we try to move more of the library towards the general matrix
> `GMatrix`? At the moment there are not very many functions which
> actually use it (and it has partial record labels), and one can't
> project out back to any of the underlying matrix forms.

I have no view on this other than to be careful about performance which reminds me: we should really do some performance measurements.

> - Can we bring the test suites and benchmarks back into the libraries
> proper? The test suite is currently lagging the major version, and on
> master doesn't compile with GHC 8.6+ (I have a PR for that).

Yes please.

> - CI looks like it's only testing one version of GHC, do we have a
> policy on how many versions we support?

No policy but I guess we should aim for a few more versions.

> - What are all of your priorities? Dominic has mentioned a few use cases around
> sparse matrices people have been working on.

Some time ago, I created hmatrix-sundials but gave it its own repo. The idea was to have an identical interface to ODE solving so that you could switch between SUNDIALS and GSL. Roman Cheplyaka has vastly improved hmatrix-sundials and it now uses suitesparse for sparse matrices.

In terms of priorities, my own are focussed on solving PDEs so I really need sparse matrices. And SUNDIALS is really a lot better for differential equations than GSL. I really like the statically typed matrices (the rows and columns being typed by their size). Are you thinking just about the base package or the other packages like glpk?

> We've also spoken about github etiquette. I agreed with Dominic that
> branches and pull requests are probably the best approach at the
> moment. I won't merge my own PRs without input from another maintainer
> at this stage, so please do have a look.

I am not totally convinced having a monorepo is the best way forward but I don’t feel strongly.

> Regards,
> Huw
> _______________________________________________
> Numeric mailing list
> Numeric at haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/numeric


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.haskell.org/pipermail/numeric/attachments/20200710/b9345c62/attachment.html>