From johnw at newartisans.com  Mon Nov  3 17:08:48 2014
From: johnw at newartisans.com (John Wiegley)
Date: Mon, 03 Nov 2014 11:08:48 -0600
Subject: GHC Trac
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F38C0EE@DB3PRD3001MB020.064d.mgd.msft.net>
 (Simon Peyton Jones's message of "Mon, 3 Nov 2014 13:28:26 +0000")
References: <618BE556AADD624C9C918AA5D5911BEF3F38C0EE@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <m2y4rsjjof.fsf@newartisans.com>

>>>>> Simon Peyton Jones <simonpj at microsoft.com> writes:

> Is it just me, or is the GHC Trac soul-destroyingly slow at the moment? IT
> takes minutes to load one page!

You may be interested in the Mylyn plugin for the Eclipse IDE, which allows
for downloading and offline browsing of issues in Trac.  That could speed up
your "hop to next bug" time, at the cost of installing Eclipse solely for that
purpose.

John

From greg at gregweber.info  Wed Nov  5 20:21:40 2014
From: greg at gregweber.info (Greg Weber)
Date: Wed, 5 Nov 2014 12:21:40 -0800
Subject: Annotations: status and documentation
Message-ID: <CAKRanNBX8_OZxnog+Bh=45xG+J6m=zZXS9wrsqCV1ZuzhfsEyw@mail.gmail.com>

I would like to be able to retrieve user-supplied annotations in Template
Haskell. The 7.8 user manual doesn't mention this ability. However, I see
this Trac page:

https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Annotations

Where it appears that this feature may exist in 7.8. I will gladly tinker
with annotations and update the user manual if someone will help me
understand what the situation is.

On a related note, it looks like this patch pointed to from the Wiki page
for annotations got ignored. Should i submit it to phabricator?
https://ghc.haskell.org/trac/ghc/ticket/8460

Thanks,
Greg Weber
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141105/f30d5c9c/attachment.html>

From greg at gregweber.info  Thu Nov  6 18:23:37 2014
From: greg at gregweber.info (Greg Weber)
Date: Thu, 6 Nov 2014 10:23:37 -0800
Subject: Annotations: status and documentation
In-Reply-To: <CAKRanNBX8_OZxnog+Bh=45xG+J6m=zZXS9wrsqCV1ZuzhfsEyw@mail.gmail.com>
References: <CAKRanNBX8_OZxnog+Bh=45xG+J6m=zZXS9wrsqCV1ZuzhfsEyw@mail.gmail.com>
Message-ID: <CAKRanNDF2gEd+TmZMpjjNaq+Nv5DRbiZPuad1anMVT9wHj-QZg@mail.gmail.com>

Should I be asking this question on a different mailing list such as
ghc-devs?

On Wed, Nov 5, 2014 at 12:21 PM, Greg Weber <greg at gregweber.info> wrote:

> I would like to be able to retrieve user-supplied annotations in Template
> Haskell. The 7.8 user manual doesn't mention this ability. However, I see
> this Trac page:
>
> https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Annotations
>
> Where it appears that this feature may exist in 7.8. I will gladly tinker
> with annotations and update the user manual if someone will help me
> understand what the situation is.
>
> On a related note, it looks like this patch pointed to from the Wiki page
> for annotations got ignored. Should i submit it to phabricator?
> https://ghc.haskell.org/trac/ghc/ticket/8460
>
> Thanks,
> Greg Weber
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141106/cd63a002/attachment.html>

From greg at gregweber.info  Thu Nov  6 19:53:25 2014
From: greg at gregweber.info (Greg Weber)
Date: Thu, 6 Nov 2014 11:53:25 -0800
Subject: Annotations: status and documentation
In-Reply-To: <CAKRanNDF2gEd+TmZMpjjNaq+Nv5DRbiZPuad1anMVT9wHj-QZg@mail.gmail.com>
References: <CAKRanNBX8_OZxnog+Bh=45xG+J6m=zZXS9wrsqCV1ZuzhfsEyw@mail.gmail.com>
 <CAKRanNDF2gEd+TmZMpjjNaq+Nv5DRbiZPuad1anMVT9wHj-QZg@mail.gmail.com>
Message-ID: <CAKRanNCSD4trXas8yNGa6P5oe6CM1CTs1Zy79QH6SRN-6tyvoA@mail.gmail.com>

I was able to get in touch with the author and I am discussing my use case
with him. Thanks!

On Thu, Nov 6, 2014 at 10:23 AM, Greg Weber <greg at gregweber.info> wrote:

> Should I be asking this question on a different mailing list such as
> ghc-devs?
>
> On Wed, Nov 5, 2014 at 12:21 PM, Greg Weber <greg at gregweber.info> wrote:
>
>> I would like to be able to retrieve user-supplied annotations in Template
>> Haskell. The 7.8 user manual doesn't mention this ability. However, I see
>> this Trac page:
>>
>> https://ghc.haskell.org/trac/ghc/wiki/TemplateHaskell/Annotations
>>
>> Where it appears that this feature may exist in 7.8. I will gladly tinker
>> with annotations and update the user manual if someone will help me
>> understand what the situation is.
>>
>> On a related note, it looks like this patch pointed to from the Wiki page
>> for annotations got ignored. Should i submit it to phabricator?
>> https://ghc.haskell.org/trac/ghc/ticket/8460
>>
>> Thanks,
>> Greg Weber
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141106/256cb14d/attachment.html>

From austin at well-typed.com  Fri Nov  7 18:16:22 2014
From: austin at well-typed.com (Austin Seipp)
Date: Fri, 7 Nov 2014 12:16:22 -0600
Subject: RFC: Dropping Windows XP support
Message-ID: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>

Hi all,

This is a quick discussion about the current system requirements for
Windows builds.

Spurred by a recent[1] LLVM discussion, I'd like to raise the question
of dropping support for Windows XP, and bumping the minimum required
version to Windows Vista or even Windows 7.

For one, Microsoft doesn't support XP anymore, so most people are
moving off it anyway. 'Soon' even XP Embedded will be obsoleted.

But second, Vista and beyond introduced useful new APIs we could use.
I was digging through the LLVM thread and two came out to me:

 1) We could switch to using slim reader/writer locks, which in some
workloads may work out better than critical sections (they'll win on
more read-heavy workloads). The downsides is there's no recursive
locking but we don't use that anyway (and recursive locks are
considered bad by many anyway[2]).

 2) We could probably use an actual condition variables API that was
introduced with Vista. Currently we use a giant EVENT object to
emulate the API, which could be replaced with the real deal.

Both of these could be nice wins for simplicity and performance I think.

I know there are some corporate users out there who this may impact,
and users as well. I'd like to know what people think. Particularly
what version we should standardize on.

FWIW, I don't plan on changing any of this until the 7.12 release at least.

[1] http://article.gmane.org/gmane.comp.compilers.llvm.devel/78419
[2] http://www.zaval.org/resources/library/butenhof1.html

-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

From david.feuer at gmail.com  Fri Nov  7 18:20:44 2014
From: david.feuer at gmail.com (David Feuer)
Date: Fri, 7 Nov 2014 13:20:44 -0500
Subject: RFC: Dropping Windows XP support
In-Reply-To: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
References: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
Message-ID: <CAMgWh9vGH2H77TeZAZHs1zW1CihqTZbb3qAbQL+yBtP=maZOwg@mail.gmail.com>

+1. Windows XP was Microsoft's most successful OS thus far, but it's pretty
much dead now. One potentially related potential concern: how will this
change affect Wine support?

On Fri, Nov 7, 2014 at 1:16 PM, Austin Seipp <austin at well-typed.com> wrote:

> Hi all,
>
> This is a quick discussion about the current system requirements for
> Windows builds.
>
> Spurred by a recent[1] LLVM discussion, I'd like to raise the question
> of dropping support for Windows XP, and bumping the minimum required
> version to Windows Vista or even Windows 7.
>
> For one, Microsoft doesn't support XP anymore, so most people are
> moving off it anyway. 'Soon' even XP Embedded will be obsoleted.
>
> But second, Vista and beyond introduced useful new APIs we could use.
> I was digging through the LLVM thread and two came out to me:
>
>  1) We could switch to using slim reader/writer locks, which in some
> workloads may work out better than critical sections (they'll win on
> more read-heavy workloads). The downsides is there's no recursive
> locking but we don't use that anyway (and recursive locks are
> considered bad by many anyway[2]).
>
>  2) We could probably use an actual condition variables API that was
> introduced with Vista. Currently we use a giant EVENT object to
> emulate the API, which could be replaced with the real deal.
>
> Both of these could be nice wins for simplicity and performance I think.
>
> I know there are some corporate users out there who this may impact,
> and users as well. I'd like to know what people think. Particularly
> what version we should standardize on.
>
> FWIW, I don't plan on changing any of this until the 7.12 release at least.
>
> [1] http://article.gmane.org/gmane.comp.compilers.llvm.devel/78419
> [2] http://www.zaval.org/resources/library/butenhof1.html
>
> --
> Regards,
>
> Austin Seipp, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141107/b42a58dd/attachment.html>

From gershomb at gmail.com  Sat Nov  8 00:19:59 2014
From: gershomb at gmail.com (Gershom B)
Date: Fri, 7 Nov 2014 19:19:59 -0500
Subject: RFC: Dropping Windows XP support
In-Reply-To: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
References: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
Message-ID: <etPan.545d61be.2ae8944a.5d3c@mbp.local>

One concern here is that even with XP falling out of support, Windows Server 2003 remains supported through July 2015, and so we should give it a little chunk of time after that falls out of support from Microsoft before we stop supporting that. I think the limitations in Server 2003 are roughly the same as XP.

http://www.microsoft.com/en-us/server-cloud/products/windows-server-2003/

However, the next Windows Server (2008) should share all Vista features.

Cheers,
Gershom


On November 7, 2014 at 1:16:39 PM, Austin Seipp (austin at well-typed.com) wrote:
> Hi all,
> 
> This is a quick discussion about the current system requirements for
> Windows builds.
> 
> Spurred by a recent[1] LLVM discussion, I'd like to raise the question
> of dropping support for Windows XP, and bumping the minimum required
> version to Windows Vista or even Windows 7.
> 
> For one, Microsoft doesn't support XP anymore, so most people are
> moving off it anyway. 'Soon' even XP Embedded will be obsoleted.
> 
> But second, Vista and beyond introduced useful new APIs we could use.
> I was digging through the LLVM thread and two came out to me:
> 
> 1) We could switch to using slim reader/writer locks, which in some
> workloads may work out better than critical sections (they'll win on
> more read-heavy workloads). The downsides is there's no recursive
> locking but we don't use that anyway (and recursive locks are
> considered bad by many anyway[2]).
> 
> 2) We could probably use an actual condition variables API that was
> introduced with Vista. Currently we use a giant EVENT object to
> emulate the API, which could be replaced with the real deal.
> 
> Both of these could be nice wins for simplicity and performance I think.
> 
> I know there are some corporate users out there who this may impact,
> and users as well. I'd like to know what people think. Particularly
> what version we should standardize on.
> 
> FWIW, I don't plan on changing any of this until the 7.12 release at least.
> 
> [1] http://article.gmane.org/gmane.comp.compilers.llvm.devel/78419
> [2] http://www.zaval.org/resources/library/butenhof1.html
> 
> --
> Regards,
> 
> Austin Seipp, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 


From mike at proclivis.com  Tue Nov 11 07:11:13 2014
From: mike at proclivis.com (Michael Jones)
Date: Tue, 11 Nov 2014 00:11:13 -0700
Subject: GHC 7.8.3 thread hang
Message-ID: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>

I am trying to debug a lockup problem (and need help with debugging technique), where hang means a thread stops at a known place during evaluation, and other threads continue.

The code near the problem is like:

       ec <- return $ encode command
       l <- return $ BSL.length ec
       ss <- return $ BSC.unpack ec

It does not matter if I use let or return, or if the length is taken after unpack. I used return so I could use this code for tracing, with strictness to try to find the exact statement that is the problem:

       traceEventIO "sendCommand"
       ec <- return $ encode command
       traceEventIO $ "sendCommand: encoded"
       l <- ec `seq` return $ BSL.length ec
       traceEventIO $ "sendCommand: size " ++ (show l)
       ss <- ec `seq` return $ BSC.unpack ec

When this runs, the program executes this many times, but always hangs under a certain condition.

For good evaluations:

7f04173ff700: cap 0: sendCommand
7f04173ff700: cap 0: sendCommand: encoded
7f04173ff700: cap 0: sendCommand: size 4
7f04173ff700: cap 0: sendCommand: unpacked
7f04173ff700: cap 0: Sending command of size 4
7f04173ff700: cap 0: Sending command of size "\NUL\EOT"
7f04173ff700: cap 0: sendCommand: sent
7f04173ff700: cap 0: sendCommand: flushed

for bad evaluation:

7f04173ff700: cap 0: sendCommand
7f04173ff700: cap 0: sendCommand: encoded

The lockup occurs when length is taken.

The difference between the working and non-working case is as follows:

A wxHaskell callback stuffs some data in a TChan. A thread started at application startup is reading the TChan and calling the code that hangs. If it did not hang, it would send it by TCP to another process/computer.

In the working case the callback pops a dialog, and passes data from one TChan to another TChan.

In the failing case, the data is used to generate strings in a wxHaskell grid, then it is parsed, and a new data is made. The new data is a combination of old and new pieces of sub data. The shape of the date is identical, because I am not making any edits to the rows.

So when data that the callback sends to TChan is unmodified, no hang. But when the data is used to make text, put it in the gui, process it, and generate new data, it hangs.

As a test I modified the code so that the text is not put into the gui. The results are the same. This indicates it has something to do with creating strings and then data from strings and mixing old and new subdata. Strings are created with show. Data is created by pattern matching and generating numbers from strings. I should also point out that in the working case, the size of the resulting string is small, say 3. In the hang case, the resulting string would be long, say 5000-10000.

I assume there are no limits to the size of ByteStrings or fundemental issues with the RTS stack/heap that require special settings.

I am using the following revisions:

GHC 7.8.3
    base ==4.7.*, 
    mtl ==2.2.1,
    containers == 0.5.5.1, 
    transformers ==0.4.1.0, 
    random == 1.0.1.1, 
    wx == 0.91.0.0, 
    wxcore == 0.91.0.0, 
    wxdirect == 0.91.0.0, 
    colour == 2.3.3, 
    stm == 2.4.2, 
    monad-loops == 0.4.2.1, 
    time == 1.4.2,
    old-locale == 1.0.0.6,
    fast-logger == 2.2.3,
    network == 2.6.0.2,
    bytestring == 0.10.4.0,
    control-monad-loop == 0.1,
    binary == 0.7.2.2,

I know that nobody can have an answer based on this. But what I am hoping is either there is some known bug, or someone can guide me in narrowing it down. The event log does not have anything unusual in it. Other threads keep running, and I can exit the application normally. The thread does not throw an exception. It just hangs.

When I run the app, I just use +RTS -v 

Perhaps there are some other options that might give more info?

? SNIPPET of log ?

7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
7fe544cfea40: cap 0: thread 5 stopped (yielding)
7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
7fe544cfea40: cap 0: thread 5 stopped (suspended while making a foreign call)
7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
7fe537eff700: cap 0: waking up thread 5 on cap 0
7fe537eff700: cap 0: thread 2 stopped (yielding)
7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
7fe544cfea40: cap 0: sendCommand
7fe544cfea40: cap 0: sendCommand: encoded
7fe544cfea40: cap 0: sendCommand: size 3                               WORKS HERE
7fe544cfea40: cap 0: sendCommand: unpacked
7fe544cfea40: cap 0: Sending command of size 3
7fe544cfea40: cap 0: Sending command of size "\NUL\ETX"
7fe544cfea40: cap 0: sendCommand: sent
7fe544cfea40: cap 0: sendCommand: flushed
7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
7fe537eff700: cap 0: thread 2 stopped (yielding)
7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
7fe537eff700: cap 0: thread 2 stopped (suspended while making a foreign call)
7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
7fe537eff700: cap 0: waking up thread 5 on cap 0

?

7fe537eff700: cap 0: fetchTelemetryServer: got lock
7fe537eff700: cap 0: thread 45 stopped (yielding)
7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
7fe537eff700: cap 0: thread 7 stopped (heap overflow)
7fe537eff700: cap 0: requesting parallel GC
7fe537eff700: cap 0: starting GC
7fe537eff700: cap 0: GC working
7fe537eff700: cap 0: GC idle
7fe537eff700: cap 0: GC done
7fe537eff700: cap 0: GC idle
7fe537eff700: cap 0: GC done
7fe537eff700: cap 0: all caps stopped for GC
7fe537eff700: cap 0: finished GC
7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
7fe537eff700: cap 0: sendCommand
7fe537eff700: cap 0: sendCommand: encoded                       PROBLEM HERE
7fe537eff700: cap 0: thread 7 stopped (heap overflow)
7fe537eff700: cap 0: requesting parallel GC
7fe537eff700: cap 0: starting GC
7fe537eff700: cap 0: GC working
7fe537eff700: cap 0: GC idle
7fe537eff700: cap 0: GC done
7fe537eff700: cap 0: GC idle
7fe537eff700: cap 0: GC done
7fe537eff700: cap 0: all caps stopped for GC
7fe537eff700: cap 0: finished GC
7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
7fe537eff700: cap 0: thread 7 stopped (yielding)
7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
7fe537eff700: cap 0: thread 2 stopped (yielding)
7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign call)
7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign call)
7fe537eff700: cap 0: running thread 45 (ThreadRunGHC)
7fe537eff700: cap 0: fetchTelemetryServer: unlock
7fe537eff700: cap 0: fetchTelemetryServer
7fe537eff700: cap 0: fetchTelemetryServer: got lock
7fe537eff700: cap 0: fetchTelemetryServer: pump seq




From carter.schonwald at gmail.com  Tue Nov 11 15:01:02 2014
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Tue, 11 Nov 2014 10:01:02 -0500
Subject: GHC 7.8.3 thread hang
In-Reply-To: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
Message-ID: <CAHYVw0wHAy7br18_4GgWQekHXficghmdVFo86tfSAgacRWAM_Q@mail.gmail.com>

what OS are you on?
what build options did you use?

On Tue, Nov 11, 2014 at 2:11 AM, Michael Jones <mike at proclivis.com> wrote:

> I am trying to debug a lockup problem (and need help with debugging
> technique), where hang means a thread stops at a known place during
> evaluation, and other threads continue.
>
> The code near the problem is like:
>
>        ec <- return $ encode command
>        l <- return $ BSL.length ec
>        ss <- return $ BSC.unpack ec
>
> It does not matter if I use let or return, or if the length is taken after
> unpack. I used return so I could use this code for tracing, with strictness
> to try to find the exact statement that is the problem:
>
>        traceEventIO "sendCommand"
>        ec <- return $ encode command
>        traceEventIO $ "sendCommand: encoded"
>        l <- ec `seq` return $ BSL.length ec
>        traceEventIO $ "sendCommand: size " ++ (show l)
>        ss <- ec `seq` return $ BSC.unpack ec
>
> When this runs, the program executes this many times, but always hangs
> under a certain condition.
>
> For good evaluations:
>
> 7f04173ff700: cap 0: sendCommand
> 7f04173ff700: cap 0: sendCommand: encoded
> 7f04173ff700: cap 0: sendCommand: size 4
> 7f04173ff700: cap 0: sendCommand: unpacked
> 7f04173ff700: cap 0: Sending command of size 4
> 7f04173ff700: cap 0: Sending command of size "\NUL\EOT"
> 7f04173ff700: cap 0: sendCommand: sent
> 7f04173ff700: cap 0: sendCommand: flushed
>
> for bad evaluation:
>
> 7f04173ff700: cap 0: sendCommand
> 7f04173ff700: cap 0: sendCommand: encoded
>
> The lockup occurs when length is taken.
>
> The difference between the working and non-working case is as follows:
>
> A wxHaskell callback stuffs some data in a TChan. A thread started at
> application startup is reading the TChan and calling the code that hangs.
> If it did not hang, it would send it by TCP to another process/computer.
>
> In the working case the callback pops a dialog, and passes data from one
> TChan to another TChan.
>
> In the failing case, the data is used to generate strings in a wxHaskell
> grid, then it is parsed, and a new data is made. The new data is a
> combination of old and new pieces of sub data. The shape of the date is
> identical, because I am not making any edits to the rows.
>
> So when data that the callback sends to TChan is unmodified, no hang. But
> when the data is used to make text, put it in the gui, process it, and
> generate new data, it hangs.
>
> As a test I modified the code so that the text is not put into the gui.
> The results are the same. This indicates it has something to do with
> creating strings and then data from strings and mixing old and new subdata.
> Strings are created with show. Data is created by pattern matching and
> generating numbers from strings. I should also point out that in the
> working case, the size of the resulting string is small, say 3. In the hang
> case, the resulting string would be long, say 5000-10000.
>
> I assume there are no limits to the size of ByteStrings or fundemental
> issues with the RTS stack/heap that require special settings.
>
> I am using the following revisions:
>
> GHC 7.8.3
>     base ==4.7.*,
>     mtl ==2.2.1,
>     containers == 0.5.5.1,
>     transformers ==0.4.1.0,
>     random == 1.0.1.1,
>     wx == 0.91.0.0,
>     wxcore == 0.91.0.0,
>     wxdirect == 0.91.0.0,
>     colour == 2.3.3,
>     stm == 2.4.2,
>     monad-loops == 0.4.2.1,
>     time == 1.4.2,
>     old-locale == 1.0.0.6,
>     fast-logger == 2.2.3,
>     network == 2.6.0.2,
>     bytestring == 0.10.4.0,
>     control-monad-loop == 0.1,
>     binary == 0.7.2.2,
>
> I know that nobody can have an answer based on this. But what I am hoping
> is either there is some known bug, or someone can guide me in narrowing it
> down. The event log does not have anything unusual in it. Other threads
> keep running, and I can exit the application normally. The thread does not
> throw an exception. It just hangs.
>
> When I run the app, I just use +RTS -v
>
> Perhaps there are some other options that might give more info?
>
> ? SNIPPET of log ?
>
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (suspended while making a foreign
> call)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: waking up thread 5 on cap 0
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: sendCommand
> 7fe544cfea40: cap 0: sendCommand: encoded
> 7fe544cfea40: cap 0: sendCommand: size 3
>  WORKS HERE
> 7fe544cfea40: cap 0: sendCommand: unpacked
> 7fe544cfea40: cap 0: Sending command of size 3
> 7fe544cfea40: cap 0: Sending command of size "\NUL\ETX"
> 7fe544cfea40: cap 0: sendCommand: sent
> 7fe544cfea40: cap 0: sendCommand: flushed
> 7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (suspended while making a foreign
> call)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: waking up thread 5 on cap 0
>
> ?
>
> 7fe537eff700: cap 0: fetchTelemetryServer: got lock
> 7fe537eff700: cap 0: thread 45 stopped (yielding)
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
> 7fe537eff700: cap 0: requesting parallel GC
> 7fe537eff700: cap 0: starting GC
> 7fe537eff700: cap 0: GC working
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: all caps stopped for GC
> 7fe537eff700: cap 0: finished GC
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: sendCommand
> 7fe537eff700: cap 0: sendCommand: encoded                       PROBLEM
> HERE
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
> 7fe537eff700: cap 0: requesting parallel GC
> 7fe537eff700: cap 0: starting GC
> 7fe537eff700: cap 0: GC working
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: all caps stopped for GC
> 7fe537eff700: cap 0: finished GC
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 7 stopped (yielding)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign
> call)
> 7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign
> call)
> 7fe537eff700: cap 0: running thread 45 (ThreadRunGHC)
> 7fe537eff700: cap 0: fetchTelemetryServer: unlock
> 7fe537eff700: cap 0: fetchTelemetryServer
> 7fe537eff700: cap 0: fetchTelemetryServer: got lock
> 7fe537eff700: cap 0: fetchTelemetryServer: pump seq
>
>
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/52e8c38a/attachment.html>

From greg at gregorycollins.net  Tue Nov 11 16:45:04 2014
From: greg at gregorycollins.net (Gregory Collins)
Date: Tue, 11 Nov 2014 08:45:04 -0800
Subject: GHC 7.8.3 thread hang
In-Reply-To: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
Message-ID: <CAHZrzdrj8KZtEA5SUSNFsY5iMGstLbRYm5rMF2w97HmyX63Ypg@mail.gmail.com>

On Mon, Nov 10, 2014 at 11:11 PM, Michael Jones <mike at proclivis.com> wrote:

>        ec <- return $ encode command
>        traceEventIO $ "sendCommand: encoded"
>        l <- ec `seq` return $ BSL.length ec
>

Your encode function probably loops on some inputs. When you call "return $
foo", foo is not forced; that doesn't happen in your example until
BSL.length forces its input. If I'm right, changing the first line to
"return $! encode command" will move the hang to before the call to
traceEventIO.

-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/602ba6d8/attachment.html>

From mike at proclivis.com  Tue Nov 11 17:38:53 2014
From: mike at proclivis.com (Michael Jones)
Date: Tue, 11 Nov 2014 10:38:53 -0700
Subject: GHC 7.8.3 thread hang
In-Reply-To: <CAHYVw0wHAy7br18_4GgWQekHXficghmdVFo86tfSAgacRWAM_Q@mail.gmail.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAHYVw0wHAy7br18_4GgWQekHXficghmdVFo86tfSAgacRWAM_Q@mail.gmail.com>
Message-ID: <BD942719-8FCC-4A4C-8707-2A47F3DAD9D0@proclivis.com>

OS: Ubuntu 14.X TLS
GHC-Options:   -rtsopts -threaded -debug -eventlog

Behaves the same with

GHC-Options:   -rtsopts -threaded -O2

Mike

On Nov 11, 2014, at 8:01 AM, Carter Schonwald <carter.schonwald at gmail.com> wrote:

> what OS are you on?
> what build options did you use?
> 
> On Tue, Nov 11, 2014 at 2:11 AM, Michael Jones <mike at proclivis.com> wrote:
> I am trying to debug a lockup problem (and need help with debugging technique), where hang means a thread stops at a known place during evaluation, and other threads continue.
> 
> The code near the problem is like:
> 
>        ec <- return $ encode command
>        l <- return $ BSL.length ec
>        ss <- return $ BSC.unpack ec
> 
> It does not matter if I use let or return, or if the length is taken after unpack. I used return so I could use this code for tracing, with strictness to try to find the exact statement that is the problem:
> 
>        traceEventIO "sendCommand"
>        ec <- return $ encode command
>        traceEventIO $ "sendCommand: encoded"
>        l <- ec `seq` return $ BSL.length ec
>        traceEventIO $ "sendCommand: size " ++ (show l)
>        ss <- ec `seq` return $ BSC.unpack ec
> 
> When this runs, the program executes this many times, but always hangs under a certain condition.
> 
> For good evaluations:
> 
> 7f04173ff700: cap 0: sendCommand
> 7f04173ff700: cap 0: sendCommand: encoded
> 7f04173ff700: cap 0: sendCommand: size 4
> 7f04173ff700: cap 0: sendCommand: unpacked
> 7f04173ff700: cap 0: Sending command of size 4
> 7f04173ff700: cap 0: Sending command of size "\NUL\EOT"
> 7f04173ff700: cap 0: sendCommand: sent
> 7f04173ff700: cap 0: sendCommand: flushed
> 
> for bad evaluation:
> 
> 7f04173ff700: cap 0: sendCommand
> 7f04173ff700: cap 0: sendCommand: encoded
> 
> The lockup occurs when length is taken.
> 
> The difference between the working and non-working case is as follows:
> 
> A wxHaskell callback stuffs some data in a TChan. A thread started at application startup is reading the TChan and calling the code that hangs. If it did not hang, it would send it by TCP to another process/computer.
> 
> In the working case the callback pops a dialog, and passes data from one TChan to another TChan.
> 
> In the failing case, the data is used to generate strings in a wxHaskell grid, then it is parsed, and a new data is made. The new data is a combination of old and new pieces of sub data. The shape of the date is identical, because I am not making any edits to the rows.
> 
> So when data that the callback sends to TChan is unmodified, no hang. But when the data is used to make text, put it in the gui, process it, and generate new data, it hangs.
> 
> As a test I modified the code so that the text is not put into the gui. The results are the same. This indicates it has something to do with creating strings and then data from strings and mixing old and new subdata. Strings are created with show. Data is created by pattern matching and generating numbers from strings. I should also point out that in the working case, the size of the resulting string is small, say 3. In the hang case, the resulting string would be long, say 5000-10000.
> 
> I assume there are no limits to the size of ByteStrings or fundemental issues with the RTS stack/heap that require special settings.
> 
> I am using the following revisions:
> 
> GHC 7.8.3
>     base ==4.7.*,
>     mtl ==2.2.1,
>     containers == 0.5.5.1,
>     transformers ==0.4.1.0,
>     random == 1.0.1.1,
>     wx == 0.91.0.0,
>     wxcore == 0.91.0.0,
>     wxdirect == 0.91.0.0,
>     colour == 2.3.3,
>     stm == 2.4.2,
>     monad-loops == 0.4.2.1,
>     time == 1.4.2,
>     old-locale == 1.0.0.6,
>     fast-logger == 2.2.3,
>     network == 2.6.0.2,
>     bytestring == 0.10.4.0,
>     control-monad-loop == 0.1,
>     binary == 0.7.2.2,
> 
> I know that nobody can have an answer based on this. But what I am hoping is either there is some known bug, or someone can guide me in narrowing it down. The event log does not have anything unusual in it. Other threads keep running, and I can exit the application normally. The thread does not throw an exception. It just hangs.
> 
> When I run the app, I just use +RTS -v
> 
> Perhaps there are some other options that might give more info?
> 
> ? SNIPPET of log ?
> 
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (suspended while making a foreign call)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: waking up thread 5 on cap 0
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 5 (ThreadRunGHC)
> 7fe544cfea40: cap 0: sendCommand
> 7fe544cfea40: cap 0: sendCommand: encoded
> 7fe544cfea40: cap 0: sendCommand: size 3                               WORKS HERE
> 7fe544cfea40: cap 0: sendCommand: unpacked
> 7fe544cfea40: cap 0: Sending command of size 3
> 7fe544cfea40: cap 0: Sending command of size "\NUL\ETX"
> 7fe544cfea40: cap 0: sendCommand: sent
> 7fe544cfea40: cap 0: sendCommand: flushed
> 7fe544cfea40: cap 0: thread 5 stopped (blocked on an MVar)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (suspended while making a foreign call)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: waking up thread 5 on cap 0
> 
> ?
> 
> 7fe537eff700: cap 0: fetchTelemetryServer: got lock
> 7fe537eff700: cap 0: thread 45 stopped (yielding)
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
> 7fe537eff700: cap 0: requesting parallel GC
> 7fe537eff700: cap 0: starting GC
> 7fe537eff700: cap 0: GC working
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: all caps stopped for GC
> 7fe537eff700: cap 0: finished GC
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: sendCommand
> 7fe537eff700: cap 0: sendCommand: encoded                       PROBLEM HERE
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
> 7fe537eff700: cap 0: requesting parallel GC
> 7fe537eff700: cap 0: starting GC
> 7fe537eff700: cap 0: GC working
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: GC idle
> 7fe537eff700: cap 0: GC done
> 7fe537eff700: cap 0: all caps stopped for GC
> 7fe537eff700: cap 0: finished GC
> 7fe537eff700: cap 0: running thread 7 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 7 stopped (yielding)
> 7fe537eff700: cap 0: running thread 2 (ThreadRunGHC)
> 7fe537eff700: cap 0: thread 2 stopped (yielding)
> 7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign call)
> 7fe544cfea40: cap 0: running thread 442 (ThreadRunGHC)
> 7fe544cfea40: cap 0: thread 442 stopped (suspended while making a foreign call)
> 7fe537eff700: cap 0: running thread 45 (ThreadRunGHC)
> 7fe537eff700: cap 0: fetchTelemetryServer: unlock
> 7fe537eff700: cap 0: fetchTelemetryServer
> 7fe537eff700: cap 0: fetchTelemetryServer: got lock
> 7fe537eff700: cap 0: fetchTelemetryServer: pump seq
> 
> 
> 
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/4d3f2634/attachment-0001.html>

From mike at proclivis.com  Tue Nov 11 19:18:03 2014
From: mike at proclivis.com (Michael Jones)
Date: Tue, 11 Nov 2014 12:18:03 -0700
Subject: GHC 7.8.3 thread hang
In-Reply-To: <CAHZrzdrj8KZtEA5SUSNFsY5iMGstLbRYm5rMF2w97HmyX63Ypg@mail.gmail.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAHZrzdrj8KZtEA5SUSNFsY5iMGstLbRYm5rMF2w97HmyX63Ypg@mail.gmail.com>
Message-ID: <6FC87AA2-6FFE-498E-B6D3-7B9B81C5FDF6@proclivis.com>

Gregory,

You are correct, it moved the problem.

I?ll study the code looking for loops, but...

one thing to note, there are two uses of the data for IO, and both would force evaluation of length/encode. One puts data in the GUI, one sends data by Ethernet. When only the GUI code is forcing IO, there is no hang. It is the Ethernet IO that causes the hang.

Because the GUI forcing does not hang, it leads me to believe there are no loops in the code.

Below is some description of overall application. -> means order of operation. T == Thread#. I note where the hang is, and what statement removal fixes it.

Mike

????


Main
T1 -> put TVar -> Start T2 -> Start T3

Async Module (TChan1/2/3 input/output wrapper around Eth based server in another process)
(This module takes a function that sends and recv data so that it is independent from communication mechanism)
(There are 2 output channels, one for response to commands in TCHan1, and one for a stream of telemetry)
T2 -> take TChan1 (config) -> 
      Serialize ->                      (Hangs here)
      Send Eth

T2 -> Read Eth -> 
      Deserialize -> 
      put TChan2 (data1)
      put TChan3 (data2)

fetchTelemetryServer (Consumes from Async Module T2 and puts data in sequence for callback)
T3 -> tryTake MVar2 (server lock) -> 
      take TChan3 (data) -> 
      put TSequence (data) -> 
      put MVar2 (server unlock)

showOptionDialogServer (Produces for Async Module T2)
(Changes the config in a dialog and send it to the server to modify telemetry definition)
wxH Menu Callback -> take MVar2 (server lock) ->
                     take MVar1 (gui lock) ->
                     take TVar (config) ->
                     convert config to strings
                     display in dialog
                     pull strings from dialog
                     build modified config 
                     put TVar (config) ->
                     put TChan1 (config) ->                    (Remove and no hang)
                     put MVar1 (gui unlock) -> 
                     put MVar2 (server lock)

performTelemetryServer
(Takes data from telemetry and prepares it for painting)
wxH Timer Callback -> take MVar1 (gui lock) ->
                      take TSequence (data) -> 
                      modify data
                      put Var (data2) -> 
                      repaint ->
                      put MVar1 (gui unlock)

onPaint
(Update the graphs in the GUI)
wx Paint Callback -> tryTake MVar1 (gui lock) -> 
                     get Var (data2) -> 
                     get TVar (config) ->
                     draw on gui -> 
                     put MVar1 (gui unlock)



On Nov 11, 2014, at 9:45 AM, Gregory Collins <greg at gregorycollins.net> wrote:

> 
> On Mon, Nov 10, 2014 at 11:11 PM, Michael Jones <mike at proclivis.com> wrote:
>        ec <- return $ encode command
>        traceEventIO $ "sendCommand: encoded"
>        l <- ec `seq` return $ BSL.length ec
> 
> Your encode function probably loops on some inputs. When you call "return $ foo", foo is not forced; that doesn't happen in your example until BSL.length forces its input. If I'm right, changing the first line to "return $! encode command" will move the hang to before the call to traceEventIO.
> 
> -- 
> Gregory Collins <greg at gregorycollins.net>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/91a32b57/attachment.html>

From allbery.b at gmail.com  Tue Nov 11 19:24:57 2014
From: allbery.b at gmail.com (Brandon Allbery)
Date: Tue, 11 Nov 2014 14:24:57 -0500
Subject: GHC 7.8.3 thread hang
In-Reply-To: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
Message-ID: <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>

On Tue, Nov 11, 2014 at 2:11 AM, Michael Jones <mike at proclivis.com> wrote:

> 7fe537eff700: cap 0: sendCommand: encoded                       PROBLEM
> HERE
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
>

Is it just me, or is that second message significant?

-- 
brandon s allbery kf8nh                               sine nomine associates
allbery.b at gmail.com                                  ballbery at sinenomine.net
unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/b1c4a6ab/attachment.html>

From mike at proclivis.com  Tue Nov 11 22:06:52 2014
From: mike at proclivis.com (Michael Jones)
Date: Tue, 11 Nov 2014 15:06:52 -0700
Subject: GHC 7.8.3 thread hang
In-Reply-To: <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
Message-ID: <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>

Those are all over the log even when it runs properly. So I assume the runtime is resizing the heap or something.

Perhaps someone knows if this is normal or not.

Mike


On Nov 11, 2014, at 12:24 PM, Brandon Allbery <allbery.b at gmail.com> wrote:

> On Tue, Nov 11, 2014 at 2:11 AM, Michael Jones <mike at proclivis.com> wrote:
> 7fe537eff700: cap 0: sendCommand: encoded                       PROBLEM HERE
> 7fe537eff700: cap 0: thread 7 stopped (heap overflow)
> 
> Is it just me, or is that second message significant?
> 
> -- 
> brandon s allbery kf8nh                               sine nomine associates
> allbery.b at gmail.com                                  ballbery at sinenomine.net
> unix, openafs, kerberos, infrastructure, xmonad        http://sinenomine.net

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/f824f990/attachment.html>

From greg at gregorycollins.net  Tue Nov 11 23:01:48 2014
From: greg at gregorycollins.net (Gregory Collins)
Date: Tue, 11 Nov 2014 15:01:48 -0800
Subject: GHC 7.8.3 thread hang
In-Reply-To: <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
 <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
Message-ID: <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>

On Tue, Nov 11, 2014 at 2:06 PM, Michael Jones <mike at proclivis.com> wrote:

> Those are all over the log even when it runs properly. So I assume the
> runtime is resizing the heap or something.


No, it means you're exhausting the heap (maybe the runtime stack for the
thread running "encode"), probably because "encode" is infinite-looping. I
think Occam's razor applies here, check that any recursion you're doing is
actually reducing the recursive argument. Perhaps you could post the code
(e.g. http://gist.github.com/)?

G


-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/2772d7dc/attachment.html>

From mike at proclivis.com  Wed Nov 12 03:24:57 2014
From: mike at proclivis.com (Michael Jones)
Date: Tue, 11 Nov 2014 20:24:57 -0700
Subject: GHC 7.8.3 thread hang
In-Reply-To: <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
 <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
 <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>
Message-ID: <0ECE95F7-1921-402B-A038-CC6A2A6776F1@proclivis.com>

Gregory,

The options in the 7.8.3 user guide says in the -Msize option that by default the heap is unlimited. I have several applications, and they all have messages like:

7fddc7bcd700: cap 2: waking up thread 7 on cap 2
7fddc7bcd700: cap 2: thread 4 stopped (yielding)
7fddcaad6740: cap 2: running thread 7 (ThreadRunGHC)
7fddcaad6740: cap 2: thread 7 stopped (heap overflow)
7fddcaad6740: cap 2: requesting parallel GC
7fddc5ffe700: cap 0: starting GC
7fddc57fd700: cap 1: starting GC
7fdda77fe700: cap 3: starting GC
7fddcaad6740: cap 2: starting GC

I assumed that when the heap ran out of space, it caused a GC, or it enlarged the heap. The programs that have these messages run for very long periods of time, and when I heap profile them, they use about 500KM to 1MB over long periods of time, and are quite stable.

As a test, I ran the hang application with profiling to see if memory jumps up before or after the hang.

What I notice is the app moves along using about 800KB, then there is a spike to 2MB at the hang. So I believe you, but I am confused about the RTS behavior and how I can have all these overflow messages in a normal application and how to tell the difference between these routine messages vs a real heap problem.

So, I dug deeper into the log. A normal execution for sending a command looks like:

7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
7f99e6ffd700: cap 0: requesting parallel GC
7f99e6ffd700: cap 0: starting GC
7f99e6ffd700: cap 0: GC working
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: all caps stopped for GC
7f99e6ffd700: cap 0: finished GC
7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: sendCommand
7f99e6ffd700: cap 0: sendCommand: encoded
7f99e6ffd700: cap 0: sendCommand: size 4
7f99e6ffd700: cap 0: sendCommand: unpacked
7f99e6ffd700: cap 0: Sending command of size 4
7f99e6ffd700: cap 0: Sending command of size "\NUL\EOT"
7f99e6ffd700: cap 0: sendCommand: sent
7f99e6ffd700: cap 0: sendCommand: flushed
7f99e6ffd700: cap 0: thread 7 stopped (blocked on an MVar)
7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 2 stopped (yielding)
7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
7f99e6ffd700: cap 0: fetchTelemetryServer
7f99e6ffd700: cap 0: fetchTelemetryServer: got lock

The thread is run, overflows, GC, runs, then blocks on an MVAr.

For a the hang case:

7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: sendCommand
7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
7f99e6ffd700: cap 0: requesting parallel GC
7f99e6ffd700: cap 0: starting GC
7f99e6ffd700: cap 0: GC working
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: all caps stopped for GC
7f99e6ffd700: cap 0: finished GC
7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
7f9a05362a40: cap 0: thread 1408 stopped (yielding)
7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
7f99e6ffd700: cap 0: requesting parallel GC
7f99e6ffd700: cap 0: starting GC
7f99e6ffd700: cap 0: GC working
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: GC idle
7f99e6ffd700: cap 0: GC done
7f99e6ffd700: cap 0: all caps stopped for GC
7f99e6ffd700: cap 0: finished GC
7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 7 stopped (yielding)
7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 2 stopped (yielding)
7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
7f99e6ffd700: cap 0: fetchTelemetryServer
7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
...
7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
7f99e6ffd700: cap 0: fetchTelemetryServer
7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
7f99e6ffd700: cap 0: fetchTelemetryServer
7f99e6ffd700: cap 0: thread 45 stopped (yielding)
7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)
7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)
7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
7f99e6ffd700: cap 0: thread 7 stopped (blocked on black hole owned by thread 7)
7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)

run, overflow, GC, run, overflow, GC, run, yield, other stuff, run blocked on black hole

And that is the last activity for thread 7.

I found a few links about black holes and such, so I?ll go off and read those and try to learn what the RTS is doing and why it can hang on a black hole. But if you have some hits, let me know. 

Mike



On Nov 11, 2014, at 4:01 PM, Gregory Collins <greg at gregorycollins.net> wrote:

> 
> On Tue, Nov 11, 2014 at 2:06 PM, Michael Jones <mike at proclivis.com> wrote:
> Those are all over the log even when it runs properly. So I assume the runtime is resizing the heap or something.
> 
> No, it means you're exhausting the heap (maybe the runtime stack for the thread running "encode"), probably because "encode" is infinite-looping. I think Occam's razor applies here, check that any recursion you're doing is actually reducing the recursive argument. Perhaps you could post the code (e.g. http://gist.github.com/)?
> 
> G
> 
> 
> -- 
> Gregory Collins <greg at gregorycollins.net>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141111/89ec52a0/attachment-0001.html>

From jwlato at gmail.com  Wed Nov 12 03:37:47 2014
From: jwlato at gmail.com (John Lato)
Date: Wed, 12 Nov 2014 03:37:47 +0000
Subject: GHC 7.8.3 thread hang
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
 <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
 <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>
 <0ECE95F7-1921-402B-A038-CC6A2A6776F1@proclivis.com>
Message-ID: <CAJ727GgkjDW3L1e2t_o870R89NdjKX+-1OA-D4FHEKgmx2tp7g@mail.gmail.com>

The blocked on black hole message is very suspicious.  It means that thread
7 is blocked waiting for another thread to evaluate a thunk.  But in this
case, it's thread 7 that created that thunk and is supposed to be doing the
evaluating.  This is some evidence that Gregory's theory is correct and
your encode function loops somewhere.

On Wed Nov 12 2014 at 11:25:30 AM Michael Jones <mike at proclivis.com> wrote:

> Gregory,
>
> The options in the 7.8.3 user guide says in the -Msize option that by
> default the heap is unlimited. I have several applications, and they all
> have messages like:
>
> 7fddc7bcd700: cap 2: waking up thread 7 on cap 2
> 7fddc7bcd700: cap 2: thread 4 stopped (yielding)
> 7fddcaad6740: cap 2: running thread 7 (ThreadRunGHC)
> 7fddcaad6740: cap 2: thread 7 stopped (heap overflow)
> 7fddcaad6740: cap 2: requesting parallel GC
> 7fddc5ffe700: cap 0: starting GC
> 7fddc57fd700: cap 1: starting GC
> 7fdda77fe700: cap 3: starting GC
> 7fddcaad6740: cap 2: starting GC
>
> I assumed that when the heap ran out of space, it caused a GC, or it
> enlarged the heap. The programs that have these messages run for very long
> periods of time, and when I heap profile them, they use about 500KM to 1MB
> over long periods of time, and are quite stable.
>
> As a test, I ran the hang application with profiling to see if memory
> jumps up before or after the hang.
>
> What I notice is the app moves along using about 800KB, then there is a
> spike to 2MB at the hang. So I believe you, but I am confused about the RTS
> behavior and how I can have all these overflow messages in a normal
> application and how to tell the difference between these routine messages
> vs a real heap problem.
>
> So, I dug deeper into the log. A normal execution for sending a command
> looks like:
>
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: sendCommand
> 7f99e6ffd700: cap 0: sendCommand: encoded
> 7f99e6ffd700: cap 0: sendCommand: size 4
> 7f99e6ffd700: cap 0: sendCommand: unpacked
> 7f99e6ffd700: cap 0: Sending command of size 4
> 7f99e6ffd700: cap 0: Sending command of size "\NUL\EOT"
> 7f99e6ffd700: cap 0: sendCommand: sent
> 7f99e6ffd700: cap 0: sendCommand: flushed
> 7f99e6ffd700: cap 0: thread 7 stopped (blocked on an MVar)
> 7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 2 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
>
> The thread is run, overflows, GC, runs, then blocks on an MVAr.
>
> For a the hang case:
>
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: sendCommand
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 2 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> ...
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> 7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> 7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: thread 45 stopped (yielding)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign
> call)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign
> call)
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (blocked on black hole owned by
> thread 7)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign
> call)
>
> run, overflow, GC, run, overflow, GC, run, yield, other stuff, run blocked
> on black hole
>
> And that is the last activity for thread 7.
>
> I found a few links about black holes and such, so I?ll go off and read
> those and try to learn what the RTS is doing and why it can hang on a black
> hole. But if you have some hits, let me know.
>
> Mike
>
>
>
> On Nov 11, 2014, at 4:01 PM, Gregory Collins <greg at gregorycollins.net>
> wrote:
>
>
> On Tue, Nov 11, 2014 at 2:06 PM, Michael Jones <mike at proclivis.com> wrote:
>
>> Those are all over the log even when it runs properly. So I assume the
>> runtime is resizing the heap or something.
>
>
> No, it means you're exhausting the heap (maybe the runtime stack for the
> thread running "encode"), probably because "encode" is infinite-looping. I
> think Occam's razor applies here, check that any recursion you're doing is
> actually reducing the recursive argument. Perhaps you could post the code
> (e.g. http://gist.github.com/)?
>
> G
>
>
> --
> Gregory Collins <greg at gregorycollins.net>
>
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141112/56d49951/attachment.html>

From mike at proclivis.com  Thu Nov 13 06:01:42 2014
From: mike at proclivis.com (Michael Jones)
Date: Wed, 12 Nov 2014 23:01:42 -0700
Subject: GHC 7.8.3 thread hang
In-Reply-To: <CAJ727GgkjDW3L1e2t_o870R89NdjKX+-1OA-D4FHEKgmx2tp7g@mail.gmail.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
 <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
 <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>
 <0ECE95F7-1921-402B-A038-CC6A2A6776F1@proclivis.com>
 <CAJ727GgkjDW3L1e2t_o870R89NdjKX+-1OA-D4FHEKgmx2tp7g@mail.gmail.com>
Message-ID: <B4227B2A-F6BD-4A93-A89B-E60741BF63D3@proclivis.com>

I am having difficulty imagining how there could be a loop that evaluates a black hole a second time by the same thread. The paper Haskell on a Shared-Memory Processor mentions that the runtime must not do this. My take on this description is that it should never happen. Even if code was recursive, it should not happen. Code might recurse forever and chew up memory until failure, but a thread creating a black hold should not trip up on its own black hole.

So I am not sure what I am looking for. There is no recursion in the encode.

Can you give some examples of what I should be looking for?

Mike


On Nov 11, 2014, at 8:37 PM, John Lato <jwlato at gmail.com> wrote:

> The blocked on black hole message is very suspicious.  It means that thread 7 is blocked waiting for another thread to evaluate a thunk.  But in this case, it's thread 7 that created that thunk and is supposed to be doing the evaluating.  This is some evidence that Gregory's theory is correct and your encode function loops somewhere.
> 
> On Wed Nov 12 2014 at 11:25:30 AM Michael Jones <mike at proclivis.com> wrote:
> Gregory,
> 
> The options in the 7.8.3 user guide says in the -Msize option that by default the heap is unlimited. I have several applications, and they all have messages like:
> 
> 7fddc7bcd700: cap 2: waking up thread 7 on cap 2
> 7fddc7bcd700: cap 2: thread 4 stopped (yielding)
> 7fddcaad6740: cap 2: running thread 7 (ThreadRunGHC)
> 7fddcaad6740: cap 2: thread 7 stopped (heap overflow)
> 7fddcaad6740: cap 2: requesting parallel GC
> 7fddc5ffe700: cap 0: starting GC
> 7fddc57fd700: cap 1: starting GC
> 7fdda77fe700: cap 3: starting GC
> 7fddcaad6740: cap 2: starting GC
> 
> I assumed that when the heap ran out of space, it caused a GC, or it enlarged the heap. The programs that have these messages run for very long periods of time, and when I heap profile them, they use about 500KM to 1MB over long periods of time, and are quite stable.
> 
> As a test, I ran the hang application with profiling to see if memory jumps up before or after the hang.
> 
> What I notice is the app moves along using about 800KB, then there is a spike to 2MB at the hang. So I believe you, but I am confused about the RTS behavior and how I can have all these overflow messages in a normal application and how to tell the difference between these routine messages vs a real heap problem.
> 
> So, I dug deeper into the log. A normal execution for sending a command looks like:
> 
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: sendCommand
> 7f99e6ffd700: cap 0: sendCommand: encoded
> 7f99e6ffd700: cap 0: sendCommand: size 4
> 7f99e6ffd700: cap 0: sendCommand: unpacked
> 7f99e6ffd700: cap 0: Sending command of size 4
> 7f99e6ffd700: cap 0: Sending command of size "\NUL\EOT"
> 7f99e6ffd700: cap 0: sendCommand: sent
> 7f99e6ffd700: cap 0: sendCommand: flushed
> 7f99e6ffd700: cap 0: thread 7 stopped (blocked on an MVar)
> 7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 2 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> 
> The thread is run, overflows, GC, runs, then blocks on an MVAr.
> 
> For a the hang case:
> 
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: sendCommand
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (heap overflow)
> 7f99e6ffd700: cap 0: requesting parallel GC
> 7f99e6ffd700: cap 0: starting GC
> 7f99e6ffd700: cap 0: GC working
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: GC idle
> 7f99e6ffd700: cap 0: GC done
> 7f99e6ffd700: cap 0: all caps stopped for GC
> 7f99e6ffd700: cap 0: finished GC
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 2 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 2 stopped (yielding)
> 7f99e6ffd700: cap 0: running thread 45 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> ...
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> 7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: fetchTelemetryServer: got lock
> 7f99e6ffd700: cap 0: fetchTelemetryServer: unlock
> 7f99e6ffd700: cap 0: fetchTelemetryServer
> 7f99e6ffd700: cap 0: thread 45 stopped (yielding)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)
> 7f99e6ffd700: cap 0: running thread 7 (ThreadRunGHC)
> 7f99e6ffd700: cap 0: thread 7 stopped (blocked on black hole owned by thread 7)
> 7f9a05362a40: cap 0: running thread 1408 (ThreadRunGHC)
> 7f9a05362a40: cap 0: thread 1408 stopped (suspended while making a foreign call)
> 
> run, overflow, GC, run, overflow, GC, run, yield, other stuff, run blocked on black hole
> 
> And that is the last activity for thread 7.
> 
> I found a few links about black holes and such, so I?ll go off and read those and try to learn what the RTS is doing and why it can hang on a black hole. But if you have some hits, let me know. 
> 
> Mike
> 
> 
> 
> On Nov 11, 2014, at 4:01 PM, Gregory Collins <greg at gregorycollins.net> wrote:
> 
>> 
>> On Tue, Nov 11, 2014 at 2:06 PM, Michael Jones <mike at proclivis.com> wrote:
>> Those are all over the log even when it runs properly. So I assume the runtime is resizing the heap or something.
>> 
>> No, it means you're exhausting the heap (maybe the runtime stack for the thread running "encode"), probably because "encode" is infinite-looping. I think Occam's razor applies here, check that any recursion you're doing is actually reducing the recursive argument. Perhaps you could post the code (e.g. http://gist.github.com/)?
>> 
>> G
>> 
>> 
>> -- 
>> Gregory Collins <greg at gregorycollins.net>
> 
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141112/85a28b3b/attachment.html>

From bulat.ziganshin at gmail.com  Thu Nov 13 14:25:49 2014
From: bulat.ziganshin at gmail.com (Bulat Ziganshin)
Date: Thu, 13 Nov 2014 17:25:49 +0300
Subject: RFC: Dropping Windows XP support
In-Reply-To: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
References: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
Message-ID: <142458433.20141113172549@gmail.com>

Hello Austin,

Friday, November 7, 2014, 9:16:22 PM, you wrote:

> For one, Microsoft doesn't support XP anymore, so most people are
> moving off it anyway. 'Soon' even XP Embedded will be obsoleted.

at  the  end  of  http://freearc.org/Statistics.aspx page you can find
stats  about  OS  used  by  ysers  of my program - archiver written in
haskell.  these  are  14%  of  *geeks* using the l;atest alpha version
(stable version of my program ddoesn't report this parameter)

so  dropping  XP support means that newer GHC can't be used anymore to
compile general-purpose programs targeting large user audience

-- 
Best regards,
 Bulat                            mailto:Bulat.Ziganshin at gmail.com


From greg at gregorycollins.net  Thu Nov 13 15:07:02 2014
From: greg at gregorycollins.net (Gregory Collins)
Date: Thu, 13 Nov 2014 07:07:02 -0800
Subject: GHC 7.8.3 thread hang
In-Reply-To: <B4227B2A-F6BD-4A93-A89B-E60741BF63D3@proclivis.com>
References: <FB8A7B58-EB4E-457A-9396-9FC6DE991245@proclivis.com>
 <CAKFCL4ULqpUHeCiGv8_txB7VpPh3VrSex7dFmoAapczFcTrN6w@mail.gmail.com>
 <4D970144-60F5-4B90-8923-262F9EAB5991@proclivis.com>
 <CAHZrzdqNzE9+dDhMynEnowiHG6cUQAo+hYwgBigo5rJOh0=rvg@mail.gmail.com>
 <0ECE95F7-1921-402B-A038-CC6A2A6776F1@proclivis.com>
 <CAJ727GgkjDW3L1e2t_o870R89NdjKX+-1OA-D4FHEKgmx2tp7g@mail.gmail.com>
 <B4227B2A-F6BD-4A93-A89B-E60741BF63D3@proclivis.com>
Message-ID: <CAHZrzdpCmcu7066iCE=8dZCXwMr8CWB1My87Cee2NL4hxOciqw@mail.gmail.com>

On Wed, Nov 12, 2014 at 10:01 PM, Michael Jones <mike at proclivis.com> wrote:

> Can you give some examples of what I should be looking for?


Without seeing the relevant source code I don't see how to help you further.

Re: blackholes, are you (or any code you're calling) using the "par"
combinator?

G
-- 
Gregory Collins <greg at gregorycollins.net>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141113/613a79c1/attachment.html>

From spam at scientician.net  Thu Nov 13 16:56:47 2014
From: spam at scientician.net (Bardur Arantsson)
Date: Thu, 13 Nov 2014 17:56:47 +0100
Subject: RFC: Dropping Windows XP support
In-Reply-To: <142458433.20141113172549@gmail.com>
References: <CAEs96n25Bf2eDCn+dNMHCgKHPQ03F6kZXFBfzEuAacR6rojpBw@mail.gmail.com>
 <142458433.20141113172549@gmail.com>
Message-ID: <m42nsg$3lh$1@ger.gmane.org>

On 2014-11-13 15:25, Bulat Ziganshin wrote:
> Hello Austin,
> 
> Friday, November 7, 2014, 9:16:22 PM, you wrote:
> 
>> For one, Microsoft doesn't support XP anymore, so most people are
>> moving off it anyway. 'Soon' even XP Embedded will be obsoleted.
> 
> at  the  end  of  http://freearc.org/Statistics.aspx page you can find
> stats  about  OS  used  by  ysers  of my program - archiver written in
> haskell.  these  are  14%  of  *geeks* using the l;atest alpha version
> (stable version of my program ddoesn't report this parameter)
> 
> so  dropping  XP support means that newer GHC can't be used anymore to
> compile general-purpose programs targeting large user audience
> 

I'm not sure I understand the version numbers, but it seems 95.46% of
your users a version from before June 26, 2010. How likely are they to
upgrade if they haven't upgraded since 2010?


From g9ks157k at acme.softbase.org  Thu Nov 13 19:02:45 2014
From: g9ks157k at acme.softbase.org (Wolfgang Jeltsch)
Date: Thu, 13 Nov 2014 21:02:45 +0200
Subject: Under what conditions are finalizers not run?
Message-ID: <1415905365.3683.47.camel@idefix>

Hi,

the documentation of System.Mem.Weak under

    <http://hackage.haskell.org/package/base-4.7.0.1/docs/System-Mem-Weak.html>

says the following:

> It is not guaranteed that a finalizer will eventually run, and no
> attempt is made to run outstanding finalizers when the program exits.

In which situations are finalizers not run? I see that they might not be
run when the program ends while the weak pointer is still alive, but are
there also other situations? The above quote seems to say that even an
implementation that ignores any finalizers would conform to the API
specification.

I think it is actually quite bad if you do not have any guarantees about
when finalizers are run. In the memo table example, for instance, this
would mean that the memo table could grow very large, which would not
just result in bad space complexity, but also bad time complexity for
lookups.

I am actually particularly interested in adding finalizers to IORefs.
Are there any stronger guarantees for finalizers attached to IORefs and
MVars?

All the best,
Wolfgang


From g9ks157k at acme.softbase.org  Thu Nov 13 19:03:39 2014
From: g9ks157k at acme.softbase.org (Wolfgang Jeltsch)
Date: Thu, 13 Nov 2014 21:03:39 +0200
Subject: Rules for class methods and Safe Haskell
In-Reply-To: <1408133439.19601.10.camel@idefix>
References: <1408133439.19601.10.camel@idefix>
Message-ID: <1415905419.3683.49.camel@idefix>


Am Freitag, den 15.08.2014, 23:10 +0300 schrieb Wolfgang Jeltsch:
> Hi,
> 
> the module Control.Arrow declares a set of rules for the Arrow class. It
> is marked ?Trustworthy?, probably to allow these rules to actually fire.
> 
> Now these rules are only correct for class instances that actually
> satisfy the arrow laws. If the author of another module defines an
> instance of Arrow that does not respect the laws, this other module
> could still be considered ?Safe? by GHC, although the rules from
> Control.Arrow are bogus now.
> 
> Is this considered a problem?
> 
> All the best,
> Wolfgang

Hi,

could someone please answer this e-mail? This issue is important for me.

All the best,
Wolfgang



From david.feuer at gmail.com  Thu Nov 13 19:12:18 2014
From: david.feuer at gmail.com (David Feuer)
Date: Thu, 13 Nov 2014 14:12:18 -0500
Subject: Rules for class methods and Safe Haskell
In-Reply-To: <1415905419.3683.49.camel@idefix>
References: <1408133439.19601.10.camel@idefix>
 <1415905419.3683.49.camel@idefix>
Message-ID: <CAMgWh9vZwwSWtKkyG2JiYSf49Ou5x0hV4JDYNNuq5NP06YP8aQ@mail.gmail.com>

That's an interesting question. I'm not even close to an expert, but I
*think* that parametricity prevents those particular rules from breaking
Safe Haskell guarantees. The laws may not *hold* for a broken instance, but
I don't *think* that lets you break type safety or IO encapsulation.
On Nov 13, 2014 2:03 PM, "Wolfgang Jeltsch" <g9ks157k at acme.softbase.org>
wrote:

>
> Am Freitag, den 15.08.2014, 23:10 +0300 schrieb Wolfgang Jeltsch:
> > Hi,
> >
> > the module Control.Arrow declares a set of rules for the Arrow class. It
> > is marked ?Trustworthy?, probably to allow these rules to actually fire.
> >
> > Now these rules are only correct for class instances that actually
> > satisfy the arrow laws. If the author of another module defines an
> > instance of Arrow that does not respect the laws, this other module
> > could still be considered ?Safe? by GHC, although the rules from
> > Control.Arrow are bogus now.
> >
> > Is this considered a problem?
> >
> > All the best,
> > Wolfgang
>
> Hi,
>
> could someone please answer this e-mail? This issue is important for me.
>
> All the best,
> Wolfgang
>
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141113/4be32384/attachment.html>

From merijn at inconsistent.nl  Fri Nov 14 04:22:11 2014
From: merijn at inconsistent.nl (Merijn Verstraaten)
Date: Thu, 13 Nov 2014 20:22:11 -0800
Subject: More flexible literate Haskell extensions (Trac #9789),
 summary on wiki
Message-ID: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>

As requested on my ticket I summarised the entire proposal on the wiki here: https://ghc.haskell.org/trac/ghc/wiki/FlexibleLiterateExtension

I don't expect a lot of disagreement on discussion, aside from minor bike shedding on the flavour of the extension. I've started implementing this already. I'm open to bikesheds on exact extension, as it shouldn't affect the implementation.

Unless there's any vehement objections, I'll produce a diff on fabricator asap.

Cheers,
Merijn

From sol at typeful.net  Fri Nov 14 06:46:10 2014
From: sol at typeful.net (Simon Hengel)
Date: Fri, 14 Nov 2014 14:46:10 +0800
Subject: More flexible literate Haskell extensions (Trac #9789), summary
 on wiki
In-Reply-To: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
Message-ID: <20141114064610.GA27317@x200>

> I don't expect a lot of disagreement on discussion, aside from minor
> bike shedding on the flavour of the extension. I've started
> implementing this already. I'm open to bikesheds on exact extension,
> as it shouldn't affect the implementation.

Joining the party late, my main use case for literate Haskell is
README.md files.  The solution I currently use is to have a symlink:

    README.lhs -> README.md

(see e.g. https://github.com/hspec/hspec-wai)

As I understand it the current proposal neither helps nor conflicts with
this use case/solution.

I leave this here as a comment mainly to indicated that:

 * the proposal as it stands does not solve my use case
 * there are other ways to solve this (namely: using symlinks)

Cheers,
Simon Hengel

From simonpj at microsoft.com  Fri Nov 14 10:29:30 2014
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 14 Nov 2014 10:29:30 +0000
Subject: More flexible literate Haskell extensions (Trac #9789), summary
 on wiki
In-Reply-To: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
Message-ID: <618BE556AADD624C9C918AA5D5911BEF3F3BDD49@DB3PRD3001MB020.064d.mgd.msft.net>

Thanks.  I don't have strong opinions about any of this.  But I would love to have an actual specification of what is proposed.  The wiki page starts with "Proposal" but lists a set of alternatives.  Later "Concrete proposal" discusses only file suffixes, and has lots of discussion of alternatives.

Would it be possible to have a section
   a) describing a single alternative, as precisely as possible.
   b) saying what the effect or meaning of proposal is

For (a), is Foo.hs still ok?  Foo.lhs?  What if both exist and/or Foo.md.hs or whatever?

For (b) what does a suffix of Foo.hs.md mean?  Presumably there is some markdown in there.  But how is it delimited?  Is md the only one proposed or are there others? Is it meant to be extensible or is there a fixed set?

In short, a *specification* of what is proposed.  I think that would be helpful for people who come to this without having participated in the discussion that led up to it.

Simon

|  -----Original Message-----
|  From: Glasgow-haskell-users [mailto:glasgow-haskell-users-
|  bounces at haskell.org] On Behalf Of Merijn Verstraaten
|  Sent: 14 November 2014 04:22
|  To: ghc-devs at haskell.org; GHC Users Mailing List
|  Subject: More flexible literate Haskell extensions (Trac #9789),
|  summary on wiki
|  
|  As requested on my ticket I summarised the entire proposal on the wiki
|  here: https://ghc.haskell.org/trac/ghc/wiki/FlexibleLiterateExtension
|  
|  I don't expect a lot of disagreement on discussion, aside from minor
|  bike shedding on the flavour of the extension. I've started
|  implementing this already. I'm open to bikesheds on exact extension,
|  as it shouldn't affect the implementation.
|  
|  Unless there's any vehement objections, I'll produce a diff on
|  fabricator asap.
|  
|  Cheers,
|  Merijn
|  _______________________________________________
|  Glasgow-haskell-users mailing list
|  Glasgow-haskell-users at haskell.org
|  http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

From jan.stolarek at p.lodz.pl  Fri Nov 14 12:00:49 2014
From: jan.stolarek at p.lodz.pl (Jan Stolarek)
Date: Fri, 14 Nov 2014 13:00:49 +0100
Subject: More flexible literate Haskell extensions (Trac #9789),
 summary on wiki
In-Reply-To: <878ujekonf.fsf@gmail.com>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
 <878ujekonf.fsf@gmail.com>
Message-ID: <201411141300.49816.jan.stolarek@p.lodz.pl>

.lhs = Literate Haskell

So maybe .mhs = Markdown Haskell instead of .lhs+md?

Jan

From sol at typeful.net  Fri Nov 14 15:33:42 2014
From: sol at typeful.net (Simon Hengel)
Date: Fri, 14 Nov 2014 23:33:42 +0800
Subject: More flexible literate Haskell extensions (Trac #9789), summary
 on wiki
In-Reply-To: <20141114075313.GA8925@x200>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
 <20141114064610.GA27317@x200>
 <3E964435-C2F8-4D0F-B7CA-31C5D2A7BC25@inconsistent.nl>
 <20141114075313.GA8925@x200>
Message-ID: <20141114153342.GA15607@x200>

On Fri, Nov 14, 2014 at 03:53:13PM +0800, Simon Hengel wrote:
> > Can you explain why this would not solve your usecase? i.e., why would
> > README.lhs.md not suffice?
> 
> I haven't tried, but my assumption was that this is not picked up by
> GitHub.

I just verified, GitHub dose not pick up README.lhs.md files as READMEs,
meaning that a README.lhs.md file will not be rendered as a projects
README file on GitHub.

Cheers,
Simon

From merijn at inconsistent.nl  Sun Nov 16 21:42:12 2014
From: merijn at inconsistent.nl (Merijn Verstraaten)
Date: Sun, 16 Nov 2014 13:42:12 -0800
Subject: More flexible literate Haskell extensions (Trac #9789),
 summary on wiki
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F3BDD49@DB3PRD3001MB020.064d.mgd.msft.net>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
 <618BE556AADD624C9C918AA5D5911BEF3F3BDD49@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <13F0D94D-E770-43C2-87FE-169E3A5ED1EA@inconsistent.nl>

Hi Simon,

Thanks for the comments. I think most of the confusion stems from people overthinking the scope of what I was proposing. I'll clear up the page a bit as it's currently conflating implementation details with semantics.

> On 14 Nov 2014, at 2:29, Simon Peyton Jones <simonpj at microsoft.com> wrote:
> Would it be possible to have a section
>   a) describing a single alternative, as precisely as possible.

The single alternative would simply be:
If GHC tries to find the source for a module Foo and none of "Foo.hs", "Foo.lhs", "Foo.hsig" or "Foo.lhsig" are found, it will accept any file with a "Foo.lhs.*" extension, i.e., "Foo.lhs.md", "Foo.lhs.tex", etc.

>   b) saying what the effect or meaning of proposal is

The proposal does NOT modify the way GHC treats the contents of files or unlits literate haskell in anyway. While I'm in favour of supporting more literate formats, that's orthogonal to this proposal.

> For (a), is Foo.hs still ok?  Foo.lhs?  What if both exist and/or Foo.md.hs or whatever?

Yes, both "Foo.hs" and "Foo.lhs" are still ok. I don't think the manual specifies what GHC does in the case "Foo.hs" AND "Foo.lhs" both exist. But the implementation prefers extensions in the following order: "hs", "lhs", "hsig" and "lhsig". I would just add the new allowed extension behind that as lower priority than the current ones.

> For (b) what does a suffix of Foo.hs.md mean?  Presumably there is some markdown in there.  But how is it delimited?  Is md the only one proposed or are there others? Is it meant to be extensible or is there a fixed set?

See my earlier point, I do *not* intend to affect the way GHC interprets/unlits the contents of files. Pandoc is already perfectly happy to work with literate files, it just currently lacks a way to determine what the content type of the non-literate bits is. Which is what I hope to deal with here.

Cheers,
Merijn

From simonpj at microsoft.com  Sun Nov 16 22:09:53 2014
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Sun, 16 Nov 2014 22:09:53 +0000
Subject: More flexible literate Haskell extensions (Trac #9789), summary
 on wiki
In-Reply-To: <13F0D94D-E770-43C2-87FE-169E3A5ED1EA@inconsistent.nl>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
 <618BE556AADD624C9C918AA5D5911BEF3F3BDD49@DB3PRD3001MB020.064d.mgd.msft.net>
 <13F0D94D-E770-43C2-87FE-169E3A5ED1EA@inconsistent.nl>
Message-ID: <618BE556AADD624C9C918AA5D5911BEF3F3C181F@DB3PRD3001MB020.064d.mgd.msft.net>

Marijn,

Thanks.  Can you make sure that you update the wiki page to reflect what you say here?  Email is transitory; the wiki page gives the *specification* of the feature, and says unambiguously what you intend.  Misunderstandings expressed in email are simply tell you how to improve the wiki page!

thanks

Simon

| -----Original Message-----
| From: Merijn Verstraaten [mailto:merijn at inconsistent.nl]
| Sent: 16 November 2014 21:42
| To: Simon Peyton Jones
| Cc: ghc-devs at haskell.org; GHC Users Mailing List
| Subject: Re: More flexible literate Haskell extensions (Trac #9789),
| summary on wiki
| 
| Hi Simon,
| 
| Thanks for the comments. I think most of the confusion stems from people
| overthinking the scope of what I was proposing. I'll clear up the page a
| bit as it's currently conflating implementation details with semantics.
| 
| > On 14 Nov 2014, at 2:29, Simon Peyton Jones <simonpj at microsoft.com>
| wrote:
| > Would it be possible to have a section
| >   a) describing a single alternative, as precisely as possible.
| 
| The single alternative would simply be:
| If GHC tries to find the source for a module Foo and none of "Foo.hs",
| "Foo.lhs", "Foo.hsig" or "Foo.lhsig" are found, it will accept any file
| with a "Foo.lhs.*" extension, i.e., "Foo.lhs.md", "Foo.lhs.tex", etc.
| 
| >   b) saying what the effect or meaning of proposal is
| 
| The proposal does NOT modify the way GHC treats the contents of files or
| unlits literate haskell in anyway. While I'm in favour of supporting more
| literate formats, that's orthogonal to this proposal.
| 
| > For (a), is Foo.hs still ok?  Foo.lhs?  What if both exist and/or
| Foo.md.hs or whatever?
| 
| Yes, both "Foo.hs" and "Foo.lhs" are still ok. I don't think the manual
| specifies what GHC does in the case "Foo.hs" AND "Foo.lhs" both exist.
| But the implementation prefers extensions in the following order: "hs",
| "lhs", "hsig" and "lhsig". I would just add the new allowed extension
| behind that as lower priority than the current ones.
| 
| > For (b) what does a suffix of Foo.hs.md mean?  Presumably there is some
| markdown in there.  But how is it delimited?  Is md the only one proposed
| or are there others? Is it meant to be extensible or is there a fixed
| set?
| 
| See my earlier point, I do *not* intend to affect the way GHC
| interprets/unlits the contents of files. Pandoc is already perfectly
| happy to work with literate files, it just currently lacks a way to
| determine what the content type of the non-literate bits is. Which is
| what I hope to deal with here.
| 
| Cheers,
| Merijn

From merijn at inconsistent.nl  Sun Nov 16 22:57:24 2014
From: merijn at inconsistent.nl (Merijn Verstraaten)
Date: Sun, 16 Nov 2014 14:57:24 -0800
Subject: More flexible literate Haskell extensions (Trac #9789),
 summary on wiki
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F3C181F@DB3PRD3001MB020.064d.mgd.msft.net>
References: <CD574F24-99A8-4276-BED2-8C3050E17332@inconsistent.nl>
 <618BE556AADD624C9C918AA5D5911BEF3F3BDD49@DB3PRD3001MB020.064d.mgd.msft.net>
 <13F0D94D-E770-43C2-87FE-169E3A5ED1EA@inconsistent.nl>
 <618BE556AADD624C9C918AA5D5911BEF3F3C181F@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <01F8A9A8-C7CC-401C-959F-46DEB7EF07A2@inconsistent.nl>

On 16 Nov 2014, at 14:09, Simon Peyton Jones <simonpj at microsoft.com> wrote:
> Thanks.  Can you make sure that you update the wiki page to reflect what you say here?  Email is transitory; the wiki page gives the *specification* of the feature, and says unambiguously what you intend.  Misunderstandings expressed in email are simply tell you how to improve the wiki page!

I had already updated the wiki with the relevant notes before replying, but apparently I forgot to mention this!

Cheers,
Merijn

From austin at well-typed.com  Tue Nov 18 16:07:43 2014
From: austin at well-typed.com (Austin Seipp)
Date: Tue, 18 Nov 2014 10:07:43 -0600
Subject: The future of the Haskell98 and Haskell2010 packages
Message-ID: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>

Hello all,

A few weeks ago, I opened up a discussion about a particular GHC bug, #9590.

This bug is concerned with the future of the Haskell98 and Haskell2010
packages, which try to embody their two respective Haskell standards.
They do this by shipping the 'exact library specification' that the
standards have.

In our discussion this past week between me, SimonM, SPJ, Herbert and
Mikolaj, we came to this discussion again since the 7.10 STABLE freeze
is almost here, and it seems to have puttered out.

In this discussion, we came to the conclusion we think these packages
should be removed from GHC for the 7.10 release. To be clear, this was
not a 100% unanimous decision or formal vote; SimonM and I supported
removal, while everyone else seemed to be rather undecided or
ambivalent.

Most of the proposed alternative solutions seemed somewhat one-off.
Furthermore, we didn't find a solution that wouldn't either A) require
some amount of GHC modifications (possibly indefinitely into the
future) to support these packages, or B) changing the definitions in
these packages to deviate from the standard.

Instead, we proposed that we instead overhaul part of the GHC users
manual, and clearly outline our deviations from the Haskell 2010
standard library.

To be clear: GHC can still typecheck, compile, and efficiently execute
Haskell 2010 code. It is merely the distribution of compatible
packages that has put us in something of a bind.

Furthermore, we aren't aware of any other compilers/platforms like
ours that try to maintain such stringent separation of these packages,
and furthermore, both of the haskell{98,2010} packages have a fairly
small number of reverse dependencies.

I'd like to hear what people think about this. It seems likely I will
move forward on this by the end of the week unless we face very strong
opposition to this idea, or someone is willing to fix #9590 somehow
for us.

-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

From eir at cis.upenn.edu  Tue Nov 18 16:44:55 2014
From: eir at cis.upenn.edu (Richard Eisenberg)
Date: Tue, 18 Nov 2014 11:44:55 -0500
Subject: The future of the Haskell98 and Haskell2010 packages
In-Reply-To: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>
References: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>
Message-ID: <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>

I support this direction. But I disagree with one statement you've made:

On Nov 18, 2014, at 11:07 AM, Austin Seipp <austin at well-typed.com> wrote:
> To be clear: GHC can still typecheck, compile, and efficiently execute
> Haskell 2010 code. It is merely the distribution of compatible
> packages that has put us in something of a bind.

GHC 7.10 will not be able to compile a Haskell2010-compliant Monad instance. In fact, as far as I can see, there is no way to write a Monad instance that is both portable to other Haskell compilers and acceptable to GHC 7.10. I think this point should be included in the manual (if I'm right).

This makes me a little sad, but I don't disagree with any of the decisions we've made along the way.

Richard

From austin at well-typed.com  Tue Nov 18 16:57:59 2014
From: austin at well-typed.com (Austin Seipp)
Date: Tue, 18 Nov 2014 10:57:59 -0600
Subject: The future of the Haskell98 and Haskell2010 packages
In-Reply-To: <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>
References: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>
 <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>
Message-ID: <CAEs96n3pF9=xjD0REkM2dyRaPft-i3DJvxGhmgKEOZLaS3CEZA@mail.gmail.com>

You're right, and something like that would be included. (I actually
meant GHC can still literally accept perfectly valid Haskell2010 code
in a syntactical sense; instances are part what I was referring to as
'compatible packages')

Actually, this reminds me of something SimonPJ mentioned yesterday
during this call, which may be able to fix this one bug, at least:

We could have -XHaskell2010 (or whatever it is) imply a subset of
RebindableSyntax, which only takes place for 'do' syntax. Then we
would also have the compiler import the haskell{98,2010} package as it
does today, with its own Monad (which does not have an Applicative
superclass constraint) and things would work.

This probably would not be a lot of work, but my main reservation I
suppose is that I don't think it's a very 'scalable' fix with the way
we seem to be going. If any other kind of similar change ever happened
again, we'd be stuck here once more since the same quick fix won't
apply. But if someone did the work for this, I'm not sure I would say
"no" to it.

Relatedly, I also wonder how long we should hold on to old standards.
It has been four years since a prior revision of the standard, and
almost 16 since Haskell 98. If Haskell 2015 happens soon, will we want
to have three standards packages for 7.12 if we don't remove them now?

On Tue, Nov 18, 2014 at 10:44 AM, Richard Eisenberg <eir at cis.upenn.edu> wrote:
> I support this direction. But I disagree with one statement you've made:
>
> On Nov 18, 2014, at 11:07 AM, Austin Seipp <austin at well-typed.com> wrote:
>> To be clear: GHC can still typecheck, compile, and efficiently execute
>> Haskell 2010 code. It is merely the distribution of compatible
>> packages that has put us in something of a bind.
>
> GHC 7.10 will not be able to compile a Haskell2010-compliant Monad instance. In fact, as far as I can see, there is no way to write a Monad instance that is both portable to other Haskell compilers and acceptable to GHC 7.10. I think this point should be included in the manual (if I'm right).
>
> This makes me a little sad, but I don't disagree with any of the decisions we've made along the way.
>
> Richard



-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

From david.feuer at gmail.com  Tue Nov 18 17:02:08 2014
From: david.feuer at gmail.com (David Feuer)
Date: Tue, 18 Nov 2014 12:02:08 -0500
Subject: The future of the Haskell98 and Haskell2010 packages
In-Reply-To: <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>
References: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>
 <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>
Message-ID: <CAMgWh9vZzf9XumdQw1DKR0S8pmhuH3GtY_=LsH4H=umok2C4Vw@mail.gmail.com>

I think you're right, and that's a strong reason to come up with an update
to the Haskell Report. Include in it, at least:

-- Big-ticket items
0. Monoid
1. Foldable, Traversable
2. Applicative
3. Applicative => Monad
-- side notes
4. inits = map reverse . scanl (flip (:)) []  -- efficiency?not optimal but
not hilariously bad
5. unwords = intercalate " "  -- increased, more intuitive laziness

On Tue, Nov 18, 2014 at 11:44 AM, Richard Eisenberg <eir at cis.upenn.edu>
wrote:

> I support this direction. But I disagree with one statement you've made:
>
> On Nov 18, 2014, at 11:07 AM, Austin Seipp <austin at well-typed.com> wrote:
> > To be clear: GHC can still typecheck, compile, and efficiently execute
> > Haskell 2010 code. It is merely the distribution of compatible
> > packages that has put us in something of a bind.
>
> GHC 7.10 will not be able to compile a Haskell2010-compliant Monad
> instance. In fact, as far as I can see, there is no way to write a Monad
> instance that is both portable to other Haskell compilers and acceptable to
> GHC 7.10. I think this point should be included in the manual (if I'm
> right).
>
> This makes me a little sad, but I don't disagree with any of the decisions
> we've made along the way.
>
> Richard
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141118/8b932272/attachment.html>

From singpolyma at singpolyma.net  Tue Nov 18 17:10:07 2014
From: singpolyma at singpolyma.net (Stephen Paul Weber)
Date: Tue, 18 Nov 2014 12:10:07 -0500
Subject: The future of the Haskell98 and Haskell2010 packages
In-Reply-To: <CAEs96n3pF9=xjD0REkM2dyRaPft-i3DJvxGhmgKEOZLaS3CEZA@mail.gmail.com>
References: <CAEs96n0gfc8UUsgakA2E+DXSzwgrNXWWAOjX5f9mu8iRpFo1tg@mail.gmail.com>
 <09109797-E902-4723-BB8C-A73DDA4F1D7F@cis.upenn.edu>
 <CAEs96n3pF9=xjD0REkM2dyRaPft-i3DJvxGhmgKEOZLaS3CEZA@mail.gmail.com>
Message-ID: <20141118171007.GF2811@singpolyma-liberty>

>RebindableSyntax

I thought this would work, but people seemed pretty sure we would need to do 
more work than RebindableSyntax to get everything in place.

-- 
Stephen Paul Weber, @singpolyma
See <http://singpolyma.net> for how I prefer to be contacted
edition right joseph
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141118/fa0a8f54/attachment.sig>

From austin at well-typed.com  Tue Nov 18 19:31:01 2014
From: austin at well-typed.com (Austin Seipp)
Date: Tue, 18 Nov 2014 13:31:01 -0600
Subject: NOTE: the primary webserver is going down for immediate maintenance.
Message-ID: <CAEs96n0zvExnfWC0GCNdkcjjfo6N8yac7bYpkE3ip3iEWGrodg@mail.gmail.com>

Hello *,

The primary haskell.org domain, www.haskell.org, is hosted on a system
which seems to have lost one of its RAID disks completely.

We were planning on moving this machine in the next few weeks to new
infrastructure, but we are now expediting this plan and will be doing
it ASAP.

As we move this server, both the webserver and the mailing system will
be going down. Please don't be alarmed if your emails aren't delivered
or things go quiet. Many services will continue to work, but we do
realize this will be upsetting for many.

You can follow the progress on #haskell-infrastructure on Freenode,
and see updates on https://status.haskell.org

If you need to download something like a GHC binary or Haskell
Platform package, you can use https://downloads.haskell.org in the
mean time, which is a new service we were hoping to announce more
officially soon, but is already working today.

Unfortunately we cannot give an expected time of completion for the
move, but we'll try to keep people well informed through IRC or
something like Reddit.

Thanks

-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

From davean at xkcd.com  Wed Nov 19 05:47:25 2014
From: davean at xkcd.com (davean)
Date: Wed, 19 Nov 2014 00:47:25 -0500
Subject: NOTE: the primary webserver is going down for immediate
 maintenance.
In-Reply-To: <CAEs96n0zvExnfWC0GCNdkcjjfo6N8yac7bYpkE3ip3iEWGrodg@mail.gmail.com>
References: <CAEs96n0zvExnfWC0GCNdkcjjfo6N8yac7bYpkE3ip3iEWGrodg@mail.gmail.com>
Message-ID: <CABFs2VjmL83+daMEFNGKGwnjtCvzs+PrpnstYBreXhwvODkTyA@mail.gmail.com>

At this time services are starting to come back.

The site is very slow due to a temporary patch while we improve the
connection to the database server.
Please report issues other then slowness to the #haskell-infrastructure on
Freenode.

Apologies for the interruption you were forced to experience.

On Tue, Nov 18, 2014 at 2:31 PM, Austin Seipp <austin at well-typed.com> wrote:

> Hello *,
>
> The primary haskell.org domain, www.haskell.org, is hosted on a system
> which seems to have lost one of its RAID disks completely.
>
> We were planning on moving this machine in the next few weeks to new
> infrastructure, but we are now expediting this plan and will be doing
> it ASAP.
>
> As we move this server, both the webserver and the mailing system will
> be going down. Please don't be alarmed if your emails aren't delivered
> or things go quiet. Many services will continue to work, but we do
> realize this will be upsetting for many.
>
> You can follow the progress on #haskell-infrastructure on Freenode,
> and see updates on https://status.haskell.org
>
> If you need to download something like a GHC binary or Haskell
> Platform package, you can use https://downloads.haskell.org in the
> mean time, which is a new service we were hoping to announce more
> officially soon, but is already working today.
>
> Unfortunately we cannot give an expected time of completion for the
> move, but we'll try to keep people well informed through IRC or
> something like Reddit.
>
> Thanks
>
> --
> Regards,
>
> Austin Seipp, Haskell Consultant
> Well-Typed LLP, http://www.well-typed.com/
> _______________________________________________
> ghc-devs mailing list
> ghc-devs at haskell.org
> http://www.haskell.org/mailman/listinfo/ghc-devs
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141119/ed69fc34/attachment-0001.html>

From v.dijk.bas at gmail.com  Fri Nov 21 07:51:04 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Fri, 21 Nov 2014 08:51:04 +0100
Subject: Type-level Replicate function
Message-ID: <CADLz5wWK0dE_3eW05Bc_XYGCyRjUhqDfr_Oz2shLw-swZsPHmQ@mail.gmail.com>

Hi,

tl;dr
How do I implement a type-level Replicate function that takes a (n ::
Nat) and a type e and produces a type-level list of n e elements.

The rest of my email gives some background on why I need this, how
it's supposed to be used and my current erroneous implementation. I'm
presenting it as a Haskell program:

{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE UndecidableInstances #-}
{-# LANGUAGE GADTs #-}

import GHC.TypeLits
import Data.Type.Equality
import Data.Type.Bool

-- For work I'm implementing a binary protocol for
-- communicating with some device. I need to be able to send
-- a packet of orders to a device.
--
-- An order is a list of addresses and command to send to
-- these addresses.
--
-- After sending the packets of orders the device should
-- send back a packet of replies. The packet of replies
-- should have a reply for each command send to each
-- address.
--
-- I would like to make the latter requirement apparent in
-- the types so that when I send a packet of orders the type
-- will tell me what packet of replies to expect.
--
-- So I introduce the OrderPacket type which is
-- parameterised with a type-level lists of commands:
data OrderPacket (cmds :: [*]) where

    -- The empty packet:
    NoOrders :: OrderPacket '[]

    -- Extending an existing packet with an order.
    --
    -- The (n :: Nat) in Order denotes the number of
    -- addresses to send the command to. Note that because I
    -- expect n command replies back I need to create a
    -- type-level list of n commands and prepend that to the
    -- existing commands:
    (:>) :: !(Order n command)
         -> !(OrderPacket commands)
         -> OrderPacket (Replicate n command :++: commands)

-- As explained, an Order is a vector of addresses of length
-- n paired with a command:
data Order (n :: Nat) command where
    Order :: (Command command)
          => !(Addresses n)
          -> !command
          -> Order n command

-- In my real implementation this class describes how to
-- encode / decode commands:
class Command command

-- This is the typical Vector of length n type specialised
-- to Address:
data Addresses (n :: Nat) where
    Nil   :: Addresses 0
    (:*:) :: !Address -> !(Addresses n) -> Addresses (n+1)

infixr 5 :*:

type Address = Int

-- Append two type-level lists.
type family (rs1 :: [*]) :++: (rs2 :: [*]) :: [*]

type instance '[]        :++: rs2 = rs2
type instance (r ': rs1) :++: rs2 = r ': (rs1 :++: rs2)


-- I'm currently using the following Replicate type-level
-- function:
type family Replicate (n :: Nat) r :: [*]

type instance Replicate (n :: Nat) r =
    If (n == 0) '[] (r ': Replicate (n-1) r)

-- However when I write some code like the following:
isNull :: OrderPacket cmds -> Bool
isNull NoOrders                = True
isNull (orderMsg :> orderMsgs) = False

-- I get the following error:

src/TypeLevelReplicate.hs:49:9:
    Type function application stack overflow; size = 201
    Use -ftype-function-depth=N to increase stack size to N
      GHC.TypeLits.EqNat
        (((n-1)-1)-1)...) 0
      ~ (((n-1)-1)-1)...) == 0)
    In the pattern: orderMsg :> orderMsgs
    In an equation for ?isNull?:
      isNull (orderMsg :> orderMsgs) = True
Failed, modules loaded: none.

-- Note that when I adapt my code to use my own type-level
-- inductive implementation of natural numbers like:

data N = Z | S N

type family Replicate (n :: N) r :: [*]

type instance Replicate Z     r = '[]
type instance Replicate (S n) r = r ': Replicate n r

-- I don't get the type function application stack overflow.

Any hints on how to make this work with Nats from GHC.TypeLits?

Thanks,

Bas

From gergo at erdi.hu  Fri Nov 21 08:21:39 2014
From: gergo at erdi.hu (=?UTF-8?B?RHIuIMOJUkRJIEdlcmfFkQ==?=)
Date: Fri, 21 Nov 2014 16:21:39 +0800
Subject: Type-level Replicate function
In-Reply-To: <CAO7EwMqNtj1ks5juePcRJr_QsRgfksOOkEqHz11Hcs+kLrR=aw@mail.gmail.com>
References: <CADLz5wWK0dE_3eW05Bc_XYGCyRjUhqDfr_Oz2shLw-swZsPHmQ@mail.gmail.com>
 <CAO7EwMqNtj1ks5juePcRJr_QsRgfksOOkEqHz11Hcs+kLrR=aw@mail.gmail.com>
Message-ID: <CAO7EwMq82dUpkLexS0c_u3_uF-j2akBUhqJcyTtE+JiGH+Rn-w@mail.gmail.com>

I've replaced the definition of Replicate with a closed type family, and
now isNull typechecks on 7.8.2:

type family Replicate (n :: Nat) t : [*] where
  Replicate 0 t = '[]
  Replicate n t = t ': Replicate (n-1) t
On Nov 21, 2014 3:52 PM, "Bas van Dijk" <v.dijk.bas at gmail.com> wrote:

Hi,

tl;dr
How do I implement a type-level Replicate function that takes a (n ::
Nat) and a type e and produces a type-level list of n e elements.

The rest of my email gives some background on why I need this, how
it's supposed to be used and my current erroneous implementation. I'm
presenting it as a Haskell program:

{-# LANGUAGE TypeOperators #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE UndecidableInstances #-}
{-# LANGUAGE GADTs #-}

import GHC.TypeLits
import Data.Type.Equality
import Data.Type.Bool

-- For work I'm implementing a binary protocol for
-- communicating with some device. I need to be able to send
-- a packet of orders to a device.
--
-- An order is a list of addresses and command to send to
-- these addresses.
--
-- After sending the packets of orders the device should
-- send back a packet of replies. The packet of replies
-- should have a reply for each command send to each
-- address.
--
-- I would like to make the latter requirement apparent in
-- the types so that when I send a packet of orders the type
-- will tell me what packet of replies to expect.
--
-- So I introduce the OrderPacket type which is
-- parameterised with a type-level lists of commands:
data OrderPacket (cmds :: [*]) where

    -- The empty packet:
    NoOrders :: OrderPacket '[]

    -- Extending an existing packet with an order.
    --
    -- The (n :: Nat) in Order denotes the number of
    -- addresses to send the command to. Note that because I
    -- expect n command replies back I need to create a
    -- type-level list of n commands and prepend that to the
    -- existing commands:
    (:>) :: !(Order n command)
         -> !(OrderPacket commands)
         -> OrderPacket (Replicate n command :++: commands)

-- As explained, an Order is a vector of addresses of length
-- n paired with a command:
data Order (n :: Nat) command where
    Order :: (Command command)
          => !(Addresses n)
          -> !command
          -> Order n command

-- In my real implementation this class describes how to
-- encode / decode commands:
class Command command

-- This is the typical Vector of length n type specialised
-- to Address:
data Addresses (n :: Nat) where
    Nil   :: Addresses 0
    (:*:) :: !Address -> !(Addresses n) -> Addresses (n+1)

infixr 5 :*:

type Address = Int

-- Append two type-level lists.
type family (rs1 :: [*]) :++: (rs2 :: [*]) :: [*]

type instance '[]        :++: rs2 = rs2
type instance (r ': rs1) :++: rs2 = r ': (rs1 :++: rs2)


-- I'm currently using the following Replicate type-level
-- function:
type family Replicate (n :: Nat) r :: [*]

type instance Replicate (n :: Nat) r =
    If (n == 0) '[] (r ': Replicate (n-1) r)

-- However when I write some code like the following:
isNull :: OrderPacket cmds -> Bool
isNull NoOrders                = True
isNull (orderMsg :> orderMsgs) = False

-- I get the following error:

src/TypeLevelReplicate.hs:49:9:
    Type function application stack overflow; size = 201
    Use -ftype-function-depth=N to increase stack size to N
      GHC.TypeLits.EqNat
        (((n-1)-1)-1)...) 0
      ~ (((n-1)-1)-1)...) == 0)
    In the pattern: orderMsg :> orderMsgs
    In an equation for ?isNull?:
      isNull (orderMsg :> orderMsgs) = True
Failed, modules loaded: none.

-- Note that when I adapt my code to use my own type-level
-- inductive implementation of natural numbers like:

data N = Z | S N

type family Replicate (n :: N) r :: [*]

type instance Replicate Z     r = '[]
type instance Replicate (S n) r = r ': Replicate n r

-- I don't get the type function application stack overflow.

Any hints on how to make this work with Nats from GHC.TypeLits?

Thanks,

Bas
_______________________________________________
Glasgow-haskell-users mailing list
Glasgow-haskell-users at haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141121/7ab157e5/attachment.html>

From a at lelf.me  Fri Nov 21 08:35:27 2014
From: a at lelf.me (Antonio Nikishaev)
Date: Fri, 21 Nov 2014 12:35:27 +0400
Subject: Type-level Replicate function
References: <CADLz5wWK0dE_3eW05Bc_XYGCyRjUhqDfr_Oz2shLw-swZsPHmQ@mail.gmail.com>
Message-ID: <m2k32p54ts.fsf@lelf.me>

Bas van Dijk <v.dijk.bas at gmail.com> writes:

> Hi,
>
> tl;dr
> How do I implement a type-level Replicate function that takes a (n ::
> Nat) and a type e and produces a type-level list of n e elements.
>
> The rest of my email gives some background on why I need this, how
> it's supposed to be used and my current erroneous implementation. I'm
> presenting it as a Haskell program:

[...]

> -- I'm currently using the following Replicate type-level
> -- function:
> type family Replicate (n :: Nat) r :: [*]
>
> type instance Replicate (n :: Nat) r =
>     If (n == 0) '[] (r ': Replicate (n-1) r)

type instance Replicate (n :: Nat) r =
    If (n == 0) '[] (r ': Replicate (If (n==0) 0 (n-1)) r)


> Any hints on how to make this work with Nats from GHC.TypeLits?



-- lelf



From v.dijk.bas at gmail.com  Fri Nov 21 08:51:53 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Fri, 21 Nov 2014 09:51:53 +0100
Subject: Type-level Replicate function
In-Reply-To: <CAO7EwMq82dUpkLexS0c_u3_uF-j2akBUhqJcyTtE+JiGH+Rn-w@mail.gmail.com>
References: <CADLz5wWK0dE_3eW05Bc_XYGCyRjUhqDfr_Oz2shLw-swZsPHmQ@mail.gmail.com>
 <CAO7EwMqNtj1ks5juePcRJr_QsRgfksOOkEqHz11Hcs+kLrR=aw@mail.gmail.com>
 <CAO7EwMq82dUpkLexS0c_u3_uF-j2akBUhqJcyTtE+JiGH+Rn-w@mail.gmail.com>
Message-ID: <CADLz5wWgCepro2f=MQy4a00o2yqq8uT7wvW-E1Jt8iBZC=OfJg@mail.gmail.com>

On 21 November 2014 09:21, Dr. ?RDI Gerg? <gergo at erdi.hu> wrote:
> I've replaced the definition of Replicate with a closed type family, and now
> isNull typechecks on 7.8.2:
>
> type family Replicate (n :: Nat) t : [*] where
>   Replicate 0 t = '[]
>   Replicate n t = t ': Replicate (n-1) t

Thanks that works beautifully!

From v.dijk.bas at gmail.com  Fri Nov 21 08:54:25 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Fri, 21 Nov 2014 09:54:25 +0100
Subject: Type-level Replicate function
In-Reply-To: <m2k32p54ts.fsf@lelf.me>
References: <CADLz5wWK0dE_3eW05Bc_XYGCyRjUhqDfr_Oz2shLw-swZsPHmQ@mail.gmail.com>
 <m2k32p54ts.fsf@lelf.me>
Message-ID: <CADLz5wVinEvcpqtquViDkDrNBLxtc5bQX7sEuccH+UzoaXXS6g@mail.gmail.com>

On 21 November 2014 09:35, Antonio Nikishaev <a at lelf.me> wrote:
> type instance Replicate (n :: Nat) r =
>     If (n == 0) '[] (r ': Replicate (If (n==0) 0 (n-1)) r)

Interesting!

From v.dijk.bas at gmail.com  Tue Nov 25 07:51:11 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Tue, 25 Nov 2014 08:51:11 +0100
Subject: Proving properties of type-level natural numbers obtained from user
 input
Message-ID: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>

Hi,

I have another type-level programming related question:

> {-# LANGUAGE GADTs #-}
> {-# LANGUAGE TypeOperators #-}
> {-# LANGUAGE ScopedTypeVariables #-}
> {-# LANGUAGE KindSignatures #-}
>
> import GHC.TypeLits

Say I have a Proxy p of some type-level natural number:

> p :: forall (n :: Nat). Proxy n
> p = Proxy

Imagine I get p from user input like this:

> main :: IO ()
> main = do
>     [arg] <- getArgs
>     let n = read arg :: Integer
>
>     case someNatVal n of
>       Nothing -> error "Input is not a natural number!"
>       Just (SomeNat (p :: Proxy n)) -> ...

I also have a function f which takes a proxy of a natural number but
it has the additional constraint that the number should be lesser than
or equal to 255:

> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
> f _ = ()

How do I apply f to p?

Obviously, applying it directly gives the type error:

> f p
<interactive>:179:1:
    Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
    The type variable ?n0? is ambiguous
    In the expression: f p
    In an equation for ?it?: it = f p

I imagine I somehow need to construct some Proof object using a function g like:

> g :: forall (n :: Nat). proxy n -> Proof
> g _ = ...

Where the Proof constructor encapsulates the (n <= 255) constraint:

> data Proof where
>     NoProof :: Proof
>     Proof :: forall (n :: Nat). (n <= 255)
>           => Proxy n -> Proof

With g in hand I can construct c which patterns matches on g p and
when there's a Proof the (n <= 255) constraint will be in scope which
allows applying f to p:

> c :: ()
> c = case g p of
>       NoProof -> error "Input is bigger than 255!"
>       Proof p -> f p

But how do I define g?

Cheers,

Bas

From lars at hupel.info  Tue Nov 25 09:36:53 2014
From: lars at hupel.info (Lars Hupel)
Date: Tue, 25 Nov 2014 10:36:53 +0100
Subject: Discovery of source dependencies without --make
Message-ID: <54744DB5.30306@hupel.info>

(cross-posted from [haskell-cafe])

Hi,

I've got a problem in our ? admittedly complex ? build process. We're
running an automated grading system for student-submitted homework
exercises. The compilation proceeds in stages:

1) Compile a library file, marked as trustworthy
2) Compile student submissions using Safe Haskell, able to access a
whitelisted set of packages and the previously compiled library files
3) Compile the test suite
4) Link everything together into an executable

The invocation is similar to this:

ghc -c -outputdir "$OUT"          -XTrustworthy Library.hs
ghc -c -outputdir "$OUT" -i"$OUT" -XSafe "$SUBMISSION"
ghc -c -outputdir "$OUT" -i"$OUT" Test_Suite.hs
ghc    -outputdir "$OUT" -i"$OUT" -o "$OUT/runner"

The second stage works well under the assumption that there's just one
single submitted file. If there's more than one, I need to specify them
in topological order wrt their module dependencies. Consider two simple
modules:

A.hs
> module A where
>
> import B
> import Library

B.hs
> module B where

Invoking GHC with "A.hs" "B.hs" will fail:

A.hs:3:1:
    Failed to load interface for ?B?

My naive attempt at solving that problem was to just insert "--make"
into the compiler flags. No luck, because according to [1, ?4.7.3],
"--make" will only look at source files and will ignore the "Library.hi"
file from the previous compilation stage. If I give the path to
"Library.hs" as well, GHC insists on recompiling it with changed flags
("-XSafe" instead of "-XTrustworthy").

Reading a bit further, I discovered the '-M' flag, however, not only
does it output Makefile-formatted output, it also ignores .hi files.

Is there any way to get the dependency discovery of '--make' without the
rest?

I know I could probably make it work if I made a package out of the
first build stage, but I really want to avoid that in order to not
increase the build complexity even further.

Cheers
Lars


[1]
<https://downloads.haskell.org/~ghc/7.6.3/docs/html/users_guide/separate-compilation.html#search-path>

From alexander.vershilov at gmail.com  Tue Nov 25 09:52:12 2014
From: alexander.vershilov at gmail.com (Alexander V Vershilov)
Date: Tue, 25 Nov 2014 13:52:12 +0400
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
Message-ID: <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>

Hi,

Following approach can work, the idea is to define a type that will
carry a proof (constraint) that we want to check. Here I have reused
Data.Tagged, but it's possible to introduce your own with concrete
constraints.

> {-# LANGUAGE DataKinds #-}
> {-# LANGUAGE GADTs #-}
> {-# LANGUAGE TypeOperators #-}
> {-# LANGUAGE KindSignatures #-}
> {-# LANGUAGE PolyKinds #-}
> {-# LANGUAGE UndecidableInstances #-}
> import GHC.TypeLits
> import GHC.Exts
> import Data.Proxy
> import Data.Tagged
> import System.Environment

New constraint carrying data type

> newtype Proof a b = Proof { unProof :: Tagged a b }

Runtime check for unknown naturals

> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
> fromSome (SomeNat p)
>    | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>    | otherwise = Nothing

Compiletime converter for known naturals

> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
> fromKnown n = Proof $ Tagged n

Function to test:

> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
> f2 _ = ()

Example of use:

> main :: IO ()
> main = do
>     [arg] <- getArgs
>     let n = read arg :: Integer
>
>     case someNatVal n of
>       Nothing -> error "Input is not a natural number!"
>       Just sn -> case fromSome sn of
>                    Just p -> return $ f2 p
>                    _ -> error "Input if larger than 255"


On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
> Hi,
>
> I have another type-level programming related question:
>
>> {-# LANGUAGE GADTs #-}
>> {-# LANGUAGE TypeOperators #-}
>> {-# LANGUAGE ScopedTypeVariables #-}
>> {-# LANGUAGE KindSignatures #-}
>>
>> import GHC.TypeLits
>
> Say I have a Proxy p of some type-level natural number:
>
>> p :: forall (n :: Nat). Proxy n
>> p = Proxy
>
> Imagine I get p from user input like this:
>
>> main :: IO ()
>> main = do
>>     [arg] <- getArgs
>>     let n = read arg :: Integer
>>
>>     case someNatVal n of
>>       Nothing -> error "Input is not a natural number!"
>>       Just (SomeNat (p :: Proxy n)) -> ...
>
> I also have a function f which takes a proxy of a natural number but
> it has the additional constraint that the number should be lesser than
> or equal to 255:
>
>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>> f _ = ()
>
> How do I apply f to p?
>
> Obviously, applying it directly gives the type error:
>
>> f p
> <interactive>:179:1:
>     Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>     The type variable ?n0? is ambiguous
>     In the expression: f p
>     In an equation for ?it?: it = f p
>
> I imagine I somehow need to construct some Proof object using a function g like:
>
>> g :: forall (n :: Nat). proxy n -> Proof
>> g _ = ...
>
> Where the Proof constructor encapsulates the (n <= 255) constraint:
>
>> data Proof where
>>     NoProof :: Proof
>>     Proof :: forall (n :: Nat). (n <= 255)
>>           => Proxy n -> Proof
>
> With g in hand I can construct c which patterns matches on g p and
> when there's a Proof the (n <= 255) constraint will be in scope which
> allows applying f to p:
>
>> c :: ()
>> c = case g p of
>>       NoProof -> error "Input is bigger than 255!"
>>       Proof p -> f p
>
> But how do I define g?
>
> Cheers,
>
> Bas
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users



-- 
Alexander

From mail at joachim-breitner.de  Tue Nov 25 10:29:14 2014
From: mail at joachim-breitner.de (Joachim Breitner)
Date: Tue, 25 Nov 2014 11:29:14 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <54744DB5.30306@hupel.info>
References: <54744DB5.30306@hupel.info>
Message-ID: <1416911354.1435.13.camel@joachim-breitner.de>

Dear Lars,


Am Dienstag, den 25.11.2014, 10:36 +0100 schrieb Lars Hupel:
> The invocation is similar to this:
> 
> ghc -c -outputdir "$OUT"          -XTrustworthy Library.hs
> ghc -c -outputdir "$OUT" -i"$OUT" -XSafe "$SUBMISSION"
> ghc -c -outputdir "$OUT" -i"$OUT" Test_Suite.hs
> ghc    -outputdir "$OUT" -i"$OUT" -o "$OUT/runner"

the only reason you do these in individual steps is that you need to
pass different flags, or are there other reasons?

Have you tried putting the pragma {-# LANGUAGE Safe #-} as the first
line into the submission file? I?m not sure how safe that actually is,
but at least


        {-# LANGUAGE Safe #-}
        {-# LANGUAGE Trustworthy #-}
        module Foo where
        
        import Unsafe.Coerce
        
is rejected by 7.6.3. This way, it could work with a single invocation
of --make.


Greetings,
Joachim

-- 
Joachim ?nomeata? Breitner
  mail at joachim-breitner.de ? http://www.joachim-breitner.de/
  Jabber: nomeata at joachim-breitner.de  ? GPG-Key: 0xF0FBF51F
  Debian Developer: nomeata at debian.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141125/f266faa8/attachment.sig>

From lars at hupel.info  Tue Nov 25 10:33:56 2014
From: lars at hupel.info (Lars Hupel)
Date: Tue, 25 Nov 2014 11:33:56 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <1416911354.1435.13.camel@joachim-breitner.de>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de>
Message-ID: <54745B14.4000705@hupel.info>

> the only reason you do these in individual steps is that you need to
> pass different flags, or are there other reasons?

Not only that ? we also want to be able to distinguish compilation
errors between "caused by students" and "we broke something". Under the
assumption that compilation for library files always succeeds, this is
not an issue for the first two stages.

> Have you tried putting the pragma {-# LANGUAGE Safe #-} as the first
> line into the submission file? I?m not sure how safe that actually is,
> but at least
> 
> 
>         {-# LANGUAGE Safe #-}
>         {-# LANGUAGE Trustworthy #-}
>         module Foo where
>         
>         import Unsafe.Coerce
>         
> is rejected by 7.6.3. This way, it could work with a single invocation
> of --make.

I'll let others comment on how safe that is since it does look a little
bit fragile. It could work, though.

Cheers
Lars

From eir at cis.upenn.edu  Tue Nov 25 18:34:38 2014
From: eir at cis.upenn.edu (Richard Eisenberg)
Date: Tue, 25 Nov 2014 13:34:38 -0500
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
Message-ID: <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>

Hi Bas,

I believe to do this "right", you would need singleton types. Then, when you discover that the number is bounded by 255, you would also discover that the type is bounded by 255, and you'd be home free.

Unfortunately, I there isn't currently a way to do comparison on GHC.TypeLits Nats with singletons. (There is a module Data.Singletons.TypeLits in the `singletons` package, but there's a comment telling me TODO in the part where comparison should be implemented.) If it were implemented, it would have to use unsafeCoerce, as there's no built-in mechanism connecting runtime numbers with TypeLits. 

If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.

The solution Alexander provides below doesn't quite build a proof, I think. Tellingly, if we omit the `natVal p <= 255` check, everything else still compiles. Thus, the `Proof` type he uses can be built even if the fact proven is false. That said, I don't know if my solution is any better, crucially relying on unsafeCoerce.

Richard

On Nov 25, 2014, at 4:52 AM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:

> Hi,
> 
> Following approach can work, the idea is to define a type that will
> carry a proof (constraint) that we want to check. Here I have reused
> Data.Tagged, but it's possible to introduce your own with concrete
> constraints.
> 
>> {-# LANGUAGE DataKinds #-}
>> {-# LANGUAGE GADTs #-}
>> {-# LANGUAGE TypeOperators #-}
>> {-# LANGUAGE KindSignatures #-}
>> {-# LANGUAGE PolyKinds #-}
>> {-# LANGUAGE UndecidableInstances #-}
>> import GHC.TypeLits
>> import GHC.Exts
>> import Data.Proxy
>> import Data.Tagged
>> import System.Environment
> 
> New constraint carrying data type
> 
>> newtype Proof a b = Proof { unProof :: Tagged a b }
> 
> Runtime check for unknown naturals
> 
>> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
>> fromSome (SomeNat p)
>>   | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>>   | otherwise = Nothing
> 
> Compiletime converter for known naturals
> 
>> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
>> fromKnown n = Proof $ Tagged n
> 
> Function to test:
> 
>> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
>> f2 _ = ()
> 
> Example of use:
> 
>> main :: IO ()
>> main = do
>>    [arg] <- getArgs
>>    let n = read arg :: Integer
>> 
>>    case someNatVal n of
>>      Nothing -> error "Input is not a natural number!"
>>      Just sn -> case fromSome sn of
>>                   Just p -> return $ f2 p
>>                   _ -> error "Input if larger than 255"
> 
> 
> On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>> Hi,
>> 
>> I have another type-level programming related question:
>> 
>>> {-# LANGUAGE GADTs #-}
>>> {-# LANGUAGE TypeOperators #-}
>>> {-# LANGUAGE ScopedTypeVariables #-}
>>> {-# LANGUAGE KindSignatures #-}
>>> 
>>> import GHC.TypeLits
>> 
>> Say I have a Proxy p of some type-level natural number:
>> 
>>> p :: forall (n :: Nat). Proxy n
>>> p = Proxy
>> 
>> Imagine I get p from user input like this:
>> 
>>> main :: IO ()
>>> main = do
>>>    [arg] <- getArgs
>>>    let n = read arg :: Integer
>>> 
>>>    case someNatVal n of
>>>      Nothing -> error "Input is not a natural number!"
>>>      Just (SomeNat (p :: Proxy n)) -> ...
>> 
>> I also have a function f which takes a proxy of a natural number but
>> it has the additional constraint that the number should be lesser than
>> or equal to 255:
>> 
>>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>>> f _ = ()
>> 
>> How do I apply f to p?
>> 
>> Obviously, applying it directly gives the type error:
>> 
>>> f p
>> <interactive>:179:1:
>>    Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>>    The type variable ?n0? is ambiguous
>>    In the expression: f p
>>    In an equation for ?it?: it = f p
>> 
>> I imagine I somehow need to construct some Proof object using a function g like:
>> 
>>> g :: forall (n :: Nat). proxy n -> Proof
>>> g _ = ...
>> 
>> Where the Proof constructor encapsulates the (n <= 255) constraint:
>> 
>>> data Proof where
>>>    NoProof :: Proof
>>>    Proof :: forall (n :: Nat). (n <= 255)
>>>          => Proxy n -> Proof
>> 
>> With g in hand I can construct c which patterns matches on g p and
>> when there's a Proof the (n <= 255) constraint will be in scope which
>> allows applying f to p:
>> 
>>> c :: ()
>>> c = case g p of
>>>      NoProof -> error "Input is bigger than 255!"
>>>      Proof p -> f p
>> 
>> But how do I define g?
>> 
>> Cheers,
>> 
>> Bas
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> Glasgow-haskell-users at haskell.org
>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 
> 
> 
> -- 
> Alexander
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
> 


From alexander.vershilov at gmail.com  Tue Nov 25 18:55:34 2014
From: alexander.vershilov at gmail.com (Alexander V Vershilov)
Date: Tue, 25 Nov 2014 22:55:34 +0400
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
 <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
Message-ID: <CAO-1Pb4bCWaSj1rBs1Qvn+WVpCC4AO8EniBfAa_fsnRbLEkwoA@mail.gmail.com>

Hi, Richard, Bas.

Maybe I didn't spell it properly but my point was to create a data
type that carry a proof
without exposing it's constructor and having clever constructor only,
then the only place
where you need to check will be that constructor.

Also it's possible to write in slightly clearer, but again it's
possible to make a mistake here
and it will be a false proof.

> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
> lessThen (SomeNat p) k
>    | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>    | otherwise = Nothing

Of cause solution using singletons could solve this problem much better.

--
Alexander

On 25 November 2014 at 21:34, Richard Eisenberg <eir at cis.upenn.edu> wrote:
> Hi Bas,
>
> I believe to do this "right", you would need singleton types. Then, when you discover that the number is bounded by 255, you would also discover that the type is bounded by 255, and you'd be home free.
>
> Unfortunately, I there isn't currently a way to do comparison on GHC.TypeLits Nats with singletons. (There is a module Data.Singletons.TypeLits in the `singletons` package, but there's a comment telling me TODO in the part where comparison should be implemented.) If it were implemented, it would have to use unsafeCoerce, as there's no built-in mechanism connecting runtime numbers with TypeLits.
>
> If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.
>
> The solution Alexander provides below doesn't quite build a proof, I think. Tellingly, if we omit the `natVal p <= 255` check, everything else still compiles. Thus, the `Proof` type he uses can be built even if the fact proven is false. That said, I don't know if my solution is any better, crucially relying on unsafeCoerce.
>
> Richard
>
> On Nov 25, 2014, at 4:52 AM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:
>
>> Hi,
>>
>> Following approach can work, the idea is to define a type that will
>> carry a proof (constraint) that we want to check. Here I have reused
>> Data.Tagged, but it's possible to introduce your own with concrete
>> constraints.
>>
>>> {-# LANGUAGE DataKinds #-}
>>> {-# LANGUAGE GADTs #-}
>>> {-# LANGUAGE TypeOperators #-}
>>> {-# LANGUAGE KindSignatures #-}
>>> {-# LANGUAGE PolyKinds #-}
>>> {-# LANGUAGE UndecidableInstances #-}
>>> import GHC.TypeLits
>>> import GHC.Exts
>>> import Data.Proxy
>>> import Data.Tagged
>>> import System.Environment
>>
>> New constraint carrying data type
>>
>>> newtype Proof a b = Proof { unProof :: Tagged a b }
>>
>> Runtime check for unknown naturals
>>
>>> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
>>> fromSome (SomeNat p)
>>>   | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>   | otherwise = Nothing
>>
>> Compiletime converter for known naturals
>>
>>> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
>>> fromKnown n = Proof $ Tagged n
>>
>> Function to test:
>>
>>> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
>>> f2 _ = ()
>>
>> Example of use:
>>
>>> main :: IO ()
>>> main = do
>>>    [arg] <- getArgs
>>>    let n = read arg :: Integer
>>>
>>>    case someNatVal n of
>>>      Nothing -> error "Input is not a natural number!"
>>>      Just sn -> case fromSome sn of
>>>                   Just p -> return $ f2 p
>>>                   _ -> error "Input if larger than 255"
>>
>>
>> On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>>> Hi,
>>>
>>> I have another type-level programming related question:
>>>
>>>> {-# LANGUAGE GADTs #-}
>>>> {-# LANGUAGE TypeOperators #-}
>>>> {-# LANGUAGE ScopedTypeVariables #-}
>>>> {-# LANGUAGE KindSignatures #-}
>>>>
>>>> import GHC.TypeLits
>>>
>>> Say I have a Proxy p of some type-level natural number:
>>>
>>>> p :: forall (n :: Nat). Proxy n
>>>> p = Proxy
>>>
>>> Imagine I get p from user input like this:
>>>
>>>> main :: IO ()
>>>> main = do
>>>>    [arg] <- getArgs
>>>>    let n = read arg :: Integer
>>>>
>>>>    case someNatVal n of
>>>>      Nothing -> error "Input is not a natural number!"
>>>>      Just (SomeNat (p :: Proxy n)) -> ...
>>>
>>> I also have a function f which takes a proxy of a natural number but
>>> it has the additional constraint that the number should be lesser than
>>> or equal to 255:
>>>
>>>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>>>> f _ = ()
>>>
>>> How do I apply f to p?
>>>
>>> Obviously, applying it directly gives the type error:
>>>
>>>> f p
>>> <interactive>:179:1:
>>>    Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>>>    The type variable ?n0? is ambiguous
>>>    In the expression: f p
>>>    In an equation for ?it?: it = f p
>>>
>>> I imagine I somehow need to construct some Proof object using a function g like:
>>>
>>>> g :: forall (n :: Nat). proxy n -> Proof
>>>> g _ = ...
>>>
>>> Where the Proof constructor encapsulates the (n <= 255) constraint:
>>>
>>>> data Proof where
>>>>    NoProof :: Proof
>>>>    Proof :: forall (n :: Nat). (n <= 255)
>>>>          => Proxy n -> Proof
>>>
>>> With g in hand I can construct c which patterns matches on g p and
>>> when there's a Proof the (n <= 255) constraint will be in scope which
>>> allows applying f to p:
>>>
>>>> c :: ()
>>>> c = case g p of
>>>>      NoProof -> error "Input is bigger than 255!"
>>>>      Proof p -> f p
>>>
>>> But how do I define g?
>>>
>>> Cheers,
>>>
>>> Bas
>>> _______________________________________________
>>> Glasgow-haskell-users mailing list
>>> Glasgow-haskell-users at haskell.org
>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>
>>
>>
>> --
>> Alexander
>> _______________________________________________
>> Glasgow-haskell-users mailing list
>> Glasgow-haskell-users at haskell.org
>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>
>



-- 
Alexander

From austin at well-typed.com  Tue Nov 25 19:46:40 2014
From: austin at well-typed.com (Austin Seipp)
Date: Tue, 25 Nov 2014 13:46:40 -0600
Subject: ANNOUNCE: GHC 7.8.4 Release Candidate 1
Message-ID: <CAEs96n0xY4p+fknvPL_y+6S5=XukbWmPs+VyTnam4+qH+ESYtQ@mail.gmail.com>

We are pleased to announce the first release candidate for GHC 7.8.4:

    https://downloads.haskell.org/~ghc/7.8.4-rc1/

This includes the source tarball and bindists for 64bit Linux. Binary
builds for other platforms will be available shortly. These binaries
and tarballs have an accompanying SHA256SUMS file signed by my GPG key
id (0x3B58D86F).

We plan to make the 7.8.4 release sometime in the next few weeks.

Please test as much as possible; bugs are much cheaper if we find them
before the release!

-- 
Regards,

Austin Seipp, Haskell Consultant
Well-Typed LLP, http://www.well-typed.com/

From brandon.m.simmons at gmail.com  Tue Nov 25 20:24:53 2014
From: brandon.m.simmons at gmail.com (Brandon Simmons)
Date: Tue, 25 Nov 2014 15:24:53 -0500
Subject: Defining a custom newByteArray primop that uses calloc?
Message-ID: <CAAyGGynCeWe5M_cc=6S+KaNrMOMtP0667s-ZkhAfoCK8zokjBA@mail.gmail.com>

In my tests, using calloc from:

    https://hackage.haskell.org/package/missing-foreign-0.1.1/docs/Foreign-Marshal-MissingAlloc.html

was about twice as fast as allocating and zeroing the same amount of
memory with `newByteArray` + any of `copy/set/fillMutableByteArray`
(all three were nearly identical). Is there a way I can reasonably
define my own `newByteArray` that uses calloc?

FWIW here are a couple of the benchmarks I'm working with in criterion:

    arrTestSet :: Int -> IO ()
    arrTestSet len = do
        let eBytes = (P.sizeOf (undefined::Int))*len
        a <- P.newAlignedPinnedByteArray
                    eBytes
                    (P.alignment (undefined :: Int))
        void $ P.setByteArray a 0 len (1::Int)

    arrTestCallocAndWrite :: Int -> IO ()
    arrTestCallocAndWrite len = do
        ptr <- callocBytes (len*(P.sizeOf(undefined::Int))) :: IO (Ptr Int)
        pokeElemOff ptr 0 1
        free ptr

Thanks,
Brandon

From v.dijk.bas at gmail.com  Wed Nov 26 07:15:46 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Wed, 26 Nov 2014 08:15:46 +0100
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <CAO-1Pb4bCWaSj1rBs1Qvn+WVpCC4AO8EniBfAa_fsnRbLEkwoA@mail.gmail.com>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
 <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
 <CAO-1Pb4bCWaSj1rBs1Qvn+WVpCC4AO8EniBfAa_fsnRbLEkwoA@mail.gmail.com>
Message-ID: <CADLz5wXQf-qy50M_CAAnK1EBrXOC9KnHwigaeG4ud4_bC3NJ9A@mail.gmail.com>

Hi Alexander,

Thanks for your answer! This provides a lot of ideas how to proceed.

I'm unsure about the following though:

> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
> lessThen (SomeNat p) k
>    | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>    | otherwise = Nothing

Doesn't this mean lessThen returns a Proxy n for all (n :: Nat) and
not just the Nat inside the SomeNat?

I also see that p is only used for comparing it to k. It's not used to
produce the return value.

Cheers,

Bas

On 25 November 2014 at 19:55, Alexander V Vershilov
<alexander.vershilov at gmail.com> wrote:
> Hi, Richard, Bas.
>
> Maybe I didn't spell it properly but my point was to create a data
> type that carry a proof
> without exposing it's constructor and having clever constructor only,
> then the only place
> where you need to check will be that constructor.
>
> Also it's possible to write in slightly clearer, but again it's
> possible to make a mistake here
> and it will be a false proof.
>
>> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
>> lessThen (SomeNat p) k
>>    | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>>    | otherwise = Nothing
>
> Of cause solution using singletons could solve this problem much better.
>
> --
> Alexander
>
> On 25 November 2014 at 21:34, Richard Eisenberg <eir at cis.upenn.edu> wrote:
>> Hi Bas,
>>
>> I believe to do this "right", you would need singleton types. Then, when you discover that the number is bounded by 255, you would also discover that the type is bounded by 255, and you'd be home free.
>>
>> Unfortunately, I there isn't currently a way to do comparison on GHC.TypeLits Nats with singletons. (There is a module Data.Singletons.TypeLits in the `singletons` package, but there's a comment telling me TODO in the part where comparison should be implemented.) If it were implemented, it would have to use unsafeCoerce, as there's no built-in mechanism connecting runtime numbers with TypeLits.
>>
>> If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.
>>
>> The solution Alexander provides below doesn't quite build a proof, I think. Tellingly, if we omit the `natVal p <= 255` check, everything else still compiles. Thus, the `Proof` type he uses can be built even if the fact proven is false. That said, I don't know if my solution is any better, crucially relying on unsafeCoerce.
>>
>> Richard
>>
>> On Nov 25, 2014, at 4:52 AM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Following approach can work, the idea is to define a type that will
>>> carry a proof (constraint) that we want to check. Here I have reused
>>> Data.Tagged, but it's possible to introduce your own with concrete
>>> constraints.
>>>
>>>> {-# LANGUAGE DataKinds #-}
>>>> {-# LANGUAGE GADTs #-}
>>>> {-# LANGUAGE TypeOperators #-}
>>>> {-# LANGUAGE KindSignatures #-}
>>>> {-# LANGUAGE PolyKinds #-}
>>>> {-# LANGUAGE UndecidableInstances #-}
>>>> import GHC.TypeLits
>>>> import GHC.Exts
>>>> import Data.Proxy
>>>> import Data.Tagged
>>>> import System.Environment
>>>
>>> New constraint carrying data type
>>>
>>>> newtype Proof a b = Proof { unProof :: Tagged a b }
>>>
>>> Runtime check for unknown naturals
>>>
>>>> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
>>>> fromSome (SomeNat p)
>>>>   | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>>   | otherwise = Nothing
>>>
>>> Compiletime converter for known naturals
>>>
>>>> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
>>>> fromKnown n = Proof $ Tagged n
>>>
>>> Function to test:
>>>
>>>> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
>>>> f2 _ = ()
>>>
>>> Example of use:
>>>
>>>> main :: IO ()
>>>> main = do
>>>>    [arg] <- getArgs
>>>>    let n = read arg :: Integer
>>>>
>>>>    case someNatVal n of
>>>>      Nothing -> error "Input is not a natural number!"
>>>>      Just sn -> case fromSome sn of
>>>>                   Just p -> return $ f2 p
>>>>                   _ -> error "Input if larger than 255"
>>>
>>>
>>> On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I have another type-level programming related question:
>>>>
>>>>> {-# LANGUAGE GADTs #-}
>>>>> {-# LANGUAGE TypeOperators #-}
>>>>> {-# LANGUAGE ScopedTypeVariables #-}
>>>>> {-# LANGUAGE KindSignatures #-}
>>>>>
>>>>> import GHC.TypeLits
>>>>
>>>> Say I have a Proxy p of some type-level natural number:
>>>>
>>>>> p :: forall (n :: Nat). Proxy n
>>>>> p = Proxy
>>>>
>>>> Imagine I get p from user input like this:
>>>>
>>>>> main :: IO ()
>>>>> main = do
>>>>>    [arg] <- getArgs
>>>>>    let n = read arg :: Integer
>>>>>
>>>>>    case someNatVal n of
>>>>>      Nothing -> error "Input is not a natural number!"
>>>>>      Just (SomeNat (p :: Proxy n)) -> ...
>>>>
>>>> I also have a function f which takes a proxy of a natural number but
>>>> it has the additional constraint that the number should be lesser than
>>>> or equal to 255:
>>>>
>>>>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>>>>> f _ = ()
>>>>
>>>> How do I apply f to p?
>>>>
>>>> Obviously, applying it directly gives the type error:
>>>>
>>>>> f p
>>>> <interactive>:179:1:
>>>>    Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>>>>    The type variable ?n0? is ambiguous
>>>>    In the expression: f p
>>>>    In an equation for ?it?: it = f p
>>>>
>>>> I imagine I somehow need to construct some Proof object using a function g like:
>>>>
>>>>> g :: forall (n :: Nat). proxy n -> Proof
>>>>> g _ = ...
>>>>
>>>> Where the Proof constructor encapsulates the (n <= 255) constraint:
>>>>
>>>>> data Proof where
>>>>>    NoProof :: Proof
>>>>>    Proof :: forall (n :: Nat). (n <= 255)
>>>>>          => Proxy n -> Proof
>>>>
>>>> With g in hand I can construct c which patterns matches on g p and
>>>> when there's a Proof the (n <= 255) constraint will be in scope which
>>>> allows applying f to p:
>>>>
>>>>> c :: ()
>>>>> c = case g p of
>>>>>      NoProof -> error "Input is bigger than 255!"
>>>>>      Proof p -> f p
>>>>
>>>> But how do I define g?
>>>>
>>>> Cheers,
>>>>
>>>> Bas
>>>> _______________________________________________
>>>> Glasgow-haskell-users mailing list
>>>> Glasgow-haskell-users at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>
>>>
>>>
>>> --
>>> Alexander
>>> _______________________________________________
>>> Glasgow-haskell-users mailing list
>>> Glasgow-haskell-users at haskell.org
>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>
>>
>
>
>
> --
> Alexander

From v.dijk.bas at gmail.com  Wed Nov 26 07:19:13 2014
From: v.dijk.bas at gmail.com (Bas van Dijk)
Date: Wed, 26 Nov 2014 08:19:13 +0100
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
 <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
Message-ID: <CADLz5wWQ9V3bg3qQYHwg71GRPPxVwSTpuwnwv9TTFk90uoBrxA@mail.gmail.com>

On 25 November 2014 at 19:34, Richard Eisenberg <eir at cis.upenn.edu> wrote:
> If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.

Thanks, I hadn't considered this yet.

Cheers,

Bas

From michael at snoyman.com  Wed Nov 26 08:54:07 2014
From: michael at snoyman.com (Michael Snoyman)
Date: Wed, 26 Nov 2014 08:54:07 +0000
Subject: ANNOUNCE: GHC 7.8.4 Release Candidate 1
References: <CAEs96n0xY4p+fknvPL_y+6S5=XukbWmPs+VyTnam4+qH+ESYtQ@mail.gmail.com>
Message-ID: <CAKA2JgJhJoDXKZ7k_0qZx3KG74nfgZk7z+qDp17iWFnyJw7-ZQ@mail.gmail.com>

On Tue Nov 25 2014 at 9:46:46 PM Austin Seipp <austin at well-typed.com> wrote:

> We are pleased to announce the first release candidate for GHC 7.8.4:
>
>     https://downloads.haskell.org/~ghc/7.8.4-rc1/
>
> This includes the source tarball and bindists for 64bit Linux. Binary
> builds for other platforms will be available shortly. These binaries
> and tarballs have an accompanying SHA256SUMS file signed by my GPG key
> id (0x3B58D86F).
>
> We plan to make the 7.8.4 release sometime in the next few weeks.
>
> Please test as much as possible; bugs are much cheaper if we find them
> before the release!
>
>
>
Just wanted to report: I ran a full Stackage build + test suite using the
release candidate, and everything passed. This build consisted of ~750
packages. You can find the list of packages at:

http://lpaste.net/115090

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141126/fd219587/attachment.html>

From svenpanne at gmail.com  Wed Nov 26 11:40:37 2014
From: svenpanne at gmail.com (Sven Panne)
Date: Wed, 26 Nov 2014 12:40:37 +0100
Subject: ANNOUNCE: GHC 7.8.4 Release Candidate 1
In-Reply-To: <CAEs96n0xY4p+fknvPL_y+6S5=XukbWmPs+VyTnam4+qH+ESYtQ@mail.gmail.com>
References: <CAEs96n0xY4p+fknvPL_y+6S5=XukbWmPs+VyTnam4+qH+ESYtQ@mail.gmail.com>
Message-ID: <CANBN=mtuKsEPbiH_7VMBvLJNFnZpYDVLt7jC7GV=NHPHr-nDuw@mail.gmail.com>

2014-11-25 20:46 GMT+01:00 Austin Seipp <austin at well-typed.com>:
> We are pleased to announce the first release candidate for GHC 7.8.4:
>
>     https://downloads.haskell.org/~ghc/7.8.4-rc1/ [...]

Would it be possible to get the RC on
https://launchpad.net/~hvr/+archive/ubuntu/ghc? This way one could
easily test things on Travis CI.

From hvr at gnu.org  Wed Nov 26 14:31:15 2014
From: hvr at gnu.org (Herbert Valerio Riedel)
Date: Wed, 26 Nov 2014 15:31:15 +0100
Subject: ANNOUNCE: GHC 7.8.4 Release Candidate 1
In-Reply-To: <CANBN=mtuKsEPbiH_7VMBvLJNFnZpYDVLt7jC7GV=NHPHr-nDuw@mail.gmail.com>
 (Sven Panne's message of "Wed, 26 Nov 2014 12:40:37 +0100")
References: <CAEs96n0xY4p+fknvPL_y+6S5=XukbWmPs+VyTnam4+qH+ESYtQ@mail.gmail.com>
 <CANBN=mtuKsEPbiH_7VMBvLJNFnZpYDVLt7jC7GV=NHPHr-nDuw@mail.gmail.com>
Message-ID: <87bnnu3ufg.fsf@gnu.org>

On 2014-11-26 at 12:40:37 +0100, Sven Panne wrote:
> 2014-11-25 20:46 GMT+01:00 Austin Seipp <austin at well-typed.com>:
>> We are pleased to announce the first release candidate for GHC 7.8.4:
>>
>>     https://downloads.haskell.org/~ghc/7.8.4-rc1/ [...]
>
> Would it be possible to get the RC on
> https://launchpad.net/~hvr/+archive/ubuntu/ghc? This way one could
> easily test things on Travis CI.

I'll put a 7.8.4rc .deb up soon (probably right after the GHC 7.10
branch has been created)

From mle+hs at mega-nerd.com  Thu Nov 27 01:55:29 2014
From: mle+hs at mega-nerd.com (Erik de Castro Lopo)
Date: Wed, 26 Nov 2014 17:55:29 -0800
Subject: Defining a custom newByteArray primop that uses calloc?
In-Reply-To: <CAAyGGynCeWe5M_cc=6S+KaNrMOMtP0667s-ZkhAfoCK8zokjBA@mail.gmail.com>
References: <CAAyGGynCeWe5M_cc=6S+KaNrMOMtP0667s-ZkhAfoCK8zokjBA@mail.gmail.com>
Message-ID: <20141126175529.f86d7a41a5eaa83696ff430b@mega-nerd.com>

Brandon Simmons wrote:

> In my tests, using calloc from:
> 
>     https://hackage.haskell.org/package/missing-foreign-0.1.1/docs/Foreign-Marshal-MissingAlloc.html
> 
> was about twice as fast as allocating and zeroing the same amount of
> memory with `newByteArray` + any of `copy/set/fillMutableByteArray`
> (all three were nearly identical). Is there a way I can reasonably
> define my own `newByteArray` that uses calloc?
> 
> FWIW here are a couple of the benchmarks I'm working with in criterion:
> 
>     arrTestSet :: Int -> IO ()
>     arrTestSet len = do
>         let eBytes = (P.sizeOf (undefined::Int))*len
>         a <- P.newAlignedPinnedByteArray

Why pinned memory?

Erik
-- 
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/

From carter.schonwald at gmail.com  Thu Nov 27 04:22:21 2014
From: carter.schonwald at gmail.com (Carter Schonwald)
Date: Wed, 26 Nov 2014 23:22:21 -0500
Subject: Defining a custom newByteArray primop that uses calloc?
In-Reply-To: <20141126175529.f86d7a41a5eaa83696ff430b@mega-nerd.com>
References: <CAAyGGynCeWe5M_cc=6S+KaNrMOMtP0667s-ZkhAfoCK8zokjBA@mail.gmail.com>
 <20141126175529.f86d7a41a5eaa83696ff430b@mega-nerd.com>
Message-ID: <CAHYVw0w9KZ32eqN103D6SKWt_cqX6RexudGDokZCoppVRCp4Cg@mail.gmail.com>

you cant provide > 1 word alignment guarantees if the bytearray isnt
pinned, ... at least with the current RTS setup. (though that would be
pretty handy if possible)

On Wed, Nov 26, 2014 at 8:55 PM, Erik de Castro Lopo <mle+hs at mega-nerd.com>
wrote:

> Brandon Simmons wrote:
>
> > In my tests, using calloc from:
> >
> >
> https://hackage.haskell.org/package/missing-foreign-0.1.1/docs/Foreign-Marshal-MissingAlloc.html
> >
> > was about twice as fast as allocating and zeroing the same amount of
> > memory with `newByteArray` + any of `copy/set/fillMutableByteArray`
> > (all three were nearly identical). Is there a way I can reasonably
> > define my own `newByteArray` that uses calloc?
> >
> > FWIW here are a couple of the benchmarks I'm working with in criterion:
> >
> >     arrTestSet :: Int -> IO ()
> >     arrTestSet len = do
> >         let eBytes = (P.sizeOf (undefined::Int))*len
> >         a <- P.newAlignedPinnedByteArray
>
> Why pinned memory?
>
> Erik
> --
> ----------------------------------------------------------------------
> Erik de Castro Lopo
> http://www.mega-nerd.com/
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141126/52be0a49/attachment.html>

From roma at ro-che.info  Thu Nov 27 10:16:35 2014
From: roma at ro-che.info (Roman Cheplyaka)
Date: Thu, 27 Nov 2014 12:16:35 +0200
Subject: Discovery of source dependencies without --make
In-Reply-To: <1416911354.1435.13.camel@joachim-breitner.de>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de>
Message-ID: <5476FA03.1070303@ro-che.info>

On 25/11/14 12:29, Joachim Breitner wrote:
> Dear Lars,
> 
> 
> Am Dienstag, den 25.11.2014, 10:36 +0100 schrieb Lars Hupel:
>> The invocation is similar to this:
>>
>> ghc -c -outputdir "$OUT"          -XTrustworthy Library.hs
>> ghc -c -outputdir "$OUT" -i"$OUT" -XSafe "$SUBMISSION"
>> ghc -c -outputdir "$OUT" -i"$OUT" Test_Suite.hs
>> ghc    -outputdir "$OUT" -i"$OUT" -o "$OUT/runner"
> 
> the only reason you do these in individual steps is that you need to
> pass different flags, or are there other reasons?
> 
> Have you tried putting the pragma {-# LANGUAGE Safe #-} as the first
> line into the submission file? I?m not sure how safe that actually is,
> but at least
> 
> 
>         {-# LANGUAGE Safe #-}
>         {-# LANGUAGE Trustworthy #-}
>         module Foo where
>         
>         import Unsafe.Coerce
>         
> is rejected by 7.6.3. This way, it could work with a single invocation
> of --make.

The only problem I see with that is that error message locations will be
a bit off, since the file being compiled is different from the file
submitted. But since we're in the hacks territory anyway, this could be
fixed up with a simple regex :-)

Roman


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141127/c356b030/attachment.sig>

From lars at hupel.info  Thu Nov 27 10:31:55 2014
From: lars at hupel.info (Lars Hupel)
Date: Thu, 27 Nov 2014 11:31:55 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <5476FA03.1070303@ro-che.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
Message-ID: <5476FD9B.6010407@hupel.info>

> The only problem I see with that is that error message locations will be
> a bit off, since the file being compiled is different from the file
> submitted. But since we're in the hacks territory anyway, this could be
> fixed up with a simple regex :-)

... or line pragmas :-)

I'm currently investigating another route though. I wrote a simple
program which parses some Haskell files with "haskell-src-exts" (we
actually even need "hse-cpp"), builds a graph with all known
dependencies, uses "topSort" from "Data.Graph" to come up with a proper
ordering and prints the source files in that order. That approach feels
a lot safer to me. It can be used like:

  ghc -c $(cabal exec topoSort "${files[@]}")

(Never mind the obvious unsafe file name handling.)

The downside is that parse errors won't be reported by GHC, but by our
preprocessing tool.

(I've attached the tool, but keep in mind it's just a quick proof of
concept.)

Cheers
Lars
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HsTopoSort.hs
Type: text/x-haskell
Size: 981 bytes
Desc: not available
URL: <http://www.haskell.org/pipermail/glasgow-haskell-users/attachments/20141127/2872d1ea/attachment.hs>

From gershomb at gmail.com  Thu Nov 27 17:59:49 2014
From: gershomb at gmail.com (Gershom B)
Date: Thu, 27 Nov 2014 12:59:49 -0500
Subject: Discovery of source dependencies without --make
In-Reply-To: <5476FD9B.6010407@hupel.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de>
 <5476FA03.1070303@ro-che.info> <5476FD9B.6010407@hupel.info>
Message-ID: <etPan.547766a4.5bdd194b.5d3c@mbp.local>

Is -M perhaps what you?ve been looking for?

https://downloads.haskell.org/~ghc/7.8.3/docs/html/users_guide/separate-compilation.html#makefile-dependencies

-g


On November 27, 2014 at 5:32:01 AM, Lars Hupel (lars at hupel.info) wrote:
> > The only problem I see with that is that error message locations will be
> > a bit off, since the file being compiled is different from the file
> > submitted. But since we're in the hacks territory anyway, this could be
> > fixed up with a simple regex :-)
>  
> ... or line pragmas :-)
>  
> I'm currently investigating another route though. I wrote a simple
> program which parses some Haskell files with "haskell-src-exts" (we
> actually even need "hse-cpp"), builds a graph with all known
> dependencies, uses "topSort" from "Data.Graph" to come up with a proper
> ordering and prints the source files in that order. That approach feels
> a lot safer to me. It can be used like:
>  
> ghc -c $(cabal exec topoSort "${files[@]}")
>  
> (Never mind the obvious unsafe file name handling.)
>  
> The downside is that parse errors won't be reported by GHC, but by our
> preprocessing tool.
>  
> (I've attached the tool, but keep in mind it's just a quick proof of
> concept.)
>  
> Cheers
> Lars
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>  


From lars at hupel.info  Fri Nov 28 08:52:32 2014
From: lars at hupel.info (Lars Hupel)
Date: Fri, 28 Nov 2014 09:52:32 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <etPan.547766a4.5bdd194b.5d3c@mbp.local>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
Message-ID: <547837D0.6020205@hupel.info>

Hi Gershom,

> Is -M perhaps what you?ve been looking for?

sadly, no. Firstly, it behaves in the same way as "--make" (i.e. only
looks at source files) and secondly, it produces a Makefile as output.

(I'd be happy though to use the GHC API if somebody could tell me
whether/where this functionality is exposed.)

Cheers
Lars


From simonpj at microsoft.com  Fri Nov 28 09:01:13 2014
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 28 Nov 2014 09:01:13 +0000
Subject: Discovery of source dependencies without --make
In-Reply-To: <547837D0.6020205@hupel.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
Message-ID: <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>

I have only been skimming this thread, but would it be worth writing a tight specification of what exactly you want?  Your original message said only "Is there any way to get the dependency discovery of '--make' without the rest", but I really don't know what that means.  Nor do I know what you mean by "it only looks at source files".  

Rather than explain by deltas from something else, it might be easier just to write down precisely what you seek.

Andrey Mokhov also wants a way to take a single Haskell module, and discover its immediate imports, a kind of non-recursive version of -M.  So 
	ghc -M-one-shot Foo.hs
would print out the list of Haskell modules that Foo imports.  (There are doubtless complications to do with CPP too, but that's the general idea.)

I don't know if that is what you want too.

Simon

|  -----Original Message-----
|  From: Glasgow-haskell-users [mailto:glasgow-haskell-users-
|  bounces at haskell.org] On Behalf Of Lars Hupel
|  Sent: 28 November 2014 08:53
|  To: Gershom B
|  Cc: glasgow-haskell-users at haskell.org
|  Subject: Re: Discovery of source dependencies without --make
|  
|  Hi Gershom,
|  
|  > Is -M perhaps what you?ve been looking for?
|  
|  sadly, no. Firstly, it behaves in the same way as "--make" (i.e. only
|  looks at source files) and secondly, it produces a Makefile as output.
|  
|  (I'd be happy though to use the GHC API if somebody could tell me
|  whether/where this functionality is exposed.)
|  
|  Cheers
|  Lars
|  
|  _______________________________________________
|  Glasgow-haskell-users mailing list
|  Glasgow-haskell-users at haskell.org
|  http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

From qdunkan at gmail.com  Fri Nov 28 09:09:57 2014
From: qdunkan at gmail.com (Evan Laforge)
Date: Fri, 28 Nov 2014 01:09:57 -0800
Subject: Discovery of source dependencies without --make
In-Reply-To: <547837D0.6020205@hupel.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de>
 <5476FA03.1070303@ro-che.info> <5476FD9B.6010407@hupel.info>
 <etPan.547766a4.5bdd194b.5d3c@mbp.local> <547837D0.6020205@hupel.info>
Message-ID: <CACbaDy6TB36hb7fv-yY-xFqd4RfErZZUFoT3a2M36vfbtjcSJg@mail.gmail.com>

Here's what I use:

http://ofb.net/~elaforge/shake/Shake/HsDeps.hs

It's a very dumb but fast parser that figures out dependencies.  I use
it for my shake build system.  Not sure if it's useful, but there it
is if it helps.

If you have a complicated build setup and you're not using shake,
maybe you should consider it :)

On Fri, Nov 28, 2014 at 12:52 AM, Lars Hupel <lars at hupel.info> wrote:
> Hi Gershom,
>
>> Is -M perhaps what you?ve been looking for?
>
> sadly, no. Firstly, it behaves in the same way as "--make" (i.e. only
> looks at source files) and secondly, it produces a Makefile as output.
>
> (I'd be happy though to use the GHC API if somebody could tell me
> whether/where this functionality is exposed.)
>
> Cheers
> Lars
>
> _______________________________________________
> Glasgow-haskell-users mailing list
> Glasgow-haskell-users at haskell.org
> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

From lars at hupel.info  Fri Nov 28 14:41:44 2014
From: lars at hupel.info (Lars Hupel)
Date: Fri, 28 Nov 2014 15:41:44 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <547889A8.9030803@hupel.info>

> Rather than explain by deltas from something else, it might be
> easier just to write down precisely what you seek.

Let's say the hypothetical feature is selected via the GHC flag
"--topo-sort". It would add a step before regular compilation and
wouldn't affect any other flag:

  ghc -c --topo-sort fileA.hs fileB.hs ...

This would first read in the specified source files and look at their
module headers and import statements. It would build a graph of module
dependencies _between_ the specified source files (ignoring circular
dependencies), perform a topological sort on that graph, and proceed
with compiling the source files in that order.

As a consequence, if there is an order in which these modules can be
successfully compiled, "--topo-sort" would choose such an order. In that
sense, the above invocation would be equivalent to

  ghc -c fileB.hs fileA.hs ...

(using some permutation of the original order specified)

Another consequence is that any invocation of GHC in the form of

  ghc -c flags... sources.hs...

with arbitrary flags would still work as usual when adding "--topo-sort".

Quoting from the user manual:

> In your program, you import a module Foo by saying import Foo. In
> --make mode or GHCi, GHC will look for a source file for Foo and
> arrange to compile it first. Without --make, GHC will look for the
> interface file for Foo, which should have been created by an earlier
> compilation of Foo.

The hypothetical "--topo-sort" flag would behave in the latter way, i.e.
it would not require a source file for an unknown dependency. Hence,
"--topo-sort" and "--make" would be conflicting options.

I hope that clears up things a bit.

Cheers
Lars

From simonpj at microsoft.com  Fri Nov 28 16:08:32 2014
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 28 Nov 2014 16:08:32 +0000
Subject: Discovery of source dependencies without --make
In-Reply-To: <547889A8.9030803@hupel.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
 <547889A8.9030803@hupel.info>
Message-ID: <618BE556AADD624C9C918AA5D5911BEF3F3EDBE8@DB3PRD3001MB020.064d.mgd.msft.net>

|  Let's say the hypothetical feature is selected via the GHC flag "--
|  topo-sort". It would add a step before regular compilation and
|  wouldn't affect any other flag:
|  
|    ghc -c --topo-sort fileA.hs fileB.hs ...
|  
|  This would first read in the specified source files and look at their
|  module headers and import statements. It would build a graph of module
|  dependencies _between_ the specified source files (ignoring circular
|  dependencies), perform a topological sort on that graph, and proceed
|  with compiling the source files in that order.

Interesting (and quite different from what I anticipated, so it's a good thing you wrote it down!).

How does that differ from ghc --make?  The only difference I can see is that
  - Modules that --make might find, but not listed on
    the command line, would not be compiled by --topo-sort
 
Simon

|  -----Original Message-----
|  From: Lars Hupel [mailto:lars at hupel.info]
|  Sent: 28 November 2014 14:42
|  To: Simon Peyton Jones
|  Cc: glasgow-haskell-users at haskell.org; Andrey Mokhov
|  Subject: Re: Discovery of source dependencies without --make
|  
|  > Rather than explain by deltas from something else, it might be
|  easier
|  > just to write down precisely what you seek.
|  
|  Let's say the hypothetical feature is selected via the GHC flag "--
|  topo-sort". It would add a step before regular compilation and
|  wouldn't affect any other flag:
|  
|    ghc -c --topo-sort fileA.hs fileB.hs ...
|  
|  This would first read in the specified source files and look at their
|  module headers and import statements. It would build a graph of module
|  dependencies _between_ the specified source files (ignoring circular
|  dependencies), perform a topological sort on that graph, and proceed
|  with compiling the source files in that order.
|  
|  As a consequence, if there is an order in which these modules can be
|  successfully compiled, "--topo-sort" would choose such an order. In
|  that sense, the above invocation would be equivalent to
|  
|    ghc -c fileB.hs fileA.hs ...
|  
|  (using some permutation of the original order specified)
|  
|  Another consequence is that any invocation of GHC in the form of
|  
|    ghc -c flags... sources.hs...
|  
|  with arbitrary flags would still work as usual when adding "--topo-
|  sort".
|  
|  Quoting from the user manual:
|  
|  > In your program, you import a module Foo by saying import Foo. In
|  > --make mode or GHCi, GHC will look for a source file for Foo and
|  > arrange to compile it first. Without --make, GHC will look for the
|  > interface file for Foo, which should have been created by an earlier
|  > compilation of Foo.
|  
|  The hypothetical "--topo-sort" flag would behave in the latter way,
|  i.e.
|  it would not require a source file for an unknown dependency. Hence,
|  "--topo-sort" and "--make" would be conflicting options.
|  
|  I hope that clears up things a bit.
|  
|  Cheers
|  Lars

From lars at hupel.info  Fri Nov 28 16:26:03 2014
From: lars at hupel.info (Lars Hupel)
Date: Fri, 28 Nov 2014 17:26:03 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F3EDBE8@DB3PRD3001MB020.064d.mgd.msft.net>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
 <547889A8.9030803@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3EDBE8@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <5478A21B.3080206@hupel.info>

> How does that differ from ghc --make?  The only difference I can see is that
>   - Modules that --make might find, but not listed on
>     the command line, would not be compiled by --topo-sort

"--make" always requires a full view on all sources. That is, any
imports which cannot be resolved from the package database are assumed
to exist as source files. Imagine the following situation:

A.hs
> module A where
>
> import B
> import Library

B.hs
> module B where

If I compile these two with "--make", it also needs "Library.hs" as
input. If I compile them without "--make", it just needs the interface
("Library.hi") as additional input.

Concretely, assuming that only "path/Library.hi" exists, this fails:

  ghc -c -ipath --make A.hs B.hs

In contrast to that, I propose that this should work:

  ghc -c -ipath --topo-sort A.hs B.hs

From simonpj at microsoft.com  Fri Nov 28 16:52:48 2014
From: simonpj at microsoft.com (Simon Peyton Jones)
Date: Fri, 28 Nov 2014 16:52:48 +0000
Subject: Discovery of source dependencies without --make
In-Reply-To: <5478A21B.3080206@hupel.info>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
 <547889A8.9030803@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3EDBE8@DB3PRD3001MB020.064d.mgd.msft.net>
 <5478A21B.3080206@hupel.info>
Message-ID: <618BE556AADD624C9C918AA5D5911BEF3F3EDD77@DB3PRD3001MB020.064d.mgd.msft.net>

I suppose that if --make found Foo.hi, but no Foo.hs, it could simply use the Foo.hi.  That would do strictly more than now. I don't know if there would be any disadvantages.

Augmenting --make's semantics sounds better to me than inventing a new compilation mode.

Maybe a feature request ticket.  Then if people like it maybe someone can implement it.

Simon

|  -----Original Message-----
|  From: Glasgow-haskell-users [mailto:glasgow-haskell-users-
|  bounces at haskell.org] On Behalf Of Lars Hupel
|  Sent: 28 November 2014 16:26
|  To: Simon Peyton Jones
|  Cc: glasgow-haskell-users at haskell.org
|  Subject: Re: Discovery of source dependencies without --make
|  
|  > How does that differ from ghc --make?  The only difference I can see
|  is that
|  >   - Modules that --make might find, but not listed on
|  >     the command line, would not be compiled by --topo-sort
|  
|  "--make" always requires a full view on all sources. That is, any
|  imports which cannot be resolved from the package database are assumed
|  to exist as source files. Imagine the following situation:
|  
|  A.hs
|  > module A where
|  >
|  > import B
|  > import Library
|  
|  B.hs
|  > module B where
|  
|  If I compile these two with "--make", it also needs "Library.hs" as
|  input. If I compile them without "--make", it just needs the interface
|  ("Library.hi") as additional input.
|  
|  Concretely, assuming that only "path/Library.hi" exists, this fails:
|  
|    ghc -c -ipath --make A.hs B.hs
|  
|  In contrast to that, I propose that this should work:
|  
|    ghc -c -ipath --topo-sort A.hs B.hs
|  _______________________________________________
|  Glasgow-haskell-users mailing list
|  Glasgow-haskell-users at haskell.org
|  http://www.haskell.org/mailman/listinfo/glasgow-haskell-users

From lars at hupel.info  Fri Nov 28 17:51:27 2014
From: lars at hupel.info (Lars Hupel)
Date: Fri, 28 Nov 2014 18:51:27 +0100
Subject: Discovery of source dependencies without --make
In-Reply-To: <618BE556AADD624C9C918AA5D5911BEF3F3EDD77@DB3PRD3001MB020.064d.mgd.msft.net>
References: <54744DB5.30306@hupel.info>
 <1416911354.1435.13.camel@joachim-breitner.de> <5476FA03.1070303@ro-che.info>
 <5476FD9B.6010407@hupel.info> <etPan.547766a4.5bdd194b.5d3c@mbp.local>
 <547837D0.6020205@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3ED457@DB3PRD3001MB020.064d.mgd.msft.net>
 <547889A8.9030803@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3EDBE8@DB3PRD3001MB020.064d.mgd.msft.net>
 <5478A21B.3080206@hupel.info>
 <618BE556AADD624C9C918AA5D5911BEF3F3EDD77@DB3PRD3001MB020.064d.mgd.msft.net>
Message-ID: <5478B61F.6020605@hupel.info>

> I suppose that if --make found Foo.hi, but no Foo.hs, it could simply
> use the Foo.hi.  That would do strictly more than now. I don't know
> if there would be any disadvantages.
> 
> Augmenting --make's semantics sounds better to me than inventing a
> new compilation mode.

Sure, that sounds even better.

> Maybe a feature request ticket.  Then if people like it maybe someone
> can implement it.

Done: <https://ghc.haskell.org/trac/ghc/ticket/9846>

I have no idea how difficult this is going to be ? if someone tells me
it's easy, I'll have a stab at it.

Thanks for your help!

Cheers
Lars

From alexander.vershilov at gmail.com  Fri Nov 28 22:27:25 2014
From: alexander.vershilov at gmail.com (Alexander V Vershilov)
Date: Sat, 29 Nov 2014 02:27:25 +0400
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <CADLz5wXQf-qy50M_CAAnK1EBrXOC9KnHwigaeG4ud4_bC3NJ9A@mail.gmail.com>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
 <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
 <CAO-1Pb4bCWaSj1rBs1Qvn+WVpCC4AO8EniBfAa_fsnRbLEkwoA@mail.gmail.com>
 <CADLz5wXQf-qy50M_CAAnK1EBrXOC9KnHwigaeG4ud4_bC3NJ9A@mail.gmail.com>
Message-ID: <CAO-1Pb6=TXPUbSgfCUp+yrYpx_Pk_+5sftWjgp9xfWCLo3iu5Q@mail.gmail.com>

Hi, Bas, Richard.

I've played a bit with example, obvously first approach contained bugs,
but seems that I have fixed it and here I have 2 approaches, one uses
unsafeCoerce (as Richard suggested) another is safe but with bad complexity:

Full file is quite big, so I'm not inlining it in mail, but here is a link:

https://github.com/qnikst/haskell-fun/blob/master/typelevel-literals/Proof.lhs

I wonder how far it's possible to go with singletons approach that I have
not tried yet.

--
Alexander

On 26 November 2014 at 10:15, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
> Hi Alexander,
>
> Thanks for your answer! This provides a lot of ideas how to proceed.
>
> I'm unsure about the following though:
>
>> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
>> lessThen (SomeNat p) k
>>    | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>>    | otherwise = Nothing
>
> Doesn't this mean lessThen returns a Proxy n for all (n :: Nat) and
> not just the Nat inside the SomeNat?
>
> I also see that p is only used for comparing it to k. It's not used to
> produce the return value.
>
> Cheers,
>
> Bas
>
> On 25 November 2014 at 19:55, Alexander V Vershilov
> <alexander.vershilov at gmail.com> wrote:
>> Hi, Richard, Bas.
>>
>> Maybe I didn't spell it properly but my point was to create a data
>> type that carry a proof
>> without exposing it's constructor and having clever constructor only,
>> then the only place
>> where you need to check will be that constructor.
>>
>> Also it's possible to write in slightly clearer, but again it's
>> possible to make a mistake here
>> and it will be a false proof.
>>
>>> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
>>> lessThen (SomeNat p) k
>>>    | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>    | otherwise = Nothing
>>
>> Of cause solution using singletons could solve this problem much better.
>>
>> --
>> Alexander
>>
>> On 25 November 2014 at 21:34, Richard Eisenberg <eir at cis.upenn.edu> wrote:
>>> Hi Bas,
>>>
>>> I believe to do this "right", you would need singleton types. Then, when you discover that the number is bounded by 255, you would also discover that the type is bounded by 255, and you'd be home free.
>>>
>>> Unfortunately, I there isn't currently a way to do comparison on GHC.TypeLits Nats with singletons. (There is a module Data.Singletons.TypeLits in the `singletons` package, but there's a comment telling me TODO in the part where comparison should be implemented.) If it were implemented, it would have to use unsafeCoerce, as there's no built-in mechanism connecting runtime numbers with TypeLits.
>>>
>>> If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.
>>>
>>> The solution Alexander provides below doesn't quite build a proof, I think. Tellingly, if we omit the `natVal p <= 255` check, everything else still compiles. Thus, the `Proof` type he uses can be built even if the fact proven is false. That said, I don't know if my solution is any better, crucially relying on unsafeCoerce.
>>>
>>> Richard
>>>
>>> On Nov 25, 2014, at 4:52 AM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Following approach can work, the idea is to define a type that will
>>>> carry a proof (constraint) that we want to check. Here I have reused
>>>> Data.Tagged, but it's possible to introduce your own with concrete
>>>> constraints.
>>>>
>>>>> {-# LANGUAGE DataKinds #-}
>>>>> {-# LANGUAGE GADTs #-}
>>>>> {-# LANGUAGE TypeOperators #-}
>>>>> {-# LANGUAGE KindSignatures #-}
>>>>> {-# LANGUAGE PolyKinds #-}
>>>>> {-# LANGUAGE UndecidableInstances #-}
>>>>> import GHC.TypeLits
>>>>> import GHC.Exts
>>>>> import Data.Proxy
>>>>> import Data.Tagged
>>>>> import System.Environment
>>>>
>>>> New constraint carrying data type
>>>>
>>>>> newtype Proof a b = Proof { unProof :: Tagged a b }
>>>>
>>>> Runtime check for unknown naturals
>>>>
>>>>> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
>>>>> fromSome (SomeNat p)
>>>>>   | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>>>   | otherwise = Nothing
>>>>
>>>> Compiletime converter for known naturals
>>>>
>>>>> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
>>>>> fromKnown n = Proof $ Tagged n
>>>>
>>>> Function to test:
>>>>
>>>>> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
>>>>> f2 _ = ()
>>>>
>>>> Example of use:
>>>>
>>>>> main :: IO ()
>>>>> main = do
>>>>>    [arg] <- getArgs
>>>>>    let n = read arg :: Integer
>>>>>
>>>>>    case someNatVal n of
>>>>>      Nothing -> error "Input is not a natural number!"
>>>>>      Just sn -> case fromSome sn of
>>>>>                   Just p -> return $ f2 p
>>>>>                   _ -> error "Input if larger than 255"
>>>>
>>>>
>>>> On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> I have another type-level programming related question:
>>>>>
>>>>>> {-# LANGUAGE GADTs #-}
>>>>>> {-# LANGUAGE TypeOperators #-}
>>>>>> {-# LANGUAGE ScopedTypeVariables #-}
>>>>>> {-# LANGUAGE KindSignatures #-}
>>>>>>
>>>>>> import GHC.TypeLits
>>>>>
>>>>> Say I have a Proxy p of some type-level natural number:
>>>>>
>>>>>> p :: forall (n :: Nat). Proxy n
>>>>>> p = Proxy
>>>>>
>>>>> Imagine I get p from user input like this:
>>>>>
>>>>>> main :: IO ()
>>>>>> main = do
>>>>>>    [arg] <- getArgs
>>>>>>    let n = read arg :: Integer
>>>>>>
>>>>>>    case someNatVal n of
>>>>>>      Nothing -> error "Input is not a natural number!"
>>>>>>      Just (SomeNat (p :: Proxy n)) -> ...
>>>>>
>>>>> I also have a function f which takes a proxy of a natural number but
>>>>> it has the additional constraint that the number should be lesser than
>>>>> or equal to 255:
>>>>>
>>>>>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>>>>>> f _ = ()
>>>>>
>>>>> How do I apply f to p?
>>>>>
>>>>> Obviously, applying it directly gives the type error:
>>>>>
>>>>>> f p
>>>>> <interactive>:179:1:
>>>>>    Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>>>>>    The type variable ?n0? is ambiguous
>>>>>    In the expression: f p
>>>>>    In an equation for ?it?: it = f p
>>>>>
>>>>> I imagine I somehow need to construct some Proof object using a function g like:
>>>>>
>>>>>> g :: forall (n :: Nat). proxy n -> Proof
>>>>>> g _ = ...
>>>>>
>>>>> Where the Proof constructor encapsulates the (n <= 255) constraint:
>>>>>
>>>>>> data Proof where
>>>>>>    NoProof :: Proof
>>>>>>    Proof :: forall (n :: Nat). (n <= 255)
>>>>>>          => Proxy n -> Proof
>>>>>
>>>>> With g in hand I can construct c which patterns matches on g p and
>>>>> when there's a Proof the (n <= 255) constraint will be in scope which
>>>>> allows applying f to p:
>>>>>
>>>>>> c :: ()
>>>>>> c = case g p of
>>>>>>      NoProof -> error "Input is bigger than 255!"
>>>>>>      Proof p -> f p
>>>>>
>>>>> But how do I define g?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Bas
>>>>> _______________________________________________
>>>>> Glasgow-haskell-users mailing list
>>>>> Glasgow-haskell-users at haskell.org
>>>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>>
>>>>
>>>>
>>>> --
>>>> Alexander
>>>> _______________________________________________
>>>> Glasgow-haskell-users mailing list
>>>> Glasgow-haskell-users at haskell.org
>>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>>
>>>
>>
>>
>>
>> --
>> Alexander



-- 
Alexander

From eir at cis.upenn.edu  Sun Nov 30 20:26:10 2014
From: eir at cis.upenn.edu (Richard Eisenberg)
Date: Sun, 30 Nov 2014 15:26:10 -0500
Subject: Proving properties of type-level natural numbers obtained from
 user input
In-Reply-To: <CAO-1Pb6=TXPUbSgfCUp+yrYpx_Pk_+5sftWjgp9xfWCLo3iu5Q@mail.gmail.com>
References: <CADLz5wWpAd+YZ-6nt294XqdRHxKi_7xosG2VuLeCgyOrYYP8Uw@mail.gmail.com>
 <CAO-1Pb6kUsMQAb5r9y4ArVOoy9x3As7zpjnfDG9WkQi2-Zepcg@mail.gmail.com>
 <59442EC1-0552-4767-A578-6B02CF805823@cis.upenn.edu>
 <CAO-1Pb4bCWaSj1rBs1Qvn+WVpCC4AO8EniBfAa_fsnRbLEkwoA@mail.gmail.com>
 <CADLz5wXQf-qy50M_CAAnK1EBrXOC9KnHwigaeG4ud4_bC3NJ9A@mail.gmail.com>
 <CAO-1Pb6=TXPUbSgfCUp+yrYpx_Pk_+5sftWjgp9xfWCLo3iu5Q@mail.gmail.com>
Message-ID: <7DEE4B6F-3B6C-4E3B-A91C-57D27CBC44D5@cis.upenn.edu>

Hi Alexander,

Nice idea to test against the set of known values. That's more type-safe than anything I've thought of. I agree that it's a bit of a painful construction, but I don't think we can do better with type-lits, as there is limited reasoning ability that GHC can manage. If you want to switch to unary naturals (`data Nat = Zero | Succ Nat`), then this can be built somewhat nicely with singletons and without unsafeCoerce. But, of course, unary naturals are very slow.

By the way, the bug in the Proof2 version is a bug in GHC 7.8.3 (only in .3 -- not in .2 or in the soon-to-be .4) that allows you to write unsaturated type families that don't work. Saying `LessThan255` without a parameter should be a syntax error, but that check was accidentally turned off for 7.8.3, leading to a bogus type error.

Thanks for sharing this work!
Richard

On Nov 28, 2014, at 5:27 PM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:

> Hi, Bas, Richard.
> 
> I've played a bit with example, obvously first approach contained bugs,
> but seems that I have fixed it and here I have 2 approaches, one uses
> unsafeCoerce (as Richard suggested) another is safe but with bad complexity:
> 
> Full file is quite big, so I'm not inlining it in mail, but here is a link:
> 
> https://github.com/qnikst/haskell-fun/blob/master/typelevel-literals/Proof.lhs
> 
> I wonder how far it's possible to go with singletons approach that I have
> not tried yet.
> 
> --
> Alexander
> 
> On 26 November 2014 at 10:15, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>> Hi Alexander,
>> 
>> Thanks for your answer! This provides a lot of ideas how to proceed.
>> 
>> I'm unsure about the following though:
>> 
>>> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
>>> lessThen (SomeNat p) k
>>>   | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>   | otherwise = Nothing
>> 
>> Doesn't this mean lessThen returns a Proxy n for all (n :: Nat) and
>> not just the Nat inside the SomeNat?
>> 
>> I also see that p is only used for comparing it to k. It's not used to
>> produce the return value.
>> 
>> Cheers,
>> 
>> Bas
>> 
>> On 25 November 2014 at 19:55, Alexander V Vershilov
>> <alexander.vershilov at gmail.com> wrote:
>>> Hi, Richard, Bas.
>>> 
>>> Maybe I didn't spell it properly but my point was to create a data
>>> type that carry a proof
>>> without exposing it's constructor and having clever constructor only,
>>> then the only place
>>> where you need to check will be that constructor.
>>> 
>>> Also it's possible to write in slightly clearer, but again it's
>>> possible to make a mistake here
>>> and it will be a false proof.
>>> 
>>>> lessThen :: KnownNat m => SomeNat -> Proxy m -> Maybe (Proof (n <= m) (Proxy n))
>>>> lessThen (SomeNat p) k
>>>>   | natVal p <= natVal k = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>>   | otherwise = Nothing
>>> 
>>> Of cause solution using singletons could solve this problem much better.
>>> 
>>> --
>>> Alexander
>>> 
>>> On 25 November 2014 at 21:34, Richard Eisenberg <eir at cis.upenn.edu> wrote:
>>>> Hi Bas,
>>>> 
>>>> I believe to do this "right", you would need singleton types. Then, when you discover that the number is bounded by 255, you would also discover that the type is bounded by 255, and you'd be home free.
>>>> 
>>>> Unfortunately, I there isn't currently a way to do comparison on GHC.TypeLits Nats with singletons. (There is a module Data.Singletons.TypeLits in the `singletons` package, but there's a comment telling me TODO in the part where comparison should be implemented.) If it were implemented, it would have to use unsafeCoerce, as there's no built-in mechanism connecting runtime numbers with TypeLits.
>>>> 
>>>> If I were you, I would just write `g` using unsafeCoerce in the right spot, instead of bothering with all the singletons, which would have to use unsafety anyway.
>>>> 
>>>> The solution Alexander provides below doesn't quite build a proof, I think. Tellingly, if we omit the `natVal p <= 255` check, everything else still compiles. Thus, the `Proof` type he uses can be built even if the fact proven is false. That said, I don't know if my solution is any better, crucially relying on unsafeCoerce.
>>>> 
>>>> Richard
>>>> 
>>>> On Nov 25, 2014, at 4:52 AM, Alexander V Vershilov <alexander.vershilov at gmail.com> wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> Following approach can work, the idea is to define a type that will
>>>>> carry a proof (constraint) that we want to check. Here I have reused
>>>>> Data.Tagged, but it's possible to introduce your own with concrete
>>>>> constraints.
>>>>> 
>>>>>> {-# LANGUAGE DataKinds #-}
>>>>>> {-# LANGUAGE GADTs #-}
>>>>>> {-# LANGUAGE TypeOperators #-}
>>>>>> {-# LANGUAGE KindSignatures #-}
>>>>>> {-# LANGUAGE PolyKinds #-}
>>>>>> {-# LANGUAGE UndecidableInstances #-}
>>>>>> import GHC.TypeLits
>>>>>> import GHC.Exts
>>>>>> import Data.Proxy
>>>>>> import Data.Tagged
>>>>>> import System.Environment
>>>>> 
>>>>> New constraint carrying data type
>>>>> 
>>>>>> newtype Proof a b = Proof { unProof :: Tagged a b }
>>>>> 
>>>>> Runtime check for unknown naturals
>>>>> 
>>>>>> fromSome :: SomeNat -> Maybe (Proof (n <= 255) (Proxy n))
>>>>>> fromSome (SomeNat p)
>>>>>>  | natVal p <= 255 = Just (Proof $ Tagged (Proxy :: Proxy n))
>>>>>>  | otherwise = Nothing
>>>>> 
>>>>> Compiletime converter for known naturals
>>>>> 
>>>>>> fromKnown :: (KnownNat n,  n <= 255) => Proxy n -> Proof (n <= 255) (Proxy n)
>>>>>> fromKnown n = Proof $ Tagged n
>>>>> 
>>>>> Function to test:
>>>>> 
>>>>>> f2 :: (c ~ (n <= 255)) => Proof c (Proxy n) -> ()
>>>>>> f2 _ = ()
>>>>> 
>>>>> Example of use:
>>>>> 
>>>>>> main :: IO ()
>>>>>> main = do
>>>>>>   [arg] <- getArgs
>>>>>>   let n = read arg :: Integer
>>>>>> 
>>>>>>   case someNatVal n of
>>>>>>     Nothing -> error "Input is not a natural number!"
>>>>>>     Just sn -> case fromSome sn of
>>>>>>                  Just p -> return $ f2 p
>>>>>>                  _ -> error "Input if larger than 255"
>>>>> 
>>>>> 
>>>>> On 25 November 2014 at 10:51, Bas van Dijk <v.dijk.bas at gmail.com> wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> I have another type-level programming related question:
>>>>>> 
>>>>>>> {-# LANGUAGE GADTs #-}
>>>>>>> {-# LANGUAGE TypeOperators #-}
>>>>>>> {-# LANGUAGE ScopedTypeVariables #-}
>>>>>>> {-# LANGUAGE KindSignatures #-}
>>>>>>> 
>>>>>>> import GHC.TypeLits
>>>>>> 
>>>>>> Say I have a Proxy p of some type-level natural number:
>>>>>> 
>>>>>>> p :: forall (n :: Nat). Proxy n
>>>>>>> p = Proxy
>>>>>> 
>>>>>> Imagine I get p from user input like this:
>>>>>> 
>>>>>>> main :: IO ()
>>>>>>> main = do
>>>>>>>   [arg] <- getArgs
>>>>>>>   let n = read arg :: Integer
>>>>>>> 
>>>>>>>   case someNatVal n of
>>>>>>>     Nothing -> error "Input is not a natural number!"
>>>>>>>     Just (SomeNat (p :: Proxy n)) -> ...
>>>>>> 
>>>>>> I also have a function f which takes a proxy of a natural number but
>>>>>> it has the additional constraint that the number should be lesser than
>>>>>> or equal to 255:
>>>>>> 
>>>>>>> f :: forall (n :: Nat). (n <= 255) => proxy n -> ()
>>>>>>> f _ = ()
>>>>>> 
>>>>>> How do I apply f to p?
>>>>>> 
>>>>>> Obviously, applying it directly gives the type error:
>>>>>> 
>>>>>>> f p
>>>>>> <interactive>:179:1:
>>>>>>   Couldn't match expected type ?'True? with actual type ?n0 <=? 255?
>>>>>>   The type variable ?n0? is ambiguous
>>>>>>   In the expression: f p
>>>>>>   In an equation for ?it?: it = f p
>>>>>> 
>>>>>> I imagine I somehow need to construct some Proof object using a function g like:
>>>>>> 
>>>>>>> g :: forall (n :: Nat). proxy n -> Proof
>>>>>>> g _ = ...
>>>>>> 
>>>>>> Where the Proof constructor encapsulates the (n <= 255) constraint:
>>>>>> 
>>>>>>> data Proof where
>>>>>>>   NoProof :: Proof
>>>>>>>   Proof :: forall (n :: Nat). (n <= 255)
>>>>>>>         => Proxy n -> Proof
>>>>>> 
>>>>>> With g in hand I can construct c which patterns matches on g p and
>>>>>> when there's a Proof the (n <= 255) constraint will be in scope which
>>>>>> allows applying f to p:
>>>>>> 
>>>>>>> c :: ()
>>>>>>> c = case g p of
>>>>>>>     NoProof -> error "Input is bigger than 255!"
>>>>>>>     Proof p -> f p
>>>>>> 
>>>>>> But how do I define g?
>>>>>> 
>>>>>> Cheers,
>>>>>> 
>>>>>> Bas
>>>>>> _______________________________________________
>>>>>> Glasgow-haskell-users mailing list
>>>>>> Glasgow-haskell-users at haskell.org
>>>>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Alexander
>>>>> _______________________________________________
>>>>> Glasgow-haskell-users mailing list
>>>>> Glasgow-haskell-users at haskell.org
>>>>> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Alexander
> 
> 
> 
> -- 
> Alexander
>