[GHC] #12368: Demand Analyzer: Cunnig plan not adhered to with aborting fixpoint interation

[GHC]

#12368: Demand Analyzer: Cunnig plan not adhered to with aborting fixpoint
interation
-------------------------------------+-------------------------------------
        Reporter:  nomeata           |                Owner:
            Type:  bug               |               Status:  patch
        Priority:  low               |            Milestone:
       Component:  Compiler          |              Version:  8.0.1
      Resolution:                    |             Keywords:
Operating System:  Unknown/Multiple  |         Architecture:
                                     |  Unknown/Multiple
 Type of failure:  None/Unknown      |            Test Case:
      Blocked By:                    |             Blocking:
 Related Tickets:                    |  Differential Rev(s):  Phab:D2392
       Wiki Page:                    |
-------------------------------------+-------------------------------------

Comment (by simonpj):

 Sorry to be slow.

 I've looked at this and I don't agree with it at all!  In the current
 code, if we hit the iteration limit we do this:
 {{{
     loop' n env pairs
       | n >= 10
       = (env, lazy_fv, orig_pairs)      -- Safe output
 }}}
 So we return an un-decorated binding `orig_pairs` but (wrongly) an
 extended environment `env`.  We should just return the un-extended
 environment!  (If a variable isn't in the environment it's treated as
 having `topSig`.)  Simple.

 Well, not totally simple. There is a nasty corner case, when we have
 nested recursive bindings:
 {{{
 f x = ...let g y = ...y... in ...
 }}}
 Suppose that

 * on the first `f` iteration we find a fixpoint for `g` which we attach to
 it.
 * But on the second `f` iteration  we fail to find a fixpoint for `g`.
 Then we should revert to `topSig`, not to the result of the first
 iteration.

 Conclusion: if the fixpoint limit is reached:

 * return an environment that simply does not mention the new binders
 * set all the binders to no-strictness-at-all

 Finally, the `loop` function in `dmdFix` would be much better if it took a
 `SigEnv` rather than an `AnalEnv`, wouldn't it?  Just a simple
 refactoring; but the `ae_sigs` field is the only bit that varies.

 OK Joachim?

 Simon

--
Ticket URL: <http://ghc.haskell.org/trac/ghc/ticket/12368#comment:8>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler