[Haskell-cafe] Reason about a Megaparsec error

[Jaro Reinders]

Aha, I think it is the 'lexeme' function that causes these problems. 'symbol = 
lexeme . string', but lexeme first skips over all the comments (even the 
multiline comments).

On 10/25/20 12:36 PM, YueCompl wrote:
> Tried this:
> 
> ```hs
>    findIt = do
>      -- ignore leading whitespaces and an optional semicolon in between
>      nbsc
>      ii0 <- getInput
>      trace ("**about*to*match*semicolon*[" <> T.unpack ii0 <> "]") $ pure ()
>      optionalSemicolon
>      trace ("**after*semicolon*attempt**") $ pure ()
>      nbsc
>      -- try get a doc comment block
>      getIt >>= \case
> ```
> 
> ```console
> $ cabal run dcp < samples/basic.txt
> Up to date
> **about*to*match*semicolon*[{##
>   # this is the module's doc comment
>   #}
> 
> # this is regular line comment to be dropped in parsing
> 
> {##
>   # this is the doc comment for  `doXXX()`
>   #}
> method doXXX() pass
> 
> {#
>   # this is regular block comment to be dropped in parsing
>   #}
> 
> {##
>   # this is the doc comment for  `doYYY()`
>   #}
> method doYYY() pass
> ]
> dcp: 10:1:
>     |
> 10 | method doXXX() pass
>     | ^
> unexpected 'm'
> expecting ';'
> 
> CallStack (from HasCallStack):
>    error, called at src/Parser.hs:154:14 in main:Parser
> ```
> 
> So it seems the very first optionalSemicolon failed without consuming anything ...
> 
> And actually I totally have no clue how the content before `method doXXX` is consumed.
> 
>> On 2020-10-25, at 19:26, Jaro Reinders <jaro.reinders at gmail.com> wrote:
>>
>> I think the optionalSemicolon will consume the '{' of the first comment and then the 'getIt' will produce Nothing because the input doesn't start with '{##' anymore, and then it will go on like that until the optionalSemicolon consumes that 'm'.
>>
>> On 10/25/20 12:21 PM, YueCompl wrote:
>>> The rerun is guarded behind an optional `L.skipBlockCommentNested "{#" "#}"` got a `Just{}` result, and only after `getIt` got a `Nothing` result.
>>>> On 2020-10-25, at 19:16, Jaro Reinders <jaro.reinders at gmail.com> wrote:
>>>>
>>>> But the 'findIt' parser will be rerun after every block comment right?
>>>>
>>>> On 10/25/20 12:10 PM, YueCompl wrote:
>>>>> Unlike `sc`, `nbsc` won't consume block comments, as it's defined as `L.space space1 (L.skipLineComment "#") empty`
>>>>>> On 2020-10-25, at 19:07, Jaro Reinders <jaro.reinders at gmail.com> wrote:
>>>>>>
>>>>>> But in the basic.txt example, won't the first nbsc consume everything up to 'method doXXX() pass'. Then the semicolon parser consumes the 'm' character, and that will never backtrack. So that 'm' character is lost.
>>>>>>
>>>>>> On 10/25/20 12:00 PM, YueCompl wrote:
>>>>>>> During the parsing of doc comments, backtracking is off on purpose, as it won't consume syntactic contents, merely whitespaces (while doc comment is a special form of block comment), the desired result is `Nothing` or `Just docCmt` without backtracking of whitespaces.
>>>>>>>> On 2020-10-25, at 18:53, Jaro Reinders <jaro.reinders at gmail.com> wrote:
>>>>>>>>
>>>>>>>> I think you have to insert a 'try' manually inside the optional, because megaparsec doesn't do backtracking by default.
>>>>>>>>
>>>>>>>> On 10/25/20 11:27 AM, YueCompl via Haskell-Cafe wrote:
>>>>>>>>> Dear Cafe,
>>>>>>>>> I find myself unable to reason about an error that `optional` in a parser err out instead of return Nothing, I asked the question at StackOverflow, and would like to seek your help here as well.
>>>>>>>>> https://stackoverflow.com/questions/64522568/why-optional-in-a-parser-can-err-out <https://stackoverflow.com/questions/64522568/why-optional-in-a-parser-can-err-out>
>>>>>>>>> https://github.com/complyue/dcp <https://github.com/complyue/dcp> is a minimum working example to reprod this error
>>>>>>>>> $ cabal run dcp:dcp < samples/basic.txt
>>>>>>>>> Up to date
>>>>>>>>> dcp: 10:1:
>>>>>>>>>     |
>>>>>>>>> 10 | method doXXX() pass
>>>>>>>>>     | ^
>>>>>>>>> unexpected 'm'
>>>>>>>>> expecting ';'
>>>>>>>>> CallStack (from HasCallStack):
>>>>>>>>>    error, called at src/Parser.hs:149:14 in main:Parser
>>>>>>>>> $
>>>>>>>>> I believe it's optionalSemicolon causing the failure:
>>>>>>>>> https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L50-L54 <https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L50-L54>
>>>>>>>>>    findIt = do
>>>>>>>>>      -- ignore leading whitespaces and an optional semicolon in between
>>>>>>>>>      nbsc >> optionalSemicolon >> nbsc
>>>>>>>>>      -- try get a doc comment block
>>>>>>>>>      getIt >>= \case
>>>>>>>>> And it's defined like this: https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L31-L32 <https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L31-L32>
>>>>>>>>> optionalSemicolon :: Parser Bool
>>>>>>>>> optionalSemicolon = fromMaybe False <$> optional (True <$ symbol ";")
>>>>>>>>> I can't reason about why it can fail like this.
>>>>>>>>> Best regards,
>>>>>>>>> Compl
>>>>>>>>> _______________________________________________
>>>>>>>>> Haskell-Cafe mailing list
>>>>>>>>> To (un)subscribe, modify options or view archives go to:
>>>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>>>>>>>> Only members subscribed via the mailman list are allowed to post.
>>>>>>>> _______________________________________________
>>>>>>>> Haskell-Cafe mailing list
>>>>>>>> To (un)subscribe, modify options or view archives go to:
>>>>>>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>>>>>>> Only members subscribed via the mailman list are allowed to post.
> 
>