[GHC] #2540: [Text.Regex] incorrect word boundary ("\\b") substitutions. Bug in regex-compat's subRegex handling of BOL flags.

[GHC]

#2540: [Text.Regex] incorrect word boundary ("\\b") substitutions. Bug in regex-
compat's subRegex handling of BOL flags.
--------------------------------+-------------------------------------------
 Reporter:  Eelis-              |          Owner:  ChrisKuklewicz
     Type:  bug                 |         Status:  assigned      
 Priority:  normal              |      Milestone:  Not GHC       
Component:  libraries (other)   |        Version:  6.8.3         
 Severity:  normal              |     Resolution:                
 Keywords:  regex regex-compat  |     Difficulty:  Unknown       
 Testcase:                      |   Architecture:  Multiple      
       Os:  Multiple            |  
--------------------------------+-------------------------------------------
Changes (by ChrisKuklewicz):

  * status:  new => assigned
  * summary:  [Text.Regex] incorrect word boundary ("\\b") substitutions =>
              [Text.Regex] incorrect word boundary ("\\b")
              substitutions. Bug in regex-compat's subRegex
              handling of BOL flags.
  * owner:  TextRegexLazy at personal.mightyreason.com => ChrisKuklewicz
  * architecture:  x86_64 (amd64) => Multiple
  * keywords:  regex => regex regex-compat
  * os:  Unknown => Multiple

Comment:

 Ah bollocks, there is a bug here but it is subtle.  The above
 complaint is actually to do with the lack of support for GNU
 extensions to regex/sed.  The regex-posix library expects to implement
 just the POSIX regular expressions and none of the different
 extensions.  This is also consistent with the BSD sed.

 The actual c-library calls in regex-posix are regcomp and regexec (and
 regfree, regerror).

 In GNU regex/sed (I tested version 4.1.5 on linux) the \b means a word
 boundary.  I assume that this is also the case in Perl and Ruby.  Thus
 \b matches only at the front of the abcdef word for these systems.

 In POSIX sed the \b is not recognized as a known escape, but is
 accepted as a literal b.  So it matches the bc in abcdef and is
 replaced by |c.

 On Mac OS 10.5.4 the equivalent to -r is -E and then:
 {{{
 $ echo "abcdef" | sed -E  's/\b(.)/|\1/g'
 a|cdef
 }}}
 With ghc version 6.8.3 on OS X I get the same answer as POSIX sed
 {{{
 Prelude> :m +Text.Regex
 Prelude Text.Regex> subRegex (mkRegex "\\b(.)") "abcdef" "|\\1"
 "a|cdef"
 }}}
 On linux I can reproduce the bug report:
 {{{
 Prelude Text.Regex>  subRegex (mkRegex "\\b(.)") "abcdef" "|\\1"
 "|a|b|c|d|e|f"
 }}}
 Note that man 3 regexec and man 7 regex on linux are not describing
 the \b behavior.  It is mis-documented.

 But there is a further problem: Change \b to `^` and it is clear that
 Text.Regex is getting the wrong answer on all systems. On OS X:

 {{{
 $ echo "abcdef" | sed -E  's/^(.)/|\1/'
 |abcdef
 Prelude Text.Regex>  subRegex (mkRegex "^(.)") "abcdef" "|\\1"
 "|a|b|c|d|e|f"
 }}}

 So there is a bug to fix with respect to `^`.  Fixing this may also
 accidentally fix the \b handling on GNU systems.  I thought I had
 added enough 'execNotBOL' (REG_NOTBOL) flags to cover all these cases,
 but regex-compat's subRegex is not clearly not clever enough.

 I will update this bug report when there is a fixed version to announce.

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/2540#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler