[GHC] #5218: Add unpackCStringLen# to create Strings from string literals

[GHC]

#5218: Add unpackCStringLen# to create Strings from string literals
---------------------------------+------------------------------------------
    Reporter:  tibbe             |        Owner:              
        Type:  feature request   |       Status:  new         
    Priority:  high              |    Milestone:  7.4.1       
   Component:  Compiler          |      Version:  7.0.3       
    Keywords:                    |     Testcase:              
   Blockedby:                    |   Difficulty:              
          Os:  Unknown/Multiple  |     Blocking:              
Architecture:  Unknown/Multiple  |      Failure:  None/Unknown
---------------------------------+------------------------------------------

Comment(by simonmar):

 Simon and I discussed this just now and found two further problems in
 addition to the extra runtime length operation described above.

 '''1. The RULES in `ByteString` for optimising literal strings currently
 do not work'''

 e.g. try

 {{{
 {-# LANGUAGE OverloadedStrings #-}
 module Foo where
 import Data.ByteString.Char8
 foo = "abc"
 }}}

 the generated code does the translation to/from `String` at runtime.
 Presumably this broke at some point, possibly due to things being inlined
 at the wrong time and/or let-floating.

 '''2. The RULES in `ByteString` never worked for non-ASCII strings'''

 This is because non-ASCII strings are UTF-8-encoded by GHC and wrapped in
 `unpackCStringUtf8`, not `unpackCString`, so the RULEs don't apply.

 ----------

 We have two solutions.

 '''1. Use quasi-quoting to declare `ByteString` literals.'''

 You would write something like

 {{{
 foo :: ByteString
 foo = [bytes| ffa3c9db77 ]
 }}}

 and the code for `bytes` would translate this directly into a call to
 `unsafePackAddress` passing a primitive literal string (`HsStringPrim`,
 aka `"..."#`).  Note that primitive literal strings are not UTF-8-encoded
 by GHC, they are just 8-bit stripped (this happens in the lexer, of all
 places).

 This would be more convenient for encoding static data into the program
 through `ByteString` literals.

 '''2. Fix the RULES for `ByteString` literals.'''

 Here's one way we could fix all the problems described above.  It's a bit
 tricky and requires changes in various places.

 First, somewhere in `base` or `ghc-prim` we have

 {{{
 newtype String8 = String8 String

 string8 :: String -> String8
 {-# INLINE [0] string8 #-}
 string8 = String8

 unpackCString :: Addr# -> String
 unpackCStringUtf8 :: Addr# -> String

 unpackCString8Len :: Addr# -> Int# -> String8
 }}}

 Next, GHC internally has RULEs that convert

 {{{
   string8 (unpackCStringUtf8 <utf8stringlit#>)
     ===> unpackCString8Len <stringlit#> <len#>

   string8 (unpackCString <asciistringlit#>)
     ===> unpackCString8Len <asciistringlit#> <len#>
 }}}

 where `<stringlit#>` is made by decoding `<utf8stringlit#>` and stripping
 out bits 8 and above from each character.

 Then in `Data.ByteString.Char8` we have:

 {{{
 module Data.ByteString.Char8 where

 packAddressLen :: Addr# -> Int# -> ByteString

 pack8 :: String8 -> ByteString

 instance IsString ByteString where
   fromString = pack . string8

 {-# RULES "pack8/unpackCString8"
   pack8 (unpackCString8 addr len) = packAddressLen addr len
  #-}
 }}}

-- 
Ticket URL: <http://hackage.haskell.org/trac/ghc/ticket/5218#comment:3>
GHC <http://www.haskell.org/ghc/>
The Glasgow Haskell Compiler