No subject

with UTF-8. No working code will be broken.

** JHC
"JHC allows unrestricted use of the Unicode character set in Haskell
source, treating input as UTF-8." [5]

** Hugs
Hugs treats input as being in the encoding specified by the current
locale, but permits Unicode only in comments and character and string
literals. [6]


* Related proposal

There is one, 5 year old, proposal that is related:
"SourceEncodingDetection" [5]. There it is proposed to detect the
encoding using an algorithm which can distinguish between UTF-8,
UTF-16 and (not always) UTF-32. It can also detect the endianness of
the document, if applicable.

I think choosing just UTF-8 is a better choice than a detection
algorithm. It places less burden on implementation writers and is even
more portable.


* Next step

Discussion! There was already some discussion on the haskell-cafe
mailing list [7].

Attached is a patch for the Haskell Report which adds a note stating
that source encodings must be UTF-8.


Regards,
Roel van Dijk


[1] - http://hackage.haskell.org/trac/haskell-prime/wiki/Process
[2] - http://www.haskell.org/onlinereport/haskell2010/haskellch2.html#x7-150002.1
[3] - http://www.unicode.org/faq/utf_bom.html#bom5
[4] - http://www.haskell.org/ghc/docs/7.0-latest/html/users_guide/separate-compilation.html#source-files
[5] - http://hackage.haskell.org/trac/haskell-prime/wiki/SourceEncodingDetection
[6] - http://cvs.haskell.org/Hugs/pages/users_guide/locale.html
[7] - http://article.gmane.org/gmane.comp.lang.haskell.cafe/87815

--e0cb4e43b285dfe30b04a01f8fef
Content-Type: application/octet-stream; name="utf8_encoding.dpatch"
Content-Disposition: attachment; filename="utf8_encoding.dpatch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gm3zr1nb0

MSBwYXRjaCBmb3IgcmVwb3NpdG9yeSBodHRwOi8vZGFyY3MuaGFza2VsbC5vcmcvaGFza2VsbDIw
MTAtcmVwb3J0OgoKVHVlIEFwciAgNSAwMDo0MjozNyBDRVNUIDIwMTEgIFJvZWwgdmFuIERpamsg
PHZhbmRpamsucm9lbEBnbWFpbC5jb20+CiAgKiBBZGRlZCBub3RlIHRoYXQgc291cmNlIGZpbGVz
IG11c3QgYmUgVVRGLTgKCk5ldyBwYXRjaGVzOgoKW0FkZGVkIG5vdGUgdGhhdCBzb3VyY2UgZmls
ZXMgbXVzdCBiZSBVVEYtOApSb2VsIHZhbiBEaWprIDx2YW5kaWprLnJvZWxAZ21haWwuY29tPioq
MjAxMTA0MDQyMjQyMzcKIElnbm9yZS10aGlzOiBiNTQ2ZjI2NWRjNjRjNGZmMDcyNGE4MzIxM2U0
ZGNhZQpdIHsKaHVuayAuL3JlcG9ydC9oYXNrZWxsLmJpYiAyNTkKICAgdGl0bGUgPSAgICAge1Vu
aWNvZGUgU3RhbmRhcmR9LA0KICAgbm90ZSA9ICAgICAge1x1cmx7aHR0cDovL3VuaWNvZGUub3Jn
L3N0YW5kYXJkL3N0YW5kYXJkLmh0bWx9fX0NCiANCitATWlzYyB7dXRmOCwNCisgIGF1dGhvciA9
ICAgIHtGLiBZZXJnZWF1IChUaGUgSW50ZXJuZXQgU29jaWV0eSl9LA0KKyAgdGl0bGUgPSAgICAg
e1VURi04fSwNCisgIG5vdGUgPSAgICAgIHtcdXJse2h0dHA6Ly93d3cuaWV0Zi5vcmcvcmZjL3Jm
YzM2MjkudHh0fX19DQorDQogQGlucHJvY2VlZGluZ3N7ZGFtYXMtbWlsbmVyODIsDQogQXV0aG9y
ID0gIkx1aXMgRGFtYXMgYW5kIFJvYmluIE1pbG5lciIsDQogVGl0bGUgPSAiUHJpbmNpcGFsIFR5
cGUtU2NoZW1lcyBmb3IgRnVuY3Rpb25hbCBQcm9ncmFtcyIsDQpodW5rIC4vcmVwb3J0L2xleGVt
ZXMudmVyYiA3NAogcHJvZ3JhbXMgYXJlIGN1cnJlbnRseSBiaWFzZWQgdG93YXJkIHRoZSBBU0NJ
SSBjaGFyYWN0ZXIgc2V0CiBcaW5kZXh7QVNDSUkgY2hhcmFjdGVyIHNldH0gdXNlZCBpbiBlYXJs
aWVyIHZlcnNpb25zIG9mIFxIYXNrZWxse30uCiAKK1NvdXJjZSBmaWxlcyBtdXN0IGJlIGVuY29k
ZWQgYXMgVVRGLTggXGNpdGV7dXRmOH0uIEltcGxlbWVudGF0aW9ucworbXVzdCBkaXNjYXJkIGFu
IGluaXRpYWwgQnl0ZSBPcmRlciBNYXJrIChCT00pIGlmIHByZXNlbnQuCisKKwogJQlSZW1vdmVk
IFNlcHQgMDIKICUgSGFza2VsbCB1c2VzIGEgcHJlLXByb2Nlc3NvciB0byBjb252ZXJ0IG5vbi1V
bmljb2RlIGNoYXJhY3RlciBzZXRzIGludG8KICUgVW5pY29kZS4gIFRoaXMgcHJlLXByb2Nlc3Nv
ciBjb252ZXJ0cyBhbGwgY2hhcmFjdGVycyB0byBVbmljb2RlIGFuZAp9CgpDb250ZXh0OgoKW2Nv
bW1lbnRlZC1vdXQgZm9ybWF0dGluZyB0d2VhawpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWls
LmNvbT4qKjIwMTAxMTE3MTAxODQzCiBJZ25vcmUtdGhpczogNTAwNmRhZDQ5YWJlZWVjODQ3NWZm
ZGIxY2EwNjAwZTQKXSAKW2ZpeCBhIFxyaWdodGFycm93IHRoYXQgc2hvdWxkIGJlIEAtPkAKU2lt
b24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMTExNzEwMTgzNQogSWdub3JlLXRo
aXM6IGQ4ZmMzYTIxODE2MDA1ODlkODJlNDU0OGZkMTQzNGU0Cl0gCltjbGFyaWZ5IHRoYXQgbWFy
c2hhbGxhYmxlIGZvcmVpZ24gcmVzdWx0IHR5cGVzIGNhbiBiZSBzeW5vbnltcwpTaW1vbiBNYXJs
b3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzIwMTIxMTMwCiBJZ25vcmUtdGhpczogNjk1
ZDZhNDVjYzk0YTk3MDU0MWYyOTE2ZWRhNTYxMwpdIApbdHlwbzogYWJzdHJhY3RsaXkgLT4gYWJz
dHJhY3RseQpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzIwMTIwNjE4
CiBJZ25vcmUtdGhpczogOThlOTVmZWE2NGIxYmMzNTMyZTUxM2I0Y2E2YTI1OQpdIApbZml4IGZv
cm1hdHRpbmcgaW4gZG9jcyBmb3IgaFNlZWsKU2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5j
b20+KioyMDEwMDcxNDE1MTcyOAogSWdub3JlLXRoaXM6IDQzMDU3ZmZhYzA3NTMwYWFjZjBjNWQ0
NzQwMmQzNjg1Cl0gClthZGQgY2FzdHtVU31DaGFyVG9DaGFyIGFuZCBjYXN0Q2hhclRve1VTfUNo
YXIKU2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxNDE1MTcxOAogSWdu
b3JlLXRoaXM6IGIzYzRiMWJmZDQ4Y2JjYjcyYzZiZjA3ZTE4N2NmZWMwCl0gCltjbGFyaWZ5IGV4
Y2VwdGlvbi1vdmVycmlkaW5nIGJlaGF2aW91ciBvZiB3aXRoRmlsZQpTaW1vbiBNYXJsb3cgPG1h
cmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzE0MTEwODI5CiBJZ25vcmUtdGhpczogNjE2ODFkOWYx
ZmNkYzFhZDYxMjEyNTllYTI3ODQxMjMKXSAKW2NvbnNpc3RlbmN5OiBmaW5hbGlzZXIgLT4gZmlu
YWxpemVyClNpbW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MTQxMTA4MTcK
IElnbm9yZS10aGlzOiAyYzQ1N2JmMTI4MWExZmE4MTQ2MDFjZGJiNWNiYjNmYQpdIApbY2xhcmlm
eSBzaG9ydGN1dHRpbmcgYmVoYXZpb3VyIG9mIGFsbC9hbnkvZWxlbQpTaW1vbiBNYXJsb3cgPG1h
cmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzE0MTEwODA1CiBJZ25vcmUtdGhpczogNjdhOTkyZjBj
NjAwMDIwNjg5NmUzNWQ5MjNmOTAxMmQKXSAKW2NsYXJpZnkgbWVhbmluZyBvZiAnYml0JwpTaW1v
biBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzE0MTEwNzUwCiBJZ25vcmUtdGhp
czogNzBmNTBmYWNkNTIyMTQxNjFiMjZhMzY5NGU3YjIyYWEKXSAKW3R5cG9zIGFuZCBwdW5jdHVh
dGlvbgpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAwNzE0MTAyNDEwCiBJ
Z25vcmUtdGhpczogMzJjYzcyMGU2ZDViNDBiYWM3ZWZhOWZmZTE4MjY0MDUKXSAKW3R5cG8KU2lt
b24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxNDEwMjQwMgogSWdub3JlLXRo
aXM6IDVkMDdjNWJjZmUyMGZkNzg2NDZjNDFlYzRhNTM4MDllCl0gClt0eXBvcyBhbmQgcHVuY3R1
YXRpb24KU2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxNDEwMjM1NQog
SWdub3JlLXRoaXM6IGE4NTVmODFlYTgyM2Y1MmFmMzUyMzllYzllOWY0MDAzCl0gClthZGQgbWlz
c2luZyBhYnMgYnJhY2tldHMgaW4gZXF1YXRpb24KU2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFp
bC5jb20+KioyMDEwMDcxNDEwMjMzMwogSWdub3JlLXRoaXM6IDVkYjE3OGM1MTYyZGZkNGI3ODlj
Mzg4MWZmMmQwMjQyCl0gClt0eXBvICJkZXBlbmRhbnQiClNpbW9uIE1hcmxvdyA8bWFybG93c2RA
Z21haWwuY29tPioqMjAxMDA3MTQxMDA2MjgKIElnbm9yZS10aGlzOiBhY2M3NzMxNjkyNDUyMjM3
MzQ3ZjUzZTlmMzI2MWQ4ZQpdIApbdHlwbzogdmFyaWJhbGUgLT4gdmFyaWFibGUKU2ltb24gTWFy
bG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxNDEwMDU1NwogSWdub3JlLXRoaXM6IGY1
NzJhMDYwN2Y0NmE0MGRlODM5OTRjOTk4NzJkNTEKXSAKWzAqKnkgaXMgMSBpZiB5IGlzIDEsIGFu
ZCAwIG90aGVyd2lzZSAocHJldmlvdXNseTogMCoqeSBpcyB1bmRlZmluZWQpClNpbW9uIE1hcmxv
dyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MTQxMDA1NDQKIElnbm9yZS10aGlzOiA4NTZk
OGQzY2FhODYyMGU1NjQzOGQ2NWRiYmE1NGFlNApdIApbW0xULi5dIC0tPiBbTFQgLi5dICAodGhl
IGZvcm1lciB3b3VsZCBwYXJzZSBhcyBhIHF1YWxpZmllZCAiLiIpClNpbW9uIE1hcmxvdyA8bWFy
bG93c2RAZ21haWwuY29tPioqMjAxMDA3MTQxMDAwNTYKIElnbm9yZS10aGlzOiAzM2IxZTBjM2I0
OTIzYTc1MzUzMjllZjMxY2M3MjlmYgpdIApbInRoaXJkIGJ1bGxldCIgLT4gImZvdXJ0aCBidWxs
ZXQiClNpbW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MTQxMDAwMjQKIEln
bm9yZS10aGlzOiBlMjVjZDgxOGM4MmRkYWZlM2FkNzIxZTFjNjk5OTVlNwpdIApbZml4IGRvdWJs
ZS1AIGluIFByZWx1ZGUgY29kZQpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIw
MTAwNzE0MDk1MjI5CiBJZ25vcmUtdGhpczogYWQxNmMxNGY5OTNhNWUxNDgxNWM2YzIyMDg5NDQw
YzIKXSAKW3R5cGVzZXR0aW5nClNpbW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAx
MDA3MTQwOTQ2MTkKIElnbm9yZS10aGlzOiA3MGMxMDI5ZTYzOGI3YzFlZWYyODg5ZTA4NTdmNTcy
ZQpdIApbdHlwbzogYm9vbGVuIC0tPiAgYm9vbGVhbgpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdt
YWlsLmNvbT4qKjIwMTAwNzE0MDk0MjMwCiBJZ25vcmUtdGhpczogMzllMWY2ZjZhZWQwNDg3OTQ2
Y2QxMGVkMjNiNWU5MGEKXSAKW2NvbnNpc3RlbmN5OiBsb3dlci1jYXNlID09PiBsb3dlcmNhc2UK
U2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxNDA5NDE1OQogSWdub3Jl
LXRoaXM6IDcwZjgyMDMzM2FmN2Q0OTY2YWM2MjM3OTBiNDkyNTIzCl0gClthZGQgcGFyYWdyYXBo
IGFib3V0IGFsaWdubWVudApTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4qKjIwMTAw
NzE0MDg1OTE3CiBJZ25vcmUtdGhpczogNjdhNzNiODE1NDZjZjE1ZjI4YThiMDkyNjg5ZGI1NTcK
XSAKW2FkZCBJbnRQdHIvV29yZFB0cgpTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWlsLmNvbT4q
KjIwMTAwNzE0MDg1ODM4CiBJZ25vcmUtdGhpczogNDlkYmVkMzVlM2Q3ZWRjZTdjZDBlY2E1MTJi
MmY4MWUKXSAKW2FkZCBmaW5hbGl6ZUZvcmVpZ25QdHIKU2ltb24gTWFybG93IDxtYXJsb3dzZEBn
bWFpbC5jb20+KioyMDEwMDcxNDA4NTgxNAogSWdub3JlLXRoaXM6IDM5MWYwMTFjOWI0Yjk0ZDM3
OWViODY3NmFiN2NjOTE0Cl0gCltpbmNvcnBvcmF0ZSBjaGFuZ2VzIHRvIHRoZSBGRkkgc3BlYyBp
ZGVudGlmaWVkIGJ5IHRoZSBIMjAxMCBjb21taXR0ZWUKU2ltb24gTWFybG93IDxtYXJsb3dzZEBn
bWFpbC5jb20+KioyMDEwMDcxNDA4NTgwMwogSWdub3JlLXRoaXM6IDYyNjZmZDRhZGM1NjBlMzMx
NDI5NGE4YmRjODIyMjM1CiBodHRwOi8vaGFja2FnZS5oYXNrZWxsLm9yZy90cmFjL2hhc2tlbGwt
cHJpbWUvd2lraS9Gb3JlaWduRnVuY3Rpb25JbnRlcmZhY2UKXSAKW2ZpeCBNYWtlZmlsZSBidWcK
U2ltb24gTWFybG93IDxtYXJsb3dzZEBnbWFpbC5jb20+KioyMDEwMDcxMzE1NDYxNQogSWdub3Jl
LXRoaXM6IDFhN2Q1MjMxMWU5NzAwMThiZGMyMDI4NWVkNzdhYWU0Cl0gCltmb3JtYXR0aW5nClNp
bW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MDgwOTI0MzAKIElnbm9yZS10
aGlzOiBlYzkxZDNhOWIzNGFkM2IxNTkxYWY3OTQ4MWU1YzFhOQpdIApbcmVtb3ZlIHRoZSBjb250
ZW50cyBpbmRlbnRhdGlvbjsgbGF0ZXIgdGV4NGh0IGRvZXMgYSBiZXR0ZXIgam9iClNpbW9uIE1h
cmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MDcxNTEzMTcKIElnbm9yZS10aGlzOiA1
N2QwNjgwYTc3YTJiM2U3MGNkOWJlNDFhZDE5MzEyYwpdIApbYWRkIG1pc3NpbmcgZGlyClNpbW9u
IE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MDcxNDQ0MzYKIElnbm9yZS10aGlz
OiA5M2JjMzlhNmY1NGQ2YjBmNDMyMDdlYzI2YTQ1MjIxMgpdIApbZG9uJ3QgZGVwZW5kIG9uIGhh
c2tlbGwuaW5kClNpbW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAxMDA3MDcxNDQy
NTMKIElnbm9yZS10aGlzOiBjOTdkMDQxZDc3ZDNiMDYwOGQ4OTJlMzYzYWNjMDZkMApdIApbNi40
LjI6IGZpeCBmb3JtYXR0aW5nClNpbW9uIE1hcmxvdyA8bWFybG93c2RAZ21haWwuY29tPioqMjAx
MDA3MDcxMDIyNTgKIElnbm9yZS10aGlzOiA0NGVlNDQyNWU3ODVhMWFkNWQwOTZjZjllMGMzYjY4
YwpdIApbVEFHIEhhc2tlbGwgMjAxMCBGaW5hbApTaW1vbiBNYXJsb3cgPG1hcmxvd3NkQGdtYWls
LmNvbT4qKjIwMTAwNzA2MTIwMTI5CiBJZ25vcmUtdGhpczogYWFhNGU2NTQ4YjVlYWI3NGQ4NmJm
NzNkNWI5MDBlNGIKXSAKUGF0Y2ggYnVuZGxlIGhhc2g6CjZkMThjNTA3NGJjYTczZGNkNDM2ODIw
N2ZkZGU4YjM2MzU0Njg1ZTgK
--e0cb4e43b285dfe30b04a01f8fef--