Perl6's newline conversion behavior seems to be by design

I found out that RT #126881 was added to the list of tickets blocking the 2015 release of Perl6.

I am happy Perl6 developers noticed the problem. But, I am worried that it may be too late to fix this issue in a way that leads to perl6 handling EOL conversion sanely at I/O boundaries.

I first noticed Perl6 was acting as if different EOL conventions did not exist about a year ago:

I still have no idea what should be fixed and where — except that I am uncomfortable with what seems to be a “everything is Unix” assumption in perl 6.

It seems, from a distance, to a person not familiar at all with how Perl6 internals development works, that it was too hard to fix the issue properly by doing EOL conversion at input-output boundaries. Therefore, they seem to have settled for manipulating string literals in a program as this commit seems to indicate:

>     -    method backslash:sym($/) { make "\n" }

> + method backslash:sym($/) { make nqp::unbox_s($*W.find_symbol([‘$?NL’])); } >
See also this commit.

I haven’t been able to locate any discussion leading to the decision to handle EOL conventions by changing string literals in a program.

This decision has unintended (at least I hope they are unintended) consequences. For example: use v6; my $x = "1\n2"; my $y ="1 2"; ($x eq $y).perl.say; outputs Bool::False on Windows using perl6 built with Visual Studio if the source file uses Unix EOLs. Presumably, on the same machine, if I were to be able to build a perl6 with Cygwin, it would work as expected.

And, no, perl does not care what EOL convention the source file uses whether I am using a perl built with Cygwin or Visual Studio (assuming you take care with your hashbang line — e.g. by terminating the argument list explicitly as in, say, #!/usr/local/bin/perl-5.19.6 --).

They do EOL conversion on input-output boundaries.

One might recommend to stick a use newline :lf in my source files. That is hardly a solution. There are many places where you want to transparently read files containing CRLF EOLs etc.

Also, consider these:

C:\> perl6 -MDigest::MD5 -e "Digest::MD5.new.md5_hex(qq[Hello\n]).say"
b73bf7d3ba1a517644661bc4bcd85f9a

C:\> perl6 -MDigest::MD5 -e "Digest::MD5.new.md5_hex(qq[Hello\xa]).say"
09f7e02f1290be211da707a266f153b3

C:\> perl6 -MDigest::MD5 -e "use newline :lf; Digest::MD5.new.md5_hex(qq[Hello\n]).say"
09f7e02f1290be211da707a266f153b3

C:\> perl -MDigest::MD5=md5_hex -E "say md5_hex(qq[Hello\n])"
09f7e02f1290be211da707a266f153b3

C:\> perl -MDigest::MD5=md5_hex -E "say md5_hex(qq[Hello\r\n])"
b73bf7d3ba1a517644661bc4bcd85f9a

In case you forgot, these are LITERAL STRINGS IN A PROGRAM!.

I am afraid it is not possible to fix this without changing the entire I/O plumbing of Perl6. Of course, I am not really qualified to make any statements about the internals of Perl6 as I haven’t followed it at all through the years.

PS: You can discuss this issue on /r/perl6.