Test failures and directory separators: Take N

Testing Inline::CPP version 0.72 on 64-bit Windows 8.1 using perl built with Microsoft Visual Studio 2013 Community Edition results in a number of test failures:

Test Summary Report
-------------------
t\08cppguess.t                      (Wstat: 0 Tests: 2 Failed: 0)
  TODO passed:   1
t\classes\07conflict_avoid.t        (Wstat: 65280 Tests: 6 Failed: 6)
  Failed tests:  1-6
  Non-zero exit status: 255
  Parse errors: No plan found in TAP output
t\grammar\11minhrt.t                (Wstat: 0 Tests: 38 Failed: 0)
  TODO passed:   23-24, 28-31
Files=52, Tests=269, 270 wallclock secs ( 0.31 usr +  0.13 sys =  0.44 CPU)
Result: FAIL
Failed 1/52 test programs. 6/269 subtests failed.
NMAKE : fatal error U1077: 'C:\opt\perl-5.22.0\bin\perl.exe' : return code '0xff'
Stop.

Focusing on the problematic test script yields, among other things:

C:\..\Inline-CPP> prove -vb t\classes\07conflict_avoid.t
t\classes\07conflict_avoid.t .. Unrecognized escape \s passed through at (eval 13) line 3.
Unrecognized escape \A passed through at (eval 13) line 3.
Unrecognized escape \T passed through at (eval 13) line 3.
Unrecognized escape \T passed through at (eval 13) line 3.
…

Anytime you see messages starting with "Unrecognized escape ...", it is a safe bet to assume that someone is eval’ing or interpolating Windows style file paths willy nilly.

While internally Windows APIs can handle either type of directory separator, you must also be ready to handle proper, platform-specific paths in your application.

Let’s take a look at one of the sources of problems in Inline::CPP’s handling and use of paths.

There is indeed an open issue on GitHub regarding similar errors. Mine are somewhat different because Windows 8 gave up on Documents and Settings and now uses directories without spaces in them to user-specific locations.

So, let’s examine where the errors stem from.

The test script, t\classes\07conflict_avoid.t opens with:

use File::Temp 'tempdir';

my $tdir = tempdir( CLEANUP => 1 );

The variable $tdir now contains a path. Depending on whether perl was built with gcc or cl and other variables which I cannot list exhaustively, it may or may not contain Windows style directory separators, and depending on Windows version and user settings, it may or may not contain spaces. Of course, paths can contain spaces even on *nix based operating systems, but Microsofts choice of “Documents and Settings” at one point made it almost certain that your software would encounter paths with spaces on Windows.

This may or may not be a problem, depending on how the variable $tdir is used.

A few lines below, we see the one example of its use:

my $foo__qux__myclass = <<"EOCPP";
#include "$tdir/Foo__Bar__MyClass.c"
  class Foo__Qux__MyClass {
    private:
      int a;
    public:
      Foo__Qux__MyClass() :a(20) {}
      int fetch () { return a; }
      int other_fetch () { Foo__Bar__MyClass mybar; return mybar.fetch(); }
  };
EOCPP

Is this the source of the problem?

No.

Sure, this may create a C++ source file with the line:

#include "C:\Documents and Settings\auser\Local Settings\Temp/Foo__Bar__MyClass.c"

but, AFAIK, both gcc and cl can handle that.

Below that, we have:

open my $fha, '>', "$tdir/Foo__Bar__MyClass.c"
  or die "Can't open file '$tdir/Foo__Bar__MyClass.c' for writing $!";

Is that the problem?

Again, no, because the file name is just passed to a Windows API that can handle both types of directory separators in paths.

In fact, I did not have to worry about these two uses, but I wanted to mention them to point out that there are occasions you are completely OK with using forward slashes as directory separators on Windows.

As you might have surmised from the mention of eval in the warnings, the problem is with the following chunk of Perl code, and its failure to anticipate backslashes and spaces in a file path:

eval qq[
  use Inline CPP =>
    "$tdir/Foo__Qux__MyClass.c", 
    filters   => 'Preprocess', 
    namespace => 'Foo', 
    classes   => {
      'Foo__Bar__MyClass' => 'Bar::MyClass',
      'Foo__Qux__MyClass' => 'Qux::MyClass'
    };
];

You might find it hard to imagine what happens after interpolating $tdir and then eval’ing the resulting string. I am used to it, encountering the assumption that all directory separators are the same and file names don’t contain spaces wayyyyy too often.

Also related: Given that the first backslash-escape in the string is \U, why don’t we get a warning for that?

Well, as $ perldoc perlrebackslash will tell you, \U means something to Perl:

C:\> perl -E "say qq{\Usinan}"
SINAN

Now the rest of the error messages from this test script make sense.

We have generated our C++ source, put it in a file in a temporary directory, but when it came time to tell Inline::CPP, we messed up a little.

What to do?

Fortunately, the simplest fix is a rather simple fix.

Use single quotes to surround the path you are interpolating into the string to be eval’ed.

That is, instead of:

eval qq[
  use Inline CPP =>
    "$tdir/Foo__Qux__MyClass.c", 
# ...

use

eval qq[
  use Inline CPP =>
    '$tdir/Foo__Qux__MyClass.c',
# ...

Of course, this assumes the path won’t end in a backslash.

That’s it!

PS: Pull request.

PPS: No comments yet, but you can discuss this post on /r/perl.