Simple portions of the Windows Port

[compnerd]

This is set of changes that are simple and non-controversial for the Windows port. This allows us to work on a smaller set of changes that have more interesting consequences (like threading and locking) without having to worry about the additional context of these patches. It also makes it easier to work on the port as it reduces the likelihood of conflicts.

[compnerd]

@swift-ci please test

[compnerd]

CC: @parkera @millenomi
As discussed on #1783, this splits off the "simple" parts of the change so that we can focus on the possibly worrisome patches.

[compnerd]

hahaha, __unused in the header is causing the issue with glibc!

[compnerd]

@swift-ci please test

[compnerd]

@swift-ci please test Linux platform

[jakepetroules]

👍

[jakepetroules]

This needs to use _wsopen_s on Windows, not open.

Unfortunately, the story with C APIs is the same as with the *A vs *W variants of Win32 functions -- you must use the wide-character variants for Unicode to work at all.

Please go through and audit any C API calls through SCF and make sure that the _w variants are used on Windows. It's probably a good idea to separate all C APIs which take strings into an abstraction layer somewhere.

I'd do something like this. Have an abstraction layer which provides both UTF-8 and UTF-16 variants of each C API that we use:

int scf_open(const char *);
int scf_wopen(const uint16_t *, size_t len); // do NOT use wchar_t, it's different size on different platforms
int scf_cfopen(CFStringRef);

In the implementations (pseudocode):

int scf_open(const char *s) {
    if (windows) {
        // We have a UTF-8 string and are on Windows, convert to UTF-16
        auto x = convert_to_utf16(s);
        return scf_wopen(x.ptr, x.length);
    } else {
        // We have a UTF-8 string and are on UNIX, no conversion needed
        return ::open(s);
    }
}

int scf_wopen(const uint16_t *s, size_t len) {
    if (windows) {
        // We have a UTF-16 string and are on Windows, no conversion needed
        return ::_wsopen_s(s);
    } else {
        // We have a UTF-16 string and are on UNIX, convert to UTF-8
        return scf_open(convert_to_utf8(s));
    }
}

int scf_cfopen(CFStringRef s) {
    if (windows) {
        return scf_wopen(CFStringGetCharactersPtr(s), CFStringGetLength(s));
    } else {
        char filename[PATH_MAX];
        CFStringGetCString(s, filename, PATH_MAX, kCFStringEncodingUTF8);
        return scf_open(filename);
    }
}

That way you can call either the UTF-8 or UTF-16 variant depending on what kind of string you have coming from the input, and it'll do the right thing on each platform in order to minimize conversions.

For example, when you have a CFString and need to pass it to a native Win32 API, you can actually avoid the UTF-8 conversion you're doing here (which again is currently broken due to UTF-8 not being supported).

[compnerd]

open is not open, it is a macro, and replaces it with _NS_open, and it does go through _wopen_s. I believe that all of the ones have already been replaced with _NS_* equivalents.

[jakepetroules]

Ah, didn't notice that. However, the approach I suggested would still be more performant on Windows because you're not converting (in many cases) from UTF-16 to UTF-8 right back to UTF-16 again. Having two roundtrips like that every time a Win32 API is called is going to add up.

But performance improvements like that can be done in a later change, so I rescind my "changes requested" for now. :)

[compnerd]

Yeah, I agree, that would be a great follow up!

code is correct, suggested perf improvements can be done in a later patch

[compnerd]

@swift-ci please test

[compnerd]

I think that I can improve some of the Windows implementations, but, lets get the basic bits in and do that in a follow up.