Support internal null characters in NSString initializers [SR-2225]

[xwu]

This PR aims to fix a discrepancy in NSString between corelibs-foundation and Apple Foundation. Namely, initialization of an NSString from a byte sequence currently calls CFStringCreateWithBytes(_:_:_:_:_:), which cannot accept null characters. Instead, switch to using CFStringCreateFromExternalRepresentation(_:_:_:).

Resolves SR-2225.

Note: I'm having trouble getting TestFoundation to compile in Xcode at the moment, but I've checked that previous builds of corelibs-foundation fail the test supplied, while Apple Foundation passes that test.

[DaveLiu888]

@xwu how are you building AppFoundation and Tests? Which snapshot are you using and Xcode version. I am having trouble building and testing as well after i update to master from 7/29?

[parkera]

The story of what happens on Darwin here is fairly complicated, which is probably why this was different in the first place, but it boils down to this:

- initWithData:(NSData *)data encoding:(NSStringEncoding)encoding {
    return (id)CFStringCreateFromExternalRepresentation(kCFAllocatorDefault, (CFDataRef)data, CFStringConvertNSStringEncodingToEncoding(encoding));
}

So this change looks correct.

[parkera]

I debugged this a bit. I'm not finished yet, but I think the discrepancy has to do with the bridging happening after the CFString initialization. The CFString call returns the same result on both platforms (reading past the NULL).

[xwu]

Wow, that's neat. CFStringGetLength() is showing that there's more characters returned from CFStringCreateWithBytes than what's printed after bridging.

For now, I'll create another PR with the change for init?(data:encoding:). I can think of maybe some hacky ways to get the rest to work, but since you're already debugging I'll defer to you.

[xwu]

@parkera I bet you've found it too, but things go awry during bridging because, if type(of: x) == _NSCFString.self, bridging is performed by calling CFStringGetCStringPtr(_:_:) followed by String(cString:). If the cost isn't too great, one solution would be to use this result only if result.utf16.count == CFStringGetLength(cf).

[xwu]

@DaveLiu888 Sorry for the delayed reply. I put off hacking on corelibs-foundation directly for a few days. Everything is now building fine for me with the 8/7 snapshot (although I did have to disable the NSString reflection test, which segfaults).

[parkera]

I actually didn't get too much further on this (we've been busy getting all of those API changes landed).

[DaveLiu888]

@xwu thanks for the update yea everything is working for me as well. 👍

[xwu]

@parkera OK, I will forge ahead and see how far I get with this. Since the solution is taking a different turn, I'll close this PR and open another one once I have something presentable. I'll roll in the init?(data:encoding:) change there as well.

[parkera]

Cool, thanks.