Feature/utf8 support #13
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chrisradek
reviewed
Jun 5, 2017
| @@ -0,0 +1,65 @@ | |||
| export function fromUtf8(input: string): Uint8Array { | |||
There was a problem hiding this comment.
Is this code taken from the Buffer module?
There was a problem hiding this comment.
This is mostly cribbed from https://github.com/google/closure-library/blob/master/closure/goog/crypt/crypt.js#L110, though I modernized it a bit. I should include a mention of that in a comment.
Merged
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds cross-platform UTF-8 support, since JS strings are UCS-2 but string-to-binary conversions always assume UTF-8 encoding in Node. Also, we want to support string-to-binary conversions and the reverse for all SDK users, not just those whose written languages are confined to the basic multilingual plane of UCS-2/UTF-16.
In node, this functionality is provided by the
buffermodule, and some browsers support a standard that defines globally availableTextEncoderandTextDecoderobjects. In other browsers and elsewhere, we have to handle the conversion manually (as is done by the V2 SDK via thebuffermodule).