Skip to content

Add tests for contentSchema #418

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 17, 2020

Conversation

ChALkeR
Copy link
Member

@ChALkeR ChALkeR commented Jul 14, 2020

These were entirely missing.
draft2019-09 only, in optional.

Top to bottom:

> Buffer.from('eyJmb28iOiAiYmFyIn0K', 'base64').toString()
'{"foo": "bar"}\n'
> Buffer.from('eyJib28iOiAyMCwgImZvbyI6ICJiYXoifQ==', 'base64').toString()
'{"boo": 20, "foo": "baz"}'
> Buffer.from('eyJib28iOiAyMH0=', 'base64').toString()
'{"boo": 20}'
> Buffer.from('e30=', 'base64').toString()
'{}'
> Buffer.from('W10=', 'base64').toString()
'[]'
> Buffer.from('ezp9Cg==', 'base64').toString()
'{:}\n'

@ssilverman ssilverman requested a review from Julian November 17, 2020 04:05
Copy link
Member

@ssilverman ssilverman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried them out. They look fine to me.

@Julian Julian merged commit 3b79a45 into json-schema-org:master Nov 17, 2020
@karenetheridge
Copy link
Member

To clarify, we do NOT expect that these should pass with a default-configured evaluator, correct? It might be helpful to put a comment in the tests explaining this -- I can imagine some implementor might assume that every test is supposed to pass, and write code accordingly, without looking closely at the spec. Perhaps the same tests can also be added to the main directory, but with every schema evaluating as valid.

The spec says:

They do not function as validation assertions; a malformed string-encoded document MUST NOT cause the containing instance to be considered invalid. (https://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.8.1)

and

Due to security and performance concerns, as well as the open-ended nature of possible content types, implementations MUST NOT automatically decode, parse, and/or validate the string contents by default. (https://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.8.2)

@karenetheridge
Copy link
Member

(edit: this file already existed in the test suite, so my concerns are not new with this PR.)

@Julian
Copy link
Member

Julian commented Nov 18, 2020

I think that's my understanding too yeah. I assume the intention is similar to format -- the tests are there for whoever decides to implement an additional optional mode to turn them into assertions.

But yeah certainly clarifications welcome there.

@ssilverman
Copy link
Member

ssilverman commented Nov 18, 2020

If, according to https://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.8.1, contentSchema doesn't function as a validation assertion, then should these additional tests always validate as true? i.e. all tests should have "valid": true? Or is that only true for validators that don't turn the "content" vocabulary on? Or only for validators that treat "content" keywords as assertions? Is that even allowed?

i.e. does this sentence:

They do not function as validation assertions; a malformed string-encoded document MUST NOT cause the containing instance to be considered invalid.

apply to all validators, those that don't have the "content" vocabulary enabled, those that have that vocabulary enabled, or those that have the vocabulary AND are configured to treat these keywords as assertions?

How is this similar or not similar to "format"?

@Julian
Copy link
Member

Julian commented Nov 18, 2020

So looking at the spec myself, yeah I guess it doesn't say as it does for format that implementations may treat these as assertions, so it indeed looks like these aren't correct (and that someone cannot implement validation on them and call it draft2019-09). So yeah please revert or change to all true, if the two of you can agree?

Certainly an implementation can choose to implement validation for these, same as they can implement whatever additional behavior they choose, but yeah the official draft doesn't include doing so.

@karenetheridge
Copy link
Member

FWIW Henry says "please don't" - see #specification on slack

@karenetheridge
Copy link
Member

When the tests are all "valid": true they may be safely located outside the optional directory.

@ssilverman
Copy link
Member

@karenetheridge #448 (I’m about to move those to non-optional.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants