gh-126505: Fix bugs in compiling case-insensitive character classes

[serhiy-storchaka]

• upper-case non-BMP character was ignored
• the ASCII flag was ignored when matching a character range whose upper bound is beyond the BMP region

• Issue: SRE ignores the ASCII flag on character ranges with non-BMP upper bound #126505

[miss-islington-app]

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-126689 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-126690 is a backport of this pull request to the 3.12 branch.

[wjssz]

    if fixup: # IGNORECASE and not LOCALE
        av = fixup(av)
        charmap[av] = 1

The charmap index, sometimes is fixup(av), sometimes is fixup(fixup(av)). Perhaps the result is the same, but it's not a good style.

You can fixup(av) in below, so that the code above doesn't need to be modified:

			if op is RANGE:
				if fixes: # not ASCII
					op = RANGE_UNI_IGNORE
				hascased = True
			else:
				assert op is LITERAL
				if not hascased and iscased(av):
					hascased = True
				av = fixup(av)  # <- fixup() here
		tail.append((op, av))
	break

[serhiy-storchaka]

fixup() is only called twice when we get a non-BMP character when len(charmap) == 256. Are you talking about this situation? Than it occurs only once, and since fixup(fixup(av)) is equal to fixup(av), it does not matter. In contrary, if write the code in a way you proposed (this is how I wrote it initially too), fixup(av) will be called twice for every non-BMP literal. The difference may be not large, but this is clearly less efficient.

[wjssz]

Ok, this is not an important issue.