-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Implement object-based token_get_all() alternative #5176
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
6284c9f
Implement TOKEN_AS_OBJECT mode
nikic b41e1ca
Locally intern strings in token_get_all()
nikic a1f7db4
Add PhpToken methods
nikic d310410
Introduce PhpToken::getAll() instead of flag
nikic 9ad94a8
Add PhpToken::__construct()
nikic 18e53f4
Allow extending PhpToken
nikic 1b6b962
Fix bool/int confusion
nikic df5fc1b
Fix initialization of extra properties
nikic b782726
Update constants and check abstract classes
nikic a852dbd
Add SKIPIF sections
nikic 8b58f41
Return null from getTokenName() if token unknown
nikic 24e5967
Add __toString() method
nikic 10590f6
Locally intern strings in token_get_all()
nikic ab7c362
Add test
nikic File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
--TEST-- | ||
PhpToken constructor | ||
--SKIPIF-- | ||
<?php if (!extension_loaded("tokenizer")) print "skip tokenizer extension not enabled"; ?> | ||
--FILE-- | ||
<?php | ||
|
||
$token = new PhpToken(300, 'function'); | ||
var_dump($token); | ||
$token = new PhpToken(300, 'function', 10); | ||
var_dump($token); | ||
$token = new PhpToken(300, 'function', 10, 100); | ||
var_dump($token); | ||
|
||
?> | ||
--EXPECT-- | ||
object(PhpToken)#1 (4) { | ||
["id"]=> | ||
int(300) | ||
["text"]=> | ||
string(8) "function" | ||
["line"]=> | ||
int(-1) | ||
["pos"]=> | ||
int(-1) | ||
} | ||
object(PhpToken)#2 (4) { | ||
["id"]=> | ||
int(300) | ||
["text"]=> | ||
string(8) "function" | ||
["line"]=> | ||
int(10) | ||
["pos"]=> | ||
int(-1) | ||
} | ||
object(PhpToken)#1 (4) { | ||
["id"]=> | ||
int(300) | ||
["text"]=> | ||
string(8) "function" | ||
["line"]=> | ||
int(10) | ||
["pos"]=> | ||
int(100) | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
--TEST-- | ||
Extending the PhpToken class | ||
--SKIPIF-- | ||
<?php if (!extension_loaded("tokenizer")) print "skip tokenizer extension not enabled"; ?> | ||
--FILE-- | ||
<?php | ||
|
||
$code = <<<'PHP' | ||
<?PHP | ||
FUNCTION FOO() { | ||
ECHO "bar"; | ||
} | ||
PHP; | ||
|
||
class MyPhpToken extends PhpToken { | ||
public int $extra = 123; | ||
|
||
public function getLoweredText(): string { | ||
return strtolower($this->text); | ||
} | ||
} | ||
|
||
foreach (MyPhpToken::getAll($code) as $token) { | ||
echo $token->getLoweredText(); | ||
|
||
if ($token->extra !== 123) { | ||
echo "Missing property!\n"; | ||
} | ||
} | ||
|
||
?> | ||
--EXPECT-- | ||
<?php | ||
function foo() { | ||
echo "bar"; | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
--TEST-- | ||
PhpToken extensions that throw during construction | ||
--SKIPIF-- | ||
<?php if (!extension_loaded("tokenizer")) print "skip tokenizer extension not enabled"; ?> | ||
--FILE-- | ||
<?php | ||
|
||
class MyPhpToken1 extends PhpToken { | ||
public $extra = UNKNOWN; | ||
} | ||
|
||
try { | ||
var_dump(MyPhpToken1::getAll("<?php foo")); | ||
} catch (Error $e) { | ||
echo $e->getMessage(), "\n"; | ||
} | ||
|
||
abstract class MyPhpToken2 extends PhpToken { | ||
} | ||
|
||
try { | ||
var_dump(MyPhpToken2::getAll("<?php foo")); | ||
} catch (Error $e) { | ||
echo $e->getMessage(), "\n"; | ||
} | ||
|
||
?> | ||
--EXPECT-- | ||
Undefined constant 'UNKNOWN' | ||
Cannot instantiate abstract class MyPhpToken2 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
--TEST-- | ||
Check that the PhpToken constructor is final | ||
--SKIPIF-- | ||
<?php if (!extension_loaded("tokenizer")) print "skip tokenizer extension not enabled"; ?> | ||
--FILE-- | ||
<?php | ||
|
||
class MyPhpToken extends PhpToken { | ||
public function __construct() { | ||
} | ||
} | ||
|
||
?> | ||
--EXPECTF-- | ||
Fatal error: Cannot override final method PhpToken::__construct() in %s on line %d |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.