Skip to content

Commit 6330ad1

Browse files
KocalKocal
committed
minor #2206 [Translator] Many performance improvements on the cache warming (Kocal)
This PR was squashed before being merged into the 2.x branch. Discussion ---------- [Translator] Many performance improvements on the cache warming | Q | A | ------------- | --- | Bug fix? | no | New feature? | no <!-- please update src/**/CHANGELOG.md files --> | Issues | Fix #... <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead --> | License | MIT The Symfony UX Translator cache warmer can be an long task, especially if your app has a lot of translations. I was able to identify some bottlenecks and fix them, with a dedicated commit for each of them. My test application have ~25k translation keys (3.119 keys * 8 locales). Of course that's a lot, but I believe some people needs to dump all their translations (e.g. if they build an SPA on Symfony). For "more classic" applications, it is recommended to [use dedicated domains for translations JS-side](https://symfony.com/bundles/ux-translator/current/index.html#configuring-the-dumped-translations). Anyway, in my application, I was able to reduce the Translator cache warming **from 2m11s** (higher than the reality, because of Blackfire profiling) **to 41s** (higher than the reality, because of Blackfire profiling): <img width="1339" alt="image" src="https://github.com/user-attachments/assets/98b5546b-0ba9-457d-b007-e8afdee7e5ac"> The vast majority of improvements come from better use of [the Symfony String component](https://symfony.com/doc/current/string.html). Blackfire profiles: 1. [Initial](https://app.blackfire.io/profiles/2e264f01-c3a6-46de-92fa-0856b2078010/graph) 2. [Re-using `s()` instances](https://app.blackfire.io/profiles/b4c5dea7-4309-4cb8-b99f-584ec3a4ddc9/graph) 3. [Removing useless `s()->slice()`](https://app.blackfire.io/profiles/f66b6bdf-39da-4d98-a966-781418b3a6f5/graph) 4. [Caching `$this->message->length()`](https://app.blackfire.io/profiles/1334e3e1-4405-44d1-a327-e446dad7d1d8/graph) as it will always return the same length 5. [Before/After](https://blackfire.io/profiles/compare/8a0df8d0-5d01-47bf-a064-bfed6783c5a3/graph): ~63% 🚀 Also, since the Translator component is still experimental, I've marked some classes as final and internal, as they should only be used internally. :) Commits ------- 02e1eb3 [Translator] Mark internal classes as internal f5e84c5 [Translator] Mark $message and $messageLength as readonly 2f13d73 [Translator] Improve performance, cache $this->message->length() to prevent unnecessary computations b582090 [Translator] Improve performance, replace array_reduce/array_keys by foreach 0bf9041 [Translator] Improve performance, remove (internally) duplicated call to `->slice($offset, 1)` 18eac63 [Translator] Improve performance, extract some logic from a loop from `IntlMessageParser#generateConstantName()` e053dd0 [Translator] Improve performance, reduce `s()` calls and inline `IntlMessageParser#offset()`
2 parents e51918f + 02e1eb3 commit 6330ad1

13 files changed

+59
-60
lines changed

‎src/Translator/src/Intl/ErrorKind.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/error.ts#L9-L77.
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class ErrorKind
2020
{

‎src/Translator/src/Intl/IntlMessageParser.php

Lines changed: 32 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -11,24 +11,30 @@
1111

1212
namespace Symfony\UX\Translator\Intl;
1313

14+
use Symfony\Component\String\AbstractString;
15+
1416
use function Symfony\Component\String\s;
1517

1618
/**
1719
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/parser.ts.
1820
*
19-
* @experimental
21+
* @internal
2022
*/
21-
class IntlMessageParser
23+
final class IntlMessageParser
2224
{
23-
private string $message;
25+
private readonly AbstractString $message;
26+
// Minor optimization, this avoid a lot of calls to `$this->message->length()`
27+
private readonly int $messageLength;
28+
2429
private Position $position;
2530
private bool $ignoreTag;
2631
private bool $requiresOtherClause;
2732

2833
public function __construct(
2934
string $message,
3035
) {
31-
$this->message = $message;
36+
$this->message = s($message);
37+
$this->messageLength = $this->message->length();
3238
$this->position = new Position(0, 1, 1);
3339
$this->ignoreTag = true;
3440
$this->requiresOtherClause = true;
@@ -112,14 +118,14 @@ private function parseMessage(int $nestingLevel, mixed $parentArgType, bool $exp
112118
*/
113119
private function parseTagName(): string
114120
{
115-
$startOffset = $this->offset();
121+
$startOffset = $this->position->offset;
116122

117123
$this->bump(); // the first tag name character
118124
while (!$this->isEOF() && Utils::isPotentialElementNameChar($this->char())) {
119125
$this->bump();
120126
}
121127

122-
return s($this->message)->slice($startOffset, $this->offset() - $startOffset)->toString();
128+
return $this->message->slice($startOffset, $this->position->offset - $startOffset)->toString();
123129
}
124130

125131
/**
@@ -365,7 +371,7 @@ private function parseIdentifierIfPossible(): array
365371
{
366372
$startingPosition = clone $this->position;
367373

368-
$startOffset = $this->offset();
374+
$startOffset = $this->position->offset;
369375
$value = Utils::matchIdentifierAtIndex($this->message, $startOffset);
370376
$endOffset = $startOffset + s($value)->length();
371377

@@ -445,9 +451,9 @@ private function parseArgumentOptions(
445451
);
446452

447453
// Extract style or skeleton
448-
if ($styleAndLocation && s($styleAndLocation['style'] ?? '')->startsWith('::')) {
454+
if ($styleAndLocation && ($style = s($styleAndLocation['style'] ?? ''))->startsWith('::')) {
449455
// Skeleton starts with `::`.
450-
$skeleton = s($styleAndLocation['style'])->slice(2)->trimStart()->toString();
456+
$skeleton = $style->slice(2)->trimStart()->toString();
451457

452458
if ('number' === $argType) {
453459
$result = $this->parseNumberSkeletonFromString(
@@ -663,7 +669,7 @@ private function parseSimpleArgStyleIfPossible(): array
663669
--$nestedBraces;
664670
} else {
665671
return [
666-
'val' => s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString(),
672+
'val' => $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString(),
667673
'err' => null,
668674
];
669675
}
@@ -676,7 +682,7 @@ private function parseSimpleArgStyleIfPossible(): array
676682
}
677683

678684
return [
679-
'val' => s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString(),
685+
'val' => $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString(),
680686
'err' => null,
681687
];
682688
}
@@ -735,7 +741,7 @@ private function tryParsePluralOrSelectOptions(
735741
return $result;
736742
}
737743
$selectorLocation = new Location($startPosition, clone $this->position);
738-
$selector = s($this->message)->slice($startPosition->offset, $this->offset() - $startPosition->offset)->toString();
744+
$selector = $this->message->slice($startPosition->offset, $this->position->offset - $startPosition->offset)->toString();
739745
} else {
740746
break;
741747
}
@@ -864,14 +870,9 @@ private function tryParseDecimalInteger(
864870
];
865871
}
866872

867-
private function offset(): int
868-
{
869-
return $this->position->offset;
870-
}
871-
872873
private function isEOF(): bool
873874
{
874-
return $this->offset() === s($this->message)->length();
875+
return $this->position->offset === $this->messageLength;
875876
}
876877

877878
/**
@@ -882,16 +883,14 @@ private function isEOF(): bool
882883
*/
883884
private function char(): int
884885
{
885-
$message = s($this->message);
886-
887886
$offset = $this->position->offset;
888-
if ($offset >= $message->length()) {
887+
if ($offset >= $this->messageLength) {
889888
throw new \OutOfBoundsException();
890889
}
891890

892-
$code = $message->slice($offset, 1)->codePointsAt(0)[0] ?? null;
891+
$code = $this->message->codePointsAt($offset)[0] ?? null;
893892
if (null === $code) {
894-
throw new \Exception("Offset {$offset} is at invalid UTF-16 code unit boundary");
893+
throw new \Exception("Offset {$offset} is at invalid UTF-16 code unit boundary.");
895894
}
896895

897896
return $code;
@@ -909,7 +908,7 @@ private function error(string $kind, Location $location): array
909908
'err' => [
910909
'kind' => $kind,
911910
'location' => $location,
912-
'message' => $this->message,
911+
'message' => $this->message->toString(),
913912
],
914913
];
915914
}
@@ -941,7 +940,7 @@ private function bump(): void
941940
*/
942941
private function bumpIf(string $prefix): bool
943942
{
944-
if (s($this->message)->slice($this->offset())->startsWith($prefix)) {
943+
if ($this->message->slice($this->position->offset)->startsWith($prefix)) {
945944
for ($i = 0, $len = \strlen($prefix); $i < $len; ++$i) {
946945
$this->bump();
947946
}
@@ -958,14 +957,13 @@ private function bumpIf(string $prefix): bool
958957
*/
959958
private function bumpUntil(string $pattern): bool
960959
{
961-
$currentOffset = $this->offset();
962-
$index = s($this->message)->indexOf($pattern, $currentOffset);
960+
$index = $this->message->indexOf($pattern, $this->position->offset);
963961
if ($index >= 0) {
964962
$this->bumpTo($index);
965963

966964
return true;
967965
} else {
968-
$this->bumpTo(s($this->message)->length());
966+
$this->bumpTo($this->messageLength);
969967

970968
return false;
971969
}
@@ -979,18 +977,18 @@ private function bumpUntil(string $pattern): bool
979977
*/
980978
private function bumpTo(int $targetOffset)
981979
{
982-
if ($this->offset() > $targetOffset) {
983-
throw new \Exception(\sprintf('targetOffset %s must be greater than or equal to the current offset %d', $targetOffset, $this->offset()));
980+
if ($this->position->offset > $targetOffset) {
981+
throw new \Exception(\sprintf('targetOffset "%s" must be greater than or equal to the current offset %d', $targetOffset, $this->position->offset));
984982
}
985983

986-
$targetOffset = min($targetOffset, s($this->message)->length());
984+
$targetOffset = min($targetOffset, $this->messageLength);
987985
while (true) {
988-
$offset = $this->offset();
986+
$offset = $this->position->offset;
989987
if ($offset === $targetOffset) {
990988
break;
991989
}
992990
if ($offset > $targetOffset) {
993-
throw new \Exception("targetOffset {$targetOffset} is at invalid UTF-16 code unit boundary");
991+
throw new \Exception("targetOffset {$targetOffset} is at invalid UTF-16 code unit boundary.");
994992
}
995993

996994
$this->bump();
@@ -1019,8 +1017,7 @@ private function peek(): ?int
10191017
}
10201018

10211019
$code = $this->char();
1022-
$offset = $this->offset();
1023-
$nextCodes = s($this->message)->codePointsAt($offset + ($code >= 0x10000 ? 2 : 1));
1020+
$nextCodes = $this->message->codePointsAt($this->position->offset + ($code >= 0x10000 ? 2 : 1));
10241021

10251022
return $nextCodes[0] ?? null;
10261023
}

‎src/Translator/src/Intl/Location.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L58-L61.
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class Location
2020
{

‎src/Translator/src/Intl/Position.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L53-L57.
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class Position
2020
{

‎src/Translator/src/Intl/SkeletonType.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L48-L51.
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class SkeletonType
2020
{

‎src/Translator/src/Intl/Type.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* Adapted from https://github.com/formatjs/formatjs/blob/590f1f81b26934c6dc7a55fff938df5436c6f158/packages/icu-messageformat-parser/types.ts#L8-L46.
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class Type
2020
{

‎src/Translator/src/Intl/Utils.php

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,10 @@
1111

1212
namespace Symfony\UX\Translator\Intl;
1313

14-
use function Symfony\Component\String\s;
14+
use Symfony\Component\String\AbstractString;
1515

1616
/**
17-
* @experimental
17+
* @internal
1818
*/
1919
final class Utils
2020
{
@@ -355,12 +355,12 @@ public static function fromCodePoint(int ...$codePoints): string
355355
return $elements;
356356
}
357357

358-
public static function matchIdentifierAtIndex(string $s, int $index): string
358+
public static function matchIdentifierAtIndex(AbstractString $s, int $index): string
359359
{
360360
$match = [];
361361

362362
while (true) {
363-
$c = s($s)->codePointsAt($index)[0] ?? null;
363+
$c = $s->codePointsAt($index)[0] ?? null;
364364
if (null === $c || self::isWhiteSpace($c) || self::isPatternSyntax($c)) {
365365
break;
366366
}

‎src/Translator/src/MessageParameters/Extractor/ExtractorInterface.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* @author Hugo Alliaume <[email protected]>
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
interface ExtractorInterface
2020
{

‎src/Translator/src/MessageParameters/Extractor/IntlMessageParametersExtractor.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
/**
1818
* @author Hugo Alliaume <[email protected]>
1919
*
20-
* @experimental
20+
* @internal
2121
*/
2222
final class IntlMessageParametersExtractor implements ExtractorInterface
2323
{

‎src/Translator/src/MessageParameters/Extractor/MessageParametersExtractor.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* @author Hugo Alliaume <[email protected]>
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class MessageParametersExtractor implements ExtractorInterface
2020
{

‎src/Translator/src/MessageParameters/Printer/TypeScriptMessageParametersPrinter.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
/**
1515
* @author Hugo Alliaume <[email protected]>
1616
*
17-
* @experimental
17+
* @internal
1818
*/
1919
final class TypeScriptMessageParametersPrinter
2020
{

‎src/Translator/src/TranslationsDumper.php

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -155,21 +155,18 @@ private function getTranslationsTypeScriptTypeDefinition(array $translationsByDo
155155
}
156156

157157
$parametersTypes[$domain] = $this->typeScriptMessageParametersPrinter->print($parameters);
158-
159158
$locales[] = $locale;
160159
}
161160
}
162161

162+
$typeScriptParametersType = [];
163+
foreach ($parametersTypes as $domain => $parametersType) {
164+
$typeScriptParametersType[] = \sprintf("'%s': { parameters: %s }", $domain, $parametersType);
165+
}
166+
163167
return \sprintf(
164168
'Message<{ %s }, %s>',
165-
implode(', ', array_reduce(
166-
array_keys($parametersTypes),
167-
fn (array $carry, string $domain) => [
168-
...$carry,
169-
\sprintf("'%s': { parameters: %s }", $domain, $parametersTypes[$domain]),
170-
],
171-
[],
172-
)),
169+
implode(', ', $typeScriptParametersType),
173170
implode('|', array_map(fn (string $locale) => "'$locale'", array_unique($locales))),
174171
);
175172
}
@@ -189,13 +186,14 @@ private function generateConstantName(string $translationId): string
189186
{
190187
static $alreadyGenerated = [];
191188

189+
$translationId = s($translationId)->ascii()->snake()->upper()->replaceMatches('/^(\d)/', '_$1')->toString();
192190
$prefix = 0;
193191
do {
194-
$constantName = s($translationId)->ascii()->snake()->upper()->replaceMatches('/^(\d)/', '_$1')->toString().($prefix > 0 ? '_'.$prefix : '');
192+
$constantName = $translationId.($prefix > 0 ? '_'.$prefix : '');
195193
++$prefix;
196-
} while (\in_array($constantName, $alreadyGenerated, true));
194+
} while ($alreadyGenerated[$constantName] ?? false);
197195

198-
$alreadyGenerated[] = $constantName;
196+
$alreadyGenerated[$constantName] = true;
199197

200198
return $constantName;
201199
}

‎src/Translator/tests/TranslationsDumperTest.php

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ public function testDump(): void
5757
export const NOTIFICATION_COMMENT_CREATED_DESCRIPTION = {"id":"notification.comment_created.description","translations":{"messages+intl-icu":{"en":"Your post \"{title}\" has received a new comment. You can read the comment by following <a href=\"{link}\">this link<\/a>","fr":"Votre article \"{title}\" a re\u00e7u un nouveau commentaire. Vous pouvez lire le commentaire en suivant <a href=\"{link}\">ce lien<\/a>"}}};
5858
export const POST_NUM_COMMENTS = {"id":"post.num_comments","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}}","fr":"{count, plural, one {# commentaire} other {# commentaires}}"},"foobar":{"en":"There is 1 comment|There are %count% comments","fr":"Il y a 1 comment|Il y a %count% comments"}}};
5959
export const POST_NUM_COMMENTS_1 = {"id":"post.num_comments.","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)","fr":"{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
60+
export const POST_NUM_COMMENTS_2 = {"id":"post.num_comments..","translations":{"messages+intl-icu":{"en":"{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)","fr":"{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
6061
export const SYMFONY_GREAT = {"id":"symfony.great","translations":{"messages":{"en":"Symfony is awesome!","fr":"Symfony est g\u00e9nial !"}}};
6162
export const SYMFONY_WHAT = {"id":"symfony.what","translations":{"messages":{"en":"Symfony is %what%!","fr":"Symfony est %what%!"}}};
6263
export const SYMFONY_WHAT_1 = {"id":"symfony.what!","translations":{"messages":{"en":"Symfony is %what%! (should not conflict with the previous one.)","fr":"Symfony est %what%! (ne doit pas rentrer en conflit avec la traduction pr\u00e9c\u00e9dente)"}}};
@@ -83,6 +84,7 @@ public function testDump(): void
8384
export declare const NOTIFICATION_COMMENT_CREATED_DESCRIPTION: Message<{ 'messages+intl-icu': { parameters: { 'title': string, 'link': string } } }, 'en'|'fr'>;
8485
export declare const POST_NUM_COMMENTS: Message<{ 'messages+intl-icu': { parameters: { 'count': number } }, 'foobar': { parameters: { '%count%': number } } }, 'en'|'fr'>;
8586
export declare const POST_NUM_COMMENTS_1: Message<{ 'messages+intl-icu': { parameters: { 'count': number } } }, 'en'|'fr'>;
87+
export declare const POST_NUM_COMMENTS_2: Message<{ 'messages+intl-icu': { parameters: { 'count': number } } }, 'en'|'fr'>;
8688
export declare const SYMFONY_GREAT: Message<{ 'messages': { parameters: NoParametersType } }, 'en'|'fr'>;
8789
export declare const SYMFONY_WHAT: Message<{ 'messages': { parameters: { '%what%': string } } }, 'en'|'fr'>;
8890
export declare const SYMFONY_WHAT_1: Message<{ 'messages': { parameters: { '%what%': string } } }, 'en'|'fr'>;
@@ -149,6 +151,7 @@ private static function getMessageCatalogues(): array
149151
'notification.comment_created.description' => 'Your post "{title}" has received a new comment. You can read the comment by following <a href="{link}">this link</a>',
150152
'post.num_comments' => '{count, plural, one {# comment} other {# comments}}',
151153
'post.num_comments.' => '{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)',
154+
'post.num_comments..' => '{count, plural, one {# comment} other {# comments}} (should not conflict with the previous one.)',
152155
],
153156
'messages' => [
154157
'symfony.great' => 'Symfony is awesome!',
@@ -178,6 +181,7 @@ private static function getMessageCatalogues(): array
178181
'notification.comment_created.description' => 'Votre article "{title}" a reçu un nouveau commentaire. Vous pouvez lire le commentaire en suivant <a href="{link}">ce lien</a>',
179182
'post.num_comments' => '{count, plural, one {# commentaire} other {# commentaires}}',
180183
'post.num_comments.' => '{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction précédente)',
184+
'post.num_comments..' => '{count, plural, one {# commentaire} other {# commentaires}} (ne doit pas rentrer en conflit avec la traduction précédente)',
181185
],
182186
'messages' => [
183187
'symfony.great' => 'Symfony est génial !',

0 commit comments

Comments
 (0)