Skip to content

Commit f259799

Browse files
bors[bot]Holger Lösken
bors[bot]
and
Holger Lösken
authored
Merge #92
92: Allowing different entities indexed into same index r=curquiza a=codedge While this PR got out of control 😄 and poses a bigger rewrite of the tests, it addresses multiple issues. ## Passing index names via command line Passing index name via command line now takes care if the name of the is already prefixed or not. Running - `bin/console meili:import --indices=prefix_indexName` - `bin/console meili:import --indices=indexName` leads to the same index creations which is `prefix_indexName` - given the `prefix` hold a value. ## Indexing different models into the same index See #90 Although MS supports importing different models into the same index, this was not easiliy possible using this package. The configuration in `meili_search.yml` would not allowe to set the same index name twice. Now you can do the following ```yaml meili_search: # Other... indices: - name: tags class: 'MeiliSearch\Bundle\Test\Entity\Tag' index_if: isPublic # Yes, we want to have links in the same index as tags # We just set the same index name 'tags' - name: tags class: 'MeiliSearch\Bundle\Test\Entity\Link' index_if: isSponsored ``` This would lead to the result of having models of `Tag` and `Link` inside the very same index `tags`. ⚠️ Make sure your identity key is unique across both models. Given you use a field `id`, the value of this needs to be unique across both models. @curquiza When the PR is approved and merged this should be documented inside a the wiki. ## Using aggregators See #90 You can now create an aggregator, register your models and will get an aggregated index created in MS. As the identity field of each model is used as identity field in the aggregated index... > Make sure your identity key is unique across both models. Given you use a field `id`, the value of this needs to be unique across both models. @curquiza When the PR is approved and merged this should be documented inside a the wiki. ## Fixing inconsistency See #89 Fixes some left of overs from refatoring `indexName` -> ìndexUid`. #89 is outdated with this PR here. ## Further changes - No separate folder to hold the test database in `tests/blog.sqlite`. Just use the `var/test.sqlite` folder which already holds cached/runtime data. - Reset the database and MS instance for each test via the `setUp` method. No need to have a dedicated `refreshDb` and `clearUp` call spread across the test. Each test should be atomic and not relying on former data. So 🗑️ everything 😃 - Update the `phpunit.xml` to the more recent format using the [`--migrate-configuration`](https://phpunit.readthedocs.io/en/9.5/textui.html?highlight=migrate#command-line-options) option. - Updating the test entities to make it easier with having properties inside the database and imported into MS. - Converting the config to a `Collection` object, see `illuminate/collection`. Makes it a lot easier to deal with, if you ask me 🙈 Co-authored-by: Holger Lösken <[email protected]>
2 parents 2643cb5 + 1611224 commit f259799

25 files changed

+516
-428
lines changed

‎.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,3 @@
44
.phpunit.result.cache
55
/var/
66
.php-cs-fixer.cache
7-
/tests/cache/blog.sqlite

‎composer.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
"php": "^7.4|^8.0",
2222
"ext-json": "*",
2323
"doctrine/doctrine-bundle": "^2.4",
24+
"illuminate/collections": "^8.47",
2425
"meilisearch/meilisearch-php": "^0.18",
2526
"symfony/filesystem": "^4.0 || ^5.0",
2627
"symfony/property-access": "^4.0 || ^5.0",

‎phpunit.xml.dist

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,27 @@
11
<?xml version="1.0" encoding="UTF-8"?>
2-
<phpunit colors="true" bootstrap="vendor/autoload.php" convertDeprecationsToExceptions="false">
3-
<php>
4-
<env name="KERNEL_CLASS" value="MeiliSearch\Bundle\Test\Kernel"/>
5-
<env name="APP_ENV" value="test"/>
6-
<env name="APP_DEBUG" value="false"/>
7-
<env name="MEILISEARCH_PREFIX" value="sf_phpunit_"/>
8-
<env name="MEILISEARCH_URL" value="http://127.0.0.1:7700"/>
9-
<env name="MEILISEARCH_API_KEY" value="masterKey"/>
10-
<env name="TRAVIS_JOB_NUMBER" value=""/>
11-
<env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>
12-
</php>
13-
<testsuites>
14-
<testsuite name="TestCase">
15-
<directory suffix=".php">tests/TestCase/</directory>
16-
</testsuite>
17-
</testsuites>
18-
<filter>
19-
<whitelist>
20-
<directory>src/</directory>
21-
<exclude>
22-
<file>src/DependencyInjection/MeiliSearchExtension.php</file>
23-
<file>src/Services/NullSearchService.php</file>
24-
</exclude>
25-
</whitelist>
26-
</filter>
2+
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" colors="true" bootstrap="vendor/autoload.php" convertDeprecationsToExceptions="false" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
3+
<coverage>
4+
<include>
5+
<directory>src/</directory>
6+
</include>
7+
<exclude>
8+
<file>src/DependencyInjection/MeiliSearchExtension.php</file>
9+
<file>src/Services/NullSearchService.php</file>
10+
</exclude>
11+
</coverage>
12+
<php>
13+
<env name="KERNEL_CLASS" value="MeiliSearch\Bundle\Test\Kernel"/>
14+
<env name="APP_ENV" value="test"/>
15+
<env name="APP_DEBUG" value="false"/>
16+
<env name="MEILISEARCH_PREFIX" value="sf_phpunit_"/>
17+
<env name="MEILISEARCH_URL" value="http://127.0.0.1:7700"/>
18+
<env name="MEILISEARCH_API_KEY" value="masterKey"/>
19+
<env name="TRAVIS_JOB_NUMBER" value=""/>
20+
<env name="SYMFONY_DEPRECATIONS_HELPER" value="disabled"/>
21+
</php>
22+
<testsuites>
23+
<testsuite name="TestCase">
24+
<directory suffix=".php">tests/TestCase/</directory>
25+
</testsuite>
26+
</testsuites>
2727
</phpunit>

‎src/Command/IndexCommand.php

Lines changed: 30 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44

55
namespace MeiliSearch\Bundle\Command;
66

7+
use Illuminate\Support\Collection;
78
use MeiliSearch\Bundle\SearchService;
89
use Symfony\Component\Console\Command\Command;
910
use Symfony\Component\Console\Input\InputInterface;
@@ -14,28 +15,47 @@
1415
*/
1516
abstract class IndexCommand extends Command
1617
{
18+
private string $prefix;
1719
protected SearchService $searchService;
1820

1921
public function __construct(SearchService $searchService, ?string $name = null)
2022
{
2123
$this->searchService = $searchService;
24+
$this->prefix = $this->searchService->getConfiguration()->get('prefix');
25+
2226
parent::__construct($name);
2327
}
2428

25-
protected function getEntitiesFromArgs(InputInterface $input, OutputInterface $output): array
29+
protected function getIndices(): Collection
30+
{
31+
return collect($this->searchService->getConfiguration()->get('indices'))
32+
->transform(function (array $item) {
33+
$item['name'] = $this->prefix.$item['name'];
34+
35+
return $item;
36+
});
37+
}
38+
39+
protected function getEntitiesFromArgs(InputInterface $input, OutputInterface $output): Collection
2640
{
27-
$entities = [];
28-
$indexNames = [];
41+
$indexNames = collect();
2942

3043
if ($indexList = $input->getOption('indices')) {
31-
$indexNames = \explode(',', $indexList);
44+
$list = \explode(',', $indexList);
45+
$indexNames = collect($list)->transform(function (string $item) {
46+
// Check if the given index name already contains the prefix
47+
if (false === strpos($item, $this->prefix)) {
48+
return $this->prefix.$item;
49+
}
50+
51+
return $item;
52+
});
3253
}
3354

3455
$config = $this->searchService->getConfiguration();
3556

36-
if ((0 === count($indexNames))
37-
&& !empty(array_keys($config['indices']))) {
38-
$indexNames = array_keys($config['indices']);
57+
if ((0 === count($indexNames)) && count($config->get('indices')) > 0) {
58+
$indexNames = $this->getIndices();
3959
}
4060

4161
if (0 === count($indexNames)) {
@@ -44,19 +64,8 @@ protected function getEntitiesFromArgs(InputInterface $input, OutputInterface $o
4464
);
4565
}
4666

47-
foreach ($indexNames as $name) {
48-
if (isset($config['indices'][$name])) {
49-
$entities[$name]['name'] = $config['indices'][$name]['class'];
50-
if (true === $input->hasOption('update-settings') && !empty($config['indices'][$name]['settings'])) {
51-
$entities[$name]['settings'] = $config['indices'][$name]['settings'];
52-
}
53-
} else {
54-
$output->writeln(
55-
'<comment>No index named <info>'.$name.'</info> was found. Check you configuration.</comment>'
56-
);
57-
}
58-
}
59-
60-
return $entities;
67+
return collect($this->getIndices())->reject(function (array $item) use ($indexNames) {
68+
return !in_array($item['name'], $indexNames->toArray(), true);
69+
});
6170
}
6271
}

‎src/Command/MeiliSearchClearCommand.php

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,18 +26,22 @@ protected function execute(InputInterface $input, OutputInterface $output): int
2626
{
2727
$indexToClear = $this->getEntitiesFromArgs($input, $output);
2828

29-
foreach ($indexToClear as $indexName => $entity) {
30-
$className = $entity['name'];
29+
/** @var array $index */
30+
foreach ($indexToClear as $index) {
31+
$indexName = $index['name'];
32+
$className = $index['class'];
3133
$array = $this->searchService->clear($className);
3234
if ('failed' === $array['status']) {
3335
$output->writeln('<error>Index <info>'.$indexName.'</info> couldn\'t be cleared</error>');
3436
} else {
35-
$output->writeln(
36-
'Cleared <info>'.$indexName.'</info> index of <comment>'.$className.'</comment> '
37-
);
37+
$output->writeln('Cleared <info>'.$indexName.'</info> index of <comment>'.$className.'</comment>');
3838
}
3939
}
4040

41+
if (0 === count($indexToClear)) {
42+
$output->writeln('Cannot clear index. Not found.');
43+
}
44+
4145
$output->writeln('<info>Done!</info>');
4246

4347
return 0;

‎src/Command/MeiliSearchImportCommand.php

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -45,43 +45,47 @@ protected function configure()
4545

4646
protected function execute(InputInterface $input, OutputInterface $output): int
4747
{
48-
$entitiesToIndex = $this->getEntitiesFromArgs($input, $output);
48+
$indexes = $this->getEntitiesFromArgs($input, $output);
4949
$config = $this->searchService->getConfiguration();
5050

51-
foreach ($entitiesToIndex as $key => $entity) {
52-
$entityClassName = $entity['name'];
51+
foreach ($indexes as $key => $index) {
52+
$entityClassName = $index['class'];
5353
if (is_subclass_of($entityClassName, Aggregator::class)) {
54-
unset($entitiesToIndex[$key]);
55-
$entitiesToIndex = array_merge(
56-
$entitiesToIndex,
54+
$indexes->forget($key);
55+
56+
$indexes = collect(array_merge(
57+
$indexes->toArray(),
5758
array_map(
5859
function ($entity) {
59-
return ['name' => $entity];
60+
return ['class' => $entity];
6061
},
6162
$entityClassName::getEntities()
6263
)
63-
);
64+
));
6465
}
6566
}
6667

67-
$entitiesToIndex = array_unique($entitiesToIndex, SORT_REGULAR);
68+
$entitiesToIndex = array_unique($indexes->toArray(), SORT_REGULAR);
6869

69-
foreach ($entitiesToIndex as $index => $entity) {
70-
$entityClassName = $entity['name'];
70+
/** @var array $index */
71+
foreach ($entitiesToIndex as $index) {
72+
$entityClassName = $index['class'];
7173
if (!$this->searchService->isSearchable($entityClassName)) {
7274
continue;
7375
}
7476

7577
$manager = $this->managerRegistry->getManagerForClass($entityClassName);
7678
$repository = $manager->getRepository($entityClassName);
7779

80+
$output->writeln('<info>Importing for index '.$entityClassName.'</info>');
81+
7882
$page = 0;
7983
do {
8084
$entities = $repository->findBy(
8185
[],
8286
null,
83-
$config['batchSize'],
84-
$config['batchSize'] * $page
87+
$config->get('batchSize'),
88+
$config->get('batchSize') * $page
8589
);
8690

8791
$responses = $this->formatIndexingResponse($this->searchService->index($manager, $entities));
@@ -97,9 +101,9 @@ function ($entity) {
97101
);
98102
}
99103

100-
if (!empty($entity['settings'])) {
101-
$indexInstance = $this->searchClient->getOrCreateIndex($config['prefix'].$index);
102-
foreach ($entity['settings'] as $variable => $value) {
104+
if (!empty($index['settings'])) {
105+
$indexInstance = $this->searchClient->getOrCreateIndex($index['name']);
106+
foreach ($index['settings'] as $variable => $value) {
103107
$method = sprintf('update%s', ucfirst($variable));
104108
if (false === method_exists($indexInstance, $method)) {
105109
throw new InvalidSettingName(sprintf('Invalid setting name: "%s"', $variable));
@@ -114,12 +118,14 @@ function ($entity) {
114118

115119
if ('failed' === $updateStatus['status']) {
116120
throw new UpdateException($updateStatus['error']);
121+
} else {
122+
$output->writeln('<info>Settings updated.</info>');
117123
}
118124
}
119125
}
120126

121127
++$page;
122-
} while (count($entities) >= $config['batchSize']);
128+
} while (count($entities) >= $config->get('batchSize'));
123129

124130
$manager->clear();
125131
}

‎src/DependencyInjection/Configuration.php

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,12 @@ public function getConfigTreeBuilder(): TreeBuilder
3838
->defaultValue('serializer')
3939
->end()
4040
->arrayNode('indices')
41-
->useAttributeAsKey('name')
4241
->arrayPrototype()
4342
->children()
43+
->scalarNode('name')
44+
->isRequired()
45+
->cannotBeEmpty()
46+
->end()
4447
->scalarNode('class')
4548
->isRequired()
4649
->cannotBeEmpty()

‎src/Engine.php

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ public function __construct(Client $client)
3232
public function index($searchableEntities): array
3333
{
3434
if ($searchableEntities instanceof SearchableEntity) {
35+
/** @var SearchableEntity[] $searchableEntities */
3536
$searchableEntities = [$searchableEntities];
3637
}
3738

@@ -70,10 +71,13 @@ public function index($searchableEntities): array
7071
public function remove($searchableEntities): array
7172
{
7273
if ($searchableEntities instanceof SearchableEntity) {
74+
/** @var SearchableEntity[] $searchableEntities */
7375
$searchableEntities = [$searchableEntities];
7476
}
7577

7678
$data = [];
79+
80+
/** @var SearchableEntity $entity */
7781
foreach ($searchableEntities as $entity) {
7882
$searchableArray = $entity->getSearchableArray();
7983
if (null === $searchableArray || 0 === \count($searchableArray)) {
@@ -137,6 +141,6 @@ public function search(string $query, string $indexUid, array $searchParams): ar
137141
*/
138142
public function count(string $query, string $indexName, array $searchParams): int
139143
{
140-
return (int) $this->client->index($indexName)->search($query, $searchParams)['nbHits'];
144+
return $this->client->index($indexName)->search($query, $searchParams)->getHitsCount();
141145
}
142146
}

‎src/Exception/InvalidEntityForAggregator.php

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,6 @@
44

55
namespace MeiliSearch\Bundle\Exception;
66

7-
/**
8-
* Class InvalidEntityForAggregator.
9-
*/
107
final class InvalidEntityForAggregator extends \LogicException
118
{
129
}

‎src/SearchService.php

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
namespace MeiliSearch\Bundle;
66

77
use Doctrine\Persistence\ObjectManager;
8+
use Illuminate\Support\Collection;
89

910
/**
1011
* Interface SearchService.
@@ -21,7 +22,7 @@ public function isSearchable($className): bool;
2122

2223
public function getSearchable(): array;
2324

24-
public function getConfiguration(): array;
25+
public function getConfiguration(): Collection;
2526

2627
/**
2728
* Get the index name for the given `$className`.
@@ -36,6 +37,8 @@ public function clear(string $className): array;
3637

3738
public function delete(string $className): ?array;
3839

40+
public function deleteByIndexName(string $indexName): ?array;
41+
3942
public function search(
4043
ObjectManager $objectManager,
4144
string $className,

‎src/SearchableEntity.php

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ public function __construct(
5151
$this->setId();
5252
}
5353

54-
public function getindexUid(): string
54+
public function getIndexUid(): string
5555
{
5656
return $this->indexUid;
5757
}

0 commit comments

Comments
 (0)