Skip to content

Commit 325e1fe

Browse files
curquizacurquiza
authored
Merge pull request #8 from meilisearch/v9
Make the scraper use the MeiliSearch v9 version
2 parents db46fc2 + 02b7ed7 commit 325e1fe

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+210
-3031
lines changed

‎.dockerignore

Lines changed: 0 additions & 21 deletions
This file was deleted.

‎CONTRIBUTING.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,8 +18,8 @@ _WARNING:_ Please be aware that the scraper sends authentication headers to ever
1818
#### Basic HTTP:
1919

2020
Basic HTTP authentication is supported by setting these environment variables:
21-
- `DOCSEARCH_BASICAUTH_USERNAME`
22-
- `DOCSEARCH_BASICAUTH_PASSWORD`
21+
- `DOCS_SCRAPER_BASICAUTH_USERNAME`
22+
- `DOCS_SCRAPER_BASICAUTH_PASSWORD`
2323

2424
#### Cloudflare Access: Identity and Access Management
2525

‎Pipfile

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,10 @@ name = "pypi"
55

66
[packages]
77
Scrapy = "==1.5.0"
8-
pyperclip = "==1.5.27"
9-
ratelimit = "==1.4.1"
108
selenium = "==3.141.0"
11-
slacker = "==0.9.60"
12-
tldextract = "==2.1.0"
139
pytest = "==3.10.0"
10+
meilisearch = "==0.9.0"
1411
requests-iap = "==0.2.0"
15-
python-helpscout-v2 = "==1.0.1"
16-
meilisearch = "*"
17-
# pyhash = "*"
1812

1913
[dev-packages]
2014
pylint = "==2.3.1"

‎Pipfile.lock

Lines changed: 83 additions & 121 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,13 @@ _Will be generalized soon for all documentations_
88

99
This project supports Python 3.6+.
1010

11-
Set both environment variables `APPLICATION_ID` and `API_KEY`.
11+
Set both environment variables `MEILISEARCH_HOST_URL` and `MEILISEARCH_API_KEY`.
1212

1313
Then, run:
1414
```bash
1515
$ pipenv install
1616
$ pipenv shell
17-
$ ./docsearch run config/config.json
17+
$ ./docs_scraper run config/config.json
1818
```
1919

2020
_WIP_

‎cli/__init__.py

Lines changed: 0 additions & 4 deletions
This file was deleted.

‎cli/scripts/publish_docker_scraper_images.sh

Lines changed: 0 additions & 11 deletions
This file was deleted.

‎cli/src/__init__.py

Whitespace-only changes.

‎cli/src/commands/__init__.py

Lines changed: 0 additions & 15 deletions
This file was deleted.

‎cli/src/commands/abstract_build_docker.py

Lines changed: 0 additions & 27 deletions
This file was deleted.

‎cli/src/commands/abstract_command.py

Lines changed: 0 additions & 65 deletions
This file was deleted.

‎cli/src/commands/bootstrap_config.py

Lines changed: 0 additions & 102 deletions
This file was deleted.

‎cli/src/commands/build_docker_scraper.py

Lines changed: 0 additions & 41 deletions
This file was deleted.

‎cli/src/commands/deploy_config.py

Lines changed: 0 additions & 22 deletions
This file was deleted.

‎cli/src/commands/deploy_docker_scraper_images.py

Lines changed: 0 additions & 13 deletions
This file was deleted.

0 commit comments

Comments
 (0)