[feature] Fuzzy location creation #1067
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added WHOIS model with the required fields. Implemented a new celery task to fetch WHOIS details using the `geoip2` web service. In order to trigger the task whenever `last_ip` changes, using existing logic of `_changed_fields` to track the changes. Closes #1032 Closes #1033 Signed-off-by: DragnEmperor <[email protected]>
Added tests for checking if `fetch_whois_details` task is called properly or not, and for checking creation of WHOIS record for a device if the last_ip recorded is public. As now whois record is also fetched whenever fetching a device, there is an increase in query count in some of the tests. Closes #1045 Signed-off-by: DragnEmperor <[email protected]>
Closes #1037 Signed-off-by: DragnEmperor <[email protected]>
Using JSONField for address gives the flexibility to format address as per convenience. Added fallbacks for typical address fields in `fetch_whois_details`. Updated docs for WhoIs feature with steps to obtain and setup the required Credentials. Signed-off-by: DragnEmperor <[email protected]>
To reduce data redundancy for duplicate IPs, WhoIs model now uses IP address as primary key and devices are mapped to it via manual lookups. Updated the test cases to reflect the same. Formatting changes in WhoIs doc page and settings variables. Signed-off-by: DragnEmperor <[email protected]>
To improve modularity and flexibility, all WhoIs related code is kept under `whois` subdirectory under `config` module. This includes tests, tasks, on delete handlers as well. Added `cache` to prevent repeated lookups during multiple WhoIs details fetch during device listing. Signed-off-by: DragnEmperor <[email protected]>
Moved all tasks to service file to keep one source. Refactored the way whois_info is fetcehd for a device. Modified the order of whois checks to keep expensive checks at the bottom. As in views like `DeviceChecksumView` whole device instance is being cached, now org settings will be fetched from db rather than through device to maintain consistency. This has led to increase in queries in some test cases. Signed-off-by: DragnEmperor <[email protected]>
Org config settings are now cached to ensure DeviceChecksumview is not degraded. This will be invalidated on org settings update/delete which are rare. Refactored code and tests for readability. Signed-off-by: DragnEmperor <[email protected]>
Migrated all receivers related to WhoIs to the model class. Includes minor refactoring as well. Signed-off-by: DragnEmperor <[email protected]>
The imports are restructured to avoid app registry errors and aid whois task discovery. Signed-off-by: DragnEmperor <[email protected]>
There was an issue that the task was triggered for a device whose last_ip is updated but without checking if we already have WhoIs for the latest ip. Have added a check to return from the task if that is the case. Signed-off-by: DragnEmperor <[email protected]>
The checks in `_need_who_is_lookup` are split for better readability Signed-off-by: DragnEmperor <[email protected]>
Fuzzy location task should be executed when creating WhoIs for a device. It should also be executed when we have an existing WhoIs and attach the device to the same location as the similar ip's devices are attached. Closes #1034 Signed-off-by: DragnEmperor <[email protected]>
DragnEmperor
changed the title
DragnEmperor
moved this to In Progress
in [GSoC25] WHOIS Information and IP Address-Based Geolocation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist
Reference to Existing Issue
Closes #1034.
Description of Changes
Fuzzy locations task is added to allow creation of such locations when WhoIs is created for the same device. Also the task should attach the device to an existing location if already WhoIs is available for that device.
Please describe these changes.