add xfeat+lighterglue and pairs from sequential

[pablovela5620]

Its been a little while since this PR got opened and I was hoping we could bring sequential matching!
#175

added exponential matching + loop closure using retrival based on this https://gitlab.ethz.ch/z-ricityslam/hloc_fork

I tested with South Building dataset since its sort of sequential (to make sure the loop closure worked).
Here is a video of the reconstruction + pairs generated. sequential matcher lead to a significant reduction in pairs created and lead to a good speedup in reconstruction!

Matching Type	Number of Pairs Generated	Reconstruction Time
Exhaustive Matching	8000	20 minutes
Vocabulary Tree Matching	6396	14 minutes
Sequential w/ Loop Closure	1543	9 minutes

sequential-reconstruction-2024-12-04_12.07.41.mp4

sequentail-matches-av1.mp4

I also included xfeat feature detector + lighterglue matcher from the great work over at https://github.com/verlab/accelerated_features

@sarlinpe let me know what I can do to help get this merged!

[robinlindh]

Hey, I have tried out the pairs_from_sequential, works very well. We use RGB data from mobile video captured at 10 FPS in a custom pipeline. Earlier we used pairs_from_poses (from some other PR on here earlier) and used 20 pairs. Now I use your pairs_from_sequential with:

    pairs_from_sequential.main(
        paths.sfm_pairs, references, None,
        window_size=6,
        quadratic_overlap=True,
        #use_loop_closure=False,
        #retrieval_path=None,
        #retrieval_interval=2,
        #num_loc=5
    )

The quadratic overlap with window size 6 gives us approximately the same range of nearby images as the 20 nearest poses, but using fewer pairs.
Running on a pretty big dataset we get a pretty big speed-up but still the same accuracy.
Haven't tried the retrieval-based loop closure yet but that will quite possibly improve our accuracy too so that looks very promising.

If anything, the window_size feels somewhat limiting currently as the same parameter controls both the number of pairs in direct sequence and the length of the furthest pair (growing quadratically). I just added some debug log on the first image to figure out what pair offsets I actually end up with as that would usually need to be handled with some care. In my case picking the window_size 6 gives a good sweet spot where the furthest pair is a few seconds away at our 10 FPS data. Would be nice to expose this a little bit more. Maybe a second parameter used for the number of quadratic overlap pairs, where the pairs are taken at +2, +4, +8 and so forth, starting from the END of the normal window size.

In any case, this is already very useful, so wanted to thank you for that!

[htchr]

Thank you @pablovela5620 for the pull request! @robinlindh have you been able to test more with your suggestions? Do you have any more insights on how this helps accuracy?

[robinlindh]

Didn't try further the thing I suggested for the quadratic pairs, we just kept using 5 for the window size which works good enough for our data that the main source of inaccuracy is not the pairs selection. I gave it a go with using the loop closure with retrieval too. That seemed to worked but I just tried it briefly and didn't see any direct improvement in the small test so haven't prioritized that more for now. We use superpoint + lightglue and the pairs from this PR, and initial poses from ARKit. Then triangulate and bundle adjust with points and some custom cost terms. That gives us pretty good accuracy for reasonable processing speeds so I don't think I will dig deeper into this PR. Definitely helpful to us, would recommend getting this merged too.

[robinlindh]

The xfeat lighterglue part I have not tested so I can't comment on that other than the theory of it looks very promising