Commit Graph

10 Commits (main)

Author SHA1 Message Date
Michael Norris eff0898a13 Enable linting: lint config changes plus arc lint command (#3966)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3966

This actually enables the linting.

Manual changes:
- tools/arcanist/lint/fbsource-licenselint-config.toml
- tools/arcanist/lint/fbsource-lint-engine.toml

Automated changes:
`arc lint --apply-patches --take LICENSELINT --paths-cmd 'hg files faiss'`

Reviewed By: asadoughi

Differential Revision: D64484165

fbshipit-source-id: 4f2f6e953c94ef6ebfea8a5ae035ccfbea65ed04
2024-10-22 09:46:48 -07:00
Matthijs Douze 32f0e8cf92 Generalize ResultHanlder, support range search for HNSW and Fast Scan (#3190)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3190

This diff adds more result handlers in order to expose them externally.
This enables range search for HSNW and Fast Scan, and nprobe parameter support for FastScan.

Reviewed By: pemazare

Differential Revision: D52547384

fbshipit-source-id: 271da5ffea6411df3d8e50641abade18bd7b774b
2024-01-11 11:46:30 -08:00
Matthijs Douze b109d086a2 Search and return codes (#3143)
Summary:
This PR adds a functionality where an IVF index can be searched and the corresponding codes be returned. It also adds a few functions to compress int arrays into a bit-compact representation.

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3143

Test Plan:
```
buck test //faiss/tests/:test_index_composite -- TestSearchAndReconstruct

buck test //faiss/tests/:test_standalone_codec -- test_arrays
```

Reviewed By: algoriddle

Differential Revision: D51544613

Pulled By: mdouze

fbshipit-source-id: 875f72d0f9140096851592422570efa0f65431fc
2023-11-25 13:57:25 -08:00
Matthijs Douze 9db182460c Relax IVFFlatDedup test (#3077)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3077

This diff relaxes some IVFFlatDedup tests where distances are slighlty different over runs.
Should fix

https://app.circleci.com/pipelines/github/facebookresearch/faiss/4709/workflows/8c8213bf-8fe0-4c4e-9a7d-991f44bf1010/jobs/25551

https://app.circleci.com/pipelines/github/facebookresearch/faiss/4709/workflows/8c8213bf-8fe0-4c4e-9a7d-991f44bf1010/jobs/25547

Reviewed By: algoriddle

Differential Revision: D49732349

fbshipit-source-id: 728b9885c6b7d6ba697ccb6bacc0abd0ee2b0679
2023-09-29 01:16:59 -07:00
Matthijs Douze b9ea339617 support range search from GPU (#2860)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2860

Optimized range search function where the GPU computes by default and falls back on gpu for queries where there are too many results.

Parallelize the CPU to GPU cloning, it seems to work.

Support range_search_preassigned in Python

Fix long-standing issue with SWIG exposed functions that did not release the GIL (in particular the MapLong2Long).

Adds a MapInt64ToInt64 that is more efficient than MapLong2Long.

Reviewed By: algoriddle

Differential Revision: D45672301

fbshipit-source-id: 2e77397c40083818584dbafa5427149359a2abfd
2023-05-16 00:27:53 -07:00
Matthijs Douze 581760302f evaluation script + IVF block search (#2781)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2781

This is a benchmarking script for keypoint matching with labelled ground-truth.

Reviewed By: alexanderguzhva

Differential Revision: D44036091

fbshipit-source-id: d9d7c089c4d172b66f33dc968c00713a1b79c2d1
2023-03-24 13:54:08 -07:00
Matthijs Douze a80c96c0de Evaluation script for hybrid CPU / GPU search
Summary:
Implementation of various combinations of coarse quantization / scaning code on CPU and GPU.

Used to generate the results of

https://github.com/facebookresearch/faiss/wiki/Hybrid-CPU-GPU-search-and-multiple-GPUs

Reviewed By: alexanderguzhva

Differential Revision: D43041802

fbshipit-source-id: 12608812ab351d60d4a6dc45be1ca493f76d4375
2023-02-15 12:55:06 -08:00
Matthijs Douze a996a4a052 Put idx_t in the faiss namespace (#2582)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2582

A few more or less cosmetic improvements
* Index::idx_t was in the Index object, which does not make much sense, this diff moves it to faiss::idx_t
* replace multiprocessing.dummy with multiprocessing.pool
* add Alexandr as a core contributor of Faiss in the README ;-)

```
for i in $( find . -name \*.cu -o -name \*.cuh -o -name \*.h -o -name \*.cpp ) ; do
  sed -i s/Index::idx_t/idx_t/ $i
done
```

For the fbcode deps:
```
for i in $( fbgs Index::idx_t --exclude fbcode/faiss -l ) ; do
   sed -i s/Index::idx_t/idx_t/ $i
done
```

Reviewed By: algoriddle

Differential Revision: D41437507

fbshipit-source-id: 8300f2a3ae97cace6172f3f14a9be3a83999fb89
2022-11-30 08:25:30 -08:00
Matthijs Douze 1829aa92a1 three small fixes (#1972)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/1972

This fixes a few issues that I ran into + adds tests:

- range_search_max_results with IP search

- a few missing downcasts for VectorTRansforms

- ResultHeap supports max IP search

Reviewed By: wickedfoo

Differential Revision: D29525093

fbshipit-source-id: d4ff0aff1d83af9717ff1aaa2fe3cda7b53019a3
2021-07-01 16:08:45 -07:00
Matthijs Douze 3dd7ba8ff9 Add range search accuracy evaluation
Summary:
Added a few functions in contrib to:
- run range searches by batches on the query or the database side
- emulate range search on GPU: search on GPU with k=1024, if the farthest neighbor is still within range, re-perform search on CPU
- as reference implementations for precision-recall on range search datasets
- optimized code to plot precision-recall plots (ie. sweep over thresholds)

The new functions are mainly in a new `evaluation.py`

Reviewed By: wickedfoo

Differential Revision: D25627619

fbshipit-source-id: 58f90654c32c925557d7bbf8083efbb710712e03
2020-12-17 17:17:09 -08:00