mirrors
/
faiss
mirror of https://github.com/facebookresearch/faiss.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
1,443 Commits
21 Branches
19 Tags
212 MiB
Commit Graph

4 Commits (main)

Author SHA1 Message Date
Michael Norris eff0898a13 Enable linting: lint config changes plus arc lint command (#3966) 
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3966

This actually enables the linting.

Manual changes:
- tools/arcanist/lint/fbsource-licenselint-config.toml
- tools/arcanist/lint/fbsource-lint-engine.toml

Automated changes:
`arc lint --apply-patches --take LICENSELINT --paths-cmd 'hg files faiss'`

Reviewed By: asadoughi

Differential Revision: D64484165

fbshipit-source-id: 4f2f6e953c94ef6ebfea8a5ae035ccfbea65ed04
 2024-10-22 09:46:48 -07:00
Maria Lomeli 0fc8456e1d Offline IVF powered by faiss big batch search (#3202) 
Summary:
This PR introduces the offline IVF (OIVF) framework which contains some tooling to run search using IVFPQ indexes (plus OPQ pretransforms) for large batches of queries using [big_batch_search](https://github.com/mlomeli1/faiss/blob/main/contrib/big_batch_search.py) and GPU faiss. See the [README](36226f5fe8/demos/offline_ivf/README.md) for details about using this framework.

This PR includes the following unit tests, which can be run with the unittest library as so:
````
~/faiss/demos/offline_ivf$ python3 -m unittest tests/test_iterate_input.py -k test_iterate_back
````
In test_offline_ivf:
````
test_consistency_check
test_train_index
test_index_shard_equal_file_sizes
test_index_shard_unequal_file_sizes
test_search
test_evaluate_without_margin
test_evaluate_without_margin_OPQ
````
In test_iterate_input:
````
test_iterate_input_file_larger_than_batch
test_get_vs_iterate
test_iterate_back

````

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3202

Reviewed By: algoriddle

Differential Revision: D52734222

Pulled By: mlomeli1

fbshipit-source-id: 61fd0084277c1b14bdae1189db8ae43340611e16
 2024-01-16 05:05:15 -08:00
Gergely Szilvasy 77c28f8ba4 Back out "Offline IVF powered by faiss big batch search" 
Summary: Revert so that the test data can be replaced with synthetic

Reviewed By: mdouze

Differential Revision: D52388604

fbshipit-source-id: c0037635a4e66c54d42400294d13d9a80610b845
 2023-12-22 03:43:05 -08:00
Maria 9a8b34e295 Offline IVF powered by faiss big batch search (#3175) 
Summary:
This PR introduces the offline IVF (OIVF) framework  which contains some tooling to run search using IVFPQ indexes (plus OPQ pretransforms) for large batches of queries using [big_batch_search](https://github.com/mlomeli1/faiss/blob/main/contrib/big_batch_search.py) and GPU faiss. See the [README](https://github.com/mlomeli1/faiss/blob/oivf/demos/offline_ivf/README.md) for details about using this framework.

This PR includes the following unit tests, which can be run with the unittest library as so:
````
~/faiss/demos/offline_ivf$ python3 -m unittest tests/test_iterate_input.py -k test_iterate_back
````

In test_offline_ivf:
````
test_consistency_check
test_train_index
test_index_shard_equal_file_sizes
test_index_shard_unequal_file_sizes
 test_search
test_evaluate_without_margin
test_evaluate_without_margin_OPQ
test_evaluate_with_margin
test_split_batch_size_bigger_than_file_sizes
test_split_batch_size_smaller_than_file_sizes
test_split_files_with_corrupted_input_file
````

In test_iterate_input:
````
test_iterate_input_file_larger_than_batch
test_get_vs_iterate
test_iterate_back
````

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/3175

Reviewed By: algoriddle

Differential Revision: D52218447

Pulled By: mlomeli1

fbshipit-source-id: 78b12457c79b02eb2c9ae993560f2e295798e7e5
 2023-12-18 15:09:31 -08:00