4 Commits

Author SHA1 Message Date
Matthijs Douze
dd814b5f14 IVF filtering based on IDSelector (no init split) (#2483)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2483

This diff changes the following:
1. all search functions now take a `SearchParameters` argument that overrides the internal search parameters
2. the default implementation for most classes throws when the params argument is non-nullptr / non-None
3. the IndexIVF and IndexHNSW classes have functioning SearchPArameters
4. the SearchParameters includes an IDSelector that can search only in a subset of the index based on a defined subset of ids

There is also some refactoring: the IDSelector was moved to its own .h/.cpp and python/__init__.py is spit in parts.

The diff is quite bulky because the search function prototypes need to be changed in all index classes.

Things to fix in subsequent diffs:

- support SearchParameters for more index types (Flat variants)

- better sub-object ownership for SearchParams (with std::unique_ptr?)

- special handling of IDSelectorRange to make it faster

Reviewed By: alexanderguzhva

Differential Revision: D39852589

fbshipit-source-id: 4988bdb5b9bee1207cd327d3f80bf5e0e2467fe1
2022-09-30 06:40:03 -07:00
Lucas Hosseini
e86bf8cae1 Enable clang-format + autofix.
Summary: Format whole codebase with clang-format.

Reviewed By: mdouze

Differential Revision: D22891341

fbshipit-source-id: 673032b2444d61026d1e2c3fa2c5659f178cf58b
2021-02-25 04:46:10 -08:00
Lucas Hosseini
36ddba9196
Facebook sync (2019-09-10) (#943)
* Facebook sync (2019-09-10)

* Fix depends Makefile target.

* Add faiss symlink for new include directives.

* Fix missing header.

* Fix tests.

* Fix Makefile.

* Update depend.

* Fix include directives spacing.
2019-09-20 18:59:10 +02:00
Lucas Hosseini
a8118acbc5
Facebook sync (May 2019) + relicense (#838)
Changelog:

- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
2019-05-28 16:17:22 +02:00