Commit Graph

10 Commits (96c868f50b268242cdbb5220e5e84dcb393b20e3)

Author SHA1 Message Date
Alexandr Guzhva 8ff1bc259d Additional C++ templates for fast sa_decode: add 8x compression level for AVX2 inline code. (#2532)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2532

Add 8x compression level, such as 'PQ64np' for 128-dim data. Prior to this change, only higher compression rates were supported.

Reviewed By: mdouze

Differential Revision: D40312821

fbshipit-source-id: 7dba4e9b8d432f5f7be618c0e7ef50dac2f88497
2022-10-17 11:14:32 -07:00
Alexandr Guzhva 2cd84aa663 Additional C++ templates for fast sa_decode: add overloads for identical coarse and fine tables (#2458)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2458

Add overloads for ::accum() for the case of each code sharing coarse quantizer centroids table and fine quantizer centroids table

Reviewed By: mdouze

Differential Revision: D39314206

fbshipit-source-id: 170a0a1c434e00c95c98151e026d1e30ac017149
2022-09-08 09:53:46 -07:00
Alexandr Guzhva c740091662 Additional C++ templates for fast sa_decode: add PQ support for IndexMinMaxDecoder (#2457)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2457

Add missing support for PQ combined with IndexMinMaxDecoder and IndexMinMaxFP16Decoder

Reviewed By: mdouze

Differential Revision: D39283622

fbshipit-source-id: 23fa1ffe3fbc2982547f4f1d9610d71c2b135666
2022-09-08 03:30:17 -07:00
Alexandr Guzhva c04088cf3d Additional C++ templates for fast sa_decode: IndexMinMaxDecoder and IndexMinMaxFP16Decoder (#2454)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2454

Add support for IndexRowwiseMinMax and IndexRowwiseMinMaxFP16 wrapped indices.

Reviewed By: mdouze

Differential Revision: D39177236

fbshipit-source-id: c205faf2c1bf844a1196fa318c629ceffe6acf52
2022-09-06 10:38:25 -07:00
Alexandr Guzhva b4924aad9c Additional C++ templates for fast sa_decode: additional overload for ::accum() (#2445)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2445

Add overloads for ::accum() to process 3 vectors per call. It is faster than processing 2 vectors per call in certain cases, at least for the AVX2 code.

Reviewed By: mdouze

Differential Revision: D39176425

fbshipit-source-id: bb39bb1f7a77442d32f20cb29281ec2e2ed2600c
2022-09-05 10:04:32 -07:00
Alexandr Guzhva abb46aceae Additional C++ templates for fast sa_decode: add IndexPQDecoder (#2444)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2444

Add IndexPQDecoder. The following codecs are supported:
* PQ[1]x8

Additionally, AVX2 and ARM versions support the following codecs:
* PQ[1]x10
* PQ[1]x16

Reviewed By: mdouze

Differential Revision: D39176423

fbshipit-source-id: b002b3d3b0533849f72f3660e8088d8dc44a66d6
2022-09-05 10:04:32 -07:00
Alexandr Guzhva 2556813131 Additional C++ templates for fast sa_decode: more codecs for Index2LevelDecoder (#2443)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2443

Add support for
* IVF[9-16 bit],PQ[1]x8 (such as IVF1024,PQ16np)
* Residual1x[9-16 bit],PQ[1]x8 (such as Residual1x9,PQ8)

Additionally, AVX2 and ARM versions support
* Residual[1]x8,PQ[2]x10
* Residual[1]x8,PQ[2]x16
* Residual1x[9-16 bit],PQ[1]x10 (such as Residual1x9,PQ16x10)
* Residual1x[9-16 bit],PQ[1]x16 (such as Residual1x9,PQ16x16)
* Residual[1]x10,PQ[2]x10
* Residual[1]x10,PQ[2]x16
* Residual[1]x16,PQ[2]x10
* Residual[1]x16,PQ[2]x16

IVF[9-16 bit],PQ[1]x10 and IVF[9-16 bit],PQ[1]x16 (such as IVF1024,PQ16x10np) are supported as well, but Faiss does not allow to train such Indices as this time.

Reviewed By: mdouze

Differential Revision: D39176424

fbshipit-source-id: 29b3d8d27a5fed0185df3e5484003fcc1521083a
2022-09-05 10:04:32 -07:00
Alexandr Guzhva ae0a9d86a7 Remove some redundant tests (#2396)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2396

Remove 1/3 of kernel tests that are likely to be redundant. This should reduce the running time for tests.

Reviewed By: mdouze

Differential Revision: D38086992

fbshipit-source-id: 468d5e9593cb7144986f763d31087b25965d45fa
2022-07-26 06:03:31 -07:00
wx257osn2@yahoo.co.jp 2952487f0e Fix test_cppcontrib_sa_decode.cpp (#2388)
Summary:
This PR contains below changes:

- Conform C++11
    - [`faiss` is written in C++11](https://github.com/facebookresearch/faiss/blob/main/CONTRIBUTING.md#coding-style), but [`faiss/cppcontrib/SaDecodeKernels-avx2-inl.h`](442d9f4a2d/faiss/cppcontrib/SaDecodeKernels-avx2-inl.h) and [the test](442d9f4a2d/tests/test_cppcontrib_sa_decode.cpp) use some C++17 features. This PR rewrites these codes to make them independent to C++17.
- Enable AVX2 on `faiss_test`
    - Currently `faiss_test` is compiled without `-mavx2` even if `-DFAISS_OPT_LEVEL=avx2` , so **`tests/test_cppcontrib_sa_decode.cpp` hasn't checked `faiss/cppcontrib/SaDecodeKernels-avx2-inl.h` at all** . This PR adds `-mavx2` to `faiss_test` if `-DFAISS_OPT_LEVEL=avx2` , so now `tests/test_cppcontrib_sa_decode.cpp` confirms `faiss/cppcontrib/SaDecodeKernels-avx2-inl.h` if `-DFAISS_OPT_LEVEL=avx2` , and does `faiss/cppcontrib/SaDecodeKernels-inl.h` if not `-DFAISS_OPT_LEVEL=avx2` .

Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2388

Reviewed By: mdouze

Differential Revision: D38005738

Pulled By: alexanderguzhva

fbshipit-source-id: b9319c585c6849e1c7a4782770f2d7ce8c0d8660
2022-07-20 23:46:31 -07:00
Alexandr Guzhva 3986ebffca fast C++ templates for sa_decode (#2354)
Summary:
Pull Request resolved: https://github.com/facebookresearch/faiss/pull/2354

A specialized code that provides 2x-3x faster Index::sa_decode for
* IVF256,PQ[1]x8np
* Residual[1]x8,PQ[2]x8

Reviewed By: mdouze

Differential Revision: D37092134

fbshipit-source-id: d848b6cf1aefa826a5ca01e41935aa5d46f5dcc7
2022-06-16 09:20:19 -07:00