Commit Graph

238 Commits (64828d28517d29738c187fe886e7a7ec558906fd)
 

Author SHA1 Message Date
Matthijs Douze 64828d2851
Update merge_to_ondisk.py 2019-08-29 15:35:39 +02:00
Matthijs Douze 0fdd38055b
Update make_trained_index.py 2019-08-29 15:35:21 +02:00
Matthijs Douze 29e0514128
Update make_index_vslice.py 2019-08-29 15:35:07 +02:00
Matthijs Douze 87c83b9d97
Update distributed_query_demo.py 2019-08-29 15:34:50 +02:00
Matthijs Douze a9a475b003
Update distributed_kmeans.py 2019-08-29 15:34:14 +02:00
Matthijs Douze 10ca6e20d4
Update combined_index.py 2019-08-29 15:33:50 +02:00
Matthijs Douze 8d08912453
Ondisk distributed index implementation (#930)
Adds the code for the distributed on-disk index
2019-08-29 13:44:08 +02:00
Lucas Hosseini 656368b5ed
Dynamically detect avx2 capabilities (#877)
* Install scipy via pip rather than system package manager

* Dynamically detect AVX2 capability

* Simplify build on TravisCI
2019-06-28 13:50:46 +02:00
Matthijs Douze 70f5b04206
Update CONTRIBUTING.md 2019-06-25 17:27:23 +02:00
Lucas Hosseini 0ca46ee474
Update README.md 2019-06-24 18:51:00 +02:00
Lucas Hosseini efa81eec76 Prepare for v1.5.3 release. 2019-06-24 17:41:35 +02:00
Eduardo Pinho 828dcea400 [C API] June 2019 update (#869)
* [c_api] use all relevant flags in compilation

* [c_api] Remove redundant IndexIVFFlat declarations

- From IndexIVF_c.h, already declared in IndexIVFFlat_c.h

* [c_api] type changes

- replace `long` with a more suitable type
- provide definitions for `faiss_component_t` and `faiss_distance_t`

* [c_api] Define CFLAGS and CUDACFLAGS
2019-06-24 13:29:38 +02:00
Lucas Hosseini 1cbc999610
Add -mavx2 -mf16c to CPUFLAGS. (#871) 2019-06-24 13:28:58 +02:00
chck fc9856a25e Fix gpu compilation with cuda 10.1 related to #751 (#865) 2019-06-21 19:43:53 +02:00
Lucas Hosseini d224d11c9d
Revert swigfaiss import for python 3 compatibility. (#868) 2019-06-20 06:13:38 +02:00
Lucas Hosseini 3896b12c65
Facebook sync (Jun 2019) (#862)
Bugfixes:
- slow scanning of inverted lists (#836).

Features:
- add basic support for 6 new metrics in CPU `IndexFlat` and `IndexHNSW` (#848);
- add support for `IndexIDMap`/`IndexIDMap2` with binary indexes (#780).

Misc:
- throw python exception for OOM (#758);
- make `DistanceComputer` available for all random access indexes;
- gradually moving from `long` to `int64_t` for portability.
2019-06-19 15:59:06 +02:00
Eduardo Pinho 2bfbead8f1 [C API] Update for compatibility with v1.5 (#858)
* [c_api] Update impl and interface for v1.5

- move IndexShards to dedicated module IndexShards_c.{h|cpp}
- remove getter/setters to unreachable fields
- reimplement faiss_IndexIVF_imbalance_factor (to use invlists)
- minor IndexIVF documentation tweaks
- Remove QueryResult, provide RangeQueryResult

* [c_api] Document FaissErrorCode

* [c_api] Update GPU impl and interface for v1.5

- Remove unavailable method setTempMemoryFraction

* [c_api] Relicense to MIT

In accordance to the rest of the project
2019-06-12 15:46:08 +02:00
Gao c4c5d6f73c Update README.md (#849)
fix typo
2019-05-31 21:02:17 +02:00
Lucas Hosseini ae09cabd6a
Fix Makefile in gpu/test (#843) 2019-05-29 14:32:05 +02:00
Lucas Hosseini dacd5a9f51 Prepare for v1.5.2 release 2019-05-28 16:22:38 +02:00
Lucas Hosseini a8118acbc5
Facebook sync (May 2019) + relicense (#838)
Changelog:

- changed license: BSD+Patents -> MIT
- propagates exceptions raised in sub-indexes of IndexShards and IndexReplicas
- support for searching several inverted lists in parallel (parallel_mode != 0)
- better support for PQ codes where nbit != 8 or 16
- IVFSpectralHash implementation: spectral hash codes inside an IVF
- 6-bit per component scalar quantizer (4 and 8 bit were already supported)
- combinations of inverted lists: HStackInvertedLists and VStackInvertedLists
- configurable number of threads for OnDiskInvertedLists prefetching (including 0=no prefetch)
- more test and demo code compatible with Python 3 (print with parentheses)
- refactored benchmark code: data loading is now in a single file
2019-05-28 16:17:22 +02:00
gensmusic 712edb043a tutorial build add CPPFLAGS which contains cuda include pathi when ./configure --with-cuda (#833) 2019-05-24 14:46:53 +02:00
Pierre Letessier 5d1ed5b6fb Fix unable to read untrained index (#817) 2019-05-07 18:14:15 +02:00
Lucas Hosseini 012954ebbf
Fix HNSW.fill_with_random_links. (#801) 2019-04-23 16:52:39 +02:00
Xuhui Sun 0a8f92d6d6 Update cpp tutorial makefile for GPU part (#774) 2019-04-11 14:20:38 +02:00
Matthijs Douze b460b0e9d5
Update INSTALL.md 2019-04-11 11:24:35 +02:00
Lucas Hosseini e811fecac6
Make clean target bourne shell compatible. (#783) 2019-04-10 15:54:57 +02:00
Lucas Hosseini 1e3e7f4262
Make Makefile bourne shell compliant. (#782) 2019-04-10 14:55:30 +02:00
Matthijs Douze f443db75f3 Merge branch 'master' of https://www.github.com/facebookresearch/faiss 2019-04-08 20:26:42 -07:00
Matthijs Douze fa91c13980 take into account torch offset when getting ptr 2019-04-08 20:26:26 -07:00
Steffen Schmitz d635aa9c65 fix: gpu compilation with cuda 10.1 (#771)
The GPU compilation fails with g++ 8.3.1 and cuda 10.1
With this proposed fix the compilation works locally

Contributes to #751
2019-04-08 20:39:21 +02:00
Lucas Hosseini 5555ae7f88
Configure install paths in Dockerfile. (#772) 2019-04-07 23:22:43 +02:00
Lucas Hosseini 7f5b22b0ff
Add conda packages metadata + tests. (#769)
+ Add conda packages metadata (now building Faiss using conda's toolchain);
+ add Dockerfile for building conda packages (for all CUDA versions);
+ add working Dockerfile building faiss on Centos7;
+ simplify GPU build;
+ avoid falling back to CPU-only version (python);
+ simplify TravisCI config;
+ update INSTALL.md;
+ add configure flag for specifying target architectures (--with-cuda-arch);
+ fix Makefile for gpu tests;
+ fix various Makefile issues;
+ remove stale file (gpu/utils/DeviceUtils.cpp).
2019-04-05 11:50:39 +02:00
Lucas Hosseini 6c1cb3cd5c Fix ivfpq demo. (#763)
Makes sense.
2019-04-03 11:24:33 +02:00
Lucas Hosseini afe0fdc161
Facebook sync (Mar 2019) (#756)
Facebook sync (Mar 2019)

- MatrixStats object
- option to round coordinates during k-means optimization
- alternative option for search in HNSW
- moved stats and imbalance_factor of IndexIVF to InvertedLists object
- range search for IVFScalarQuantizer
- direct unit8 codec in ScalarQuantizer
- renamed IndexProxy to IndexReplicas and moved to main Faiss
- better support for PQ code assignment with external index
- support for IMI2x16 (4B virtual centroids!)
- support for k = 2048 search on GPU (instead of 1024)
- most CUDA mem alloc failures throw exceptions instead of terminating on an assertion
- support for renaming an ondisk invertedlists
- interrupt computations with ctrl-C in python
2019-03-29 16:32:28 +01:00
Jeff Johnson a9959bf63b
Merge pull request #746 from facebookresearch/automated_fixup_code_of_conduct_file_exists
Adding Code of Conduct file
2019-03-25 11:43:16 -07:00
Facebook Community Bot 65d3d1f129 OSS Automated Fix: Addition of Code of Conduct 2019-03-22 11:13:52 -07:00
Lucas Hosseini 7b07685338
Update INSTALL.md
Remove stale CMake reference.
2019-03-06 12:12:03 +01:00
Marcin Tolysz 4b3210575b Improve arm64 support. (#676)
* Improve Arm64 support.

Make library compile and run on Arm64 8.2 (Nvidia Xavier arm+cuda)
`configure` is regenerated

* remove simd from utils.cpp
2019-02-15 13:13:49 +01:00
Arun-George-Zachariah b06cf4128d Upgrading pip in the Docker Container (#711) 2019-02-15 13:12:54 +01:00
Minjoon Seo c6a53c078b Bug fix: data needed to be added with ids (#723)
When data is being added to each index block, it needs to explicitly identify ids.
2019-02-10 05:44:05 +01:00
Lucas Hosseini 4e8d324922
Fix Travis build on OSX. (#692) 2019-01-15 21:47:02 +01:00
Matthijs Douze 702ad532db
Update README.md 2019-01-15 19:24:48 +01:00
Matthijs Douze fd0f0671d2
Update README.md 2019-01-15 19:22:16 +01:00
Corey J. Nolet f5feac149c For #669. Adding install target to gpu Makefile (#688) 2019-01-11 17:34:39 +01:00
Bonnie Varghese eb6c4cc96a Allow setting nprobe for IndexIDMap->IndexIVF (#673)
There we go
2019-01-09 13:00:36 +01:00
Lucas Hosseini f417a53628
Fix CI tests (#687)
* Fix test_transfer_invlists.cpp

* Fix relative imports.

* Fix test_index_accuracy.py.

* Use default OSX version.

* Allow osx gcc6 build to fail.
2019-01-08 17:52:36 +01:00
Eduardo Pinho 27a4ba0d93 [C API] Multi-GPU functions (#628)
* [c_api] minor tweak on Index_c header imports

* [c_api] Multi GPU constructs

- Add constructor & destructor for `FaissGpuMultipleClonerOptions`
- Add function `faiss_index_cpu_to_gpu_multiple`
- Add function `faiss_index_cpu_to_gpu_multiple_with_options`

* [c_api] fix doc comments in GpuClonerOptions

* [c_api] Improve cpu_to_gpu_multiple prototype

* [c_api] Improve cpu_to_gpu_multiple prototype

* [c_api] Add dtor definition for GpuMultipleClonerOptions
2018-12-23 18:46:07 +01:00
Lucas Hosseini ac7005b6ef Remove CMake. (#645) 2018-12-23 18:45:16 +01:00
Jeff Johnson aafc7b0457
Merge pull request #672 from irachex/patch-1
Add missing `\` in gpu Makefile
2018-12-22 19:36:05 -05:00