712de55a20
Reformat Markdown code blocks ( #13023 )
2024-05-18 15:52:21 +02:00
1bcd17ee69
Bump contributor-assistant/github-action from 2.3.2 to 2.4.0 in /.github/workflows ( #13006 )
...
Bump contributor-assistant/github-action in /.github/workflows
Bumps [contributor-assistant/github-action](https://github.com/contributor-assistant/github-action ) from 2.3.2 to 2.4.0.
- [Release notes](https://github.com/contributor-assistant/github-action/releases )
- [Commits](https://github.com/contributor-assistant/github-action/compare/v2.3.2...v2.4.0 )
---
updated-dependencies:
- dependency-name: contributor-assistant/github-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-13 11:46:16 +02:00
28e06aa391
Centralize ENV variable definition in utils/general.py ( #13004 )
...
* Centralize ENV variable definition in utils/general.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update export.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-05-12 22:16:54 +02:00
d1ea6b1efa
Revert CI `pip install` retries to default ( #13002 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-05-12 18:48:42 +02:00
331c39e3c3
Add `pip install --retries 3` to CI to resolve transients ( #13001 )
...
* Add `pip install --retries 3` to CI to resolve transients
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-05-12 18:25:53 +02:00
2f6b91ee8c
Set `TORCH_CPP_LOG_LEVEL=ERROR` for reduced verbosity ( #12989 )
...
* modify torch cpp log level to Error to avoid annoying print
* Auto-format by https://ultralytics.com/actions
---------
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-05-12 17:10:18 +02:00
920c721ea7
Backport compatibility with TensorRT version 10 from yolov8 ( #12984 )
...
Add compatibility with TensorRT version 10.
Based on the is_trt10 code in yolov8.
2024-05-05 11:55:12 +02:00
b599ae42d9
Add Turkish and Vietnamese Docs ( #12972 )
...
* Add Turkish and Vietnamese Docs
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update README.md
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update pyproject.toml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-28 17:18:39 +02:00
4456c95352
Bump slackapi/slack-github-action from 1.25.0 to 1.26.0 in /.github/workflows ( #12948 )
...
* Bump slackapi/slack-github-action in /.github/workflows
Bumps [slackapi/slack-github-action](https://github.com/slackapi/slack-github-action ) from 1.25.0 to 1.26.0.
- [Release notes](https://github.com/slackapi/slack-github-action/releases )
- [Commits](https://github.com/slackapi/slack-github-action/compare/v1.25.0...v1.26.0 )
---
updated-dependencies:
- dependency-name: slackapi/slack-github-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-27 13:22:02 +02:00
2236169145
Bump gunicorn from 19.10.0 to 22.0.0 in /utils/google_app_engine ( #12929 )
...
* Bump gunicorn from 19.10.0 to 22.0.0 in /utils/google_app_engine
Bumps [gunicorn](https://github.com/benoitc/gunicorn ) from 19.10.0 to 22.0.0.
- [Release notes](https://github.com/benoitc/gunicorn/releases )
- [Commits](https://github.com/benoitc/gunicorn/compare/19.10.0...22.0.0 )
---
updated-dependencies:
- dependency-name: gunicorn
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* [Snyk] Security upgrade gunicorn from 19.10.0 to 22.0.0 (#12938 )
fix: utils/google_app_engine/additional_requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-GUNICORN-6615672
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-04-17 20:41:59 -07:00
cf8b67b7e9
Update merge-main-into-prs.yml ( #12922 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-14 07:36:44 -07:00
51dc1af6e8
Update merge-main-into-prs.yml ( #12921 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-14 06:30:37 -07:00
6c8f693352
Update merge-main-into-prs.yml ( #12920 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-14 06:23:17 -07:00
a3ddc1795e
Update to `ultralytics>=8.1.47` ( #12919 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-14 06:19:07 -07:00
77aa0e61af
Create merge-main-into-prs.yml ( #12918 )
...
* Create merge-main-into-prs.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update merge-main-into-prs.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-14 06:14:49 -07:00
21f8f94d11
Sort imports with Ruff and iSort ( #12915 )
...
* Sort imports with Ruff and iSort
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-04-13 08:21:03 -07:00
d07d0cf633
Create cla.yml ( #12899 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-04-09 14:52:45 +02:00
ae4ef3b267
[Snyk] Security upgrade pillow from 9.5.0 to 10.3.0 ( #12868 )
...
* fix: requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6514866
* Update requirements.txt
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-04-07 00:46:25 +02:00
ac6c4383bc
Update Discord and Contributing Guide URLs ( #12847 )
...
* Update Discord and Contributing Guide URLs
* Update __init__.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-03-24 18:31:58 +01:00
db125a2017
Update export.py to `tensorflow<=2.15.1` ( #12803 )
...
* Update export.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update export.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-03-10 13:49:37 +01:00
a428ff7689
Update pyproject.toml ( #12740 )
...
* Update pyproject.toml
sio
Signed-off-by: Abhishek Sharma <140809134+AbhishekSharma-17@users.noreply.github.com>
* Update pyproject.toml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Abhishek Sharma <140809134+AbhishekSharma-17@users.noreply.github.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-03-10 13:28:53 +01:00
d6900cde52
[Snyk] Security upgrade wheel from 0.32.2 to 0.38.0 ( #12791 )
...
fix: requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-WHEEL-3180413
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
2024-03-10 13:27:53 +01:00
a083914a19
Limit TF export to `tensorflow<2.15.1` ( #12800 )
...
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* bump tests to 3.12
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update ci-testing.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-03-10 13:17:51 +01:00
b2ffe05569
Add Discord badge ( #12783 )
...
* Add Discord badge
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-03-04 21:12:53 +01:00
b24177bf99
A minor correction in a comment ( #12782 )
...
* A minor correction in a comment
I added the 'h' in 'https' in the link to the label smoothing issue.
Signed-off-by: Kumar Selvakumaran <62794224+kumar-selvakumaran@users.noreply.github.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Kumar Selvakumaran <62794224+kumar-selvakumaran@users.noreply.github.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-03-04 21:11:17 +01:00
b939236170
Correct the output dir in dataloaders.py ( #12771 )
...
Correct the output dir in dataloaders.py
Signed-off-by: Hongbo <12580159+ya0guang@users.noreply.github.com>
2024-03-01 12:32:29 +01:00
574331f984
Replace inline comments with docstrings ( #12764 )
...
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Add docstrings
* Add docstrings
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Update plots.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-02-26 23:16:32 +01:00
41603da162
Add docstrings to YOLOv5 functions ( #12760 )
...
* Add docstrings to top level files
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Add docstrings
* Auto-format by https://ultralytics.com/actions
* Update activations.py
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Auto-format by https://ultralytics.com/actions
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-02-25 14:04:01 +01:00
ca00c34c2a
Fix docs typos in callbacks.py ( #12750 )
...
* Update callbacks.py
Signed-off-by: imyhxy <imyhxy@gmail.com>
* Update general.py
Signed-off-by: imyhxy <imyhxy@gmail.com>
---------
Signed-off-by: imyhxy <imyhxy@gmail.com>
2024-02-25 13:57:41 +01:00
95ebf68f92
Bump nick-invision/retry from 2 to 3 in /.github/workflows ( #12707 )
...
Bumps [nick-invision/retry](https://github.com/nick-invision/retry ) from 2 to 3.
- [Release notes](https://github.com/nick-invision/retry/releases )
- [Changelog](https://github.com/nick-fields/retry/blob/master/.releaserc.js )
- [Commits](https://github.com/nick-invision/retry/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: nick-invision/retry
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-08 19:48:28 +01:00
875d92782e
[Snyk] Security upgrade pillow from 9.5.0 to 10.2.0 ( #12697 )
...
* fix: requirements.txt to reduce vulnerabilities
The following vulnerabilities are fixed by pinning transitive dependencies:
- https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6219984
- https://snyk.io/vuln/SNYK-PYTHON-PILLOW-6219986
* Auto-format by https://ultralytics.com/actions
* Update requirements.txt
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: snyk-bot <snyk-bot@snyk.io>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-02-01 23:56:35 +01:00
9cdbd1de6b
Bump slackapi/slack-github-action from 1.24.0 to 1.25.0 in /.github/workflows ( #12679 )
...
Bump slackapi/slack-github-action in /.github/workflows
Bumps [slackapi/slack-github-action](https://github.com/slackapi/slack-github-action ) from 1.24.0 to 1.25.0.
- [Release notes](https://github.com/slackapi/slack-github-action/releases )
- [Commits](https://github.com/slackapi/slack-github-action/compare/v1.24.0...v1.25.0 )
---
updated-dependencies:
- dependency-name: slackapi/slack-github-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 13:09:36 +01:00
4d5c0a403b
Update format.yml ( #12682 )
...
* Update format.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update format.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-29 11:53:02 +01:00
4878541d43
Ultralytics Actions with OpenAI GPT-4 PR Summary ( #12678 )
...
* Update Ultralytics Actions
* Update format.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-28 23:52:03 +01:00
72749c1ce2
Link checks SSL insecure robustness ( #12675 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-27 18:49:40 +01:00
050c72cbba
YAML reformat ( #12648 )
...
* YAML reformat
* Auto-format by Ultralytics actions
---------
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-01-18 17:53:34 +01:00
703d37ef79
Update requirements.txt to `numpy>=1.23.5` ( #12641 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-17 01:19:22 +01:00
c8846622b1
Update jax requirement from <=0.4.21 to <=0.4.23 ( #12594 )
...
* Update jax requirement from <=0.4.21 to <=0.4.23
Updates the requirements on [jax](https://github.com/google/jax ) to permit the latest version.
- [Release notes](https://github.com/google/jax/releases )
- [Changelog](https://github.com/google/jax/blob/main/CHANGELOG.md )
- [Commits](https://github.com/google/jax/compare/jaxlib-v0.1.32...jax-v0.4.23 )
---
updated-dependencies:
- dependency-name: jax
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
* Update pyproject.toml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-17 00:04:45 +01:00
32e04a86f4
Update jaxlib requirement from <=0.4.21 to <=0.4.23 ( #12593 )
...
Updates the requirements on [jaxlib](https://github.com/google/jax ) to permit the latest version.
- [Release notes](https://github.com/google/jax/releases )
- [Changelog](https://github.com/google/jax/blob/main/CHANGELOG.md )
- [Commits](https://github.com/google/jax/compare/jaxlib-v0.1.32...jaxlib-v0.4.23 )
---
updated-dependencies:
- dependency-name: jaxlib
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-17 00:01:19 +01:00
7d9a117d07
Update links.yml to --scheme https ( #12625 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-13 22:42:33 +01:00
22e9afef74
Python refactor and simplification ( #12624 )
...
* Python code cleanup
* Auto-format by Ultralytics actions
---------
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-01-13 22:34:05 +01:00
de64179cd4
Improved OpenVINO quantization code ( #12614 )
...
* Improved OpenVINO quantization code
* Auto-format by Ultralytics actions
---------
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
2024-01-12 01:01:34 +01:00
9abbef522f
Update banners for YOLOv8 release v8.1.0 ( #12605 )
...
* Auto-format by Ultralytics actions
* updated git banner
* Update README.md
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update README.zh-CN.md
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-10 16:16:40 +01:00
4733b4df90
Update links.yml ( #12601 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-08 18:55:43 +01:00
ad97dc6a6d
Update Actions with Lychee and GitHub Token ( #12592 )
...
* Add Ultralytics Actions
* Auto-format by Ultralytics actions
* Update format.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
Co-authored-by: UltralyticsAssistant <web@ultralytics.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-08 01:29:14 +01:00
f56355d321
Update links.yml ( #12588 )
...
* Update links.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Update links.yml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-07 01:43:03 +01:00
c42aba5b2f
Delete .pre-commit-config.yaml ( #12578 )
...
* Delete .pre-commit-config.yaml
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Delete setup.cfg
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
---------
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-03 11:36:55 +01:00
dfa884f9db
Create pyproject.toml ( #12577 )
...
Signed-off-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-03 11:35:32 +01:00
f6a781ac76
Use named imports in models.yolo ( #12576 )
2024-01-03 11:34:34 +01:00
46ae996cb1
Bugfix: update dataloaders.py to fix Multi-GPU DDP RAM multiple-cache issue ( #10383 )
...
* Update dataloaders.py
This is to address (and hopefully fix) this issue: Multi-GPU DDP RAM multiple-cache bug #3818 (https://github.com/ultralytics/yolov5/issues/3818 ). This was a very serious and "blocking" issue until I could figure out what was going on. The problem was especially bad when running Multi-GPU jobs with 8 GPUs, RAM usage was 8x higher than expected (!), causing repeated OOM failures. Hopefully this fix will help others.
DDP causes each RANK to launch it's own process (one for each GPU) with it's own trainloader, and its own RAM image cache. The DistributedSampler used by DDP (https://github.com/pytorch/pytorch/blob/master/torch/utils/data/distributed.py ) will feed only a subset of images (1/WORLD_SIZE) to each available GPU on each epoch, but since the images are shuffled between epochs, each GPU process must still cache all images. So I created a subclass of DistributedSampler called SmartDistributedSampler that forces each GPU process to always sample the same subset (using modulo arithmetic with RANK and WORLD_SIZE) while still allowing random shuffling between epochs. I don't believe this disrupts the overall "randomness" of the sampling, and I haven't noticed any performance degradation.
Signed-off-by: davidsvaughn <davidsvaughn@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update dataloaders.py
move extra parameter (rank) to end so won't mess up pre-existing positional args
* Update dataloaders.py
removing extra '#'
* Update dataloaders.py
sample from DDP index array (self.idx) in mixup mosaic
* Merging self.indices and self.idx (DDP indices) into single attribute (self.indices).
Also adding SmartDistributedSampler to segmentation dataloader
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Multiply GB displayed by WORLD_SIZE
---------
Signed-off-by: davidsvaughn <davidsvaughn@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
2024-01-03 08:15:07 +01:00
Glenn Jocher
dependabot[bot]
inisis
Nick Martin
Abhishek Sharma
Kumar Selvakumaran
Hongbo
imyhxy
Paula Derrenger
Adrian Boguszewski
Muhammad Rizwan Munawar
Aarni Koskela
davidsvaughn