Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 3 additions & 18 deletions docs/credits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ dependencies from the AgML project configuration.
| dict2xml | >=1.7.6 |
| ipywidgets | >=8.1.5 |
| rich | >=14.0.0 |
| datasets | * |
| transformers | * |

## Development dependencies

Expand All @@ -40,21 +42,4 @@ dependencies from the AgML project configuration.
| coverage | >=7.6.1 |
| interrogate | >=1.7.0 |

## Docs dependencies

| Project | Version |
| --- | --- |
| mkdocs | >=1.5.0 |
| mkdocs-material | >=9.5.0 |
| markdown-callouts | >=0.4.0 |
| markdown-exec | >=1.7.0 |
| mkdocs-gen-files | >=0.5.0 |
| mkdocs-literate-nav | >=0.6.0 |
| mkdocs-coverage | >=1.0.0 |
| mkdocstrings | >=0.24.0 |
| mkdocstrings-python | >=1.8.0 |
| mkdocs-git-revision-date-localized-plugin | >=1.2.0 |
| mkdocs-minify-plugin | >=0.8.0 |

For the fully generated credits (including resolved versions and licenses), refer to the
upstream generator in the AgML repository at `scripts/docs/gen_credits.py`.
For the full dependency list with resolved versions and licenses, see [pyproject.toml](https://github.com/Project-AgML/AgML/blob/main/pyproject.toml) in the AgML repository.
27 changes: 27 additions & 0 deletions docs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,10 @@ train_ds.as_torch_dataset()

You're now ready to use AgML for training your own models! Luckily, AgML comes with a training module that enables quick-start training of standard deep learning models on agricultural datasets. Training a grape detection model is as simple as the following code:

:::warning Deprecation Notice
`agml.models` is deprecated and will be removed in a future release. The example below still works but will emit a `DeprecationWarning` at import time.
:::

```python
import agml
import agml.models
Expand All @@ -126,6 +130,29 @@ model = agml.models.DetectionModel(num_classes=loader.num_classes)
model.run_training(loader)
```

## HuggingFace Integration

AgML now includes a `HuggingFaceDataLoader` for loading datasets directly from the [Hugging Face Hub](https://huggingface.co/datasets) into native HuggingFace `DatasetDict` objects, compatible with `transformers` and `diffusers` pipelines.

```python
from agml.data import HuggingFaceDataLoader

# Load a dataset from the Hub
loader = HuggingFaceDataLoader("org/dataset-name")

# Load a specific config/subset
loader = HuggingFaceDataLoader("org/dataset-name", config="augmented")

# Split into train/val/test with optional stratification
dataset = loader.split(val_size=0.1, test_size=0.1, stratify_cols="label")
# Returns a DatasetDict with 'train', 'val', and 'test' splits

# Access the underlying DatasetDict at any time
dataset = loader.dataset
```

The `HuggingFaceDataLoader` automatically casts image-like columns to the HuggingFace `Image` type for decoded pixel access. Stratified splitting is supported across one or more columns using a composite key.

## Public Datasets

AgML contains a wide variety of public datasets from various locations across the world:
Expand Down
535 changes: 434 additions & 101 deletions static/data/hf_datasets.json

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading