-
Notifications
You must be signed in to change notification settings - Fork 391
docs: document knowledge data storage limits for cloud plans #935
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
dc3288a
docs: add knowledge storage limit page for cloud
RiskeyL 0d4deea
docs: make knowledge upload limits plan-based on cloud pages
RiskeyL c8d356f
docs: add plan-based file size limits to knowledge api endpoints
RiskeyL 159f94f
chore: add knowledge data storage ui label to glossary
RiskeyL 3a06273
docs: align storage limit page title with ui label
RiskeyL File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| --- | ||
| title: Knowledge Data Storage Limit | ||
| sidebarTitle: Data Storage Limit | ||
| description: What counts toward your plan's knowledge storage, what happens when you reach the limit, and how to free space | ||
| --- | ||
|
|
||
| Every document indexed with the **High Quality** [index method](/en/cloud/use-dify/knowledge/create-knowledge/setting-indexing-methods) stores its chunks as vectors (the numeric form that powers retrieval), and those vectors count toward your workspace's knowledge storage. | ||
|
|
||
| Each plan includes a fixed amount: | ||
|
|
||
| | Plan | Storage | | ||
| |:-----|:--------| | ||
| | Sandbox | 50 MB | | ||
| | Professional | 5 GB | | ||
| | Team | 20 GB | | ||
|
|
||
| The workspace owner and admins can check current usage in **Settings** > **Billing** > **Knowledge Data Storage**. | ||
|
|
||
| ## What Consumes Storage | ||
|
|
||
| Usage depends on how many vectors your documents produce, not on their raw file size. The more chunks a document is split into, the more vectors it produces. | ||
|
|
||
| <Info> | ||
| In [Parent-child mode](/en/cloud/use-dify/knowledge/create-knowledge/chunking-and-cleaning-text), every child chunk becomes a vector. | ||
| </Info> | ||
|
|
||
| The embedding model matters too: each model outputs vectors of a fixed length, listed in the provider's documentation as its number of dimensions. For example, OpenAI's `text-embedding-3-large` uses nearly twice the storage per chunk of `text-embedding-3-small`. | ||
|
|
||
| As a result, a document can use several times its own file size in storage once indexed. | ||
|
|
||
| Documents indexed with the **Economical** method use keyword indexing instead of vectors and don't consume storage. | ||
|
|
||
| Usage reflects the actual storage your vectors occupy, measured periodically, so a delay is expected before the number changes after an upload or deletion. | ||
|
|
||
| ## When an Upload Would Exceed the Limit | ||
|
|
||
| Every upload is checked before indexing. When the estimated total storage after the upload would exceed your plan's capacity, the document is kept in the document list but not indexed. | ||
|
|
||
| To bring it under the limit, split the file into smaller uploads, or raise the chunk size in the document's [chunk settings](/en/cloud/use-dify/knowledge/create-knowledge/chunking-and-cleaning-text) so it produces fewer chunks, though larger chunks make each retrieved passage longer and less precise. | ||
|
|
||
| Alternatively, upgrade your plan for more capacity, then try again. | ||
|
|
||
| ## When Storage Is Full | ||
|
|
||
| Once usage reaches your plan's limit, the workspace can no longer add knowledge content. | ||
|
|
||
| Uploading documents, adding or editing chunks, restoring archived documents, and re-indexing operations such as changing chunk settings are all blocked, including in knowledge bases that use **Economical** indexing. | ||
|
|
||
| To restore these actions, free space or upgrade your plan. | ||
|
|
||
| If a downgrade or an expired subscription leaves your data above the new plan's limit, nothing is deleted: your apps can still retrieve from these knowledge bases, but adding content stays blocked until usage fits the limit or you upgrade again. | ||
|
|
||
| ## Free Up Storage | ||
|
|
||
| Delete the documents or knowledge bases you no longer need. Their vectors are removed, and the space returns to your quota. | ||
|
|
||
| The usage shown in **Settings** > **Billing** refreshes on a delay, so the number takes a while to drop after a deletion. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.