Feature Request: Selected Model Format Downloads and Zip Downloads

[james-333i]

In testing various models, I'm noticing that the downloader is downloading ALL files for models including:
original/compiled
original/packaged
split_einsum/compiled (or sometimes split_einsum_v2)
split_einsum/packaged

This potentially leads to 3-4 times the download size / storage required

Ideally we'd specify the variant we want to download and only download that. I think only the compiled version is necessary.

Separately, sometimes the files in each folders are zipped. Ideally if a zip is detected, it would be expanded after download automatically.

[tattn]

Thank you for your feedback.

It seems that this can be achieved by making the globs API provided in FileDownloader available for use in LLMSession as well.

LocalLLMClient/Sources/LocalLLMClientUtility/FileDownloader.swift

Line 52 in 085e287

case huggingFace(id: String, globs: Globs)

I will consider implementing this.

[james-333i]

Thanks. The primary goal would be to limit the total download size and support models that store files on huggingface as zip files. I guess LLMSession could also support choosing which variant to use if multiple are downloaded but I think usually you wouldn't want to download multiple variants due to space and you'd just pick one.

[tattn]

I’ve added an API that allows you to pass the Source of FileDownloader directly.
By specifying the necessary files using globs, you can avoid downloading unnecessary files.
Please try it out on the main branch.

https://github.com/tattn/LocalLLMClient/blob/main/Sources/LocalLLMClientMLX/LLMSession%2BMLX.swift#L20

[james-333i]

Thank you. Could you please provide an example for this so that it is clear?

[tattn]

The following is the default globs configuration.

let session = LLMSession(model: .mlx(source: .huggingFace(
    id: "mlx-community/Qwen3-1.7B-4bit",
    globs: ["*.safetensors", "*.json"]
)))

Please configure the globs so that unnecessary files are excluded. Specify the patterns in a format supported by fnmatch, which is the internal implementation of globs.
https://pubs.opengroup.org/onlinepubs/9699919799/functions/fnmatch.html

[james-333i]

Thanks. A few points of feedback / questions:

1. This covers selectively choosing files but does not seem to cover cases where there is a zip file such as:
   https://huggingface.co/Narsa3/coreml-Qwen2.5-0.5B-Instruct-4bit/tree/main

Correct me if I'm wrong, but I don't believe you are handling expanding zip files automatically after download.

2. Will this globs markup support specifying folder path? I can't tell from the fnmatch document. Would this work?

globs: ["original/compiled/."]

Take this one for example:
https://huggingface.co/apple/coreml-stable-diffusion-v1-5-palettized/tree/main

I could either choose a zip file option and have it expand or would want to specify the a specific folder and the files in it such as original/compiled.

[tattn]

The two reference links you provided are for CoreML models, not MLX.
These are entirely different types of models and are not supported by this library.

Is there a repository with a similar structure that uses an MLX model?

[james-333i]

Ok. I built my own stable diffusion pipeline since this library did not support it and that's what I'm using these models for. I don't have other examples off the top of my head but having unzip capabilities seems useful.

I have a custom downloader right now so this isn't high priority. I was just hoping to use a centralized method for all downloads.

[tattn]

Thank you for sharing the situation!

This library is currently focused on llama.cpp, MLX, and Foundation Models, and I’d like to concentrate on supporting use cases that are needed for those.

Regarding the downloader, I’m also open to considering it if many use cases require it for those libraries.
This is because I believe it would become difficult to maintain if I tried to support every possible case.

[tattn]

It might be a good idea to use a custom downloader suited to your app’s use case to handle the download, and then use this library’s feature to load the model from a local path.

let localURL = URL(fileURLWithPath: "/path/to/your/model")
let session = LLMSession(model: .mlx(url: localURL))