AWQ (Llama3.1/3.3-8B/70B)

[dreaming-panda]

[dreaming-panda]

Refer
https://github.com/Infini-AI-Lab/UMbreLLa/blob/721da76b58cb5518756dda2231facf6033ad04e4/umbrella/quantization/awq_utils.py#L5

https://github.com/Infini-AI-Lab/UMbreLLa/blob/721da76b58cb5518756dda2231facf6033ad04e4/umbrella/models/llama.py#L222
and

https://github.com/Infini-AI-Lab/UMbreLLa/blob/721da76b58cb5518756dda2231facf6033ad04e4/umbrella/models/llama_layer.py#L165

[annanyapr]

@dreaming-panda I have raised this PR which should implement AWQ #11 . Can you give me access to a server so that I can verify these changes. I see that we have some commands like numactl and I would ideally want to execute them on a preconfigured server.

[annanyapr]

Can you also take a look at #11 @dreaming-panda