[AI][ML][LLM]Run Llama 3B through Docker Model

 Install necessary plugin:

# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

# echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# apt-get update

# apt-get install docker-model-plugin

Pull:

# docker model pull ai/llama3.2

Run in Interactive mode:

# docker model run ai/llama3.2

Exit Interactive mode:

> /bye

Run in Single-prompt mode:

# docker model run ai/llama3.2 "Explain how Docker containers work in one sentence."

-----Failed to carry out from here:

Enable OpenAI mode:

# docker desktop enable model-runner --tcp 12434

Test OpenAI:

# curl http://localhost:12434/engines/v1/models

Remotely use through OpenAI:

# curl http://localhost:12434/engines/v1/chat/completions \

  -H "Content-Type: application/json" \

  -d '{

    "model": "ai/llama3.2",

    "messages": [{"role": "user", "content": "Hello!"}]

  }'