Ai Engineering Core / Exploring Models

Ways to Use LLMs

🗺️ Three Ways to Use LLMs

1. 💬 Chat Interfaces — Product Usage

This is the most familiar way — using a finished AI product with a UI. You're not touching the raw model directly; you're using a polished product built on top of it.

Examples: ChatGPT, Claude app, Gemini app

What's included under the hood:

  • 🖥️ A user interface

  • 🧠 Conversation memory

  • 🌐 Web search

  • 🛠️ Built-in tools and features

💡 When you use ChatGPT, you're not using GPT directly — you're using a full product that happens to be powered by GPT.

☁️ 2. Cloud APIs — Programmatic Access

How developers interact with models through code — send a request, get a response.

Two sub-categories

Call a model provider's endpoint over HTTP. You write the code, they run the model.

  • Requires an API key from the provider

  • Billed per token (input + output)

  • Examples: OpenAI, Anthropic, Cohere, Mistral

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Explain gradient descent"}]
)
print(response.content[0].text)

Libraries that sit on top of an API and add application-level features — memory, tools, retrieval, and agent logic. The model still runs in the cloud; the framework just structures how you talk to it.

  • LangChain is the most popular — adds conversation memory, document Q&A (RAG), and tool use

  • Swap the underlying model (GPT-4 → Claude) without rewriting your app

  • Good for: chatbots, document Q&A, autonomous agents

from langchain_openai import ChatOpenAI
from langchain.chains import ConversationChain

llm = ChatOpenAI(model="gpt-4o")       # model runs on OpenAI's servers
chain = ConversationChain(llm=llm)     # adds memory, history
chain.predict(input="What is backpropagation?")

Other options

Managed cloud platforms — AWS Bedrock, Google Vertex AI, Azure AI — let enterprises access multiple models through their existing cloud account, with added compliance, billing control, and private networking.

API routing platforms — like OpenRouter — let you call many providers through a single unified API, making it easy to switch models or optimise for cost.

Key benefit of this whole category: no infrastructure to manage. You write code, they handle the servers, GPUs, and uptime.

3. 💻 Local Inference — Run It Yourself

Running the model entirely on your own machine — no cloud, no API, no internet needed.

Benefits:

  • 🛡️ Full privacy

  • ✈️ Works offline

  • 💸 Zero API costs

  • 🔬 Complete control and experimentation

Two main ways to run models locally:

🤗 HuggingFace Transformers

  • Load model architecture, weights, and tokenizer directly in Python

  • Highly flexible with massive model selection

  • Best for advanced users who want full control

from transformers import AutoModel

🦙 Ollama

  • Simplified, beginner-friendly local model runtime

  • Pre-packaged models with optimized execution

  • Comes with a local API interface out of the box

Your App → Local API (Ollama) → Model on Your Machine

⚡ Feature

🦙 Ollama

🤗 HuggingFace

Ease of use

Very easy

Moderate

Flexibility

Limited

Very high

Performance

Optimized

Depends

Model selection

Curated

Massive

Local API

✅ Built-in

❌ Manual setup

📖 Key Terms to Remember

  • 🔢 Parameters — learned numbers that store the model's knowledge (e.g. 7B = 7 billion parameters)

  • ⚖️ Weights — the actual numerical values inside the model; weights = knowledge

  • ⚙️ Inference — Inference is simply running a trained model on new input to get an output.

  • 🗜️ Distillation — compressing knowledge from a large model into a smaller one