Routers & Abstraction Layers

As we wrap up direct provider calls, let's look at the final category — routers and abstraction layers.

🧐 What's the Difference?

🔀 OpenRouter in Action

OpenRouter is one of the most popular routers — and you may have already been using it throughout this lab.

Why use it?

• 🔑 One API key for all providers

• 💳 Single billing account

• 🧪 Easy model switching — just change the model name

• 🆓 Some models are completely free

▶️ Create a new code cell and run it:

response = openrouter.chat.completions.create(model="z-ai/glm-4.5", messages=tell_a_joke)
display(Markdown(response.choices[0].message.content))

Here we used GLM-4.5 — an open-source model from Chinese startup ZhipuAI, accessed entirely through OpenRouter.

💡 All model pricing, performance benchmarks, and availability are listed on openrouter.ai

🧩 LLM Abstraction Layers & Frameworks

🔗 LangChain

• 🏆 The most famous LLM framework — often the reason people take LLM engineering courses

• 🏗️ Heavyweight abstraction layer with many powerful built-in concepts

• 🔌 Lets you swap between different model providers by importing different langchain_<provider> packages

• 📦 Usage pattern:

  ▶️ Create a new code cell and run it:

  from langchain_openai import ChatOpenAI
  
  llm = ChatOpenAI(model="gpt-5-mini")
  response = llm.invoke(tell_a_joke)
  
  display(Markdown(response.content))

• ⚠️ Has a steep learning curve — there's a lot to learn before it feels natural

• ✅ Covered in depth in Later Sections

⚡ LiteLLM

• 🪶 Lightweight counterpart to LangChain — does what it says on the tin

• 🎯 Single unified interface to call any model from any provider

• 🧪 Usage pattern:

  ▶️ Create a new code cell and run it:

  from litellm import completion
  response = completion(model="openai/gpt-4.1", messages=tell_a_joke)
  reply = response.choices[0].message.content
  display(Markdown(reply))

• 🔀 Provider format: provider/model-name

  • AWS Bedrock → bedrock/...

  • Azure → azure/...

  • Google Vertex → vertex_ai/...

• 💰 Automatically reports input tokens, output tokens, and cost per call

• 🏭 Great for production systems — can track per-user token spend across models

💸 API Cost Reality Check

• Costs are quoted per million tokens — individual calls are tiny

• Example: a full Hamlet (shakespearean play) (~53K tokens) call cost ~0.5 cents

• Don't fear API costs for dev/testing, but do monitor unit economics at scale

• LiteLLM makes tracking per-call costs trivially easy