Trying Other Model Providers Chat Completion API

🌟 The Rise of OpenAI-Compatible Endpoints

OpenAI's Chat Completions API became so widely adopted that other AI providers started building their own endpoints with the exact same structure. These are called "OpenAI Compatible Endpoints". 🤝

For example, Google created one at:

<https://generativelanguage.googleapis.com/v1beta/openai/>

🔑 The Smart Decision by OpenAI

OpenAI took it a step further — they allowed developers to reuse the same Python SDK for other providers, simply by swapping out the base URL and API key.

So instead of learning a brand new library for every provider, you can do this:

gemini = OpenAI(
    base_url="<https://generativelanguage.googleapis.com/v1beta/openai/>",
    api_key="AIz...."
)
gemini.chat.completions.create(...)2

⚠️ Important: Even though OpenAI appears in the code, no OpenAI model is being used here. You're simply borrowing their lightweight SDK as a communication tool to reach Google's servers. 🌐

🧪 Try Google Gemini

🔑 Step 1: Setup API Key

• Visit Google AI Studio : https://aistudio.google.com/

• Generate your API key : https://aistudio.google.com/api-keys

• Add it to your .env file and save 💾

GOOGLE_API_KEY=AIz...

🤫 Step 2: Loading Environment Variables

▶️ Create a new code cell and run it:

GEMINI_BASE_URL = "<https://generativelanguage.googleapis.com/v1beta/openai/>"

load_dotenv(override=True)

google_api_key = os.getenv("GOOGLE_API_KEY")

if not google_api_key:
    print("No API key was found - please be sure to add your key to the .env file, and save the file! Or you can skip the next 2 cells if you don't want to use Gemini")
elif not google_api_key.startswith("AIz"):
    print("An API key was found, but it doesn't start AIz")
else:
    print("API key found and looks good so far!")

🏃‍♂️ Step 3 : Call Gemini Endpoint

▶️ Create a new code cell and run it:

gemini = OpenAI(base_url=GEMINI_BASE_URL, api_key=google_api_key)

response = gemini.chat.completions.create(model="gemini-2.5-flash-lite", messages=[{"role": "user", "content": "Tell me a fun fact"}])

response.choices[0].message.content

🧪 Try Ollama

Here's the exciting part — Ollama runs right on your local machine, and it too exposes an OpenAI-compatible endpoint! 🏠

🛋️ Step 1: Setup up Ollama

First, make sure Ollama is running.

▶️ Create a new code cell and run it:

requests.get("<http://localhost:11434>").content

If the following check doesn't return "Ollama is running", then:

▶️ Create a new code cell and run it:

!ollama serve

⬇️ Step 2 — Download Llama 3.2 from Meta

▶️ Create a new code cell and run it:

!ollama pull llama3.2

💡 Note: If your machine has limited RAM, use llama3.2:1b instead.
❌ Avoid llama3.3 or llama4 — they're too large for most local machines.

🔌 Step 3 — Connect Using the OpenAI SDK

▶️ Create a new code cell and run it:

OLLAMA_BASE_URL = "<http://localhost:11434/v1>"

ollama = OpenAI(base_url=OLLAMA_BASE_URL, api_key='ollama')

🧪 Step 4 — Try It Out

▶️ Create a new code cell and run it:

response = ollama.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Tell me a fun fact"}]
)
print(response.choices[0].message.content)

🤖 Bonus — Try DeepSeek R1

▶️ Create a new code cell and run it:

!ollama pull deepseek-r1:1.5b

▶️ Create a new code cell and run it:

response = ollama.chat.completions.create(
    model="deepseek-r1:1.5b",
    messages=[{"role": "user", "content": "Tell me a fun fact"}]
)
print(response.choices[0].message.content)

🌐 All of this runs 100% locally — no internet, no cloud, no API costs. Just your machine doing the work! 🚀

✅ Special Note

The ! tells Jupyter to run the command in the terminal/shell instead of treating it as Python code.

!ollama pull deepseek-r1:1.5b  # runs in terminal
ollama pull deepseek-r1:1.5b   # would crash — not valid Python ❌

🎯 Think of ! as saying: "Hey, run this in the command line, not Python."

🛑 How to Stop Ollama from Running in the Background Always

If Ollama starts automatically without you running ollama serve, here's how to stop it and prevent it from auto-starting:

🪟 Windows

Stop it:
Right-click the Ollama icon in the bottom-right system tray → click Quit Ollama

Prevent auto-start:
Open Task Manager (Ctrl + Shift + Esc) → go to Startup Apps tab → find Ollama → right-click → Disable

🍎 macOS

Stop it:
Click the Ollama icon in the top menu bar → click the ⚙️ gear icon → select Quit Ollama

Prevent auto-start:
Open System Settings → search for Login Items → under Allow in the Background, find Ollama → toggle it Off

🐧 Linux

Stop it:

sudo systemctl stop ollama

Prevent auto-start:

sudo systemctl disable ollama