Streaming Results

🌊 What is Streaming?

Streaming is the process of displaying AI-generated content as it's being created, rather than waiting for the full response to be ready.

⏳ Without Streaming

📤 User sends request
       ↓
🧠 Model thinks...
       ↓
✅ Model finishes entire response
       ↓
👀 User finally sees output

The user stares at a blank screen until everything is complete. 😐

⚡ With Streaming

📤 User sends request
       ↓
🚀 Model starts generating
       ↓
🔤 Token 1 appears
🔤 Token 2 appears
🔤 Token 3 appears
   ...

The response gradually appears on screen, creating the familiar ⌨️ typewriter effect — just like you see in ChatGPT! ✨

🤔 Why Does Streaming Work?

LLMs naturally produce text one token at a time 🪙. For example:

👤 User: Tell me about React

🤖 Model generates:

React
React is
React is a
React is a JavaScript
React is a JavaScript library
...

Since the model is already working incrementally under the hood 🔧, it makes perfect sense to show the output incrementally too — no waiting required! 🎉

⚙️ Lets add streaming in our `create_brochure` function

▶️ Create a new code cell and run it: ( In same File - part4.ipynb)

def stream_brochure(company_name, url):
    stream = openai.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content": brochure_system_prompt},
            {"role": "user", "content": get_brochure_user_prompt(company_name, url)}
          ],
        stream=True
    )    
    response = ""
    display_handle = display(Markdown(""), display_id=True)
    for chunk in stream:
        response += chunk.choices[0].delta.content or ''
        update_display(Markdown(response), display_id=display_handle.display_id)

This function generates a company brochure using AI, word by word, and displays it in real time.

1️⃣ Send a request to GPT

stream = openai.chat.completions.create(..., stream=True)

Asks GPT-4 to write a brochure, with stream=True so it sends back the response chunk by chunk instead of all at once.

2️⃣ Set up a live display

response = ""
display_handle = display(Markdown(""), display_id=True)

Creates an empty placeholder on screen, ready to be updated.

Part	Meaning
`Markdown("")`	Creates an empty markdown display
`display_id=True`	Gives it a unique ID so we can find it later
`display_handle`	Stores that ID/reference for future use

3️⃣ Build the response live

for chunk in stream:
    response += chunk.choices[0].delta.content or ''
    update_display(Markdown(response), display_id=...)

Each chunk = a few new words. They get added to the response and the display refreshes instantly — creating the ⌨️ typewriter effect.

Chunk 1: "Hello"
Response: "Hello"

Chunk 2: " World"
Response: "Hello World"

Part	Meaning
`Markdown(response)`	The new content to show
`display_id=display_handle.display_id`	Finds the same whiteboard using its name tag
`update_display`	Erases & rewrites the whiteboard with new content

🧠 In One Line:

Send prompt → receive words one by one → keep updating the screen until the brochure is complete.

▶️ Create a new code cell and run it:

stream_brochure("Codeveda", "<https://codeveda.dev>")