# Stable Diffusion Experiments
course: Module 4 — Generative AI & LLMs
module: Module-4-Generative-AI-LLMs
type: notebook
source_url: https://personal-learn.armco.dev/files/Module-4-Generative-AI-LLMs/General/21_June/Stable_Diffusion_Experiments.ipynb
---
[cell 1 code]
!pip install diffusers[torch] transformers accelerate safetensors matplotlib
[cell 2 markdown]
Step 1: Install dependencies
[cell 3 code]
"""Demo_02_Stable_Diffusion_Prompt_Engineering.ipynb
# **Prompt Engineering with Stable Diffusion**
### **Objective:**
To understand how changing the text prompt affects the generated image quality, style, lighting, and level of detail.
---
### **Steps to perform:**
1. Setup the Environment
2. Import the necessary libraries
3. Load the Stable Diffusion model
4. Define different prompt styles
5. Generate and compare images
"""
[cell 4 markdown]
Step 2: Import libraries
[cell 5 code]
import os
import torch
import matplotlib.pyplot as plt
from diffusers import StableDiffusionPipeline
[cell 6 markdown]
Step 3: Load the pretrained model
[cell 7 code]
model_id = "runwayml/stable-diffusion-v1-5"
pipeline = StableDiffusionPipeline.from_pretrained(
model_id,
torch_dtype=torch.float16
)
print("✅ Model loaded successfully.")
[cell 8 markdown]
Step 4: Move pipeline to GPU
[cell 9 code]
device = "cuda" if torch.cuda.is_available() else "cpu"
pipeline = pipeline.to(device)
print(f"✅ Pipeline moved to device: {device}")
[cell 11 code]
base_subject = "A futuristic sports car"
prompt_variations = [
"A futuristic sports car",
"A futuristic sports car, cinematic lighting",
"A futuristic sports car, cinematic lighting, ultra realistic",
"A futuristic sports car, cinematic lighting, ultra realistic, 8k, highly detailed",
"A futuristic sports car, cyberpunk city background, neon lights, cinematic, highly detailed"
]
[cell 12 code]
os.makedirs("prompt_engineering_outputs", exist_ok=True)
generated_images = []
for i, prompt in enumerate(prompt_variations):
print(f"\n🎨 Generating image {i+1}")
print("Prompt:", prompt)
image = pipeline(
prompt,
num_inference_steps=10
).images[0]
image_path = f"prompt_engineering_outputs/prompt_variation_{i+1}.jpg"
image.save(image_path)
print(f"✅ Saved: {image_path}")
generated_images.append(image)
[cell 13 code]
fig, axes = plt.subplots(1, len(generated_images), figsize=(20, 5))
for ax, img, prompt in zip(axes, generated_images, prompt_variations):
ax.imshow(img)
ax.axis("off")
ax.set_title(prompt[:40] + "...", fontsize=9)
plt.tight_layout()
plt.show()
[cell 14 code]
# 2. Experiment: num_inference_steps
# Change this line:
# image = pipeline(prompt).images[0]
# To this:
image = pipeline(
prompt,
num_inference_steps=30
).images[0]
# Then try different values:
# image = pipeline(
# prompt,
# num_inference_steps=10
# ).images[0]
# image = pipeline(
# prompt,
# num_inference_steps=50
# ).images[0]
# Meaning:
# 10 steps = faster, less detailed
# 30 steps = balanced
# 50 steps = slower, more refined
[cell 15 code]
# 3. Experiment: guidance_scale
# Change this line:
# image = pipeline(prompt).images[0]
# To this:
# image = pipeline(
# prompt,
# guidance_scale=7.5
# ).images[0]
# Now try:
# image = pipeline(
# prompt,
# guidance_scale=3
# ).images[0]
# image = pipeline(
# prompt,
# guidance_scale=12
# ).images[0]
# Meaning:
# 3 = more creative, may ignore prompt
# 7.5 = balanced
# 12 = follows prompt strongly
# 4. Experiment: same prompt, different guidance values
# Instead of changing image_prompts, keep one prompt and compare guidance.
# Replace your image_prompts with:
# image_prompts = [
# "A red dragon flying over a snowy mountain",
# "A red dragon flying over a snowy mountain",
# "A red dragon flying over a snowy mountain"
# ]
# Add this list below it:
# guidance_values = [3, 7.5, 12]
# Change your loop from:
# for i, prompt in enumerate(image_prompts):
# print(f"\n🎨 Generating image {i+1} for prompt: '{prompt}'")
# image = pipeline(prompt).images[0]
# To this:
# for i, prompt in enumerate(image_prompts):
# guidance = guidance_values[i]
# print(f"\n🎨 Generating image {i+1}")
# print(f"Prompt: {prompt}")
# print(f"Guidance Scale: {guidance}")
# image = pipeline(
# prompt,
# guidance_scale=guidance
# ).images[0]
# Now the notebook compares the same prompt with different guidance values.
# 5. Experiment: same prompt, different inference steps
# Replace your image_prompts with:
# image_prompts = [
# "A futuristic city on Mars, cinematic, highly detailed",
# "A futuristic city on Mars, cinematic, highly detailed",
# "A futuristic city on Mars, cinematic, highly detailed"
# ]
# Add this list below it:
# step_values = [10, 30, 50]
# Change your loop to:
# for i, prompt in enumerate(image_prompts):
# steps = step_values[i]
# print(f"\n🎨 Generating image {i+1}")
# print(f"Prompt: {prompt}")
# print(f"Inference Steps: {steps}")
# image = pipeline(
# prompt,
# num_inference_steps=steps
# ).images[0]
# This compares:
# 10 steps vs 30 steps vs 50 steps
# 6. Experiment: seed / reproducibility
# Seed means same starting noise.
# Add this before the loop:
# seed = 42
# Change this:
# image = pipeline(prompt).images[0]
# To this:
# generator = torch.Generator(device=device).manual_seed(seed)
# image = pipeline(
# prompt,
# generator=generator
# ).images[0]
# Now every time you run with the same prompt and same seed, you should get the same or very similar image.
# 7. Experiment: same prompt, different seeds
# Replace your image_prompts with:
# image_prompts = [
# "A magical castle floating in the sky, fantasy art",
# "A magical castle floating in the sky, fantasy art",
# "A magical castle floating in the sky, fantasy art"
# ]
# Add this below:
# seed_values = [10, 42, 99]
# Change your loop to:
# for i, prompt in enumerate(image_prompts):
# seed = seed_values[i]
# print(f"\n🎨 Generating image {i+1}")
# print(f"Prompt: {prompt}")
# print(f"Seed: {seed}")
# generator = torch.Generator(device=device).manual_seed(seed)
# image = pipeline(
# prompt,
# generator=generator
# ).images[0]
# This shows that:
# same prompt + different seed = different image
# 8. Experiment: negative prompt
# Add this before the loop:
# negative_prompt = "blurry, low quality, distorted, bad anatomy, ugly, deformed"
# Change this:
# image = pipeline(prompt).images[0]
# To this:
# image = pipeline(
# prompt,
# negative_prompt=negative_prompt
# ).images[0]
# Example prompt list:
# image_prompts = [
# "A realistic portrait of a warrior king, cinematic lighting",
# "A beautiful fantasy princess, detailed face",
# "A superhero standing in a futuristic city"
# ]
# Negative prompt tells the model what to avoid.