chore: Update AI Home Renovation Agent to utilize Gemini 3 Flash and Gemini 3 Pro, enhancing multimodal capabilities. Modify tools to support artifact management and ensure layout preservation in renderings.

2026-04-28 14:18:51 -05:00 · 2026-01-03 17:01:21 -08:00
parent a1030c00e7
commit 44efabeac6
3 changed files with 149 additions and 60 deletions
--- a/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/README.md
+++ b/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/README.md
@@ -3,12 +3,12 @@
 ### 🎓 FREE Step-by-Step Tutorial 
 **👉 [Click here to follow our complete step-by-step tutorial](https://www.theunwindai.com/p/build-an-ai-home-renovation-planner-agent-using-nano-banana) and learn how to build this from scratch with detailed code walkthroughs, explanations, and best practices.**

-A multi-agent system built with Google ADK that analyzes photos of your space, creates personalized renovation plans, and generates photorealistic renderings using Gemini 2.5 Flash's multimodal capabilities.
+A multi-agent system built with Google ADK that analyzes photos of your space, creates personalized renovation plans, and generates photorealistic renderings using Gemini 3 Flash and Gemini 3 Pro's multimodal capabilities.

 ## Features

 - **🔍 Smart Image Analysis**: Upload room photos and inspiration images - agent automatically detects and analyzes them
- **🎨 Photorealistic Rendering**: Generates professional-quality images of your renovated space using Gemini 2.5 Flash
+- **🎨 Photorealistic Rendering**: Generates professional-quality images of your renovated space using Gemini 3 Pro
 - **💰 Budget-Aware Planning**: Tailors recommendations to your budget constraints
 - **📊 Complete Roadmap**: Provides timeline, budget breakdown, contractor list, and action checklist
 - **🤖 Multi-Agent Orchestration**: Demonstrates Coordinator/Dispatcher + Sequential Pipeline patterns
--- a/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/agent.py
+++ b/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/agent.py
@@ -1,6 +1,6 @@
 """AI Home Renovation Planner - Coordinator/Dispatcher Pattern with Multimodal Vision

-This demonstrates ADK's Coordinator/Dispatcher Pattern with Gemini 2.5 Flash's multimodal
+This demonstrates ADK's Coordinator/Dispatcher Pattern with Gemini 3 Flash's multimodal
 capabilities where a routing agent analyzes requests and delegates to specialists:

 - General questions → Quick info agent
@@ -26,7 +26,7 @@ from .tools import (

 search_agent = LlmAgent(
    name="SearchAgent",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Searches for renovation costs, contractors, materials, and design trends",
    instruction="Use google_search to find current renovation information, costs, materials, and trends. Be concise and cite sources.",
    tools=[google_search],
@@ -108,7 +108,7 @@ def calculate_timeline(

 info_agent = LlmAgent(
    name="InfoAgent",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Handles general renovation questions and provides system information",
    instruction="""
 You are the Info Agent for the AI Home Renovation Planner.
@@ -135,7 +135,7 @@ Be enthusiastic about home improvement and helpful!

 rendering_editor = LlmAgent(
    name="RenderingEditor",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Edits existing renovation renderings based on user feedback",
    instruction="""
 You refine existing renovation renderings.
@@ -165,6 +165,10 @@ Call: edit_renovation_rendering(
 Be SPECIFIC in prompts - vague = poor results!

 After editing, briefly confirm the change.
+
+**IMPORTANT - DO NOT use markdown image syntax!**
+- Do NOT output `![image](filename.png)` or similar markdown image links
+- Simply confirm the edit was successful and mention the artifact is available in the artifacts panel
 """,
    tools=[edit_renovation_rendering, list_renovation_renderings],
 )
@@ -176,7 +180,7 @@ After editing, briefly confirm the change.

 visual_assessor = LlmAgent(
    name="VisualAssessor",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Analyzes room photos and inspiration images using visual AI",
    instruction="""
 You are a visual AI specialist. Analyze ANY uploaded images and detect their type automatically.
@@ -198,6 +202,16 @@ AUTOMATICALLY DETECT:
 - Key problems: [what needs fixing]
 - Improvement opportunities: [quick wins, major changes]

+**CRITICAL - DOCUMENT EXACT LAYOUT (for preservation in rendering):**
+- Window positions: [e.g., "large window on left wall above sink", "skylight in center"]
+- Door positions: [e.g., "doorway on right side"]
+- Cabinet layout: [e.g., "L-shaped upper and lower cabinets along back and left walls"]
+- Appliance positions: [e.g., "stove centered on back wall", "refrigerator on right"]
+- Sink location: [e.g., "under window on left wall"]
+- Counter layout: [e.g., "continuous counter along back and left walls"]
+- Special features: [e.g., "skylight", "breakfast bar", "island"]
+- Camera angle in photo: [e.g., "shot from doorway looking into kitchen"]
+
 ## For INSPIRATION images:
 **Inspiration Style:**
 - Style name: [modern farmhouse/minimalist/industrial/etc.]
@@ -237,9 +251,19 @@ Room Details:
 - Key Issues: [problems to address]
 - Improvement Opportunities: [suggested improvements]
 - Budget Constraint: $[amount if mentioned, or "Not specified"]
+
+**EXACT LAYOUT TO PRESERVE (critical for rendering):**
+- Windows: [exact positions and sizes]
+- Doors: [exact positions]
+- Cabinets: [configuration and placement - upper/lower, which walls]
+- Appliances: [stove, fridge, dishwasher positions]
+- Sink: [location]
+- Counter layout: [shape and coverage]
+- Special features: [skylights, islands, breakfast bars, etc.]
+- Camera angle: [perspective of the original photo]
 ```

-Be DETAILED in your analysis - this drives the quality of the generated rendering later.
+Be EXTREMELY DETAILED about the layout - the rendering must match this layout EXACTLY while only changing surface finishes.
 """,
    tools=[AgentTool(search_agent), estimate_renovation_cost],
 )
@@ -247,31 +271,49 @@ Be DETAILED in your analysis - this drives the quality of the generated renderin

 design_planner = LlmAgent(
    name="DesignPlanner",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Creates detailed renovation design plan",
    instruction="""
 Read from state: room_analysis, style_preferences, room_type, key_issues, opportunities, budget_constraint

 Create SPECIFIC, ACTIONABLE design plan tailored to their situation.

+**CRITICAL RULE - PRESERVE EXACT LAYOUT:**
+The design plan must KEEP THE EXACT SAME LAYOUT as the current room. DO NOT suggest:
+- Moving appliances to different locations
+- Reconfiguring cabinet positions
+- Adding or removing windows/doors
+- Changing the room's footprint or structure
+- Adding islands or removing existing features
+
+ONLY specify changes to SURFACE FINISHES applied to the existing layout:
+- Paint colors for existing walls and cabinets
+- New countertop material on existing counters
+- New flooring in the same floor area
+- New backsplash on existing walls
+- New hardware on existing cabinets
+- Lighting upgrades (can add under-cabinet lights, replace fixtures in same positions)
+
 ## Design Plan

 **Budget-Conscious Approach:**
 - If budget_constraint exists: Prioritize changes that give max impact for the money
 - Separate "must-haves" vs "nice-to-haves"

-**Design Specifications:**
- **Layout**: [keep same/modify - be specific about changes]
- **Colors**: [exact colors with names - "Benjamin Moore Simply White OC-117"]
- **Materials**: [specific products - "Shaker white cabinets", "Carrara quartz countertops"]
- **Flooring**: [type, color, installation]
- **Lighting**: [fixture types, placement, purpose]
- **Storage**: [solutions for identified needs]
- **Appliances**: [if applicable - keep/replace/upgrade]
- **Key Features**: [backsplash, hardware, special elements]
+**Design Specifications (surface finish changes ONLY - no layout changes):**
+- **Layout**: PRESERVE EXACTLY AS-IS (reference Visual Assessor's layout documentation)
+- **Cabinet Color**: [exact paint color with code - applied to EXISTING cabinets]
+- **Wall Color**: [exact paint color with code]
+- **Countertops**: [material and color - applied to EXISTING counter layout]
+- **Flooring**: [type, color - same floor area]
+- **Backsplash**: [material, pattern - same wall areas]
+- **Hardware**: [handles, pulls - replace on existing cabinets]
+- **Lighting**: [fixture upgrades in same positions, add under-cabinet if applicable]
+- **Appliances**: [keep existing OR replace with similar size in SAME locations]
+- **Key Features**: [decorative elements only]

 **Style Consistency:**
-If inspiration photo provided: Match that aesthetic precisely
+If inspiration photo provided: Match that aesthetic precisely using ONLY surface finish changes
 If no inspiration: Use style_preferences from state

 Use calculate_timeline tool with room_type and renovation_scope.
@@ -281,17 +323,23 @@ Use calculate_timeline tool with room_type and renovation_scope.
 ```
 DESIGN COMPLETE

-Renovation Scope: [cosmetic/moderate/full/luxury]
-Design Approach: [preserve_layout/reconfigure_layout]
+Renovation Scope: [cosmetic/moderate - NO structural changes]
+Layout: PRESERVED EXACTLY (no changes to cabinet positions, appliance locations, or room structure)
+
+Surface Finish Changes:
+- Cabinets: [color change only]
+- Walls: [paint color]
+- Countertops: [material/color]
+- Flooring: [type/color]
+- Backsplash: [material/pattern]
+- Hardware: [style/finish]
+- Lighting: [upgrades]

 Materials Summary:
-[Detailed list with product names]
-
-Design Plan Summary:
-[All specifications from above]
+[Detailed list with product names and color codes]
 ```

-Be SPECIFIC with product names, colors, dimensions. This drives the rendering quality.
+Be SPECIFIC with product names, colors, dimensions. The rendering must show the EXACT same layout with only the surface finishes changed.
 """,
    tools=[calculate_timeline],
 )
@@ -299,7 +347,7 @@ Be SPECIFIC with product names, colors, dimensions. This drives the rendering qu

 project_coordinator = LlmAgent(
    name="ProjectCoordinator",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Coordinates renovation timeline, budget, execution plan, and generates photorealistic renderings",
    instruction="""
 Read conversation history to extract:
@@ -342,21 +390,31 @@ Build an EXTREMELY DETAILED prompt that incorporates:
 **Prompt Structure:**
 "Professional interior photography of a renovated [room_type]. 

-Current Space Context: [If Visual Assessor analyzed photos, mention key layout features to preserve]
+**CRITICAL - PRESERVE EXACT LAYOUT**: The room must maintain the EXACT same layout, structure, and spatial arrangement as the original photo:
+- Same window positions and sizes
+- Same door locations
+- Same cabinet configuration and placement
+- Same appliance positions (stove, sink, refrigerator in same spots)
+- Same architectural features (skylights, alcoves, etc.)
+- Same room dimensions and proportions
+- Same camera angle/perspective as the original photo

-Design Specifications:
+ONLY change the surface finishes, colors, materials, and decorative elements - NOT the structure or layout.
+
+Design Specifications (changes to apply to the EXISTING layout):
 - Style: [exact style from design plan]
- Colors: [specific color names with codes - e.g., 'Benjamin Moore Simply White OC-117 on walls']
- Cabinets/Fixtures: [exact specifications - e.g., 'white shaker style cabinets with brushed nickel hardware']
- Countertops: [material and color - e.g., 'Carrara marble-look quartz countertops']
- Flooring: [type and color - e.g., 'light oak luxury vinyl plank flooring']
- Backsplash: [pattern and material - e.g., 'white subway tile in classic running bond']
- Lighting: [specific fixtures - e.g., 'recessed LED lights plus pendant lights over island']
- Appliances: [if applicable - e.g., 'stainless steel appliances']
- Key Features: [all important elements from design]
+- Cabinet Color: [specific color names with codes - e.g., 'Benjamin Moore Simply White OC-117' - apply to EXISTING cabinets in their current positions]
+- Wall Color: [specific paint color]
+- Countertops: [material and color - apply to EXISTING counter layout]
+- Flooring: [type and color - same floor area]
+- Backsplash: [pattern and material - same wall areas]
+- Hardware: [handles, pulls - replace on existing cabinets]
+- Lighting: [specific fixtures - same positions or clearly specified additions]
+- Appliances: [keep existing OR specify replacements in SAME locations]
+- Key Features: [all important elements]

-Camera: Wide-angle interior photography, eye-level perspective
-Quality: Photorealistic, 8K, professional interior design magazine, natural lighting, bright and airy"
+Camera: Match the EXACT camera angle from the original photo
+Quality: Photorealistic, 8K, professional interior design magazine, natural lighting"

 Parameters:
 - prompt: [your ultra-detailed prompt above]
@@ -366,6 +424,12 @@ Parameters:
 **After generating:**
 Briefly describe (2-3 sentences) key features visible in the rendering and how it addresses their needs.

+**IMPORTANT - DO NOT use markdown image syntax!**
+- Do NOT output `![image](filename.png)` or similar markdown image links
+- Do NOT try to display the image inline with markdown
+- Simply mention that the rendering has been generated and saved as an artifact
+- The user can view the artifact through the artifacts panel
+
 **Note**: Image editing from uploaded photos has limitations in ADK Web. We generate fresh renderings based on detailed descriptions from the analysis.
 """,
    tools=[generate_renovation_rendering, edit_renovation_rendering, list_renovation_renderings],
@@ -390,7 +454,7 @@ planning_pipeline = SequentialAgent(

 root_agent = LlmAgent(
    name="HomeRenovationPlanner",
-    model="gemini-2.5-flash",
+    model="gemini-3-flash-preview",
    description="Intelligent coordinator that routes renovation requests to the appropriate specialist or planning pipeline. Supports image analysis!",
    instruction="""
 You are the Coordinator for the AI Home Renovation Planner.
--- a/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/tools.py
+++ b/advanced_ai_agents/multi_agent_apps/ai_home_renovation_agent/tools.py
@@ -110,7 +110,7 @@ class GenerateRenovationRenderingInput(BaseModel):


 class EditRenovationRenderingInput(BaseModel):
-    artifact_filename: str = Field(..., description="The filename of the rendering artifact to edit.")
+    artifact_filename: str = Field(default=None, description="The filename of the rendering artifact to edit. If not provided, uses the last generated rendering.")
    prompt: str = Field(..., description="The prompt describing the desired changes (e.g., 'make cabinets darker', 'add pendant lights', 'change floor to hardwood').")
    asset_name: str = Field(default=None, description="Optional: specify asset name for the new version (defaults to incrementing current asset).")
    reference_image_filename: str = Field(default=None, description="Optional: filename of a reference image to guide the edit. Use 'latest' for most recent upload.")
@@ -124,7 +124,7 @@ async def generate_renovation_rendering(tool_context: ToolContext, inputs: Gener
    """
    Generates a photorealistic rendering of a renovated space based on the design plan.
    
-    This tool uses Gemini 2.5 Flash's image generation capabilities to create visual 
+    This tool uses Gemini 3 Pro's image generation capabilities to create visual 
    representations of renovation plans. It can optionally use current room photos 
    and inspiration images as references.
    """
@@ -134,7 +134,10 @@ async def generate_renovation_rendering(tool_context: ToolContext, inputs: Gener
    logger.info("Starting renovation rendering generation")
    try:
        client = genai.Client()
-        inputs = GenerateRenovationRenderingInput(**inputs)
+        
+        # Handle inputs that might come as dict instead of Pydantic model
+        if isinstance(inputs, dict):
+            inputs = GenerateRenovationRenderingInput(**inputs)
        
        # Handle reference images (current room photo or inspiration)
        reference_images = []
@@ -163,31 +166,40 @@ async def generate_renovation_rendering(tool_context: ToolContext, inputs: Gener
        
        Original description: {inputs.prompt}
        
+        **CRITICAL REQUIREMENT - PRESERVE EXACT LAYOUT:**
+        The generated image MUST maintain the EXACT same room layout, structure, and spatial arrangement described in the prompt:
+        - Keep all windows, doors, skylights in their exact positions
+        - Keep all cabinets, counters, appliances in their exact positions
+        - Keep the same room dimensions and proportions
+        - Keep the same camera angle/perspective
+        - ONLY change surface finishes: paint colors, cabinet colors, countertop materials, flooring, backsplash, hardware, and decorative elements
+        - DO NOT move, add, or remove any structural elements or major fixtures
+        
        Enhance this to be a professional interior photography prompt that includes:
-        - Specific camera angle (wide-angle, eye-level perspective)
-        - Exact colors and materials mentioned
-        - Realistic lighting (natural light from windows, fixture types)
-        - Spatial layout and dimensions
-        - Texture and finish details
+        - Specific camera angle (match original photo perspective if described)
+        - Exact colors and materials mentioned (apply to existing surfaces)
+        - Realistic lighting (natural light from existing windows, fixture types)
+        - Maintain existing spatial layout and dimensions
+        - Texture and finish details for the new materials
        - Professional interior design photography quality
        
        Aspect ratio: {inputs.aspect_ratio}
        """
        
        if reference_images:
-            base_rewrite_prompt += "\nUse the provided reference image(s) as inspiration for style, layout, or visual elements."
+            base_rewrite_prompt += "\n\n**Reference Image Layout:** The reference image shows the EXACT layout that must be preserved. Match the camera angle, room structure, window/door positions, and furniture/appliance placement EXACTLY. Only change the surface finishes and colors."
        
-        base_rewrite_prompt += "\n\n**Important:** Output your prompt as a single detailed paragraph optimized for photorealistic interior rendering."
+        base_rewrite_prompt += "\n\n**Important:** Output your prompt as a single detailed paragraph optimized for photorealistic interior rendering. Emphasize that the layout must remain unchanged."
        
        # Get enhanced prompt
        rewritten_prompt_response = client.models.generate_content(
-            model="gemini-2.5-flash", 
+            model="gemini-3-flash-preview", 
            contents=base_rewrite_prompt
        )
        rewritten_prompt = rewritten_prompt_response.text
        logger.info(f"Enhanced prompt: {rewritten_prompt}")

-        model = "gemini-2.5-flash-image"
+        model = "gemini-3-pro-image-preview"
        
        # Build content parts
        content_parts = [types.Part.from_text(text=rewritten_prompt)]
@@ -229,6 +241,7 @@ async def generate_renovation_rendering(tool_context: ToolContext, inputs: Gener
                inline_data = chunk.candidates[0].content.parts[0].inline_data
                
                # Create a Part object from the inline data
+                # The inline_data already contains the mime_type from the API response
                image_part = types.Part(inline_data=inline_data)
                
                try:
@@ -247,7 +260,7 @@ async def generate_renovation_rendering(tool_context: ToolContext, inputs: Gener
                    
                    logger.info(f"Saved rendering as artifact '{artifact_filename}' (version {version})")
                    
-                    return f"✅ Renovation rendering generated successfully!\n\nSaved as: **{artifact_filename}** (version {version} of {inputs.asset_name})\n\nThis photorealistic rendering shows your renovated space based on the design plan."
+                    return f"✅ Renovation rendering generated successfully!\n\nThe rendering has been saved and is available in the artifacts panel. Artifact name: {inputs.asset_name} (version {version}).\n\nNote: The image is stored as an artifact and can be accessed through the session artifacts, not as a direct image link."
                    
                except Exception as e:
                    logger.error(f"Error saving artifact: {e}")
@@ -282,14 +295,25 @@ async def edit_renovation_rendering(tool_context: ToolContext, inputs: EditRenov

    try:
        client = genai.Client()
-        inputs = EditRenovationRenderingInput(**inputs)
+        
+        # Handle inputs that might come as dict instead of Pydantic model
+        if isinstance(inputs, dict):
+            inputs = EditRenovationRenderingInput(**inputs)
+        
+        # Get artifact_filename from session state if not provided
+        artifact_filename = inputs.artifact_filename
+        if not artifact_filename:
+            artifact_filename = tool_context.state.get("last_generated_rendering")
+            if not artifact_filename:
+                return "❌ No artifact_filename provided and no previous rendering found in session. Please generate a rendering first or specify the artifact filename."
+            logger.info(f"Using last generated rendering from session: {artifact_filename}")
        
        # Load the existing rendering
-        logger.info(f"Loading artifact: {inputs.artifact_filename}")
+        logger.info(f"Loading artifact: {artifact_filename}")
        try:
-            loaded_image_part = await tool_context.load_artifact(inputs.artifact_filename)
+            loaded_image_part = await tool_context.load_artifact(artifact_filename)
            if not loaded_image_part:
-                return f"❌ Could not find rendering artifact: {inputs.artifact_filename}"
+                return f"❌ Could not find rendering artifact: {artifact_filename}"
        except Exception as e:
            logger.error(f"Error loading artifact: {e}")
            return f"Error loading rendering artifact: {e}"
@@ -307,7 +331,7 @@ async def edit_renovation_rendering(tool_context: ToolContext, inputs: EditRenov
                if reference_image_part:
                    logger.info(f"Using reference image for editing: {ref_filename}")

-        model = "gemini-2.5-flash-image"
+        model = "gemini-3-pro-image-preview"

        # Build content parts
        content_parts = [loaded_image_part, types.Part.from_text(text=inputs.prompt)]
@@ -337,7 +361,7 @@ async def edit_renovation_rendering(tool_context: ToolContext, inputs: EditRenov
                asset_name = current_asset_name
            else:
                # Extract from filename
-                base_name = inputs.artifact_filename.split('_v')[0] if '_v' in inputs.artifact_filename else "renovation_rendering"
+                base_name = artifact_filename.split('_v')[0] if '_v' in artifact_filename else "renovation_rendering"
                asset_name = base_name
        
        version = get_next_version_number(tool_context, asset_name)
@@ -361,6 +385,7 @@ async def edit_renovation_rendering(tool_context: ToolContext, inputs: EditRenov
                inline_data = chunk.candidates[0].content.parts[0].inline_data
                
                # Create a Part object from the inline data
+                # The inline_data already contains the mime_type from the API response
                edited_image_part = types.Part(inline_data=inline_data)
                
                try:
@@ -379,7 +404,7 @@ async def edit_renovation_rendering(tool_context: ToolContext, inputs: EditRenov
                    
                    logger.info(f"Saved edited rendering as artifact '{edited_artifact_filename}' (version {version})")
                    
-                    return f"✅ Rendering edited successfully!\n\nSaved as: **{edited_artifact_filename}** (version {version} of {asset_name})\n\nThe rendering has been updated based on your feedback."
+                    return f"✅ Rendering edited successfully!\n\nThe updated rendering has been saved and is available in the artifacts panel. Artifact name: {asset_name} (version {version}).\n\nNote: The image is stored as an artifact and can be accessed through the session artifacts, not as a direct image link."
                    
                except Exception as e:
                    logger.error(f"Error saving edited artifact: {e}")