{ "name": "Compare different LLM responses side-by-side with Google Sheets", "nodes": [ { "id": "", "name": "When chat message received", "type": "@n8n/n8n-nodes-langchain.chatTrigger", "position": [ -7400, 3040 ] }, { "id": "", "name": "Loop Over Items", "type": "n8n-nodes-base.splitInBatches", "position": [ -5960, 3040 ] }, { "id": "", "name": "Simple Memory", "type": "@n8n/n8n-nodes-langchain.memoryBufferWindow", "position": [ -4880, 3000 ] }, { "id": "", "name": "Chat Memory Manager", "type": "@n8n/n8n-nodes-langchain.memoryManager", "position": [ -4980, 3180 ] }, { "id": "", "name": "Sticky Note", "type": "n8n-nodes-base.stickyNote", "position": [ -8120, 2600 ], "parameters": { "width": 640, "height": 1180, "content": "## Easily Compare LLMs Using OpenAI and Google Sheets\n\nThis workflow allows you to **easily evaluate and compare the outputs of two language models (LLMs)** before choosing one for production.\n\nIn the" } }, { "id": "", "name": "OpenRouter Chat Model", "type": "@n8n/n8n-nodes-langchain.lmChatOpenRouter", "position": [ -5180, 3000 ] }, { "id": "", "name": "Sticky Note1", "type": "n8n-nodes-base.stickyNote", "position": [ -7220, 2620 ], "parameters": { "width": 360, "height": 580, "content": "## Define Models to Compare\n\nThis node defines the array of model IDs to be compared.\n\nIn this template, we compare two models using the OpenRouter API. You can modify the list by specifying the full " } }, { "id": "", "name": "Sticky Note2", "type": "n8n-nodes-base.stickyNote", "position": [ -6500, 2620 ], "parameters": { "width": 360, "height": 580, "content": "## Set model, sessionId, chatInput, sessionIdBase\n\nThis node prepares the variables used during the loop that queries each model.\n\n- **model**: The ID of the model being used in the current iteration." } }, { "id": "", "name": "Set model, sessionId, chatInput, sessionIdBase", "type": "n8n-nodes-base.set", "position": [ -6380, 3040 ] }, { "id": "", "name": "AI Agent", "type": "@n8n/n8n-nodes-langchain.agent", "position": [ -5480, 3180 ] }, { "id": "", "name": "Sticky Note3", "type": "n8n-nodes-base.stickyNote", "position": [ -5600, 3160 ], "parameters": { "width": 540, "height": 520, "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n## AI Agent\n\nThis AI Agent is connected to OpenRouter Models. The model is selected dynamically from the variable `{{$json.model}}`, defined earlier.\n\nMemory is isolated per model usi" } }, { "id": "", "name": "Sticky Note4", "type": "n8n-nodes-base.stickyNote", "position": [ -5040, 3160 ], "parameters": { "width": 380, "height": 520, "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n## Chat Memory Manager\n\nThis node handles retrieval of prior context for the chat session. It helps with qualitative evaluation by storing context that’s injected into the Google Sheet" } }, { "id": "", "name": "Sticky Note5", "type": "n8n-nodes-base.stickyNote", "position": [ -4640, 3160 ], "parameters": { "width": 380, "height": 760, "content": "\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n## Prepare Data for Chat and Google Sheets\n\nThis node sets the following fields:\n\n- **output**: The model's response, formatted for chat display with visual separation to make comparis" } }, { "id": "", "name": "Concatenate Chat Answers", "type": "n8n-nodes-base.summarize", "position": [ -5300, 2620 ] }, { "id": "", "name": "Sticky Note6", "type": "n8n-nodes-base.stickyNote", "position": [ -5080, 2120 ], "parameters": { "width": 460, "height": 500, "content": "## Add Model Results to Google Sheet\n\nThis Google Sheets step records both model responses along for evaluation.\n\n⚠️ Depending on the length of model responses, you may need to adjust row height or co" } }, { "id": "", "name": "Group Model Outputs for Evaluation", "type": "n8n-nodes-base.aggregate", "position": [ -5300, 2440 ] }, { "id": "", "name": "Add Model Results to Google Sheet", "type": "n8n-nodes-base.googleSheets", "position": [ -4940, 2440 ] }, { "id": "", "name": "Prepare Data for Chat and Google Sheets", "type": "n8n-nodes-base.set", "position": [ -4500, 3180 ] }, { "id": "", "name": "Define Models to Compare", "type": "n8n-nodes-base.set", "position": [ -7100, 3040 ] }, { "id": "", "name": "Split Models into Items", "type": "n8n-nodes-base.splitOut", "position": [ -6760, 3040 ] }, { "id": "", "name": "Set Output for Chat UI", "type": "n8n-nodes-base.set", "position": [ -4940, 2620 ] } ], "connections": { "AI Agent": { "main": [ [ { "node": "Chat Memory Manager", "type": "main", "index": 0 } ] ] }, "Simple Memory": { "ai_memory": [ [ { "node": "Chat Memory Manager", "type": "ai_memory", "index": 0 }, { "node": "AI Agent", "type": "ai_memory", "index": 0 } ] ] }, "Loop Over Items": { "main": [ [ { "node": "Concatenate Chat Answers", "type": "main", "index": 0 }, { "node": "Group Model Outputs for Evaluation", "type": "main", "index": 0 } ], [ { "node": "AI Agent", "type": "main", "index": 0 } ] ] }, "Chat Memory Manager": { "main": [ [ { "node": "Prepare Data for Chat and Google Sheets", "type": "main", "index": 0 } ] ] }, "OpenRouter Chat Model": { "ai_languageModel": [ [ { "node": "AI Agent", "type": "ai_languageModel", "index": 0 } ] ] }, "Split Models into Items": { "main": [ [ { "node": "Set model, sessionId, chatInput, sessionIdBase", "type": "main", "index": 0 } ] ] }, "Concatenate Chat Answers": { "main": [ [ { "node": "Set Output for Chat UI", "type": "main", "index": 0 } ] ] }, "Define Models to Compare": { "main": [ [ { "node": "Split Models into Items", "type": "main", "index": 0 } ] ] }, "When chat message received": { "main": [ [ { "node": "Define Models to Compare", "type": "main", "index": 0 } ] ] }, "Group Model Outputs for Evaluation": { "main": [ [ { "node": "Add Model Results to Google Sheet", "type": "main", "index": 0 } ] ] }, "Prepare Data for Chat and Google Sheets": { "main": [ [ { "node": "Loop Over Items", "type": "main", "index": 0 } ] ] }, "Set model, sessionId, chatInput, sessionIdBase": { "main": [ [ { "node": "Loop Over Items", "type": "main", "index": 0 } ] ] } } }