ai_pipeline from yesterday processes one text at a time. Your post-survey has fifty open-text responses to a question about research barriers. In your current workflow, how do you code all fifty?
Read each response, assign it a theme from my codebook, enter the code in a spreadsheet. Two minutes per response, an hour forty for fifty. And that's assuming I'm consistent across the batch.
A list comprehension calling the agent for each text gives you all fifty classifications in one expression — same coding schema applied consistently across every response:
agent = Agent(model, system_prompt="Classify this research barrier into one category: funding, time, access, skills, or other.")
return [agent.run_sync(t).output.strip().lower() for t in texts]
Does the same agent object get reused for each call, or does each iteration create a new agent?
The agent object is reused. You create it once outside the comprehension — the system_prompt is fixed for all calls. Each .run_sync(t) is a fresh call to the model with a new user prompt t. Creating the agent inside the comprehension would work but wastes construction overhead on every iteration.
So the agent is my coded inter-rater, and the list comprehension is applying it to every transcript. That's inter-rater reliability at scale — same coder, fifty transcripts, consistent criteria.
The analogy is exact. One agent, one coding schema, consistent application across the batch.
Fifty responses, five categories, one line of code. That's my entire qualitative coding round done before the coffee cools.
Spot-check with a random sample:
import random
sample = random.sample(list(zip(texts, labels)), 5)
for text, label in sample:
print(label, "|", text[:50])Batch classification has systematic biases — validate before reporting.
agent = Agent(
model,
system_prompt="Classify into: funding, time, access, skills, or other."
)
return [agent.run_sync(t).output.strip().lower() for t in texts]Construct the agent outside the comprehension. The system prompt is fixed; only the user prompt changes per iteration. Reusing the agent avoids repeated object construction overhead.
After batch classification, random-sample 10–15 responses and compare the agent's codes against yours. Calculate an agreement rate. For systematic reviews, aim for >80% agreement before reporting AI-assisted coding in your methods section. Flag systematic disagreements for prompt refinement.
ai_pipeline from yesterday processes one text at a time. Your post-survey has fifty open-text responses to a question about research barriers. In your current workflow, how do you code all fifty?
Read each response, assign it a theme from my codebook, enter the code in a spreadsheet. Two minutes per response, an hour forty for fifty. And that's assuming I'm consistent across the batch.
A list comprehension calling the agent for each text gives you all fifty classifications in one expression — same coding schema applied consistently across every response:
agent = Agent(model, system_prompt="Classify this research barrier into one category: funding, time, access, skills, or other.")
return [agent.run_sync(t).output.strip().lower() for t in texts]Does the same agent object get reused for each call, or does each iteration create a new agent?
The agent object is reused. You create it once outside the comprehension — the system_prompt is fixed for all calls. Each .run_sync(t) is a fresh call to the model with a new user prompt t. Creating the agent inside the comprehension would work but wastes construction overhead on every iteration.
So the agent is my coded inter-rater, and the list comprehension is applying it to every transcript. That's inter-rater reliability at scale — same coder, fifty transcripts, consistent criteria.
The analogy is exact. One agent, one coding schema, consistent application across the batch.
Fifty responses, five categories, one line of code. That's my entire qualitative coding round done before the coffee cools.
Spot-check with a random sample:
import random
sample = random.sample(list(zip(texts, labels)), 5)
for text, label in sample:
print(label, "|", text[:50])Batch classification has systematic biases — validate before reporting.
agent = Agent(
model,
system_prompt="Classify into: funding, time, access, skills, or other."
)
return [agent.run_sync(t).output.strip().lower() for t in texts]Construct the agent outside the comprehension. The system prompt is fixed; only the user prompt changes per iteration. Reusing the agent avoids repeated object construction overhead.
After batch classification, random-sample 10–15 responses and compare the agent's codes against yours. Calculate an agreement rate. For systematic reviews, aim for >80% agreement before reporting AI-assisted coding in your methods section. Flag systematic disagreements for prompt refinement.
Create a free account to get started. Paid plans unlock all tracks.