Skip to content
automationprompt-engineering

Building Your First AI-Powered Test Case Generator

📅 2026-01-12âąī¸ 12 minâœī¸ By Oleg Neskoromnyi

You don't need to be a machine learning expert to build an AI-powered test case generator. In fact, you can build one in an afternoon using the OpenAI API, some Python, and your QA expertise.

I built mine in 4 hours. It now generates 80% of my test cases, and I spend my time on the 20% that requires critical thinking.

Let me show you exactly how.

Prerequisites: Basic Python knowledge, OpenAI API key ($5 credit is enough to start), and your existing test case templates.

Why Build Your Own Instead of Using ChatGPT Directly?

Good question. Here's why a custom generator wins:

  1. Consistency - Same format every time, matches your team's template
  2. Context - Pre-loaded with your domain knowledge and business rules
  3. Batch processing - Generate test cases for 10 features in minutes
  4. Version control - Track prompt improvements over time
  5. Integration - Connect to Jira, TestRail, or your test management tool

Think of it as "ChatGPT, but tailored to your specific testing needs."

Architecture Overview

Here's what we're building:

|architecture.py
# Simple architecture
User Input (Feature Description)
  ↓
Prompt Template (Your QA expertise)
  ↓
OpenAI API (GPT-4)
  ↓
Test Cases (Your format)
  ↓
Output (CSV, JSON, or direct to test tool)

Step 1: Set Up Your Environment

First, install the OpenAI Python library:

|setup.sh
# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install openai python-dotenv

# Create .env file
echo "OPENAI_API_KEY=your_key_here" > .env

Never commit your API key to git! Always use environment variables or a .env file (and add it to .gitignore).

Step 2: Create Your Prompt Template

This is where your QA expertise shines. The better your prompt, the better your test cases.

|prompt_template.py
from string import Template

# Your QA expertise goes here
PROMPT_TEMPLATE = Template("""
You are a senior QA engineer. Generate comprehensive test cases for the following feature.

Feature Name: $feature_name
Description: $description
User Story: $user_story

Requirements:
$requirements

Generate test cases in this exact format:

Test Case ID | Title | Preconditions | Steps | Expected Result | Priority | Type

Include:
- Positive test cases (happy path)
- Negative test cases (error handling)
- Edge cases and boundary conditions
- Security considerations
- Performance edge cases

Use Given-When-Then format for steps.
Priority: Critical, High, Medium, Low
Type: Functional, Security, Performance, Usability

Generate 15-20 test cases covering all scenarios.
""")

Notice what I included:

  • Clear role - "You are a senior QA engineer"
  • Exact format - No surprises in output
  • Specific categories - Positive, negative, edge cases
  • My team's conventions - Given-When-Then, priority levels

Step 3: Build the Generator

Now let's write the actual generator:

|test_case_generator.py
import os
from openai import OpenAI
from dotenv import load_dotenv
from prompt_template import PROMPT_TEMPLATE

load_dotenv()

class TestCaseGenerator:
  def __init__(self):
      self.client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

  def generate(self, feature_name, description, user_story, requirements):
      """Generate test cases for a feature"""

      # Fill in the prompt template
      prompt = PROMPT_TEMPLATE.substitute(
          feature_name=feature_name,
          description=description,
          user_story=user_story,
          requirements=requirements
      )

      # Call OpenAI API
      response = self.client.chat.completions.create(
          model="gpt-4",  # or "gpt-4-turbo" for faster/cheaper
          messages=[
              {
                  "role": "system",
                  "content": "You are an expert QA engineer specializing in test case design."
              },
              {
                  "role": "user",
                  "content": prompt
              }
          ],
          temperature=0.7,  # Balance creativity and consistency
          max_tokens=2000
      )

      return response.choices[0].message.content

  def save_to_csv(self, test_cases, filename):
      """Save generated test cases to CSV"""
      with open(filename, 'w') as f:
          f.write(test_cases)
      print(f"Test cases saved to {filename}")

# Usage
if __name__ == "__main__":
  generator = TestCaseGenerator()

  test_cases = generator.generate(
      feature_name="User Login",
      description="Email/password authentication with remember me option",
      user_story="As a user, I want to log in with my email and password so I can access my account",
      requirements="""
      - Email and password fields required
      - "Remember me" checkbox (optional)
      - Forgot password link
      - Account lockout after 5 failed attempts
      - Session timeout after 30 minutes of inactivity
      """
  )

  generator.save_to_csv(test_cases, "login_test_cases.csv")
  print(test_cases)

Cost Optimization: Start with GPT-4-turbo (cheaper and faster). Only use GPT-4 if you need deeper reasoning for complex features.

Step 4: Enhance with Domain Knowledge

Here's where you make it powerful. Add your domain-specific knowledge:

|domain_knowledge.py
# Add this to your generator class
class TestCaseGenerator:
  def __init__(self, domain_context=None):
      self.client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
      self.domain_context = domain_context or self.default_domain_context()

  def default_domain_context(self):
      """Your domain expertise"""
      return """
      Domain: E-commerce web application

      Standard Security Checks:
      - SQL injection attempts
      - XSS attacks
      - CSRF protection
      - Session hijacking

      Common Edge Cases:
      - Unicode characters in inputs
      - Very long strings (>1000 chars)
      - Special characters in email
      - Concurrent user sessions

      Compliance Requirements:
      - GDPR data handling
      - PCI DSS for payment data
      - WCAG 2.1 AA accessibility

      Performance Baselines:
      - Page load: <2 seconds
      - API response: <500ms
      - Support 1000 concurrent users
      """

  def generate(self, feature_name, description, user_story, requirements):
      # Prepend domain context to every prompt
      full_prompt = f"{self.domain_context}\n\n{PROMPT_TEMPLATE.substitute(...)}"
      # ... rest of the method

Now every test case generation includes your domain expertise automatically.

Step 5: Add Batch Processing

Generate test cases for multiple features at once:

|batch_processor.py
import json
from test_case_generator import TestCaseGenerator

def batch_generate(features_file):
  """Generate test cases for multiple features from JSON"""

  generator = TestCaseGenerator()

  # Load features from JSON
  with open(features_file, 'r') as f:
      features = json.load(f)

  results = {}

  for feature in features:
      print(f"Generating test cases for: {feature['name']}...")

      test_cases = generator.generate(
          feature_name=feature['name'],
          description=feature['description'],
          user_story=feature['user_story'],
          requirements=feature['requirements']
      )

      results[feature['name']] = test_cases

      # Save individual files
      filename = f"test_cases_{feature['name'].lower().replace(' ', '_')}.csv"
      generator.save_to_csv(test_cases, filename)

  return results

# Usage: python batch_processor.py features.json
if __name__ == "__main__":
  import sys
  batch_generate(sys.argv[1])

Example features.json:

|features.json
[
{
  "name": "User Login",
  "description": "Email/password authentication",
  "user_story": "As a user, I want to log in...",
  "requirements": "- Email required\n- Password required..."
},
{
  "name": "Product Search",
  "description": "Search products by keyword",
  "user_story": "As a user, I want to search...",
  "requirements": "- Search bar\n- Filters..."
}
]

Step 6: Integrate with Your Tools

Connect to Jira, TestRail, or your test management tool:

|jira_integration.py
from jira import JIRA

class JiraIntegration:
  def __init__(self, server, email, api_token):
      self.jira = JIRA(server=server, basic_auth=(email, api_token))

  def create_test_case(self, project_key, test_case):
      """Create a test case in Jira"""
      issue_dict = {
          'project': {'key': project_key},
          'summary': test_case['title'],
          'description': self.format_test_case(test_case),
          'issuetype': {'name': 'Test'},
      }

      new_issue = self.jira.create_issue(fields=issue_dict)
      return new_issue.key

  def format_test_case(self, test_case):
      """Format test case for Jira"""
      return f"""
      *Preconditions:*
      {test_case['preconditions']}

      *Steps:*
      {test_case['steps']}

      *Expected Result:*
      {test_case['expected_result']}

      *Priority:* {test_case['priority']}
      *Type:* {test_case['type']}
      """

# Usage
jira = JiraIntegration(
  server="https://your-domain.atlassian.net",
  email="your-email@example.com",
  api_token="your_api_token"
)

jira.create_test_case("PROJ", test_case)

For TestRail integration, check out the testrail-api Python package. The pattern is similar to Jira.

Real-World Example: Before vs After

Before (Manual):

  • Time: 2-3 hours per feature
  • Test cases: 8-12 (missing edge cases)
  • Consistency: Varies by mood

After (AI Generator):

  • Time: 15 minutes per feature (including review)
  • Test cases: 15-20 (comprehensive)
  • Consistency: Same format every time

My workflow now:

  1. Copy user story from Jira (30 seconds)
  2. Run generator (2 minutes)
  3. Review and enhance with domain knowledge (10 minutes)
  4. Import to TestRail (3 minutes)

Common Issues and Solutions

Issue 1: Inconsistent Output Format

Solution: Be extremely specific in your prompt:

|solution.md
Bad: "Generate test cases"
Good: "Generate test cases in this EXACT format: [format]
Do not add any extra text before or after the table."

Issue 2: Generic Test Cases

Solution: Add more domain context and examples:

|context.py
# Add example test cases to your prompt
EXAMPLE_TEST_CASE = """
Example of a good test case:
TC-001 | Login with valid credentials | User has account | GIVEN user on login page WHEN enters valid email and password THEN redirected to dashboard | Success | Critical | Functional
"""

Issue 3: Missing Edge Cases

Solution: Explicitly ask for them:

|edge-cases.md
"Also consider these edge cases:
- Very long inputs (>1000 characters)
- Special characters: <>"'&
- Concurrent requests
- Network timeouts
- Database failures"

Cost Estimation

Based on my usage:

  • Per feature: $0.10 - $0.30 (GPT-4-turbo)
  • Monthly (20 features): $2 - $6
  • Time saved: 30-40 hours/month

ROI: If your hourly rate is $50, you save $1,500-$2,000/month for a $6 investment.

Next Steps: Making It Production-Ready

Once you have the basics working:

  1. Add error handling - API timeouts, rate limits
  2. Implement caching - Don't regenerate identical test cases
  3. Version control - Track prompt changes in git
  4. Add UI - Streamlit or Flask for non-technical users
  5. Metrics tracking - How many test cases generated, time saved

Start small: Get it working for one feature first. Then expand. Don't try to build the perfect system on day one.

The Code Repository

Want the complete working code? I've created a GitHub template with:

  • ✅ Complete generator code
  • ✅ Example prompts
  • ✅ Jira/TestRail integrations
  • ✅ Docker setup
  • ✅ Documentation

Coming soon on the contact page - drop your email if interested!


Ready to build yours? Start with the basic generator and iterate. The key is making it work for YOUR domain and YOUR team's format.

Questions? Reach out on the contact page. I'd love to hear what you build!

Continue Reading

Claude Code /btw and /voice changed how I talk to my terminal

Read more →

I Did a Live AI Demo at a QA Meetup. It Failed.

Read more →

15 Years of Finding Bugs Taught Me How to Build Software

Read more →

Stay Connected

Subscribe and get instant access to 50 free AI prompts for software testers — plus new articles on AI-powered testing, automation strategies, and quality leadership. No spam, unsubscribe anytime.