In the previous parts of this series, we explored the potential of AI agents in software testing, discussed real-world applications, and delved into essential AI concepts. While commercial AI testing tools dominate the market, open-source alternatives are now making professional-grade AI testing accessible to everyone. Now, let's get practical and explore how you can integrate open-source AI technologies into your own projects.
Open-Source AI Integration with OpenRouter and DeepSeek-R1
OpenRouter is an open-source API platform that allows developers to access a wide range of AI models, including cutting-edge models like DeepSeek-R1. Unlike proprietary APIs from companies like OpenAI or Google, OpenRouter provides free access to many models, making it an excellent choice for developers looking to experiment with AI without high costs. If you prefer not to use OpenRouter's API, you can also host the model yourself, which may require additional computational resources on your behalf.
Step 1: Sign Up for OpenRouter API Key
- Go to OpenRouter.ai and sign up for an account.
- Navigate to your profile picture at the top right corner and select "Keys" from the dropdown menu.
- Click on the "Create Key" button.
- Enter a Name and click on "Create".
- Copy the Key and write it down somewhere safe.
Step 2: Load API Key into Your Project
- You can use OpenAI Python package since it supports OpenRouter API endpoint (mimics OpenAI format)
- Import OpenAI SDK and set base_url to OpenRouter API.
- Make sure to paste your own API Key (e.g. from OpenRouter) under "PASTE_YOUR_API_KEY_HERE".
from openai import OpenAI client = OpenAI( base_url="https://openrouter.ai/api/v1", api_key="PASTE_YOUR_API_KEY_HERE", )
API Key Security Warning: Never commit API keys to version control. Use environment variables or secret managers.
Step 3: Use the New Open Source Model DeepSeek-R1
- You can change the model from "deepseek/deepsek-r1:free" to other models you prefer, but be warned that this may involve costs.
- Check the costs of the models under Models
- You can also check the number of tokens in your API Key completions by clicking on "Activity" or "View Activity" under the options of your API Keys
- Test the DeepSeek R1 model yourself by changing the prompt after "content":
completion = client.chat.completions.create(
extra_headers={
"HTTP-Referer": "<YOUR_SITE_URL>", # Optional. Site URL for rankings on openrouter.ai.
"X-Title": "<YOUR_SITE_NAME>", # Optional. Site title for rankings on openrouter.ai.
},
model="deepseek/deepseek-r1:free",
messages=[
{
"role": "user",
"content": "Generate Playwright tests for a login page with email/password fields."
}
]
)
print(completion.choices[0].message.content)
Combine and run these two code snippets to test your own prompts with DeepSeek-R1.
Leveraging DeepSeek-R1 model with Playwright Testing Framework
Playwright is a powerful end-to-end testing framework for web applications. When combined with DeepSeek-R1, it enables automated test generation and execution, streamlining your testing workflow.
Generate tests
Use DeepSeek-R1 to analyze your codebase and generate relevant test cases based on its understanding of the context. These test cases can then be executed using Playwright to ensure your application behaves as expected.
Example: Generating Automated Tests with DeepSeek-R1
Here’s an example of a generated automated test for an online shop application based on Python Flask. This demonstrates how DeepSeek-R1 can produce comprehensive and functional test cases:
import unittest
import sqlite3
from unittest.mock import patch
from flask import url_for, request
from main import app, getLoginDetails, is_valid, allowed_file, parse
# Configure the app for testing
app.config['TESTING'] = True
app.config['WTF_CSRF_ENABLED'] = False
app.config['SERVER_NAME'] = 'localhost'
app.config['APPLICATION_ROOT'] = '/'
class TestFlaskApp(unittest.TestCase):
def setUp(self):
self.client = app.test_client()
self.ctx = app.app_context()
self.ctx.push()
# Create a test database
conn = sqlite3.connect('test_database.db')
conn.execute('''DROP TABLE IF EXISTS users''')
conn.execute('''DROP TABLE IF EXISTS products''')
conn.execute('''DROP TABLE IF EXISTS kart''')
conn.execute('''CREATE TABLE users (
userId INTEGER PRIMARY KEY AUTOINCREMENT,
email TEXT UNIQUE NOT NULL,
password TEXT NOT NULL,
firstName TEXT NOT NULL,
lastName TEXT NOT NULL,
address1 TEXT NOT NULL,
address2 TEXT,
zipcode TEXT NOT NULL,
city TEXT NOT NULL,
state TEXT NOT NULL,
country TEXT NOT NULL,
phone TEXT NOT NULL
)''')
conn.execute('''CREATE TABLE products (
productId INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
price REAL NOT NULL,
description TEXT,
image TEXT,
stock INTEGER NOT NULL,
categoryId INTEGER NOT NULL
)''')
conn.execute('''CREATE TABLE kart (
kartId INTEGER PRIMARY KEY AUTOINCREMENT,
userId INTEGER NOT NULL,
productId INTEGER NOT NULL,
FOREIGN KEY(userId) REFERENCES users(userId),
FOREIGN KEY(productId) REFERENCES products(productId)
)''')
conn.commit()
conn.close()
def tearDown(self):
# Clean up the test database
conn = sqlite3.connect('test_database.db')
conn.execute('DROP TABLE IF EXISTS users')
conn.execute('DROP TABLE IF EXISTS products')
conn.execute('DROP TABLE IF EXISTS kart')
conn.commit()
conn.close()
self.ctx.pop()
def test_root_route(self):
response = self.client.get('/')
self.assertEqual(response.status_code, 200)
def test_get_login_details(self):
with self.client.get(url_for('getLoginDetails')) as response:
self.assertEqual(response.status_code, 200)
data = response.json
self.assertIn('loggedIn', data)
self.assertIn('firstName', data)
self.assertIn('noOfItems', data)
def test_is_valid(self):
# Create test user
conn = sqlite3.connect('test_database.db')
conn.execute('''
INSERT INTO users (email, password, firstName, lastName, address1, city, state, country, phone, zipcode)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
''', ('test@example.com', 'password', 'John', 'Doe', '123 Street', 'City', 'State', 'Country', '1234567890', '12345'))
conn.commit()
conn.close()
# Test valid credentials
self.assertTrue(is_valid('test@example.com', 'password'))
# Test invalid credentials
self.assertFalse(is_valid('invalid@example.com', 'password'))
def test_allowed_file(self):
# Test allowed file types
self.assertTrue(allowed_file('test.jpg'))
self.assertTrue(allowed_file('test.png'))
self.assertFalse(allowed_file('test.txt'))
def test_parse_function(self):
data = [(1, 'Product 1', 100, 'Description 1', 'image1.jpg', 5),
(2, 'Product 2', 200, 'Description 2', 'image2.jpg', 3),
(3, 'Product 3', 300, 'Description 3', 'image3.jpg', 2)]
parsed_data = parse(data)
self.assertEqual(len(parsed_data), 1)
self.assertEqual(len(parsed_data[0]), 3)
def test_add_to_cart(self):
with self.client.session_transaction() as session:
session['email'] = 'test@example.com'
with patch('sqlite3.connect') as mock_connect:
mock_cursor = mock_connect.return_value.cursor.return_value
mock_cursor.execute.return_value.fetchone.return_value = (1,)
response = self.client.get(url_for('addToCart', productId=1))
self.assertEqual(response.status_code, 302)
def test_remove_from_cart(self):
with self.client.session_transaction() as session:
session['email'] = 'test@example.com'
with patch('sqlite3.connect') as mock_connect:
mock_cursor = mock_connect.return_value.cursor.return_value
mock_cursor.execute.return_value.fetchone.return_value = (1,)
response = self.client.get(url_for('removeFromCart', productId=1))
self.assertEqual(response.status_code, 302)
def test_login_route(self):
# Test login form
response = self.client.get(url_for('loginForm'))
self.assertEqual(response.status_code, 200)
# Test login with valid credentials
with patch('main.is_valid', return_value=True):
response = self.client.post(url_for('login'), data={
'email': 'test@example.com',
'password': 'password'
})
self.assertEqual(response.status_code, 302)
# Test login with invalid credentials
with patch('main.is_valid', return_value=False):
response = self.client.post(url_for('login'), data={
'email': 'invalid@example.com',
'password': 'password'
})
self.assertEqual(response.status_code, 200)
def test_register_route(self):
# Test registration form
response = self.client.get(url_for('registrationForm'))
self.assertEqual(response.status_code, 200)
# Test registration with valid data
response = self.client.post(url_for('register'), data={
'email': 'test@example.com',
'password': 'password',
'firstName': 'John',
'lastName': 'Doe',
'address1': '123 Street',
'city': 'City',
'state': 'State',
'country': 'Country',
'phone': '1234567890'
})
self.assertEqual(response.status_code, 200)
# Test registration with existing email
response = self.client.post(url_for('register'), data={
'email': 'test@example.com',
'password': 'password',
'firstName': 'John',
'lastName': 'Doe',
'address1': '123 Street',
'city': 'City',
'state': 'State',
'country': 'Country',
'phone': '1234567890'
})
self.assertEqual(response.status_code, 200)
if __name__ == '__main__':
unittest.main()
Generated with DeepSeek R1 (d-32B) Model
Example Output
Here is a short preview of the output of the executed tests.
Conclusion
Open-source AI offers a powerful and accessible path to integrating AI into your software testing processes. By experimenting with tools like DeepSeek-R1, and leveraging the capabilities of open-source testing frameworks like Playwright, you can significantly improve the efficiency and effectiveness of your testing efforts. While these open-source tools lower the barrier to AI testing, remember they complement rather than replace manual testing. Start with non-critical workflows, validate outputs, and gradually expand usage as confidence grows.