Why shared test inboxes make E2E suites flaky
End-to-end tests for authentication flows — account registration, email verification, magic links, and password resets — are notoriously brittle when teams reuse a single static Gmail or catch-all mailbox. Parallel CI workers race against one another, pull stale OTP codes from previous runs, or hit rate limits on the same address.
Provisioning a brand-new mailbox per test case through a free temp mail API guarantees complete isolation. Each Playwright or pytest run receives a unique address on a live standard (.com) or academic (.edu.pl) domain and reads only the messages sent to that specific test execution.
The 4-step disposable email API lifecycle
Tempboxy exposes a zero-friction JSON REST interface at /api/v1 requiring no OAuth dance or credit card:
- Discover live domains:
GET /api/v1/domainsreturns currently active domains across both upstream providers. - Provision an isolated inbox:
POST /api/v1/inboxeswith{"provider": "tempmail", "ttl": "10m"}returns a uniqueaddressand signedinbox_id. - Poll for code or full messages:
GET /api/v1/inboxes/:id/codereturns the newest extracted OTP and confirmation link, while/messagesreturns full HTML, plain text, normalized attachments, and tracker security scores. - Burn on teardown:
DELETE /api/v1/inboxes/:idremoves the temporary address cleanly once the test passes.
Complete Playwright OTP test example
Using Playwright's native expect.poll helper avoids brittle hard-coded waitForTimeout sleeps and retries cleanly until SMTP delivery completes:
import { test, expect } from '@playwright/test';
const API = 'https://tempboxy.com/api/v1';
test('new user completes email OTP verification', async ({ page, request }) => {
const createRes = await request.post(`${API}/inboxes`, {
data: { provider: 'tempmail', ttl: '10m' },
});
expect(createRes.status()).toBe(201);
const { address, inbox_id } = await createRes.json();
try {
await page.goto('/signup');
await page.getByLabel('Email').fill(address);
await page.getByRole('button', { name: 'Continue' }).click();
let otpCode = null;
await expect.poll(async () => {
const pollRes = await request.get(`${API}/inboxes/${inbox_id}/code`);
const payload = await pollRes.json();
otpCode = payload.code;
return otpCode;
}, { timeout: 30000, intervals: [2000, 3000, 4000] }).toMatch(/^\d{4,8}$/);
await page.getByLabel('Verification code').fill(otpCode);
await page.getByRole('button', { name: 'Verify' }).click();
await expect(page).toHaveURL(/dashboard/);
} finally {
await request.delete(`${API}/inboxes/${inbox_id}`);
}
});
Python requests & pytest helper
For backend integration tests in Python, a tiny polling helper handles both numeric OTPs and password-reset URLs across either provider pool:
import time
import requests
API = "https://tempboxy.com/api/v1"
def wait_for_verification(inbox_id: str, timeout_s: int = 30) -> dict:
deadline = time.time() + timeout_s
while time.time() < deadline:
res = requests.get(f"{API}/inboxes/{inbox_id}/code", timeout=10).json()
if res.get("code") or res.get("verification_link"):
return res
time.sleep(3)
raise TimeoutError("Timed out waiting for verification email")
Auditing HTML templates, attachments & trackers
Beyond checking that an email arrived, engineering teams often need to inspect transactional templates visually before release. Every message returned by GET /api/v1/inboxes/:id/messages includes a security object reporting accidental 1x1 tracking pixels, UTM parameters, and spam-risk score alongside normalized attachments.
For manual visual QA, open the developer email testing inbox to inspect real incoming emails at full desktop width or 375px mobile width, inspect raw HTML markup, and download RFC-5322 .eml archives for bug reports. You can also test deliverability DNS records beforehand using the Email Domain MX Checker.
Best practices for TTL and cleanup in CI
- Use short TTLs in CI: Pass
"ttl": "10m"for ephemeral test runs so unused recovery tokens expire promptly. - Always delete in a
finallyblock: CallingDELETE /api/v1/inboxes/:idfrees temporary mailboxes immediately. - Poll responsibly: Back off to 3–4 second intervals rather than tight loops; SMTP handshakes rarely complete in under 2 seconds.