Will AI Replace Your Software Testing Job?

Today, we’re tackling a big question: With the rise of code generators and AI that can make UI elements based on a single drawing, will AI take your software testing job?

The short answer is no. Today we can safely tell you that no, AI won’t take your job. Instead, we think it will enhance the way you work. Let’s see how we can make AI a powerful ally for software testers.

Role of AI in Software Testing

We like to think of AI in the testing world’s context as a controlled chaos maker. We can simulate some parts of being human to introduce some unpredictability in our tests and see if our app mitigates against the chaos.

Instead of fighting AI, let’s embrace it. One of the ways we can do that is if we can have it write test cases for us, execute it, then tell us if it passed or not.

We can then easily move those generated tests to a framework like cypress or to a continuous integration environment like GitHub Actions.

So for the rest of the article, we’ll show you how you can have ai make a test case that:

  1. Generates and sends an email.
  2. Reads the contents of said email from the user’s inbox.
  3. Ensures that contents of 1 and 2 are the same.

To be safe and not risk sending any emails out to customers, we’ll use Mailsac and its API.

Making the Assistant

So for this walkthough we’re going to be using OpenAI’s “Assistant” feature. Assistants are a little bit more of a halfway point between the chat AI we all know, and an AI agent that is more autonomous.

We want to give this AI the ability to generate and run some code and read files, but not necessarily make any autonomous decisions.

For the instructions we’ll give it some system prompts to let it know under what context it should be operating under.

We’ll use the latest gpt-4o model and enable the code interpreter tools. That should allow it to read, run and execute some code for us.

Now let’s go ahead and start generating our test cases.

We’ll use the prompt

You are creating a code test that will return true of false if a certain criteria passes. The language for > all tests should be in javascript. Use dotenv for your API keys.

The criteria is:

  1. Generate an email message and send it via nodemailer using google’s SMTP servers. Send it to [email protected].
  2. Wait 2 seconds for the email to send from Google then use the mailsac API to check the subject and
    content of the email.
  3. Make the test pass if it matches what was sent in 1 and fail if it doesn’t match.

Keep in mind that you can easily tweak the SMTP to use your own service. We’ll stick with Google’s as its the most accessible for this walkthrough.

Generating the Test Case

Fire it off to the AI Assistant and let it generate the test case. In this run it generated a mocha test for me. In previous iterations it didn’t generate a whole mocha test so I’ll ask it to just use node.

As you can see it pretty much walks you through the whole project creation process. Let’s go ahead and do our setup and install.

mkdir email_test
cd email_test
npm init -y
npm install nodemailer dotenv axios

The libraries it wants to use are pretty standard. (You could make an argument against axios but hey, let’s roll with it).

And now we’ll use a .env file to hold our API keys. You’ll need an application password for the google SMTP.

You can find where to add that in your Google account.

You’ll also need a mailsac key.

The .env file is now:


Now copy all of your keys and email onto the .env file.

Let’s start working through the code itself


Looking through the code it looks fine, with 2 exceptions: It needs to be guided on the API endpoint (to use /api/text instead of /api/messages) and it the object retrived is the text itself, so we can remove the nonexistant child textBody

Now let’s go ahead and run the test.

Check on mailsac to visually make sure an email was sent.

You can even make the test fail on purpose to ensure it’s working as intended.

It works!

Extending AI

From here it’s easy to incorporate these kinds of tests into specific continuous integration frameworks of your choice.

You could use cypress and work with existing libraries or an existing framework to run your tests.

Or you could use GitHub actions to fully run your now AI generated tests in the cloud. We have a guide showing you how to do just that.


As you can see it still took a human element to know what to want to test, frame it, and even tweak the AI’s output a bit to make sure that what it was generating made sense.

Tester’s aren’t going anywhere anytime soon, especially if you leverage AI to do the work for you while you focus on whats important to test and why.

How You Can Avoid CC Errors That Could Violate GDPR

Developing and testing an application is difficult enough without the stress of GDPR, CAN-SPAM, accidental information leak, etc. But it doesn’t have to be.

In this article I’ll walk you through how our email platform could’ve helped prevent Running Warehouse from exposing thousands of their customer’s emails.

Background on the Running Warehouse Incident

Now, this isn’t meant to poke fun at a company’s mistake but really to highlight just how easy these mistakes are to miss. Last month, I received two emails from “Running Warehouse” about an update to their terms of service. Nothing really unusual at first glance… Until you notice that about 1000 other customer’s email addresses had been sent along in the CC field. And to add even more insult to injury, they did this TWICE. So now I have a list of about 2000 of Running Warehouse’s customers that I’m sure the customers themselves are not happy about.

Really not a good look for a company who could be facing a class action lawsuit for a previous data breach back in 2021. 

So let’s look at the implications of this easy to do mistake:

  1. Potential GDPR violations
  2. Loss of customer trust
  3. Company reputation lost via social media, youtube videos about you, forums, etc.

Prevent these errors with Mailsac

Obviously no company wants that! This is where mailsac steps in. We help prevent these kinds of easy mistakes, but at the application level. We have a set of features where you can hook up your continuous integration with our API to ensure the email you want to send is the one the recipient actually received.

So let’s walk through how we’d prevent this if this email was coming from our application.

Technical Deep-Dive

Here’s what we’ll do:

  1. Setup a sample application that is meant to simulate sending emails.
  2. Setup our Mailsac API Key.
  3. Send a sample email.
  4. Use the API to ensure our cc and email fields match what we sent.

Let’s get started.

Sample Application

The sample application we'll be using
  • To stand in place of your application, we’ll use this dead simple node app whose sole purpose is to just send an email based on what you place in a form. 
  • To simulate our email service API key, we’ll use our Application Key we generated for our gmail account. You can do this too if you want to step through this part. You can find it under your Google account -> security -> 2 FA -> App Password or just search for app passwords.
  • Add your credentials in the config.js file directly.
    • Again, while we embed the code here, you should really use a dot-env library.
  • Most importantly, for our example here, let’s use a mailsac temporary address. We have ad hoc inbox creation abilities so for now, let’s say we’ll send it to [email protected]
  • Fire up the app

Setup our Mailsac API Key

Before we fire off an email, let’s add some quick code to check whether what we send is the same or not once it’s received.

Mailsac API Dashboard

The Email CC Checker Snippet

  await setTimeout( async () => {
        const mailsac = new Mailsac({ headers: { "Mailsac-Key":  process.env.MAILSAC_API } })
        const results = await mailsac.messages.listMessages("[email protected]")
        const messages = results.data;
        if (messages[0].cc.length <= 0){
            console.error("This email has no CC. That's Good!")
        if (messages[0].subject === req.body.subject){
            console.error("This has the correct subject. That's Good!")
      }, 1000);

Normally we’d place it in an .env file, but for our purposes here, we’ll just place it directly in the headers.

Send Sample Email

Alright now let’s fire this up and fire off an email

  • Let’s make the content and subject something specific like:
    • Subject: This email is intended only for emailcontentcheck
    • Content: Hey, this email is strictly for [email protected]. No others on cc field.
  • Fire it off
  • Wait for the check to fire off and print out comparison / success
Showing a successful email cc check.
Showing a successful email cc check.

Use the API to ensure our cc and email fields match what we sent

Right after our code fires off, it should print out the message comparison result. But now, let’s make sure our CC fields are also empty

      await setTimeout( async () => {
        const mailsac = new Mailsac({ headers: { "Mailsac-Key": process.env.MAILSAC_KEY } })
        const results = await mailsac.messages.listMessages("[email protected]")
        const messages = results.data;
        if (messages[0].cc.length <= 0){
            console.error("This email has no CC. That's Good!")
        if (messages[0].subject === req.body.subject){
            console.error("This has the correct subject. That's Good!")
      }, 1000);

We’re even throwing in a subject line check. You can check out the full API documentation to see how you can check for empty BCCs, content checks, and more.

Fire it off again and enjoy having an automated way test your email content, cc fields, and more!

Wrap Up

Finding the right words to use in an email is already hard enough, don’t complicate it by worrying about whether the email went through,  if it works right, if it accidentally cc’s people, etc. 

Just let mailsac handle the double checking for you.

And if you want to dive even deeper into email testing, we have a full set of articles on integrating with Cypress or integrating other testing tools like Selenium with GitHub Actions for a fully automated testing pipeline.

The Cost of Email Mistakes: 5 Case Studies You Need to Know

The Cost of Email Mistakes: 5 Case Studies You Need to Know

Email is a crucial communication tool for businesses, even in 2024. It’s simple and effective. But in its simplicity we can find ourselves complacent in how we use it. Or even who we send our emails to.

In this article, we’ll examine five real-life case studies where emails went wrong and caused significant problems for the organization that sent it. From exposing private information to damaging a company’s reputation, these examples highlight the potential risks of email errors. By understanding these incidents, businesses can learn the importance of proper email management and take steps to prevent similar mistakes. These case studies serve as a reminder that attention to detail is essential in all aspects of communication.

Case Study #1 – HBO Max’s “Integration Test Email” Incident

HBO says: "Yes, it was the intern"

In June 2021, HBO Max accidentally sent an “Integration Test Email #1” to a large number of its subscribers. As you can guess, the email was meant for internal testing but mistakenly went out to real customers. The incident caused widespread confusion and snark on social media. Some people pointed out the optics of mistakenly sending out an email to your entire client base, but most took the email fail in jest:

Tweet of someone that reads: "For the #1 Integration Test Email in ur life <3 @hbomax"

Hilariously, the official HBO Max twitter account had this explanation:

Tweet that reads: "We mistakenly sent out an empty test email to a portion of our HBO Max mailing list this evening. We apologize for the inconvenience, and as the jokes pile in, yes, it was the intern. No, really. And we’re helping them through it. ❤️"

We don’t blame the intern, but we do cast some judgment on their complete lack of email service testing! Using disposable email addresses for internal testing could have prevented this. But hey, we did at least get some genuine entertainment.  

Case Study 2 – Running Warehouse CC vs BCC

Header that reads "Running Warehouse vs BCC and CC"

Sadly this case is not as harmless. On May 2024 RunningWarehouse, a discount shoe seller, sent out what should have been a routine email about updating their terms of service.

As you can see, this wasn’t just sent to a single customer. While not the entire customer list, approximately 999 customers is pretty indicative of an email testing batching failure. And it didn’t go unnoticed either, as there was at least one 30 post thread on arguably the internet’s largest running forum

The issue is still relatively fresh as of this writing, but it’s safe to assume that Running Warehouse did not intend to leak 1000 of its customer’s email addresses to each other due to someone mistaking CC vs BCC. And even more seriously, it could even be a violation of GDPR under “Sharing Email Addresses Without Consent” according to GDPR’s Data Breach Guidelines.

Case Study 3 – University Backtracks Acceptance Letters

Banner image that reads "Sorry, Wrong Recipient"

In January 2019, the University of South Florida St. Petersburg mistakenly sent acceptance emails to nearly 680 applicants, although only 250 had been accepted. As you can imagine this caused confusion, disappointment, and reputational damage to the university. The affected students and their families were left feeling misled, and the university had to invest considerable time and resources to address the fallout from the erroneous communication. 

You may not be shocked to hear that this isn’t the first time this has happened, as Columbia University also accidentally sent 270 emails to students who had not been officially accepted. 

These cases are a clear cut impact of email errors on institutional credibility and the importance of systematic verification before sending mass emails. Systems sending out emails absolutely need to ensure that the emails they intend to send are 100% accurate.

Case Study 4 – Australian Leaks Citizen Emails

Banner text that reads "Australia vs BCC and CC"

In September 2020, an employee at Australia’s Department of Foreign Affairs and Trade (DFAT) accidentally exposed the personal details of almost 3,000 citizens by not using the BCC field in an email. This revealed the email addresses of Australians stranded abroad due to COVID-19 travel restrictions. The mishandling of this sensitive information led to major privacy concerns and required immediate action, such as attempting to recall the email and asking recipients to delete it. 

While not directly tied to a system, temporary email accounts could be used in place to ensure the intended audience and message gets to its audience intact. 

Case Study 5 – Serco Accidentally Shares Contract Tracers

Banner text that reads: "Serco vs BCC and CC"

In May 2020, Serco, a business services and outsourcing company, accidentally exposed the email addresses of nearly 300 newly recruited COVID-19 contact tracers by using the CC field instead of BCC. This breach of privacy led to concerns about the security of personal data and put the company’s data protection practices under scrutiny. Serco apologized and announced that they would review and update their procedures to prevent similar incidents in the future. 

Yet another case of manual implementations of email lists without temporary email addresses, email outbound trapping services, or message integrity verification (ie, only 1 person in the to field, nothing in the cc field).

Stop Customer Data Leaks & Company Reputation Hits With Mailsac

We offer several benefits that can help prevent incidents like the ones described above:

  • Enhanced Security: Temporary emails protect more than just attacks and spam. By using a disposable email address, you can send test emails either manually or via your system to ensure the message arrives exactly as intended.
  • Privacy Protection: Mailsac helps keep personal and sensitive email addresses private. This is especially useful for companies conducting internal tests or handling sensitive information. 
  • Minimized Clutter: Using temporary emails for one-time use reduces inbox clutter around testing. It’s beneficial for both individuals and teams, as it keeps testing cycles organized. You can even split your testing into campaigns by using unique email addresses you control with every testing cycle with our Zero-Setup subdomains feature.
  • Testing and Development: Mailsac is ideal for testing campaigns, systems, and internal communications. Temporary emails can be used during development phases to catch errors and ensure that only verified emails are sent to real users. We have fully featured APIs to integrate between any systems.

Disposable email services are an essential tool for modern communication. They can help prevent costly mistakes and ensure that PR issues around email remain in the past. For those looking to enhance their email security and efficiency, we’re an excellent choice. Give Mailsac a try today and see how it can benefit your email needs.

Streamline Email Testing in Healthcare with Mailsac

In healthcare, email communication intersects with patient care and data security. The margin for error is virtually nonexistent. Mailsac’s SaaS platform offers a robust email delivery suite, tailored to meet the rigorous demands of healthcare IT security, compliance, and operational efficiency. Mailsac has been integrated into systems at many enterprises in the healthcare industry, and has been relied on 24/7 for email testing for over a decade.

Enhanced Security and Compliance

Security Audits: Mailsac is built to pass IT department security audits, aligning with healthcare standards like HIPAA. Its infrastructure ensures patient data is safeguarded during email testing.

Data Privacy: With disposable email addresses, Mailsac supports testing that avoids exposing real patient information, maintaining privacy and compliance.

Collaborative Testing Environment

Unified Inbox: Devs and QAs can collaborate effectively using Mailsac’s unified inbox feature, which consolidates test emails from both persistent and temporary email accounts into a single view.

SSO Integration: Simplify access while enhancing security with Single Sign-On (SSO), allowing seamless integration into existing healthcare IT ecosystems.

Automated and Efficient Testing

CI/CD Integration: Mailsac’s API automates email tests within CI/CD pipelines, reducing manual effort and accelerating development cycles in fast-paced healthcare settings.

Scalable Solutions: Whether scaling up operations or integrating new services, Mailsac’s scalable platform adapts to the evolving needs of healthcare enterprises, ensuring email testing is never a bottleneck. There’s no need to manage server resources, deployments or upgrades.

Comprehensive Compatibility Testing

Device and Platform Coverage: Guarantee that critical communications are accessible across all devices and platforms used by healthcare professionals and patients alike.

Deliverability Assurance

Inbox Placement: Rigorous testing with Mailsac ensures healthcare emails achieve high deliverability, crucial for appointment reminders, test results, and other sensitive communications.

Optimizing Patient Communication

Email Optimization: Test and refine email content for clarity and engagement, ensuring messages to patients are both accessible and actionable.


For healthcare organizations, Mailsac offers a precision toolset for email testing — ensuring security, efficiency, and compliance while enhancing collaborative efforts between QA and development teams. Since 2012, Mailsac has practiced technical excellence and has helped hundreds of customers in healthcare manage their unique challenges.

10 Common Email Testing Pitfalls and How to Avoid Them

Email testing is an essential step in the QA process, ensuring that communications reach their intended recipients accurately and efficiently. However, even experienced QA teams can fall into common traps that undermine their efforts. Here are ten frequent email testing pitfalls and strategic ways to avoid them, streamlining your workflow and enhancing email reliability.

1. Ignoring Mobile Responsiveness


Not testing how emails render on mobile devices, leading to formatting issues or poor user experiences.


Use email testing tools that simulate various mobile devices and screen sizes to ensure your emails look great everywhere.

2. Overlooking Email Client Diversity


Focusing on a single email client, ignoring the fact that your audience uses a wide range of email services with different rendering engines.


Test emails across multiple clients (like Gmail, Outlook, Yahoo) to identify and fix client-specific issues.

3. Neglecting Deliverability Tests


Assuming emails reach the inbox without verifying, risking them being flagged as spam.


Conduct deliverability tests with tools that provide insights into spam scores and help optimize for better inbox placement.

4. Underestimating the Importance of Content Clarity


Creating content that’s confusing or misleading, leading to poor user engagement.


Ensure your messages are clear, concise, and actionable. Test variations to see which performs best in terms of user engagement.

5. Skipping Accessibility Checks


Forgetting to make emails accessible to all users, including those with disabilities.


Include text alternatives for images, use sufficient contrast ratios, and test with screen readers to ensure accessibility.

6. Failing to Test Links and Attachments


Assuming all links and attachments work without thoroughly testing them, which could lead to a frustrating user experience.


Manually check each link and attachment in different environments to ensure functionality and security.

7. Ignoring Email Load Times


Overloading emails with high-resolution images or complex HTML, leading to slow loading times.


Optimize images and streamline code to improve load times, ensuring a smooth user experience.

8. Forgetting to Validate Email Lists


Sending tests to outdated or incorrect email addresses, skewing testing results.


Regularly cleanse and validate your email lists to ensure accuracy and relevance.

9. Overlooking Privacy and Compliance


Neglecting privacy laws and email regulations, risking legal issues and damaged reputation.


Stay informed about regulations like GDPR and CAN-SPAM, ensuring your email practices are compliant.

10. Not Leveraging Automation


Performing repetitive tests manually, which is time-consuming and prone to human error.


Incorporate automated testing workflows to save time, reduce errors, and increase efficiency.

In Conclusion

By being mindful of these common pitfalls and implementing the suggested solutions, QA teams can significantly improve their email testing processes. Tools like Mailsac offer zero-configuration custom private domains, comprehensive Swagger REST APIs, and a generous free tier, making it easier for teams to test emails effectively and efficiently. Remember, the goal is not just to send emails but to ensure they are delivered, readable, and engaging across all devices and clients.

Have Playwright Automatically Write Tests For You with Codegen

These days, we have better options than writing our test specs by hand.

The open source community has released a variety of frameworks to relieve us from that particular tedium: Cypress, Selenium and Pupeteer. And in this video companion guide, I’ll focus on Playwright. Specifically, how playwright can help automate a lot of the boilerplate involved in writing test specs with CodePen.

The video will do a quick walk you through playwright in itself. This article will provide the core login.spec.ts file I used and where to go from next.

But before we do that, it’s worth mentioning that efficient testing isn’t just about the right tools; it’s also about the seamless integration of these tools into your existing systems. That’s where Mailsac comes in. Our platform offers unique capabilities for email testing within your automated workflows, making it an excellent complement to Playwright for end-to-end testing solutions.

With Mailsac, you can ensure not just the functionality but also the integrity of email interactions in your applications, all within the automation framework you’ll establish with Playwright.

So, What Is Playwright?

Playwright is an open-source automation library created by Microsoft. It’s designed to enable developers and testers to write reliable and efficient tests for web applications.

It’s cross platform and officially compatible with the major browsers.

Diving Into Codegen

Start by installing playwright inside your project

npm init playwright@latest

I’ll be using the defaults for this guide. After installation you’ll have these files generated in your project:

Files generated
For more: https://playwright.dev/docs/test-configuration

Putting Codegen Through Its Paces

First Run

Fire up codegen via the built in console

npx playwright codegen

As you click around your application you’ll see codegen record each click based on its css class.

It will build each line as you go about your test. In our video, we perform a login with an incorrect set of credentials and a correct set.

Our login.spec.ts file

Before our manual edits, this is what codegen generated for us:

import { test, expect } from '@playwright/test';

test('test', async ({ page }) => {
  await page.goto('http://localhost:3000/');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await page.getByLabel('Email Address').click();
  await page.locator('form div').filter({ hasText: 'Email AddressEmail Address' }).getByRole('paragraph').click();
  await page.getByLabel('Email Address').click();
  await page.getByLabel('Email Address').fill('[email protected]');
  await page.getByLabel('Email Address').press('Tab');
  await page.getByLabel('Password').fill('password');
  await page.getByLabel('Password').press('Enter');
  await page.getByLabel('Email Address').click();
  await page.getByLabel('Email Address').fill('[email protected]');
  await page.getByLabel('Email Address').press('Tab');
  await page.getByLabel('Password').fill('password123');
  await page.getByLabel('Password').press('Enter');
  await page.getByRole('button', { name: 'Settings' }).click();
  await page.getByText('Note: Your email mailsac.demo').click({
    button: 'middle'
  await page.getByRole('button', { name: 'Next boilerplate' }).click();

All we had to do was manually add was the highlighted lines (13 and 19) to turn it into a real test:

import { test, expect } from '@playwright/test';

test('test', async ({ page }) => {
  await page.goto('http://localhost:3000/');
  await page.getByRole('button', { name: 'Sign in' }).click();
  await page.getByLabel('Email Address').click();
  await page.locator('form div').filter({ hasText: 'Email AddressEmail Address' }).getByRole('paragraph').click();
  await page.getByLabel('Email Address').click();
  await page.getByLabel('Email Address').fill('[email protected]');
  await page.getByLabel('Email Address').press('Tab');
  await page.getByLabel('Password').fill('password');
  await page.getByLabel('Password').press('Enter');
  await expect(page).toHaveURL('http://localhost:3000/login');
  await page.getByLabel('Email Address').click();
  await page.getByLabel('Email Address').fill('[email protected]');
  await page.getByLabel('Email Address').press('Tab');
  await page.getByLabel('Password').fill('password123');
  await page.getByLabel('Password').press('Enter');
  await expect(page).toHaveURL('http://localhost:3000/home');
  await page.getByRole('button', { name: 'Settings' }).click();
  await page.getByText('Note: Your email mailsac.demo').click({
    button: 'middle'
  await page.getByRole('button', { name: 'Next boilerplate' }).click();

Running the Test

Running the test by default shows no visual progress. But if you’d like to see the browser run through your steps visually, you’ll need to issue the command:

npx playwright test --headed

Where to go next

The most natural next step is integrating playwright tests with a continuous integration platform like Travis or Github Actions. Plugging playwright into a CI system like Github Actions is fully supprted by playwright natively.

Another possible progression is using playwright to test critical paths in your application like user registration or password reset flows. We have a full guide on how to do that with another framework, Cypress.

If you want us to explore how you can integrate playwright with email testing and Github Actions or any other potential playwright integrations, let us know on our forums. We’ve only scratched the very surface of what playwright can do.

Until next time.

Integrating Email Testing into Your CI/CD Pipeline: A Step-by-Step Guide

Continuous Integration/Continuous Deployment (CI/CD) pipelines are at the heart of modern software development practices, enabling teams to automate testing and deployment processes to increase efficiency and reliability. Integrating email testing into your CI/CD pipeline is crucial for applications that rely on email functionality, ensuring that every update maintains or enhances the quality of email communications. This guide walks you through the process step by step, ensuring a seamless integration of email testing into your CI/CD workflows.

Step 1: Understand Your Email Testing Needs

Before diving into integration, clearly define what aspects of your email functionality need testing. This might include deliverability, content accuracy, responsiveness, and interaction with email clients. Understanding these needs helps in selecting the right tools and defining the scope of testing within your pipeline.

Step 2: Select the Right Email Testing Tools

Choose tools that offer API integration capabilities and can simulate various email scenarios. Tools like Mailsac allow for seamless integration into CI/CD pipelines, offering features such as disposable email addresses, REST API access for automated testing, and custom domain testing capabilities.

Step 3: Set Up Your Email Testing Environment

Configure your chosen email testing tool within your development and staging environments. Ensure it’s capable of capturing and analyzing the emails sent by your application during the automated testing phase. This setup should mimic your production environment as closely as possible to ensure accurate results.

Step 4: Integrate Email Testing into Your CI/CD Pipeline

Incorporate email testing tasks into your CI/CD pipeline configuration. This involves:

  • Triggering Email Tests: Automate the sending of emails based on specific triggers within your pipeline, such as a successful build or deployment to a staging environment.
  • Monitoring and Logging: Ensure your email testing tool captures detailed logs and analytics, providing insights into the success or failure of email deliveries and content rendering.
  • Analyzing Results: Set up mechanisms to analyze the results of email tests, identifying issues like failed deliveries, content errors, or rendering issues across email clients.

Step 5: Automate Feedback Loops

Implement automated feedback loops to alert developers and QA teams of any issues detected during email testing. This can be done through integration with project management tools, sending notifications via email, Slack, or other communication platforms used by your team.

Step 6: Continuous Monitoring and Optimization

Even after successful integration, continuously monitor the performance and effectiveness of your email testing within the CI/CD pipeline. Use insights gained from testing to refine and optimize email functionality, ensuring ongoing improvement and adherence to best practices.

Step 7: Document and Educate Your Team

Document the integration process, tools used, and best practices for email testing within your CI/CD pipeline. Educate your development and QA teams on the importance of email testing and how it fits into the broader context of quality assurance and software reliability.


Integrating email testing into your CI/CD pipeline is a strategic move that enhances the quality assurance process for applications relying on email communications. By following this step-by-step guide, your team can ensure that email functionality remains robust, reliable, and responsive to the needs of your users. Tools like Mailsac can play a pivotal role in this process, offering the flexibility, ease of integration, and comprehensive testing capabilities necessary to meet the demands of modern software development practices.

The Complete Checklist for Email Testing in QA Projects

Email remains a critical communication tool in business, making its reliability and functionality crucial for a wide range of applications. For QA teams, ensuring that emails are sent, received, and displayed as intended across various environments and platforms is a non-negotiable part of the software development process. This checklist provides a comprehensive guide for QA professionals to thoroughly test email functionality, ensuring a seamless user experience.

1. Preparation and Planning

  • Define Objectives: Clearly outline what you need to test, such as deliverability, content, links, and attachments.
  • Identify Email Scenarios: List all the scenarios in which an email would be sent. This includes transactional emails, marketing emails, notifications, and any others specific to your application.
  • Understand the Audience: Know the email clients (e.g., Gmail, Outlook) and devices (e.g., smartphones, tablets) your audience uses.

2. Functional Testing

  • Deliverability: Ensure emails reach the recipient’s inbox, not the spam folder.
  • Send and Receive Verification: Confirm that emails are sent and received without errors in all scenarios.
  • Link and Attachment Testing: Check all links and attachments for correct functionality and security.
  • Formatting and Layout: Verify that emails display correctly across different email clients and devices.
  • Accessibility Testing: Ensure emails are accessible, including alternative text for images and readable fonts for those with visual impairments.

3. Content and Design Testing

  • Spelling and Grammar: Verify the accuracy of content, including spelling and grammar.
  • Branding Consistency: Ensure the email design aligns with your brand’s guidelines and messaging.
  • Responsive Design: Test emails on various screen sizes to ensure the design is responsive and elements are clickable.

4. Security and Compliance Testing

  • Data Protection: Confirm that personal data is handled securely, in compliance with laws like GDPR.
  • Email Authentication: Check for SPF, DKIM, and DMARC records to prevent phishing and ensure sender authenticity.

5. Performance Testing

  • Load Testing: Assess the system’s ability to handle high volumes of emails without performance degradation.
  • Speed Testing: Evaluate the time taken to send, receive, and load emails, ensuring it meets user expectations.

6. Integration and Automation Testing

  • Integration Checks: Verify that email functionality integrates seamlessly with other systems and workflows.
  • Automation Suitability: Identify processes that can be automated, such as regression tests for email functionalities.

7. User Acceptance Testing

  • Real User Simulation: Conduct tests that mimic real-user scenarios to ensure the email meets user needs and expectations.
  • Feedback Collection: Gather and incorporate feedback from actual users to refine email functionality.


Email testing is a critical component of QA that ensures your application communicates effectively and reliably with users. By following this comprehensive checklist, QA teams can systematically address and rectify potential issues, enhancing the overall user experience.

Interested in simplifying and accelerating your email testing process? Mailsac offers a robust platform designed to streamline email testing for QA teams. With features like disposable email addresses, zero-setup custom domain support, and extensive API access, Mailsac enables you to focus on what matters most—delivering a flawless product. Try Mailsac for free today, getting test email in seconds, and discover how we can elevate your QA email testing to the next level.

Transform Your Email Testing with Cypress and Mailsac

If you’re in the business of making or testing software, you’re well aware of the critical role email functionality can play in applications. Particularly in sensitive workflows like user sign-ups and password resets. Testing these features however can often be a complex and daunting task. This is where Cypress and Mailsac come in. Together these two streamline the email testing process; cypress driving it and mailsac capturing and validating any emails that may come out of your testing.

Join me as we explore how you can leverage these powerful tools to automate and enhance your testing workflow, ensuring a seamless user experience in your applications.

This article acts as a companion guide to the video linked at the top. Here, we’ll quickly walk through the topics mentioned in the video but also expand some areas and provide the code that the video uses.

The general path we’ll take in this guide is:

  1. Walk through our application’s reset flow
  2. Safeguard against sending emails to real customers
  3. Automate the password reset process with subject verification with cypress.

Let’s get started!

Setting the Stage: Manually Testing our Next.js App

We’ll focus on a simple next.js application equipped with local authentication features. Instead of starting from scratch, we’ll use Martin Persson’s Next.js Authentication Template. The concepts we’ll test are a common part of almost every application, and by using his template we’ll have a starting point (most) of us can agree on. Martin’s application comes complete with login functionality, member-exclusive sections, and importantly, a password reset flow.

Our goal is to show how we can streamline testing the password reset flow, ensuring that branch of your code behaves the same way every time. By the end of this, you’ll have a clear blueprint for applying these techniques in your projects.

Regular Password Reset Speedrun

Let’s do a quick walk through on how we would reset our password in our app.

Create an Account

The app doesn’t do any email validation out of the box but it does create and store account credentials in its database.

Reset Password To a Real Email Address

This is a standard password reset flow. The main thing to note here is that the application is sending out transactional emails to real email addresses.


During continuous testing we could be building our application and testing its capabilities tens to hundreds of times per month. We’ll need to ensure we don’t send any emails real customers.

This is where Mailsac comes in.

Safeguarding Against Sending Emails to Real Customers

Mailsac has an email capture feature that enables customers to “reroute” any emails generated by your application to an inbox on the mailsac platform. Additional testing capabilities are available including verification of delivered content and ensuring the emails that get sent have the correct subject.

Start Capturing Emails from your Application

Getting started is extremely easy. Just point your application’s SMTP settings to the mailsac’s servers and all outbound emails will be captured.

Our sample application reads in a .env file:

MAIL_PASSWORD="k_99dfuuifjd"         # Your Mailsac API Key
MAIL_USER="mailsacuser"              # Your Mailsac Username

Going through the password reset flow in our application again, we successfully have no emails in our real gmail inbox, and an email in the Mailsac inbox.

Also, we did not create this inbox ahead of time. It was done ad-hoc as the email came into mailsac.

Success. Now we are safe against sending out real emails to customers. Let’s begin the automated testing processes with Cypress

Automatic Testing with Cypress

Our goal with cypress is to automate the clicking of a password reset link, and to verify that an email was actually initiated and sent by our application.

Installing and Configuring Cypress

Let’s start by installing cypress

npm cypress install

And then opening up the cypress testing center

npx cypress open

You’ll be greeted by what kind of testing type to initiate. In our case, we’ll use end to end testing

And the included Chrome browser

We are then greeted by the testing specs list

And now the real work begins.

Configure Cypress

Our configuration of cypress will need mailsac’s client and dotenv library. Cypress will need this to have the ability to read our application’s .env file

npm install @mailsac/api dotenv

We can now define what exactly a how exactly to execute a checkMailsacMail and deleteAllEmails function

import { defineConfig } from "cypress"
import { Mailsac } from "@mailsac/api"
import * as dotenv from 'dotenv';

export default defineConfig({
  e2e: {
    // https://docs.cypress.io/guides/references/configuration#e2e
    baseUrl: "http://localhost:3000",
    defaultCommandTimeout: 15000,
    // Whether or not test isolation is enabled to ensure a clean browser context between tests.
    testIsolation: false,
    setupNodeEvents(on,config ) {
      on('task', {
        async checkMailsacMail(address) {
          const mailsac = new Mailsac({ headers: { "Mailsac-Key": process.env.MAIL_PASSWORD } })
          const messages = await mailsac.messages.listMessages(address);
          return messages.data;
        async deleteAllEmails(address) {
          const mailsac = new Mailsac({ headers: { "Mailsac-Key": process.env.MAIL_PASSWORD } })
          const messages = await mailsac.messages.deleteAllMessages(address);
          if(messages.status === 204) {
            return true;
          } else {
            return false;
  component: {
    devServer: {
      framework: "next",
      bundler: "webpack",
  // Settings
  env: {
    baseUrl: "http://localhost:3000",
    username: "[email protected]",
    password: "password123",

The contents of this file are fully explained in the video (Starting at 6:24) but to call out the highlighted lines (13-30):


The checkMailsacMail function initiates a mailsac client and uses the API key provided from the .env file variable, MAIL_PASSWORD. It then calls the listMessages function and passes it up to the calling function to do as it pleases. The function returns a JSON response as outlined in the Mailsac ListMessages documentation.


Similar to checkMailsacMail but of course, deletes all messages in an inbox. Note that the return is a 204 to confirm deletion.

Let’s move on to the actual password testing spec

Password Reset Flow Test Spec

The spec itself is placed in the cypress/e2e/password_reset_flow_success.cy.ts path and contains:

describe('Password Reset Change', () => {
  it('Should successfully change the password', () => {

    // Find a button with class and contains text
    cy.get(".MuiButton-root").contains("Sign in").click()

    // The new url should include
    cy.url().should("include", "/login")

    const { username, password } = Cypress.env()

    cy.get("a").contains("Forgot password").click()

    cy.url().should("include", "/forgot-password")

    cy.get("input[name=email]").should("have.value", username);

    cy.url().should("include", "/login")    

    cy.task("checkMailsacMail", username).then((messages) => {
      const resetUrl = messages[0].links[0];
      const subject = messages[0].subject;
      const originalInbox = messages[0].originalInbox;

      expect(subject).to.eq("next-boilerplate: Reset your password.");

      cy.url().should("include", resetUrl);

      cy.task("deleteAllEmails", username).then((result) => {


Cypress will run through this spec file line by line and execute the steps we manually ran through at the beginning of the guide. The highlighted lines are really the structure of the returned JSON from mailsac’s ListMessages API

Final Automation

The the spec file in place, go ahead and run the spec

Wrap up

Combining Cypress and Mailsac is like giving your email testing a super boost. We’ve walked you through the nitty-gritty of automating a password reset flow, making sure your emails hit the mark without bothering real customers. It’s all about making your software solid while keeping things simple and stress-free. Give it a try, save time, and keep your users smiling.

If you have any questions about the guide or if you get stuck feel free to ask us anything in our forums.

November 2023 Release Notes

Throttling notices in every inbox

Custmers will start seeing informational messages in the inbox view https://mailsac.com/inbox/{{ email address }} when there has been throttling imposed on the inbox or sender to the inbox.

Paying customers will very rarely experience throttling. In almost all cases, throttling happens because they were sending to a public inbox, not a custom domain or private address.

We only throttle incoming messages to protect the stability of our service for all of our customers. These protections have been in place for years, but were not transparent to customers.

If you are on a paid plan and you are seeing throttling messages, reach out to [email protected], we can help you configure a custom domain or private addresses. These both can be done in seconds with no need for DNS changes.

If you are on our free tier and seeing these messages, this is a nudge for you to sign up for a paid plan. We would love to have you as a customer.