Spring Cleaning with Mailsac API

My garden has been planted and the days are getting warmer. Next task, cleaning up. Mailsac won’t help you organize you garage or avoid the accumulation of clutter. But it can help you get rid of old emails.

The Delete All Messages in a Domain Endpoint released on March 23, 2021 can help you keep your private domain storage under the storage limit. It’s useful for deleting all messages before running integration tests.

This post will detail the different methods of bulk message deletion.

Cleanup a Custom Domain

All messages in a custom domain can irrevocably be deleted.

From the Dashboard select Custom Domains Select Domain to Mange → Advanced tab → Irreversibly Delete Email

The REST API has a commensurate endpoint for deleting all messages in a domain. It can be invoked by calling an HTTP POST https://mailsac.com/api/domains/{domain}/delete-all-domain-mail , where {domain} is the custom domain.

Cleanup a Private Inbox

All messages in a Private Inbox can be deleted from the website or the REST API. Both methods will not delete starred messages.

From the From the Dashboard select Manage Email Addresses Select Email Address Purge Inbox

Messages in a Private Inbox can be deleted using the endpoint for deleting messages in a private inbox. The endpoint can be used with the HTTP DELETE method on https://mailsac.com/api/addresses/{email}/messages where {email} is the private email address.

Deleting Individual Messages

Individual messages can be deleted from both private and non-private inboxes.

From the From the Dashboard enter an inbox in the inbox viewer form and select Check The Mail!. From the Inbox, Click the message → Select Delete Permanently Delete

Individual messages can be deleted using the delete message REST API. The endpoint can be called using an HTTP DELETE with the URL https://mailsac.com/api/addresses/{email}/messages/{messageId} . The {email} parameter is the email address and the {messageId} is identifier for the message to be deleted. The messageId can be found by using the list messages in an inbox REST API endpoint.

Industry Standard API Documentation

Mailsac API Documentation has been converted over to use OpenAPI.

Code Examples

The new format gives developers and quality assurance testers code samples for common programming languages (curl, Node, Python, PHP, Go, etc).

Explore the API Using Swagger UI

The API can be explored without writing any code using the Swagger UI Explorer. All that is needed to get started is a free Mailsac API Key (requires sign up).

In the Swagger UI Explorer, choose the Authorize button and enter your Mailsac API Key


Use the “Try it out” button to interact with any of the REST API endpoints. A curl example, request URL, response body, and headers are provided.

New and Improved SMTP Header REST API Endpoint

Devs and Quality Assurance Testers Can Easily Validate Mail Headers

The SMTP header endpoint provides quality assurance testers with the option to view an email message’s SMTP headers in parsed formats that easily integrate with automated testing frameworks.

Problem

Developers and QAs are often asked to validate contents of emails. This can include from address, links, and subject. For many organizations this can be a manual process of checking the email and validating if the test criteria has been met.

Solution

Mailsac’s new message header endpoint provides SMTP headers in 3 formats:

1. JSON object format, grouped by lowercased header key. This format is easily consumed by industry standard tools such as Selenium.

{
  "received": [
    "from 107.174.234.77 by frontend1-172-31-29-224 via 172.31.42.57 with HTTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon Dec 24 2018 15:29:06 GMT+0000 (Coordinated Universal Time)",
    "from 107.174.234.77 by smtp-in2-172-31-42-57 via 172.31.23.10 (proxy) with SMTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon, 24 Dec 2018 15:29:06 UTC",
  ],
  "from": [
    "grem@hello.msdc.co"
  ],
  "to": [
    "cd@mailsac.com"
  ],
  "subject": [
    "invitation to collaborate"
],
  "date": [
    "Mon, 24 Dec 2018 15:29:06 +0000"
  ]
}

2. Ordered JSON array format. This formats pre-parses the headers, but maintains the original order, while still handling duplicate headers such as Received.

?format=ordered-json

[
  {
    "name": "received",
    "value": "from 107.174.234.77 by frontend1-172-31-29-224 via 172.31.42.57 with HTTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon Dec 24 2018 15:29:06 GMT+0000 (Coordinated Universal Time)"
  },
  {
    "name": "received",
    "value": "from 107.174.234.77 by smtp-in2-172-31-42-57 via 172.31.23.10 (proxy) with SMTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon, 24 Dec 2018 15:29:06 UTC"
  },
...
  {
    "name": "to",
    "value": "cd@mailsac.com"
  },
]

3. Plaintext original format. This format is useful when you are interested in parsing or inspecting the email headers yourself, and do not wish to download the entire message.


?format=plain

Received: from 107.174.234.77 by frontend1-172-31-29-224 via 172.31.42.57 with HTTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon Dec 24 2018 15:29:06 GMT+0000 (Coordinated Universal Time)
Received: from 107.174.234.77 by smtp-in2-172-31-42-57 via 172.31.23.10 (proxy) with SMTP id 8m7iqeiZKJ3MzwTwUQlU for <cd@mailsac.com>; Mon, 24 Dec 2018 15:29:06 UTC
...
To: cd@mailsac.com

“We are currently using the REST API headers endpoint in support between our own microservices. Our POP3 server fetches headers of message to implement the POP3 TOP command.” — Michael Mayer, Partner Forking Software LLC

Getting Started

The message header endpoint /api/messages/:messageId/headers is available on all Mailsac plans (including our free tier). See our API Specification for more information.

This code example could can be modified to view the headers for the first email message on an inbox calendartrinity@mailsac.com. Make sure to insert your API Key and change the email address to an email address you which is public or reserved by your account.

const superagent = require('superagent') // npm install superagent

const mailsac_api_key = 'YOUR_API_KEY_HERE' // change this!

superagent
  .get('https://mailsac.com/api/addresses/calendartrinity@mailsac.com/messages')
  .set('Mailsac-Key', mailsac_api_key)
  .then((messages) => {
      const messageId = messages.body[0]._id
      superagent
          .get('https://mailsac.com/api/addresses/calendartrinity@mailsac.com/messages/' + messageId + '/headers')
          .set('Mailsac-Key', mailsac_api_key)
           .then((response) => {
               console.log(response.body)
           })
  })
  .catch(err => console.error(err))

/**
{
  received: [
    'from [ by fireroof via ::1 with HTTP id bo4xdVji_oqEixBO0gGLbvIoe for <calendartrinity@mailsac.com>; Wed, 28 Oct 2020 23:05:29 GMT',
    'from [ fireroof with SMTP id bo4xdVji_oqEixBO0gGLbvIoe for <calendartrinity@mailsac.com>; Wed, 28 Oct 2020 16:05:29 PDT'
  ],
  'x-mailsac-inbound-version': [ '' ],
  date: [ 'Wed, 28 Oct 2020 16:05:29 -0700' ],
  to: [ 'calendartrinity@mailsac.com' ],
  from: [ 'bclinton@mailsac.com' ],
  subject: [ 'test Wed, 28 Oct 2020 16:05:29 -0700' ],
  'message-id': [ '<20201028160528.2893005@fireroof>' ],
  'x-mailer': [ 'swaks v20190914.0 jetmore.org/john/code/swaks/' ]
}
**/

Retiring TLS 1.0 and TLS 1.1

Our REST APIs and website will require the use of TLSv1.2 on October 24, 2020. TLS 1.2 was published as RFC 5246 in 2008.

All major web browsers (Chrome, Edge, Internet Explorer, Firefox, and Safari) have already disabled TLS 1.0 and TLS 1.1. Most modern programming languages have support for TLS 1.2.

Integrations written in Java 6 and Python 2.6 do not have TLS 1.2 support.

References

REST Compliant Response From Email Validation API

The GET Method on the REST API endpoint /api/validations/addresses/:addressToValidate has been updated to return a JSON object. Previously, this endpoint was returning an array. The POST Method response remains unchanged.

An example GET request and response has been added to the Mailsac REST API documentation.

Email validation via the Mailsac Website is still available to all registered customers. An email address can be checked for valid format and if it is associated with any known disposable email services.

New Delete Messages Flag When Releasing a Private Email Address

When releasing a private address, there is now an option to “empty the inbox” – deleting all messages associated with the private address. You can find the option by clicking the settings button when managing your private addresses.

The API endpoint DELETE /api/addresses/:email now supports the query string deleteAddressMessages. When deleteAddressMessages=true is passed, all messages associated with the inbox will be deleted.

Please note that all messages are deleted, including starred/saved messages. It is immediate and irreversible.

Authentication Changes

Mailsac now requires authentication on all API routes and many parts of the website. The API key for your account can be created and viewed from the dashboard. If you are using the website to view emails you will need to create an account and sign in to view the body, images, and headers.

API Authentication

Many API routes required authentication prior to this change. If you are a customer with an existing API key and using it to make your API requests, there are no changes you need to make.

If you were using anonymous access, you will need to create an account and create an API key.

There are three methods for authenticating to the API. HTTP Header, Query String Parameter, and Request JSON Body. To use the HTTP Header create the request header Mailsac-Key and use the value of your API key. To use the query string parameter append the query string parameter _mailsacKey to the query section (after ?) in the url. Example: https://mailsac.com/api/addresses/test@example.com/messages?_mailsacKey=eoj1mn7x5y61w0egs25j6xrv
During a POST or PUT operation a JSON field _mailsacKey can be used.

For a complete list of API Routes check out the API documentation.

Website Authentication

The content of the most recent email received is still available without logging in. Older messages, images, and headers will require an account. You can register for an account for free.

Open Source Spam Processing

Mailsac has added a new spamminess indicator in the API.

New messages will have a spam property that’s a score between 0.0 and 1.0. 1.0 indicates a high likelihood of spam. The system will get smarter over time.

The main component of spam detection is a time tested approach called Naive Bayes Classifier. The core of the code is open source at github.com/mailsac/spam-classifier, though a little special sauce is applied, too. This classifier library is ready to use and includes a simple API server and a terminal training tool.

In the works are additional spam classification projects using three different deep learning techniques: random forest, a long short term neural network, and a combination recurrent network + liquid state machine with spike dependent plasticity. The goal is to open source the useful pieces of the coming spam classifiers.

Parsed Links on Messages API

Message metadata will now include pre-parsed links. This enhancement should make automation even easier.

 

For example, say you want to test email signup confirmations on your website. Normally you would need to request the inbox for the test inbox:

GET /api/addresses/mytestemail@example.com/messages
[{
  _id: 'y39sak29gaer-0',
  subject: 'Confirm your email address',
  from: [{ address: 'noreply@sampleservice.com' }],
  to: [{ address: 'mytestemail@example.com' }]
  // more fields
}]

then download the full email or body:

GET /api/dirty/mytestemail@example.com/y39sak29gaer-0
<div>
  <a href="https://www.sampleservice.com/email-confirmation/39349234">Confirm your email!</a>
</div>

then parse the contents and search for the link.

Now the message metadata from the first request will also include a links array of strings:

GET /api/addresses/mytestemail@example.com/messages
[{
  _id: 'y39sak29gaer-0',
  subject: 'Confirm your email address',
  from: [{ address: 'noreply@sampleservice.com' }],
  to: [{ address: 'mytestemail@example.com' }]
  links: ['https://www.sampleservice.com/email-confirmation/39349234']
}]

This feature should save you a roundtrip to the API and let you not worry about properly parsing HTML links.

 

API Reference

 

Faster response times on message routes

A LRU caching layer, active on each API, was added recently. Since email contents are stored separately from the database and every other microservice, many API requests will get faster. We expect faster response times for recently received messages, messages that are requested repeatedly, and any API request where the message is re-parsed in-flight.

Mailsac stores metadata about every message in MongoDB. In the interest of saving disk and RAM, the full email contents are stored across file servers. When a piece of the email is requested via the API, the full file is fetched and parsed into the desired format. Believe it or not, for Mailsac, CPU is far less resource constrained than disk and RAM, so this trade-off makes sense.

For example, when the email headers are requested, the API begins fetching the full message from the file server, parses the SMTP package, formats the headers, and sends them to the client. Fetching attachments is similar – when an attachment is requested, the API gets the original email from the file server where it is stored, extracts the attachment, and transfers it to the client.

This process involves a lot of disk IO and network latency. With the LRU cache, we seeing 80 ms – 600 ms faster requests, and the cache is used on about 45% of requests that re-parse the email.