Parsed Links on Messages API

Message metadata will now include pre-parsed links. This enhancement should make automation even easier.

 

For example, say you want to test email signup confirmations on your website. Normally you would need to request the inbox for the test inbox:

GET /api/addresses/[email protected]/messages
[{
  _id: 'y39sak29gaer-0',
  subject: 'Confirm your email address',
  from: [{ address: '[email protected]' }],
  to: [{ address: '[email protected]' }]
  // more fields
}]

then download the full email or body:

GET /api/dirty/[email protected]/y39sak29gaer-0
<div>
  <a href="https://www.sampleservice.com/email-confirmation/39349234">Confirm your email!</a>
</div>

then parse the contents and search for the link.

Now the message metadata from the first request will also include a links array of strings:

GET /api/addresses/[email protected]/messages
[{
  _id: 'y39sak29gaer-0',
  subject: 'Confirm your email address',
  from: [{ address: '[email protected]' }],
  to: [{ address: '[email protected]' }]
  links: ['https://www.sampleservice.com/email-confirmation/39349234']
}]

This feature should save you a roundtrip to the API and let you not worry about properly parsing HTML links.

 

API Reference

 

15 minute partial API outage due apparent DOS

For about 15 minutes (8:13 am – 8:27 am PDT), our API was flooded with traffic due to hundreds of thousands of email attempts from 4 IP addresses. Nearly all emails were received, but HTTP requests for the API and UI frequently timed out. We do not know the percentage of requests that timed out, but it was quite high. The API is load balanced and only one API was timing out frequently.

We blocked the bad IPs immediately upon seeing traffic logs (~8:19), but because our custom IP blocking service relies on the API to fetch the blacklist, and the API was not fully responsive to HTTP on one leg, it took a while for the changes to propagate to all five inbound SMTP servers.

Faster response times on message routes

A LRU caching layer, active on each API, was added recently. Since email contents are stored separately from the database and every other microservice, many API requests will get faster. We expect faster response times for recently received messages, messages that are requested repeatedly, and any API request where the message is re-parsed in-flight.

Mailsac stores metadata about every message in MongoDB. In the interest of saving disk and RAM, the full email contents are stored across file servers. When a piece of the email is requested via the API, the full file is fetched and parsed into the desired format. Believe it or not, for Mailsac, CPU is far less resource constrained than disk and RAM, so this trade-off makes sense.

For example, when the email headers are requested, the API begins fetching the full message from the file server, parses the SMTP package, formats the headers, and sends them to the client. Fetching attachments is similar – when an attachment is requested, the API gets the original email from the file server where it is stored, extracts the attachment, and transfers it to the client.

This process involves a lot of disk IO and network latency. With the LRU cache, we seeing 80 ms – 600 ms faster requests, and the cache is used on about 45% of requests that re-parse the email.

Enhanced Outbound Sending API

Until now, all transactional messages sent through the Mailsac API only allowed text. With the latest release, the API supports HTML and attachments. Additionally supported is sending an entire raw SMTP message over the REST API.

New Transactional Email Fields

 

HTML

You can provide full HTML in the html field when doing a POST /api/outgoing-messages. This was not possible in previous versions of the API.

Attachments

Attachments can be sent and linked into the html by including an array of attachment objects in the attachments field.

Example Inline Attachment
{
  "contentDisposition": "attachment",
  "content": "3asfji32gia...93as==",
  "encoding": "base64",
  "filename": "2561a.jpg",
}
Example Attached File
{
  "cid": "[email protected].",
  "contentDisposition": "inline; filename=2561a.jpg",
  "content": "3asfji32gia...93as==",
  "encoding": "base64",
  "filename": "2561a.jpg",
}

Received Headers

Persisting received headers is now possible – send a single string or array of received headers. This allows you to receive SMTP in your application, pre-parse it, and pass along received information.

Raw SMTP over HTTP

 

If you want to test SMTP parsing or have an application that generates raw SMTP messages, put the entire SMTP contents into the raw field when doing a POST /api/outgoing-messages.

Mailsac will parse the headers, text, HTML, and attachments, then deliver it for you.

All other fields will be ignored when you send a raw message.

 

 

Read the API docs for sending transactional emails over HTTP

 

API Reference Overhaul

The API documentation received a fresh new look using a fork of the popular Slate docs framework.

new-docs-look
Significant update to our documentation

API endpoints should be easier to navigate. Many request and response examples were added. Endpoints for downloading attachments are easier to find.

In the coming weeks there will be updates to the docs for the SMTP Web Socket API and the Webhook API, including new examples and guides.

View the new API Docs

POP server release

A new POP3 server has been deployed. It was rewritten from the ground up in Golang. The source code will be released to GitHub soon.

The original Node.js POP3 server consumed far more memory than we felt necessary (120 MB upon startup, 400 MB under regular load), and the codebase relied on some packages which were outdated due to the author re-licensing dependencies under restrictive licenses, and removing some of the source code from GitHub. We had to pin to older versions which were initially published under the MIT license. There are still other Mailsac SMTP services packages stuck on older SMTP packages; these will be migrated to Go soon also. We hope to open source as much as possible again soon – the Mailsac GitHub project is far out of date from mailsac.com.

Mailsac POP3 settings