Email integration is tricky. It’s something that’s great in theory because it’s a medium and interface that almost everyone is comfortable with. In some ways, it’s convenient, while in other ways, it can be tedious. The real challenge is understanding and appreciating where it should fit in with an application. We definitely view it as an important piece of the puzzle, but we’re cautious about the potential problems that it introduces. This blog post goes over some of the details of our execution. We also have a changelog post that addresses specifics of what we’ve added.
Concerns & Considerations
Email is an interesting feature to tackle. It has a fair amount of quirks that need to be accounted for in order to successfully handle all of the curveballs that get thrown at you. There’s a degree of overlap between some of these, but they’re just different enough to justify addressing them separately.
Email Errors
What happens when an email fails? This could be caused by deliverability problems, parsing issues, or a variety of other mistakes.
Partial Failures
What happens when it only partially fails? On a web page, you might get a single validation error. With an email, do we reject the whole thing? Accept the part that works?
Attachments & Signature Files
We’d love to accept attachments via email, but there’s a really big problem with that since a not insignificant number of people have complex signature files with corporate logos and other stuff. Automatically attaching these to every issue or comment isn’t acceptable, but filtering them out is nearly impossible without ruining the experience.
Communicating Errors to Users
It’s important that we communicate with users when we encounter a problem handling an email. The project could be archived, the account could be frozen, or the person may not actually have access to a given project. We need to proactively let people know when things go wrong, and email is the only way for us to do that which leads to the next challenge.
Asynchronous Nature of Email
When sending error notifications, we need to explain which email caused the error. If the user fired off three quick emails and only one of them failed, they need to know which one. (We actually need to improve this as we’re not doing a great job right now.)
Encoding
This one isn’t impossible to work with, but it’s important to get right. Otherwise, the issue creation will fail silently. If the encoding is wrong, the message can be truncated incorrectly and end up including too much or even worse, too little, of the email body or subject.
Email Client Varieties
Just like browsers, email clients all have their little quirks. From how they handle forwarded messages to their default encoding, they present a lot of curveballs that need to be tested against.
Spam
If you accept email from any source, you’re going to encounter spam. If you don’t accept email from any source, you’re limiting the functionality. Finding the right balance of tolerance and security can be challenging.
How it Works Now
Getting Your Custom Email Address
Every user gets a unique email address for each project that they are assigned to. That email address is a secret accessible only by them, and we cross-reference every email with sending address as an added layer of spam prevention. Sifter’s inbound email is explicitly not designed to work as a generic support email address for an indeterminate number of users. That is, it won’t work so well as a help desk. It is designed to provide an alternative to the web interface for users with Sifter accounts. That may change in the future, but for now, Sifter is heavily designed around the concept of accountability and the assumption that everyone that needs a user account in order to participate in a project.
On every new issue page, there’s a link to the instructions explaining how to email issues to a project. (Figure 1) The subsequent page includes the unique email address, the ability to download a vCard with the email address, and some basic instructional text about how to use the more advanced options. (Figure 2).
Setting an Issue Subject
We extract the issue subject from the subject line of the email. However, we make sure to remove and “Re:” or “Fwd:” text so that the subject isn’t cluttered up.
Updating Issue Attributes via Email with Hash Tags
When creating or updating issues, you can update the assignee or priority for the issue. When updating an existing issue, you can also update the status. We don’t allow you to set a status for new issues because they default to open. We’ve created a fairly simple syntax for updating issues via email through hash tags included on the first line of your email. If it’s not on the first line, we won’t recognize it.
For example…
#resolve #a:gdimon
…would resolve an issue and assign it to me. If you use tags, they must be on the first line of the email. You don’t have to include tags with every email, though. You can just do a quick text update if you’d like.
Updating Issue Status
When updating the status, we explicitly make sure to recognize several variations on the status names so that you don’t have to remember exactly which variation we use. For instance, when reopening an issue, we’ll gladly accept any of the following.
#reopen, #reopens, #reopened, #open, #opens, #opened
Similarly, you can resolve issues with…
#resolve, #resolves, #resolved
…and close issues with…
#close, #closes, #closed
Updating Issue Assignee
You can also assign or unassign issues with #a:username. You’ll need to know usernames, but we include a list for each project at the bottom of the instructions page for each email address. So, for example…
#a:gdimon
…would assign an issue to me while…
#unassign
…would unassign an issue.
Updating Priority
With priority, we recognize the X-Priority email header value that can be set using your email client. So instead of using a hash tag, you can use your email client’s interface. (Figure 3) We understand that there are inherent limitations to this, but ideally, priority shouldn’t be changing significantly anyways.
Error Handling
With basic usage of the email integration, errors should be few and far between. However, because failing silently would create significant confusion, we’ve made sure to carefully and explicitly catch errors and notify the sender of the problem so that they can address it. For instance, if the project has been archived, it’s important to let users know that they can no longer submit issues to that project. Similarly, because we tie the custom email address to the sender, we have to make sure that users send emails from the correct address. If they accidentally use an alternate address, we need to let them know that we couldn’t find a match.
Right now, when we encounter these errors, we explain what the problem was, and include information about the specific email that caused the problem. Including the original email content is key because if they have fired off several emails, they need to be able to easily identify which one caused the problems. This also makes it easier for them to resubmit in the case of a simple typo or other correctable problem.
On Attachments
Right now, we’ve explicitly chosen not to handle attachments. We’d definitely like to, but in order to accept attachments via email, we’d also be accepting all of the junk that comes along with some signature files. For now, we don’t think that anyone wants to have somebody’s company logo or legal policy automatically attached to every single comment and issue because it’s in their email signature.
So far, we’ve thought of two options. The first would be to require a specific syntax for attachment names so that we could safely ignore any files that aren’t named according to a strict set of rules. However, we really don’t like that idea as it becomes imposing and inconvenient. It’s incredibly unlikely that we’ll go down that path.
The second option is for us to do some heavy duty research into the matter to see if there are some recognizable patterns that would enable us to safely ignore signature-related attachments without accidentally losing valid attachments. The key part is ensuring that we don’t incorrectly ignore valid attachments.
We’d love to do attachments, but until we’re confident that we can pull it off without flooding accounts with signature file attachments, we’re going to hold off. It’s still the early days of email integration, and we want to play it safe.