The Developer's Guide to Building Notification Systems: Routing and Preferences

Your CTO handed you a project to revamp or build your product’s notification system recently. You realized the complexity of this project around the same time as you discovered that there’s not a lot of information online on how to do it. Companies like LinkedIn, Uber, and Slack have large teams of over 25 employees working just on notifications, but smaller companies like yours don’t have that luxury. So how can you meet the same level of quality with a team of one? This is the third post in our series on how you, a developer, can build or improve the best notification system for your company. It follows the first post about identifying user requirements and designing with scalability and reliability in mind. In this piece, we will learn about setting up routing and preferences.

Notifications serve a range of purposes, from delivering news to providing crucial security alerts that require immediate attention. A reliable notification system both enables valuable interactions between an organization and its customers and prospects and also drives user engagement. These systems combine software engineering with the art of marketing to the right people at the right time.

Building a service capable of dynamically routing notifications and managing preferences is vital to any notification system. But if you’ve never built a system like this, it might be difficult to figure out what the requirements are and where the edge cases lie.

In this article, you’ll learn invaluable points to consider when building your own routing service. You’ll understand the requirements for multi-channel support and in choosing the right API providers. You’ll also learn how to design user preferences so that you can make the most out of each message.

Multi-channel support: a necessity

Let’s say that you have just built a web-based application. The first channel that you’ll use to connect with your users is likely email because of how ubiquitous it is. However, with the diversification of channels and depending on your use case, email might not be the most efficient notification channel for you. Compared to other channels, emails typically have a low delivery rate, a low open rate, and a high time to open rate. It’s not uncommon for people to take a full day to even notice your email. If your email gets to the user, it might take awhile before they open it, if at all.

To engage with your users more effectively, you’ll want to support channels across a broad range of systems not limited to any one application or device. It’s vital to understand not only which channels are most relevant for you but also for your users. If you opt to use Telegram and your users don’t have it, it won’t be a very useful channel to interact with them. Multi-channel support is also vital because while you might pick appropriate channels today, you won’t know which channels you will need to support in the future. Typically, the more appropriate channels you support, the higher the chances of intersecting with applications your users actually use now and in the future.

Choosing notification channels and providers

You’ll have to select relevant channels and appropriate providers for each channel. For example, two core providers for mobile push notifications are Apple Push Notification Service (APNs) and Firebase Cloud Messaging (FCM). APNs only supports Apple devices while Firebase supports both Android and iOS as well as Chrome web apps.

In the world of email providers, SendGrid, Mailgun, and Postmark are all popular but there are hundreds more. All email APIs differ in what they offer, both in supported functionality and API flexibility. Some providers, like Mailgun, only support transactional emails triggered by user activity. Other providers, like SendGrid and Sendinblue, offer both transactional and marketing emails. If your company opts for a provider that can handle both, you’ll still want to separate the traffic sources, by using different email addresses or domains, to aid email deliverability. If you only have one domain for sending both types of emails and the domain gets flagged as spam, your critical transactional emails will also be affected. Whichever provider you choose, you’ll still want to meticulously verify your DKIM, SPF, and DMARC checks, and domain and IP blacklisting using your own tools or a site like Mail-Tester.

Making requests and receiving responses also differs with each email API provider. Some providers, like Amazon SES, require the developer to handle sending attachments, while others, like Mailgun, provide fields in the API schema for including attachment files directly.

There are minute variances in formatting HTTPS requests. The maximum payload sizes range from 10MB with Amazon SES API and up to 50MB with Postmark. There are also differences between the rate limits for requests.

In terms of API responses, Amazon SES provides a message identifier when an email is sent successfully through the API, but, for example, SendGrid returns an empty response in that situation. The HTTP response codes also differ slightly depending on the provider. For example, AWS SES uses the response code 200 for successful email send operations, while Sendinblue uses 201, and SendGrid uses 202.

No matter which provider you end up choosing, don’t build your application solely to fit their logic and specifications. If you do so, it will be much more difficult to change providers in the future as you’ll have to overhaul your backend. It’s crucial to invest in a layer of abstraction based on your own paradigm.

Dynamically routing notifications between channels

How do you determine which channels to use and when? Just because you’re able to use email, SMS and mobile push doesn’t mean that you should use all of them simultaneously, since doing so carries a high risk of annoying your users. This is where you begin to formulate an algorithm to route messages between the different channels and the different providers within each channel. The algorithm needs to be robust to handle delivery failures and other errors. For example, if the user hasn’t engaged with a push notification after a day, do you resend it or use email instead?

You can begin constructing the algorithm using basic criteria. For example, if there is no phone number, eliminate SMS as an option for that user. If email is the primary channel, opting to send at 10 a.m. or 1 p.m. local time typically improves read rates. If the user is present or active in the app, consider sending an in-app push notification instead of an email. Finally, and especially important, get your user’s preferences for how and when they want to be contacted and integrate these preferences into your routing service.

Adding user preferences to your system

Once you’ve got your channels, providers, and routing algorithm figured out, you need to think about providing users with granular control over notification preferences instead of just a binary opt-in/opt-out switch.

Consider this: if you only allow opting in to or out of all notifications at once, your users might unsubscribe from all your communications because they find one specific notification annoying. As a result, you will lose out on valuable user engagement.

With granular control over preferences, a user identifies exactly how and when they hear from you. If a user doesn’t like email but wants SMS messages (not common, but possible!), they can adjust their preferences and keep the SMS line of communication open. Every enabled notification channel is another opportunity to engage the user in a way that’s productive for them. From the end user’s perspective, it’s empowering to control how and when they are contacted.

Note that for some channels, the user’s preferences should be ignored. For instance, two-factor authentication should go to SMS or mobile push regardless of the user’s preference for email. The possibility to override the default logic should be incorporated into your algorithm while you are designing your routing engine.

If you want to take user engagement further, allow users to opt-in/opt-out of specific channels, frequency, timing and topics. You can allow them to set up their preferences based on time of day, frequency per period, or to specify more than one email address. You can give them the option to receive transactional, digest emails, daily newsletters, or only the critical ones. You can also allow them to redirect their notifications to another address, for example if the user is out of office.

Granular preferences also extend past the dominion of developers and the user’s experience. Granularity of consent is becoming part of privacy compliance laws in Europe and in the state of California and might follow elsewhere in the future. Separately, granular preferences are an extremely advantageous analytical tool for the marketing team to improve brand strategy and personalization efforts. Is there a particular channel or topic that seems to be more popular? That information can be highly helpful to pivot in line with your users and grow your company.

Tips for future-proof maintenance

When you’re starting with notifications for a new product, there is nothing wrong with sticking to one channel and one provider. The most important principle to keep in mind is to design your notification system so that you can expand it in the future. You should leave the door open to include more providers when you need them.

Don’t assume that API paradigms are the same for each provider or notification type. For example, you want to send an email, and if delivery fails to send a push notification instead. But you won’t get a 400 HTTP response from the email provider in case of failure. The provider will retry your email over a couple of days. Instead, you’ll want to include webhooks or queues to notify you of the failure, and you’ll need to track the state of the message here. If you make blanket assumptions about how API calls work or how errors are returned, you’ll have trouble adapting to a different paradigm in the future. Instead, you can add a layer of abstraction on top of the API.

It’s also invaluable to centralize the way you call the provider APIs. If you spread out calls to an API throughout your code base, it will be more difficult to integrate other channels or API providers in the future. Let’s say you’re starting with email and AWS SES as the provider. In two years’ time, you might decide to integrate mobile push notifications as well. What might that look like? The incurred technical debt will include scouring the code base for all instances of calls to the AWS SES API before you can integrate mobile push as an additional channel. But with centralized calls, you’ll have more consistent, cleaner, and reusable code as you grow.

How many notification channels should you have?

Typically, having three or four channels that are relevant to your product is an ideal scenario for a mature product. When you intersect channels with the preferences and availability of users, you create higher levels of complexity for your algorithm. Offering many channels for notifications might become too complex to maintain. But offering too few channels might harm your chances of interacting with users since some channels might not be viable for all users. For instance, you might decide to offer email and push notifications. But if a user didn’t download your product, your interaction with them is limited only to email.

Best technologies for routing and preferences engines

It ultimately pays to choose technologies that will be a good fit for your routing and preferences needs. There will be a great deal of asynchronous programming, as the routing service will often be waiting to receive responses for each function. You’ll want to pick a language or a framework that allows you to respond to async events at scale.

The routing service also involves considerable state tracking, as most of the routing will depend on waiting on a response for each notification before changing state. The routing service will also need to be re-activated every time it receives a response from a provider and will need to determine if the notification was sent successfully or if it has to pursue next steps. See the example below of how a notification function’s state might be tracked.

At Courier, we use AWS Lambda. Since our usage tends to come in bursts, serverless technology allows us to adjust and scale for changes in demand throughout each day as well as handle asynchronous operations efficiently.

Don’t forget: compliance in notification routing

When creating your own routing and preferences service, you will need to ensure that whichever channels you implement are fully compliant with applicable laws. For example, there are legal mandates on how users may be contacted or how they can unsubscribe from contact.

For commercial email messages, the CAN-SPAM Act of 2003 is a federal United States law that spells out distinct rules and gives recipients a way to stop all contact. Penalties can cost as much as $16,000 per email in violation. This law also outlines requirements such as not using misleading header information or subject lines, identifying ads, and telling recipients how they can opt out of all future email from you. The opt-out process itself is strictly regulated.

For SMS, the United States Telephone Consumer Protection Act (TCPA) of 1991 sets forth rules against telemarketing and SMS marketing. Under this law, businesses cannot send messages to a recipient without their consent. This consent needs to be explicit and documented. The consent is also twofold: recipients need to consent to receiving SMS marketing messages and they need to consent to receiving them on their mobile device. Recipients need to be provided a description of what they are subscribing to, how many messages they should expect, a link to the terms and conditions of the privacy policy, and instructions on how to opt-out.

In California especially, the California Consumer Privacy Act (CCPA) of 2018 provides additional rights for California residents only. These rights include the right to know which information a company has collected about them and how it’s used as well as the right to delete it or to opt-out of the sale of this information. Information that qualifies under the consumers’ right-to-know includes names, email addresses, products purchased, browsing history, geolocation information, fingerprints, and anything else that can be used to infer preferences. Should a consumer request this information, the company has to share the preceding 12 months of records, and also include sources of this information and with whom it was shared and why. In 2020, California Privacy Rights Act (CPRA) of 2020 amended the CCPA. The CRPA provides further consumer rights to limit the use and disclosure of their personal information.

Other countries have their own compliance laws for businesses reaching out to leads and customers. Canada has its Anti-Spam Legislation (CASL). The European Union has the General Data Protection Regulation (GDPR) which now also covers granularity of consent. The United Kingdom has its own regulations along with the GDPR, the Privacy and Electronic Communications Regulations (PECR) and Data Protection Act.

Compliance itself needs to be integrated at the developer level. Providers, like SendGrid, don’t know what you’re sending. It’s up to the developer to ensure that all applicable compliance laws are followed for their choice of channels.

Conclusion

Building a notification system into a product is not for everyone. The process is time-consuming, complex, and expensive. The level of notification customizability and routing options you decide to implement will ultimately dictate a preference for either maximizing user engagement or optimizing cost. A startup with a product that hasn’t yet found its product-market fit has to focus on finding early customers and getting their feedback. But established companies with a proven customer base will have concerns related to more complex routing logic, future-proofing and compliance. This would require more functionality and higher maintenance costs.

This piece taught us about the necessity of sending data for notifications to the right people, at the right frequency, at the right time and how this can be done through routing and customized preferences. Tune in for the next post in this series to learn about observability and analytics to monitor the functioning and performance of your in-house notifications system. To stay in the loop about upcoming content, subscribe below or follow us @trycourier!