Raymond See Headshot

Raymond See

You can’t really invest in a data team too early. That’s something we learned as a small but growing team. We invested in a data team at an early stage, so we could establish product usage trends, create business insights, and identify areas to improve our product. 

If you’re an early-stage startup, start building your data strategy team and architecture as soon as you can. You don’t have to go big—a one-person data team can make a massive difference to your success as you grow.

## Hire a Generalist First

When it comes to launching your data team, you might be tempted to hire an expert from the largest brand name you possibly can. Our advice: find someone with early-stage startup experience and has worked with data at scale too.

There are wonderful people with brilliant minds working on large enterprise data teams, but these large teams often have their employees specialize in specific parts of their data platform. That makes sense when you have 10 data engineers on a team or when you have a large team of machine learning data scientists working on pricing algorithms. But when you’ve only got one data role for the entire organization, you need someone who can do a little bit of everything.  

It’s important to first hire an all-rounder who can create data insights, develop business hypotheses, and create a scalable data architecture from scratch. You need someone ready to start building the data team from the ground up, preferably someone who can hire out the rest of your team once you’re ready to do so.

This first hire should be someone who has experience building (or at least working on) a data team at an early-stage startup. They need to be able to generate value quickly and have the business acumen to ask the right questions. They should be able to approach data in an 80/20 manner to deliver immediate results instead of diving into the hardest problems that you have.

For the data engineering side, it’s common to hire a consultant or contractor to help set up your architecture. We decided to complement our first hire with a consultant. This way, we have the best of both worlds – an expert in data engineering part time, and a generalist who can maintain data architecture on a day to day basis. We set up strong foundations and also kept our knowledge in house.

## A Small Data Team Can Answer Big Questions 

Even if you can’t hire right away for all the roles you want on your data team long term, you can do a lot with only one or two of the right people. The key is thinking in terms of scalability and efficiency. What helps your data team do more with less? And how will this work when your team is bigger? 

Scalable analyses are extremely important with a small data team. By thinking ahead about how your processes and workflows will function as your team grows, you’ll save time and create valuable solutions for some of your employees’ most time-consuming issues. 

For example, we immediately realized that our sales team had no visibility into prospective customer product usage, and it was going to come up often. So we built our sales reps a workspace dash so they can pull the data themselves. Our data team’s job is to enable data-driven decision-making. When we saw an opportunity for a one-time data investment that could lead to hundreds of saved hours on the sales side, we acted on it.  

A small data team should also be focused on preventing issues in the future. If you only have the headcount for a one-person data team at the moment, keep them focused on your biggest problems but also budget time to maintain and scale your data architecture to reduce tech debt.  

## The Tools Our Data Team Uses 

From the beginning, our goal was to be forward-thinking with every decision. 

Like all startups, we wanted to generate immediate ROI. We use a centralized data warehouse that is our single source of truth – nothing hurts data teams more than having fragmented or untrustworthy data. We’re able to consolidate our data thanks to automations and integrations. This helps us provide valuable insights to the entire company with low lift.

![Courier Data Pipeline](//images.ctfassets.net/z7iqk1q8njt4/4cl15zd8abN9xXQGiSrGAX/c38faa16b0ef71ee923c344fc266a4c3/image.png)

We pipe in all of our data using ELT tools like [Segment](https://segment.com/) (which is also our data collection tool), [Fivetran](https://www.fivetran.com/), and Snowflake external stages.

[Snowflake](https://www.snowflake.com/) is the brains of the operation, all of our data gets piped there and we use that as our Single Source of Truth. Establishing this single source of truth is critical for ensuring that all of our metrics are the same across all systems. This helps build trust across our stakeholders and makes it easy for us to join data from multiple sources. For example, while feature usage is helpful for us to look at in isolation, combining this with payment data will create insights on what customers are willing to pay for.

We use [dbt](https://docs.getdbt.com/) to clean all of our raw data into a more usable format. For example, our dim_user table has ~20 tables powering it under the hood, that have a lot of critical info on our customers, such as how many notifications they send and what plan type they are on. By making this so readily available, our non-technical users can quickly draw business insights. We can quickly look at non-paying or self-serve customers that are sending a lot of notifications to identify business contract prospects. Our customer success team also built reports to look at the health of our business contract customers. We also use dbt to clean our data of sensitive information.

We use [Census](https://getcensus.com/) to send our warehouse data back to places such as HubSpot and Courier. This allows our sales and marketing teams to access this information on the platform that they are using on a daily basis.

[Metabase](https://www.metabase.com/) is our BI tool. It allows us to create dashboards, write SQL queries, and provide a self-serve format for non-technical users. We’ve found that creating these kinds of self-serve data dashboards allows users to answer a lot of their own questions. They are quick to spin up without large investments upfront like teams that use Looker or Tableau. 

For example, we have a KPI dash that shows the top metrics across the company. We can also track how individual workspaces are performing and identify workspaces that we can target for our sales team. We have also brought data-driven decision-making to our product development by deep-diving into our customers’ product usage. One example is that we found that a lot of our users were creating their own Courier instance instead of joining their colleague’s existing Courier instance. Using this insight, our product team went back to the drawing board to create a page that surfaces Courier workspaces that are created by the same business email domain so you can easily request access. 

![data-courier-workspace](//images.ctfassets.net/z7iqk1q8njt4/3YqJYHW2TZvNZKECRcxRLp/aa3270ad15f19508ccc4163f0eecbcba/data-courier-workspace.png)

We also use [Propel](https://propeldata.com/) to power our in-app analytics to provide visibility for our end users. One way that we use Propel is to deliver template analytics for our Business-tier customers. Propel allows our engineers to easily put GraphQL visualizations on top of our metrics from our warehouse.

![data-courier-analytics](//images.ctfassets.net/z7iqk1q8njt4/21zTz6npIrfOc1bmnswwuq/428adf42da4af54d0389b0899c52f780/data-courier-analytics.png)

[Eppo](https://www.geteppo.com/) is our experimentation analytics platform. The way that they define experiments and relevant metrics makes it really easy to scale experimentation without much overhead – it can take as few as five minutes to setup an additional experiment. Having an integration with our warehouse is key – no longer is experimentation a blackbox, and we get to have our internal data layer for free for us to do deep dives. No longer do you need to go to your data team to ask for specific cuts such as “How is this feature affecting paid vs free users?” or “Do we see differences in the US vs Rest of World?” as you get this out of the box with a few clicks. This unlocks more time for our data team and creates a better experience for our Product, Engineering and Design stakeholders.

## Our Data Team Is Evolving Again

Now, we’re entering a period of growth, so naturally, we’re evolving our data approach again. Our current goal is to enable our data scientists to build subject matter expertise while still maintaining centralized data standards. 

Our current structure is centralized, with a senior analytics engineer reporting to the head of data. It works now since we’re still in the earlier stages of building out a data team. But when the team expands, we will be transitioning to a hybrid model.

Data scientists will cover certain business areas (like the product team), but they’ll still report to the Head of Data. This will allow the data scientists to build subject matter expertise while still maintaining centralized data standards as we continue to grow. Because the data team will always be changing and growing along with the rest of the organization, you need standards that will scale with the business.

If you’d like to learn more about how we approach data standardization, check out our blog post on [how Courier became HIPAA compliant](https://www.courier.com/blog/how-courier-became-hipaa-compliant/).

Courier is a notification service that centralizes all of your templates and messaging channels in one place which increases visibility and reduces engineering time. 

Start Routing Notifications Today!

Book with data flying into/out of it through computers, phones, and other devices.

data-header

How tools like Segment, Metabase, Snowflake, Census, and others, can help establish a data team from the very early stages in a startup.

data-thumbnail

Tools and Techniques to Establish Your Data Team Early

Prakhar Srivastav

Motivational quotes were quite the rage back in the day when MMS & email forwarding were popular. I remember my parents forwarding me at the start of every morning. Fast forward to today, if you are lucky, you are part of some forward group on your messaging app of choice (Whatsapp, Telegram, etc.).

Inspired by the same idea, today we are going to build a service that sends our friends and family an AI generated motivational quote-of-the-day. Rather than hardcoding a list of motivational quotes, we are going to use a machine learning model to generate a quote on demand, so that we never run out of quotes to share!

![QOTD Final Project](//images.ctfassets.net/z7iqk1q8njt4/1R04JX4FjNAccZFLrmr9SF/467f50f89c928e7665e81ee25fb69c19/qotd_pic_1.png)

## Instructions

### Part 1: Using AI to generate motivational quotes

#### OpenGPT2 and Language Models
OpenAI GPT-2 model was proposed in Language Models are Unsupervised Multitask Learners by Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever. It’s a causal transformer pre-trained using language modeling on a very large corpus of ~40 GB of text data.

To simplify this, at a high level OpenAI GPT2 is a large language model that has been trained on massive amounts of data. This model can be used to predict the next token in a given sequence.

If that sounds too complicated, don't worry, you don't need to know any Machine Learning or AI to follow along with this project. Libraries such as [Hugging Face](https://huggingface.co) make using this model in our app very easy.

#### Hugging Face

We'll use the [Hugging Face](https://huggingface.co/) library to load and serve the ML model that will generate the quotes for us. Hugging Face makes it very easy to use transformer models (of which GPT2 is a type) in our projects without any knowledge of ML or AI. As mentioned earlier, GPT2 is a general purpose language model which means that it is good at predicting generic text given an input sequence. In our case, we need a model more suited for generating quotes. To do that, we have two options:

1. We can fine-tune the GPT2 model by using our own text for which we'll need a good dataset of quotes.
2. Or we can find an *already* existing model which has been fine-tuned with some quotes.

Luckily, in our case there’s a fine-tuned model that has been trained on the 500k quotes dataset - https://huggingface.co/nandinib1999/quote-generator

With Hugging Face, using this model is as easy as as creating a tokenizer

```python
from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline

tokenizer = AutoTokenizer.from_pretrained("nandinib1999/quote-generator")
```

then, constructing a model from the pretrained model

```python
model = AutoModelWithLMHead.from_pretrained("nandinib1999/quote-generator")
```

and finally, constructing the generator which we can use to generate the quote

```python
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

# use a starting prompt
generator("Keep an open mind and")
[{'generated_text': 'Keep an open mind and a deep love for others'}]
```

#### Building an API to serve the model

Now that we have a way to generate quotes for us, we have to think about how we can use this in our app. There are multiple ways to go about building this.

1. Load the model everytime we want to run the script to send the script.
2. Create an API or service that serves this GPT2 model to generate quotes for us on demand. 

A key plus point of the second option is that once the model is loaded the API can respond to us quickly and can be used in other applications as well. FWIW, the first option is a totally valid approach as well.

We can use [Fast API](https://fastapi.tiangolo.com/) to build a quick serving API. Here's what that looks like

```python

# in file api.py

from pydantic import BaseModel
from fastapi import FastAPI, HTTPException
from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline

## create the pipeline
tokenizer = AutoTokenizer.from_pretrained("nandinib1999/quote-generator")
model = AutoModelWithLMHead.from_pretrained("nandinib1999/quote-generator")
generator = pipeline("text-generation", model=model, tokenizer=tokenizer)

app = FastAPI()

class QuoteRequest(BaseModel):
    text: str

class QuoteResponse(BaseModel):
    text: str

### Serves the Model API to generate quote
@app.post("/generate", response_model=QuoteResponse)
async def generate(request: QuoteRequest):
    resp = generator(request.text)
    if not resp[0] and not resp[0]["generated_text"]:
        raise HTTPException(status_code=500, detail='Error in generation')
    return QuoteResponse(text=resp[0]["generated_text"])
```

Let's test it out

```sh
$ uvicorn api:app

INFO:     Started server process [40767]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
```

Now we can start sending requests to the `/generate` endpoint that will generate a quote for us.

![QOTD Part 2](//images.ctfassets.net/z7iqk1q8njt4/1TXDqA7VRfbwZRNDH76Kms/f6cce20ca8d4f5aaa9ecd313976488a3/qotd2.png)

### Part 2: Building the Quote Generator

Now that we have a way to generate quotes on demand, we can stop here and start working on sending this via [Courier](http://courier.com/). But who are we kidding, no one reads text anymore! We can make this interesting by using a nice image and placing our quote on it to make it look like a poster.

#### Generate quote

Given our API, we can now do the following to generate a quote

```py
from random import choice
# feel free to add more starting prompts for more variety
canned_seeds = ["Always remember to", "Start today with", "It is okay to"]
seed = choice(canned_seeds)
resp = requests.post('http://127.0.0.1:8000/generate', data=json.dumps({"text": seed}))
return resp.json()["text"]
```

#### Downloading the background image

The first challenge is getting a beautiful background image for our quote. For that, we'll use the Unsplash API that provides a nice endpoint to return a random image matching a query. Opening https://source.unsplash.com/random/800×800/?nature in our browser returns a nice nature image. 

To keep things interesting, we can use different query terms such as stars, etc. Here's the how the code for downloading our background image looks like - 

```py
from random import choice
image_backgdrops = ['nature', 'stars', 'mountains', 'landscape']
backdrop = choice(image_backdrops)
response = requests.get("https://source.unsplash.com/random/800×800/?"+ backdrop, stream=True)
# write the output the img.png on our filesystem
with open('img.png', 'wb') as out_file:
    shutil.copyfileobj(response.raw, out_file)
del response
```

#### Creating the image with the quote

Ok, now we have our background image and a quote which means we can work on assembling the final image that will be sent to the recipients. At a high level we want to place some text on an image but even this simple task can be challenging. For starters, there are a number of questions for us to answer

1. How will the text be placed on the image?
2. What about wrapping the text?
3. What color should the text be so that it is visible on the background image?
4. How do we do this for images with varying widths and heights?

The answers to some of these questions are more complicated than others. To keep it simple, we'll put the text in the center, and do some wrapping so that it looks good. Finally, we'll use a light color text for now. For all image manipulation, we'll use Python Image Library (PIL) to make this easy for us.

```py
# use the image we downloaded in the above step
img = Image.open("img.png")
width, height = img.size
image_editable = ImageDraw.Draw(img)

# wrap text
lines = textwrap.wrap(text, width=40)

# get the line count and generate a starting offset on y-axis
line_count = len(lines)
y_offset = height/2 - (line_count/2 * title_font.getbbox(lines[0])[3])

# for each line of text, we generate a (x,y) to calculate the positioning
for line in lines:
    (_, _, line_w, line_h) = title_font.getbbox(line)
    x = (width - line_w)/2
    image_editable.text((x,y_offset), line, (237, 230, 211), font=title_font)
    y_offset += line_h
img.save("result.jpg")
print("generated " + filename)
return filename
```

This generates the final image called `result.jpg`

#### Uploading the image
For the penultimate step, we need to upload the image so that we can use that with Courier. In this case, I'm using Firebase Storage but you can feel free to use whatever you like.

```py
import firebase_admin
from firebase_admin import credentials
from firebase_admin import storage

cred = credentials.Certificate('serviceaccount.json')
firebase_admin.initialize_app(cred, {...})

bucket = storage.bucket()
blob = bucket.blob(filename)
blob.upload_from_filename(filename)
blob.make_public()
return blob.public_url
```

## Step 3: Integrating with Courier

Finally, we have everything we need to start sending our awesome quotes to our friends and family. We can use Courier to create a good looking email template.

[Start by creating an account.](https://app.courier.com/signup)

#### Creating the template in Courier

![QOTD Courier Email Template Preview](//images.ctfassets.net/z7iqk1q8njt4/NvMlJ4UVAaWZHzIOooG0P/879d8e96f866f09d7a5d0b582a79562a/qotd3.png)

#### Sending the message

Sending a message with Courier is as easy as it gets. While Courier has its own SDKs that can make integration easy, I prefer using their API endpoint to keep things simple. With my `AUTH_TOKEN` and `TEMPLATE_ID` in hand, we can use the following piece of code to send our image

```py
import requests

headers = {
    "Accept": "application/json",
    "Content-Type": "application/json",
    "Authorization": "Bearer {}".format(os.environ['COURIER_AUTH_TOKEN'])
}
message={
    "to": { "email": os.environ["COURIER_RECIPIENT"] },
    "data": {
        "date": datetime.today().strftime("%B %d, %Y"),
        "img": image_url ## this is image url we generated earlier
    },
    "routing": {
        "method": "single",
        "channels": [
            "email"
        ]
    },
    "template": os.environ["COURIER_TEMPLATE"]
}
requests.post("https://api.courier.com/send", json={"message": message}, headers=headers)
```

The API key can be found in [Settings](https://app.courier.com/settings) and the Template ID can be found in the [template design's](https://app.courier.com/designer) settings. And that's it!

## Conclusions

This tutorial demonstrated how easy it is to get started with machine learning & Courier.

If you want to go ahead and improve this project, here are some interesting ideas to try

- Better background image: Use a term from the generated quote to search for an image?
- Better background color for the text: Use better colors for the text. One cool idea is to use the complimentary color from the image's main color. You can use k-means clustering to find that out.
- Adding more channels : Extends this to messages on messaging clients and sms!

## About the Author

[Prakhar](https://prakhar.me) is a senior software engineer at Google where he works on building developer tools. He's a passionate open-source developer and loves playing the guitar in his free time.

## Quick Links

🔗 [Courier Docs](https://www.courier.com/docs/)

🔗 [Hugging Face](https://huggingface.co/)

🔗 [Fast API](https://fastapi.tiangolo.com/)

🔗 [Unsplash API](https://unsplash.com/developers)

QOTD Header

Courier and OpenGPT2 in action: build a service that sends friends and family an AI generated motivational quote of the day.

QOTD Thumbnail

Develop a Motivational QOTD with Courier and GPT2

Aydrian Avatar

Aydrian Howard

GET /messages Example

Fetching a Single Message

Retrieving Message History

During our August 19th Courier Live, [Tony Nguyen](https://twitter.com/tone81) from our engineering team joined me to talk about my favorite Courier feature, the logs. We presented a high level overview of the logs and dove into the details of each timeline event in the [Courier Application](https://www.trycourier.app). We then covered how you could retrieve the same information using the [Messages API](https://docs.courier.com/reference/messages-api).

Check out the video below to watch us:

* Explain the basics of the Courier Logs
* Walk through the lifecycle of a notification
* Explore the notification statuses
* Use the [Messages API](https://docs.courier.com/reference/messages-api) to retrieve message events and history

[Courier Live: Getting the Most out of Courier Logs](https://www.youtube.com/embed/YhzlsiXzovg)
Be sure to Like the video and Subscribe to our [YouTube channel](https://youtu.be/0hnc8PUcWTg).

## Retrieving Log Details Programmatically

While Courier provides a great graphical experience for viewing their logs, sometimes you need to be able to grab that information as part of a backend process. This can be done using the [Messages API](https://docs.courier.com/reference/messages-api).

### Listing Messages

You can retrieve a pageable list of log messages using [GET /messages](https://docs.courier.com/reference/messages-api#getmessages).

```shell
curl --request GET \
  --url https://api.courier.com/messages \
  --header 'authorization: Bearer <AUTH_TOKEN>'```

### Fetching a single Message

Once you have a messageId, either from the call above or the response of a [send call](https://docs.courier.com/reference/send-api), you can use it to get more details using [GET /messages/{message_id}](https://docs.courier.com/reference/messages-api#getmessagebyid).

```shell
curl --request GET \
  --url https://api.courier.com/messages/1-5f3d7aa7-e4b8f9ba9a4672ba5d46754c \
  --header 'authorization: Bearer <AUTH_TOKEN>'```

### Getting Message History

If you would like to retrieve the details provided in the log details timeline, you can use [GET /messages/{message_id}/history](https://docs.courier.com/reference/messages-api#getmessagehistorybyid).

```shell
curl --request GET \
  --url https://api.courier.com/messages/1-5f3d7aa7-e4b8f9ba9a4672ba5d46754c/history \
  --header 'authorization: Bearer <AUTH_TOKEN>'```

You can learn more about the Courier API from the [Courier API Reference Documentation](https://docs.courier.com/reference).

Is there something you’d like to see us do using Courier? Let us know and it might be the subject of our next Courier Live. We stream a new Courier Live every Wednesday at noon Pacific. Follow us on [Twitch](https://twitch.tv/trycourier) to be notified when we go live.

-Aydrian

Getting the Most out of Courier Logs

Retrieving Log Details Programmatically

Listing Messages

Fetching a single Message

Getting Message History

More from Courier

Build your first notification in minutes

Build your first notification in minutes

Platform

Use Cases

Explore

Company