Rails 6 - Action Mailbox tryout

There are cases when application receives large number of inbound emails. And if we have to process those emails, then one has to go through each email in order to perform operations.

Let’s consider a use case of HR domain. An organization has sent an email asking candidates to submit resumes for an opening. All candidates are replying to this email with their resumes as attachments. Now, we need to upload these resumes to cloud and create a database entry for each resume.

Steps to achieve this:

We’ll go through each mail.
We’ll download the resume.
Now, we’ll upload the resume to cloud and create entry in the resume table.

It looks like a tedious task.

Introducing Action Mailbox

Rails 6 has introduced Action Mailbox for processing inbound emails. It routes incoming emails to controller-like mailboxes for processing in Rails. It supports all major platforms like Mailgun, Mandrill, Postmark and SendGrid.

Installation and Implementation

Let’s create a new application feedback_collector by running the following command.

> rails new feedback_collector

Install action_mailbox by running rails action_mailbox:install. This command will also install Active Storage, which stores the emails which come in. It saves these emails and keep track of it whether it has been processed or not.

Then it loads an Active Job to process it and delete that email once it is done. Even after deletion, it keeps track of the id and checksum to avoid the processing of the same email if it comes again.

> cd feedback_collector
> rails action_mailbox:install
#=> Copying application_mailbox.rb to app/mailboxes
#=>      create  app/mailboxes/application_mailbox.rb
#=> Copied migration 20191021075823_create_active_storage_tables.active_storage.rb from active_storage
#=> Copied migration 20191021075824_create_action_mailbox_tables.action_mailbox.rb from action_mailbox

Above command generates two migrations for action_mailbox and active_storgage. And it also creates application_mailbox.

Now let’s generate scaffolds for User, Product and Feedback.

> rails g scaffold User name email
> rails g scaffold Product title
> rails g scaffold Feedback user:references product:references content:text

Run the migrations:

> rails db:migrate

action_mailbox table’s schema contains columns like status, message_id and message_checksum. status can be pending, processing, finished or bounced. message_id and message_checksum are there to avoid duplication.

ApplicationMailbox class looks like this.

class ApplicationMailbox < ActionMailbox::Base
  # routing /something/i => :somewhere
end

Here we can define routes for emails. For ex:

class ApplicationMailbox < ActionMailbox::Base
  # routing /something/i => :somewhere
  routing  :all => :feedbacks
end

We have specified one route for redirecting all the emails to FeedbacksMailbox. We can also specify routes in regex format. For ex:

class ApplicationMailbox < ActionMailbox::Base
  # routing /something/i => :somewhere
  routing  /feedback\-.+@example.com/i => :feedbacks
end

This expression can match emails like feedback-Ahdhc12@example.com, feedback-5264yYxjg@example.com.

Now let’s create FeedbacksMailbox.

> rails g mailbox Feedbacks
#=> Running via Spring preloader in process 623
#=>       create  app/mailboxes/feedbacks_mailbox.rb
#=>       invoke  test_unit
#=>       create    test/mailboxes/feedbacks_mailbox_test.rb

FeedbacksMailbox class contains a method named process, to process the emails. In this class, we can access mail object.

If we have to perform some operations before processing the email, we can do that with before_processing callback. For ex:

class FeedbacksMailbox < ApplicationMailbox
  before_processing :user
  
  def process
  end

  def user
    @user ||= User.find_by(email: mail.from)
  end
end

Now, we have got the user using mail.from. But to save the feedback for a product, we’ll need the product_id.

To get the product_id, we can specify the reply email’s regex in such a way that it contains the product_id. For ex:

  RECIPIENT_FORMAT = /feedback\-(.+)@example.com/i

If reply email is feedback-1234@example.com, then using above regex we can get 1234 as product_id.

Now, Let’s try to process the email and save the user’s feedback.

class FeedbacksMailbox < ApplicationMailbox
  RECIPIENT_FORMAT = /feedback\-(.+)@example.com/i

  before_processing :user
  
  def process
    # Creating the feedback
    # mail.decoded returns the email body if mail is not multipart 
    # else we'll use mail.parts[0].body.decoded
    # and in our case that is feedback
    if mail.parts.present?
      Feedback.create user_id: @user.id, product_id: product_id, content: mail.parts[0].body.decoded
    else
      Feedback.create user_id: @user.id, product_id: product_id, content: mail.decoded
    end
  end

  def user
    @user ||= User.find_by(email: mail.from)
  end

  def product_id
    # There can be multiple recipients, 
    # so finding the one which matches the RECEIPIENT_FORMAT
    
    recipient = mail.recipients.find { |r| RECIPIENT_FORMAT.match?(r) }
    
    # Returns the first_match and that is product_id
    # For Ex: recipient = "feedback-1234@example.com"
    # Then it'll return 1234
    recipient[RECIPIENT_FORMAT, 1]
  end
end

As we have access to mail object, we can also read multipart email or attachments if there are any.

Testing on Development Environment

To test this on development enviroment, we can simply go to http://localhost:3000/rails/conductor/action_mailbox/inbound_emails/new and deliver an inbound email. Based on the to email, it’ll route to mailbox and process the email.

Configuration for Production

To configure Action Mailbox for the prodcution environment, we need to specify the ingress in config/environment/production.rb.

Let’s consider ingress as postmark.

config.action_mailbox.ingress = :postmark

We also need to generate a strong password that Action Mailbox can use to authenticate requests to the postmark ingress. We have to store password in the encrypted credentials as ingress_password.

action_mailbox:
  ingress_password: PASSWORD

Instead of storing in credentials, we can also provide this password in the RAILS_INBOUND_EMAIL_PASSWORD environment variable.

Now, we need to configure inbound webhook to forward inbound emails to /rails/action_mailbox/postmark/inbound_emails with the username actionmailbox and the password we previously generated. Following will be our webhook URL:

https://actionmailbox:PASSWORD@example.com/rails/action_mailbox/postmark/inbound_emails

Live Example

For example, we’ll use following services.

Sendgrid (Mailing Service)
Freenom (Domain Registration Service)
ngrok (Provides Public URL for exposing local web server)

All these services are free.

Setup

Let’s create accounts on SendGrid, Freenom and ngrok.
For ngrok installation, please refer to steps mentioned in this guide.

Now, let’s register a free domain by using this link.
Enter any domain like actionmailbox in the search box and hit Check Availability button. It’ll give all the free options and select any option. Now, you can view your domain in the My Domains section under Services nav item.

After this, we need to authenticate our domain on SendGrid. For authentication, follow these steps:

Click here
Select DNS host as Other Host (Not Listed) and type DNS Host as freenom
Click Next
Enter domain name in the From Domain textbox
Click Next
It’ll show three CNAME records that we need to add in the domain management section of Freenom
Go to My Domains section on Freenom in a new tab
Click Manage Freenom DNS tab
Now, we need to add those three CNAME records here. Copy Host into Name, Value into Target and select Type as CNAME.
Also create a MX record. We can keep name as blank, type as MX, target as mx.sendgrid.net and priority as 10.
Now go back to SendGrid tab, check the checkbox and click Verify.
If it fails then wait for 15-20 minutes and again click Verify.
Visit this link and it’ll show status as Verified for our domain.

Next step is to add following SMTP configuration in development.rb

config.action_mailer.smtp_settings = {
  :user_name => SENDGRID_USERANME,
  :password => SENDGRID_PASSWORD,
  :domain => OUR DOMAIN,
  :address => 'smtp.sendgrid.net',
  :port => 587,
  :authentication => :plain,
  :enable_starttls_auto => true
}
config.action_mailer.delivery_method = :smtp
config.action_mailer.perform_deliveries = true
config.action_mailer.raise_delivery_errors = false

Start the server by running the command rails s.

After that, run ./ngrok http 3000 in a new tab.

> ./ngrok http 3000
ngrok by @inconshreveable                                                                                               

Session Status                online
Account                       ROMIL MEHTA (Plan: Free)
Version                       2.3.35
Region                        United States (us)
Web Interface                 http://127.0.0.1:4040
Forwarding                    http://386e42cd.ngrok.io -> http://localhost:3000
Forwarding                    https://386e42cd.ngrok.io -> http://localhost:3000

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

From the Forwarding value above, our server public url is 386e42cd.ngrok.io.
Make a note of your URL for the below example.

Now, we’ll follow the steps mentioned in the Conguration for Production section.

Add ingress as sendgrid

config.action_mailbox.ingress = :sendgrid

Create an ingress password and add in the credentials.

action_mailbox:
  ingress_password: PASSWORD

Go to Inbound Parse section of sendgrid and click Add Host & URL.
Select the domain, set destination url as https://actionmailbox:INGRESS_PASSWORD@SERVER_PUBLIC_URL/rails/action_mailbox/postmark/inbound_emails and check the option for POST the raw, full MIME message.

Example

Let’s create a mailer for asking the feedback of the product by running the following command:

> rails g mailer Feedback
Running via Spring preloader in process 48769
      create  app/mailers/feedback_mailer.rb
      invoke  erb
      create    app/views/feedback_mailer
      invoke  test_unit
      create    test/mailers/feedback_mailer_test.rb
      create    test/mailers/previews/feedback_mailer_preview.rb

Add the following code related to sending mail in app/mailers/feedback_mailer.rb

class FeedbackMailer < ApplicationMailer
  default from: FROM_MAIL_ADDRESS

  def send_email
    mail(to: ANY_USERS_EMAIL, reply_to: REPLY_TO_MAIL_ADDRESS, subject: 'Mailbox Test', body: 'Provide feedback for the product by replying to this mail')
  end
end

In the above example, format for REPLY_TO_MAIL_ADDRESS should be feedback-#{PRODUCT_ID}@#{SERVER_PUBLIC_URL} and PRODUCT_ID should be the product for which we are asking the feedback.

We can trigger the feedback email by running the following command.

> FeedbackMailer.send_email.deliver_now

By running the above command, user will receive an email.

We have already setup the routes for feedback mailbox. So if user replies to this email, then it’ll call the process method of FeedbackMailbox and will process the email.

Summary

We looked at the basics of Action Mailbox, how to install, implement and configure it. And also provided the setup for example similar to Production.