Web, Applications agiles et Ruby on Rails

A Ruby Gateway for Postfix (in order to build a Rails Webmail)

Edit : This code is globally out of date and pretty hazardous... I now use the awesome Astrotrain plugin to retrieve E-Mails into Ruby on Rails' apps, but feel free to read my blog post anyway :-)

Well, it took me almost two weeks to get the whole thing to work, and regarding to the poorly documentation I have had to deal with, I think it could be a good idea to share this experience.

The goal is to retrieve incoming and outgoing mail from Postfix, then push them into a Database. First of all, let’s configure Postfix. I assume you already have a working installation of Postfix. Postfix uses the “vmail” user (yes, I use the MySQL virtual Mail package).

Add this to your /etc/postfix/master.cf :

mail_filter unix -     n       n       -       -       pipe
 flags=hq user=vmail argv=/usr/bin/ruby1.8 /var/spool/filter/mail_filter.rb

smtp      inet  n       -       -       -       -       smtpd
 -o content_filter=mail_filter:

Now, every mail (incoming AND outgoing!) will be “piped” to the /var/spool/filter/mail_filter.rb Ruby script. By the way, I could -of course- have used any language like Shell or Perl, but I read some interesting docs about Tmail, and I now really love Ruby, so… why not ?

First, you’ll need a Mail parser to read the mail content : install TMail

sudo gem install tmail

Ok, let’s create this Ruby filter, we’ll store it where we just told to Postfix to look at

/var/spool/vmail/mail_filter.rb

And now, let’s read mails !

#Read the message
message = $stdin.read
#Parse it
mail = TMail::Mail.parse(message)

…easy, hu ? Now, let’s see what’s inside this mail

#Extract what we need...
from_list = "" #the "from" field is also an array 
mail.from.each do |addr|
  from_list.length > 1 ? from_list << addr + "," : from_list << addr
end        

to_list = ""
mail.to.each do |addr|
  to_list.length > 1 ? to_list << addr + "," : to_list << addr
end

if mail.cc != nil 
  cc_list = ""
  mail.cc.each do |addr|
    cc_list.length > 1 ? cc_list << addr + "," : cc_list << addr
  end
end

if mail.bcc != nil
  bcc_list = ""
  mail.to.each do |addr|
    bcc_list.length > 1 ? bcc_list << addr + "," : bcc_list << addr
  end
end

Every mail now piped to the Mail Filter, WE DO HAVE to handle file writing if we want to retrieve mail using pop3 ! Skip this if you don’t want to duplicate mails to the filesystem and in the DB

#Get the domain name
@domain_name = to_list.split('@')[1]

#Write the mail to the file system so we still can access it via pop3
myFile = File.open("/var/spool/vmail/#{@domain_name}/#{to_list.split('@')[0]}/new/#{Time.now.to_i}", "w")
myFile.write message
myFile.close

DB Connection. I use the exact domain name for the database name as convention, this now allow me to link the mail to the proper database.

# connect to the MySQL server, Warning : The domain name MUST BE the same as DB name
begin
  dbh = Mysql.real_connect("localhost", "DB_user", "DB_password", @domain_name.split('.')[0])     
rescue
  Process.exit
end

And now we insert the mail body, assuming this is a simple text mail

if mail.parts.length == 0
  #no attachment, only a text body
  req = dbh.query("INSERT INTO emails (filename,from_field,to_field,cc_field,cci_field,subject,content,created_at,updated_at,status,direction,email_account_id) 
              VALUES (
              'nom de fichier',
              '#{from_list}',
              '#{to_list}',
              '#{cc_list}',
              '#{bcc_list}',
              '#{mail.subject}',
              '#{mail.body}',
              now(),
              now(),
              'new',
              'incoming',
              '1')
              ")

But, what about attachments ?! Hehe… this part is quite a bit F***** tricky and causes me some headaches because of the lack of documention

  
else
  #email.attachments are TMail::Attachment
  #but they ignore a text/mail parts.
  @i = 0
  @email_id = 0
  mail.parts.each_with_index do |part, index|
    if @i == 0
      #Push mail text body into DB
      req = dbh.query("INSERT INTO emails (filename,from_field,to_field,cc_field,cci_field,subject,content,created_at,updated_at,status,direction,email_account_id) 
                  VALUES (
                  'nom de fichier',
                  '#{from_list}',
                  '#{to_list}',
                  '#{cc_list}',
                  '#{bcc_list}',
                  '#{mail.subject}',
                  '#{part.body}',
                  now(),
                  now(),
                  'new',
                  'incoming',
                  '1')
                  ")
    else
      filename = (part['content-location'] && part['content-location'].body) || part.sub_header("content-type", "name") || part.sub_header("content-disposition", "filename")

      #get email_id and create the sub dir for attachments
      if @email_id == 0 
         @email_id = dbh.insert_id()
         system "mkdir /home/guillaume/#{@domain_name.split('.')[0]}/tmp/attachments/#{@email_id}"
         system "chmod -R 0770 /home/guillaume/#{@domain_name.split('.')[0]}/tmp/attachments/#{@email_id}"
      end
      
      dbh.query("INSERT INTO attachments (created_at,updated_at,filename,email_id,content_type)
                VALUES (
                now(),
                now(),
                '#{filename}',
                #{@email_id},
                '#{part.content_type}'
                )
      ")
      #write attachments to the file system
      File.open("/home/guillaume/#{@domain_name.split('.')[0]}/tmp/attachments/#{@email_id}/#{filename}",File::CREAT|File::TRUNC|File::WRONLY,0777) do |f|
        f.write(part.body)
        system "chmod 0770 /home/guillaume/#{@domain_name.split('.')[0]}/tmp/attachments/#{@email_id}/#{filename}"
      end    
    end
    @i += 1
  end
end

And voila ! As you can see, I store Attachments on the filesystem, this is my own way to do things as I do hate to insert huge blobs into Databases, but do what you want…

You can retrieve the whole script here and adapt it to you needs, hope this could help…