Filter spam to a folder with procmail and qmail

Posted by on Feb 25, 2009 in Mail | 33 comments

This post is inspired by Russ Whittmann and Europheus’ solutions , with these goals/improvements in mind:

  • Move spam marked by spamassassin into a special Spam folder in each account (flabergasted this isn’t standard on something like PLESK!)
  • Allow for quota, which is available on some PLESK servers (I previously thought it was available on all servers until recently) .
  • Make a solution with as minimal manual labour as possible outside the PLESK control panel, which relates most specifically to having to edit and create .qmail and .procmailrc files each time you change or add an email account.
  • Avoid having to have a special .procmailrc file for every user, one recipe for all!

Why?

The problem with Russ’ solution is that should you want to use procmail to filter spam and other emails into respective folders, replacing the .qmail file with just

| /usr/local/psa/bin/psa-spamc accept
| preline /usr/bin/procmail -m -o .procmailrc domain.com user

gets rid of quota restrictions, which is normally controlled by

| /usr/bin/deliverquota ./Maildir

this is far from ideal on a server with many email accounts.

A point to note
People that aren’t using psa-spamassassin (e.g. people using qmail-scanner) you don’t need that first line, “| /usr/local/psa/bin/psa-spamc accept” in your .qmail file, since qmail-scanner already already takes it though spamassassin.

Europheus has a (pretty good) stab at making administration of the tweaks to all email accounts on the server at once, but there are limitations to this too because it overrides all settings written by PLESK onto the .qmail file and assumes that everyone wants a mailbox on their account and nothing more. Some people may want have mail redirects, mailgroups etc. Between these two and some imagination, a solution much closer to my needs arose.

The solution is to pipe in the quota command within the .procmailrc recipe, and also pass variables into the procmail recipe so that all qmail files can reference one procmailrc file. Here’s what I did:

The .procmailrc file

There are a couple of ways of doing this. I’ve personally chosen not to include spam deletion after 30 days on here and add it as a shell script run from crontab every day, so you can seperate as much as possible from mail handling, and also avoid running a command EVERY time a spam message is received. I supply both options:

Click here to view/download the .procmailrc file which autodeletes spam within the recipe (the link sends you to a script with indented code)

Click here to view/download the .procmailrc file and Click here to view/download the .spam-purge file, which deletes spam from all accounts that is older than 30 days

Put these in your /var/qmail folder. You can change the location, but then remember to change the .qmail-repair script to reflect the location in the .qmail file.

Don’t know why but I kept getting an error from procmail in the qmail log about not being able to change directory to /var/qmail/popuser, so when I created a directory and made it owned by popuser, the problem went away…. anyone know where that would come from?

The .qmail-repair script

In order to make these changes play nice with PLESK, I created a script, loosely based on Europheus’ one so that it will “repair” .qmail files rewritten by PLESK. Basically, all it does is replace the line in .qmail which runs it through deliverquota and runs it through the server-wide .procmailrc file. Each .qmail file is given a slightly tailored line so arguments containing the user and domain are passed to the .procmail file.

There is mention in some places that PLESK completely replaces the content of the .qmail file. This is not the case from at least PLESK 8.6 (tested) and possibly lower (not tested), where it just removes the line in question, such as redirects.

There is one limitation to this script. If you disable the mailbox of a user in PLESK, this won’t disable it in the .qmail file since PLESK looks for the line it added, which it doesn’t find since it got modified by this script. This means mail will still be delivered into the mailbox, although the user won’t be able to connect anymore via POP or IMAP. If you want to disable a mailbox be sure to also remove the line pointing to the procmail recipe, and when enabling the mailbox you shouldn’t have any problems by doing it within PLESK.

Click here to view/download the .qmail-repair script.

Put this in your /var/qmail folder as well. Again, the location is a suggestion, I like to keep all these qmail manipulation files in one place

Finally, instead of a crontab entry, you can add this to PLESK’s even manager so it only runs when a mail account is changed, therefore requiring the .qmail file to be repaired. You can do this by going to Server > Event Manager and having it run the script on these conditions : Mail account created & Mail account updated. (Updated – thanks atomicturtle from www.atomicrocketturtle.com)

33 Comments

  1. THis is great!!
    I was surfing for “deleting SPAM mail” solution, and finally got it here.
    We use sendmail/procmail, spamassassin, spamassassin-milter.

  2. Finally found what I am looking for, but there is one thing on my mind.
    This might really sound like a nOOb question, but do I have to create the Spam folder myself or is that taken care of by the scripts provided.
    Kind regards

    Florian

    • It’s actually a good question :) The spam folders will be created automatically with these scripts. Other ones we initially used did not, which is partially why we made these.

  3. Thank you for your quick reply.
    Now I have a second question: who needs to be the owner and which group to the scripts need to belong to?

    Kind regards
    Florian

  4. I’m not sure if it actually matters but I have it as the same owner as the mailboxes, in this case popuser.

  5. What am I missing here.
    I tried to run the .qmail-repair script from the command line in var/qmail.
    First I got -bash: ./.qmail-repair: /bin/sh^M: bad interpreter: No such file or directory After a little research I found the dos2 unix tool to get rid of the ^M.
    head -1 .qmail-repair | od -c
    0000000 # ! / b i n / s h r n
    to # ! / b i n / s h n
    now I get
    qmail]# ./.qmail-repair
    ./.qmail-repair: line 1: ?#!/bin/sh: No such file or director
    any ideas
    Thank you for your patience and help

    Kind regards
    Florian

  6. From the sounds of it your system is either having problems running a shell script, or you have some spaces or lack of them in places.

    My advice is you add the file again to your system. Copy and paste the text contents of the script on here either into vi or with something like WinSCP, which allows file text editing on the fly.

  7. Thx that solved this problem. I created all three files through Plesk and set the owner to popuser
    Going back to my folder creation question. Will the .Spam folder be create when I run the repair script or when Spam is delivered for the first time and which script does actually create the folder?

    Thank you so much
    Florian

  8. Yes, it will create folders automatically. The procmail recipe should do it for you.

  9. I just found that in order to see the Spam folder in MS Entourage you have subscribe to this folder. Oh I love M$.
    It works perfect now. (as far as I can tell)
    THX so much unfortunately you don’t have a donate button

    Kind regards
    Florian

  10. No problem, we’re putting this stuff out to help people… if you’d like to donate something a link to this page or netweblogic.com would be great, thanks!

  11. Didn’t seem to work for me, maybe I am a little lame when it comes to these things, I dropped the said files into the correct folder (via SFTP), I then logged into Plesk and create two event management actions. Is there anything else I need to do, I still get spam?

    Thanks
    Lee

  12. @Lee
    Have you run the .qmail-repair script once? It must be run so it changes the .qmail files for users. If you added the events correctly, changing the password or something similar to a mail account on your server should do the trick.

  13. Hi!

    Your script really save my host! :)

    But Spam folder is not deleted each 30 days (btw i set it to 5)

    * ? test -d $SPAMDIR && find $SPAMDIR/new -mtime +5 -exec rm {} ; && false

    Is this correct?

    • Yes, all you have to do is change the -mtime +x to whatever number of days you want. It can be 5, 30, whatever you want :)

    • Hi again.

      Now it stop to make .Spam folder in news mail accounts :'(

      spam mails into .Spam folder are deleted, but not into news accounts

      If i do sh .qmail-repair no error showed.

      If i do sh .procmail:

      quota: User is doesn't exist.
      .procmailrc: line 5: MAILDIR: command not found
      .procmailrc: line 6: SHELL: command not found
      .procmailrc: line 7: SPAMDIR: command not found
      .procmailrc: line 10: :0: command not found
      .procmailrc: line 19: syntax error near unexpected token `}'
      .procmailrc: line 19: ` { }'

      Any tip?

    • Hi Chencho,
      Can't really comment without having the full picture, but I'd recommend looking up http://netweblogic.com/linux/plesk-linux/restor… then make sure you've done all the above and rerun the scripts that reset the .qmail files.

  14. Hi again!

    Anyone knows if this guide works with Plesk 9.3?

    Thanks.

    • I've been told it works on 9 but I can't confirm it. I'm pretty sure it won't work if you're using postfix instead of qmail though.

    • At Plesk side i see:

      Servidor SMTP (Postfix)

      Can i switch to qmail?

      I have qmail installed over yum, and it's started, but i see i have both running (qmail and postfix) as services, but seems Plesk are using postfix.

    • I can't comment unfortunately, as I've not been able to use v9 much. However, I'm fairly certain you can do this, but you should probably make the switch via the control panel somehow, not via yum.

    • /usr/local/psa/admin/sbin/autoinstaller –select-release-current –install-component qmail

      Now i's using qmail.

      I'm going to test your script.

  15. i do something wrong… i think.

    Now i'm ussing qmail, centos 5.4, plesk 9 (but now i'm doing all by console).

    That's what i'm doing (as root):

    cd /var/qmail
    wget .procmail, .spam-purge, .qmail-repair
    sh .procmail

    *********************
    : command not found:
    .procmailrc: line 4: MAILDIR: command not found
    .procmailrc: line 5: SHELL: command not found
    .procmailrc: line 6: SPAMDIR: command not found
    : command not found:
    : command not found: :0
    : command not found0: {
    : command not found2: :0
    .procmailrc: line 13: alias: bin: not found
    .procmailrc: line 13: alias: boot: not found
    .procmailrc: line 13: alias: control: not found
    .procmailrc: line 13: alias: mailnames: not found
    .procmailrc: line 13: alias: plugins: not found
    .procmailrc: line 13: alias: popuser: not found
    .procmailrc: line 13: alias: queue: not found
    .procmailrc: line 13: alias: users: not found
    .procmailrc: line 13: alias: ^X-Spam-Status:: not found
    : not found: line 13: alias: Yes.*
    : command not found4: {
    : command not found6: :0
    .procmailrc: line 17: alias: bin: not found
    .procmailrc: line 17: alias: boot: not found
    .procmailrc: line 17: alias: control: not found
    .procmailrc: line 17: alias: mailnames: not found
    .procmailrc: line 17: alias: plugins: not found
    .procmailrc: line 17: alias: popuser: not found
    .procmailrc: line 17: alias: queue: not found
    .procmailrc: line 17: alias: users: not found
    .procmailrc: line 17: alias: !?: not found
    .procmailrc: line 17: alias: test: not found
    .procmailrc: line 17: alias: -d: not found
    : not found: line 17: alias:
    : command not found8:
    : command not found9:
    .procmailrc: line 21: :0: command not found
    .procmailrc: line 22: syntax error near unexpected token `|'
    'procmailrc: line 22: ` | /usr/bin/deliverquota “$SPAMDIR”
    *************************

    sh .qmail-repair

    *************************
    : command not found 4:
    'qmail-repair: line 5: syntax error near unexpected token `{
    'qmail-repair: line 5: `create_qmail() {
    *************************

    sh .spam-purge (no erros are showed)

    • Some things about my server:

      locate .qmail
      /usr/share/qmail-scanner/contrib/logrotate.qmail-scanner
      /var/drweb/.qmail
      /var/qmail/.qmail-repair
      /var/qmail/alias/.qmail-anonymous
      /var/qmail/alias/.qmail-drweb
      /var/qmail/alias/.qmail-drweb-daemon
      /var/qmail/alias/.qmail-mailer-daemon
      /var/qmail/alias/.qmail-mailman
      /var/qmail/alias/.qmail-postmaster
      /var/qmail/alias/.qmail-root

      which procmail
      /usr/bin/procmail

      I dont add something like:

      |preline /usr/bin/procmail -m -o .procmailrc

      Into .qmail file.

    • Hi Chencho,
      Unfortunately, I can't offer individual help to this level. Sorry.

    • No matter ;)

  16. Great post, thanks a lot!! :-)

    One question though: why have you removed the line that actually calls Spamassassin? After adding the lines below to the .procmailrc file Procmail actually started calling Spamassassin

    # Run SpamAssassin
    :0fw
    | spamassassin

    • Well, it depends on your setup but this was primarily written for plesk (8.7) and spamassassin gets called regardless. Same goes when you install qmail-scanner as in my other tutorial.

      Thanks for posting the extra line needed, that'll help some people I'm sure.

  17. Hi,
    great site – I am very interested in this. I placed the two scripts into /var/qmail, made them executable, owned by popuser. In plesk's event manager I have inserted “sh /var/qmail/.gmail-repair” – user popuser – for the events “Mail account created & Mail account updated”.
    But: When mail accounts are being updated or when I start the ./qmail-repair-script manually (no error messages given)… nothing happens. I still have the messages with ****SPAM**** in the mail accounts' inboxes.
    I am using plesk 9.3.0.
    Anyone…any idea? Would be great.
    Greetings from Germany!

    • No more answers required – it works! (Even with Plesk 9.5.2!) Thanks to all.

    • With qmail? I switch form postfix to qmail because after update 9.3 to 9.5.2 postfix dont send mails to internall domains.

      Now i'm with qmail and so interested in this scripts

    • With qmail, yes. I have been using the scripts above without any changes.

  18. well known An explanation of how Procmail delivers to various folder formats is in the latest … Hints for using procmail as a mail filtering agent from