So here I was, setting up IMAP and SMTP (for untrusted networks) and I needed a way to whitelist access for specific users.
Just because I'm using my systems shadow and passwd files for auth.

Luckily Dovecot supports Lua for passdb authentication...

With this snippet that you can add to your 10-auth.conf,
Dovecot will execute the given Lua program on an authentication attempt.
If it fails, the whole authentication failed, otherwise it just keeps its current
authentication state. This check also cannot be skipped.

passdb {
  driver = lua
  args = file=[path to script]
  skip = never
  result_success = continue
  result_failure = return-fail
  result_internalfail = return-fail
}
    
This way even when the user trying to auth exists, and one of the passdb entries
passed the password check, the login will fail if the script fails.
On the other side, just because a user is whitelisted, the login isn't successful,
it just doesn't downright fail.
Another auth stage needs to be successful to pass.

And here is the associated Lua code to handle the whitelist check:

local WHITELIST_FILE = "[path to your whitelist]"

function auth_passdb_lookup(req)
  for user in io.lines(WHITELIST_FILE) do
    if user == req.username then return dovecot.auth.PASSDB_RESULT_NEXT, {} end
  end
  return dovecot.auth.PASSDB_RESULT_USER_DISABLED, "this user cannot access services"
end

function script_init()
  return 0
end

function script_deinit()
end
    
This one is pretty self explainatory so I just leave it at that.

Have fun setting up your mail server.