After installing the SMTP plugin for two sites, I noticed my server's ram usage was slowly increasing, (Roughly about 1MB an hour) but never decreasing. At one point it got to over 600MB so I reset the server and the same issue came up after about two days.
I traced the memory increase back to the Apache process that runs Wordpress. After disabling the SMTP plugin, the ram usage started dropping back down to normal.