How to Build a Custom Java Mail Server From Scratch Building your own mail server from scratch is an excellent way to understand the underlying protocols of the internet. While production-grade email delivery requires complex security compliance (like SPF, DKIM, and DMARC), creating a functional Java-based mail server will teach you how socket programming, multithreading, and network protocols interact.
This guide will walk you through building a minimal SMTP (Simple Mail Transfer Protocol) server to receive emails and a POP3 (Post Office Protocol) server to retrieve them using core Java. Part 1: Architecture of a Mail Server
A complete mail server system consists of two primary components:
SMTP Server (Receiver/Sender): Listens on a port (traditionally port 25), accepts incoming text commands from email clients, and saves the message body.
Mail Repository: A simple storage system (like a local directory structure) where messages are saved as text files under specific user accounts.
POP3/IMAP Server (Retriever): Listens on another port (POP3 uses port 110) to let email clients authenticate and download stored messages. Part 2: Building the SMTP Server
The SMTP protocol relies on a plain-text conversational exchange between a client and a server. The server waits for connections, processes commands (like HELO, MAIL FROM, RCPT TO, and DATA), and stores the resulting stream.
Here is a simplified, multi-threaded SMTP server implementation:
import java.io.; import java.net.; import java.util.UUID; public class JavaMailServer { private static final int SMTP_PORT = 2525; // Using 2525 to avoid root permissions public static void main(String[] args) { System.out.println(“Starting SMTP Mail Server on port ” + SMTP_PORT + “…”); try (ServerSocket serverSocket = new ServerSocket(SMTP_PORT)) { while (true) { Socket clientSocket = serverSocket.accept(); new Thread(new SmtpHandler(clientSocket)).start(); } } catch (IOException e) { System.err.println(“Server error: ” + e.getMessage()); } } } class SmtpHandler implements Runnable { private Socket socket; public SmtpHandler(Socket socket) { this.socket = socket; } @Override public void run() { try ( BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream())); PrintWriter out = new PrintWriter(socket.getOutputStream(), true) ) { // Send initial greeting out.println(“220 MyCustomJavaMailServer Ready”); String line; String recipient = “”; StringBuilder emailData = new StringBuilder(); boolean isReadingData = false; while ((line = in.readLine()) != null) { System.out.println(“Client: ” + line); if (isReadingData) { if (line.equals(“.”)) { isReadingData = false; saveEmail(recipient, emailData.toString()); out.println(“250 OK Message accepted for delivery”); } else { emailData.append(line).append(” “); } continue; } String command = line.toUpperCase(); if (command.startsWith(“HELO”) || command.startsWith(“EHLO”)) { out.println(“250 Hello ” + socket.getInetAddress().getHostName()); } else if (command.startsWith(“MAIL FROM:”)) { out.println(“250 2.1.0 Sender OK”); } else if (command.startsWith(“RCPT TO:”)) { // Extract recipient email address recipient = line.substring(line.indexOf(“<”) + 1, line.indexOf(“>”)).trim(); out.println(“250 2.1.5 Recipient OK”); } else if (command.equals(“DATA”)) { isReadingData = true; out.println(“354 Start mail input; end with Use code with caution. Part 3: Building the POP3 Server
Once an email is saved into the mailstore/ folder, the recipient needs a way to fetch it. POP3 is a simple protocol designed for this exact use case. It allows clients to check available messages (STAT), list them (LIST), retrieve them (RETR), and flag them for deletion (DELE).
Below is a basic POP3 listener that reads from our custom storage:
import java.io.; import java.net.; public class JavaPop3Server { private static final int POP3_PORT = 1110; // Custom port to avoid permission restrictions public static void main(String[] args) { System.out.println(“Starting POP3 Server on port ” + POP3_PORT + “…”); try (ServerSocket serverSocket = new ServerSocket(POP3_PORT)) { while (true) { Socket clientSocket = serverSocket.accept(); new Thread(new Pop3Handler(clientSocket)).start(); } } catch (IOException e) { System.err.println(“POP3 Server error: ” + e.getMessage()); } } } class Pop3Handler implements Runnable { private Socket socket; private String currentUser = “”; public Pop3Handler(Socket socket) { this.socket = socket; } @Override public void run() { try ( BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream())); PrintWriter out = new PrintWriter(socket.getOutputStream(), true) ) { out.println(“+OK POP3 server ready”); String line; while ((line = in.readLine()) != null) { String[] tokens = line.split(” “); String command = tokens[0].toUpperCase(); if (command.equals(“USER”)) { currentUser = tokens[1]; out.println(“+OK User accepted”); } else if (command.equals(“PASS”)) { // In a production server, validate the password here out.println(“+OK Mailbox locked and ready”); } else if (command.equals(“STAT”)) { File userDir = new File(“mailstore/” + currentUser); File[] files = userDir.listFiles(); int count = (files != null) ? files.length : 0; out.println(“+OK ” + count + “ messages”); } else if (command.equals(“LIST”)) { File userDir = new File(“mailstore/” + currentUser); File[] files = userDir.listFiles(); if (files == null || files.length == 0) { out.println(“+OK 0 messages”); } else { out.println(“+OK ” + files.length + “ messages”); for (int i = 0; i < files.length; i++) { out.println((i + 1) + “ ” + files[i].length()); } out.println(“.”); } } else if (command.equals(“RETR”)) { int msgNum = Integer.parseInt(tokens[1]) - 1; File userDir = new File(“mailstore/” + currentUser); File[] files = userDir.listFiles(); if (files != null && msgNum >= 0 && msgNum < files.length) { out.println(“+OK ” + files[msgNum].length() + “ octets”); try (BufferedReader reader = new BufferedReader(new FileReader(files[msgNum]))) { String mailLine; while ((mailLine = reader.readLine()) != null) { out.println(mailLine); } } out.println(“.”); } else { out.println(“-ERR No such message”); } } else if (command.equals(“QUIT”)) { out.println(“+OK Sayonara”); break; } else { out.println(“-ERR Unknown command”); } } } catch (IOException e) { System.err.println(“POP3 Handler error: ” + e.getMessage()); } } } Use code with caution. Part 4: Testing the Server
You can easily test your custom mail system locally without needing a third-party email client. Open your terminal or command prompt and use telnet or nc (netcat). Testing SMTP (Sending an Email) Run JavaMailServer. Connect using: telnet localhost 2525 Type the following conversation:
HELO localhost MAIL FROM:[email protected] RCPT TO:[email protected] DATA Subject: Hello from my custom Java Mail Server! This is a scratch-built mail server test. . QUIT Use code with caution.
Look inside your root project folder. You will find a file path structured like mailstore/alex@://test.com containing the email content. Testing POP3 (Reading the Email) Run JavaPop3Server. Connect using: telnet localhost 1110 Type the following conversation: USER [email protected] PASS password STAT RETR 1 QUIT Use code with caution. Moving to Production: Critical Additions
What we built above is an educational foundation. If you want to connect this setup to the live web and deal with modern mail clients, you will need to implement:
TLS/SSL Encryption (SSLSocket): Modern email communication strictly forbids sending plaintext passwords and content over the open internet. You must wrap your standard sockets with secure sockets (SMTPS on port 465 or STARTTLS on port 587).
Database Integration: Instead of reading files straight from directories (which can create performance bottlenecks and race conditions), store your emails, logs, and account credentials inside a relational database like PostgreSQL or MySQL.
DNS Records & Security Standards: Real mail servers will block your server unless you configure public records for your domain. You must set up MX Records (pointing to your server IP), SPF (listing authorized sender IPs), DKIM (signing emails cryptographically), and DMARC rules.
By writing these socket listeners yourself, you now have a direct understanding of what happens behind the scenes every time you hit “Send” in an email application.
Leave a Reply