Building a lightweight data transfer system using email in client-server architecture

[NetworkExperts]

Enterprises, especially in banking, avoid public APIs due to security. In low-connectivity scenarios, email-based asynchronous transfer ensures secure, reliable communication without real-time access.

In many enterprise environments—particularly in industries such as banking and finance—organizations are often reluctant to expose or consume publicly accessible REST application programming interfaces (APIs) due to strict security and compliance requirements. This creates a challenge for enabling secure data exchange between distributed client systems and centralized servers. Additionally, in constrained connectivity scenarios, such as field operations where personnel deliver assets (for example, laptops) to end customers, continuous internet access is not always available. For example, delivery agents may capture transaction data by scanning barcodes through a client application, which then stores the information locally and queues it in an email outbox. These emails remain unsent until the agent returns to an office environment with network connectivity, at which point they are transmitted to a central system for processing. Such use cases highlight the need for a reliable, asynchronous, and secure data transfer mechanism that can function without real-time connectivity, leverage existing enterprise tools like email and avoid dependency on externally exposed services.

 
Figure 1. Architecture diagram

The system is designed with two main components:

• Client application
• Server application

The communication between them is handled entirely through email, making the system simple, firewall friendly, and highly adaptable.

Client-side implementation

The client application is built using a combination of:

• VBScript (Or any .net based framework which supports library to interact with Microsoft Outlook)
• HTML

Since the application runs on user machines with Microsoft Outlook installed, it leverages Outlook as a medium to send data.

How it works:

1. The client prepares data in comma-separated values (CSV) format.
2. Using VBScript, it interacts with Outlook installed locally.
3. The CSV file is attached to an email.
4. The email is sent to a dedicated mailbox.

 Why this approach?

• No need for direct server connectivity
• Works in restricted network environments
• Leverages existing enterprise tools (Outlook)

Email as the transport layer

Instead of using APIs or sockets, the system uses email as a queueing and transport mechanism.

Each email contains:

• A CSV attachment (data payload)
• A subject line (used for routing and identification)

This makes the system loosely coupled and easy to extend.

Server-side processing

On the server side, a background service—referred to as a spooler service—handles incoming emails.

Responsibilities of the spooler:

1. Connect to the mailbox
2. Fetch unread emails
3. Download attachments
4. Pass data to the processing layer

Integration with Azure EWS REST API

The spooler service uses the Exchange Web Services (EWS) REST API provided by Microsoft Azure to access emails programmatically.

 Key capabilities:

• Authenticate with the mailbox
• Retrieve emails
• Download attachments
• Mark emails as processed

This approach ensures:

• Secure access
• Scalability
• Compatibility with enterprise email systems

Intelligent data processing

Once emails are downloaded, the server application processes them based on the email subject line.

Example:

Subject line	Action performed
IMPORT_CUSTOMER	Process customer data
UPDATE_PRICING	Update pricing records
SYSTEM_LOGS	Store logs for analysis

This design allows:

• Easy extensibility
• Clear separation of concerns
• Rule-based processing

End-to-end flow

1. Client generates CSV data
2. Email sent through Outlook to dedicated mailbox
3. Spooler service fetches email using EWS API
4. Attachment downloaded
5. Subject line evaluated
6. Data processed accordingly

Advantages of this approach

• Firewall friendly: It uses standard email protocols.
• Decoupled architecture: Client and server are loosely connected.
• Reliable delivery: Email systems provide retry and storage.
• Auditability: Emails act as a natural audit trail.
• Low infrastructure dependency

Considerations and limitations

• Latency: Email is not real time.
• Attachment size limits
• Error handling: It needs retry and duplicate handling logic.

Additional security measures to be taken

1. Restrict who can send to the mailbox

Allowed senders (allowed)

• Configure mailbox to accept mail only from specific email addresses or domains
• Example: only allow client-app@company.com

Internal-only restriction

• Block all external senders
• Accept emails only from your organization

Transport (mail flow) rules

• Reject emails if:
• Sender not in approved list
• Subject doesn’t match expected pattern
• Attachments are not CSV

Meet the author:

Nataraj G N, Subject Matter Expert, HPE