CSharp Setup Instructions

  • WARNING: VISUAL EDITOR BREAKS KEY LINKS ON THIS PAGE!
  • YOU MUST DISABLE VISUAL EDITOR IN YOUR ACCOUNT SETTINGS BEFORE EDITING.
    • SWITCHING out of Visual Edit still breaks links
  • If you do break links, edit in Wiki Editor and replace %20 in any links to page anchors to with spaces

Home

Understanding Direct

This document assumes you are familiar with core concepts such as:
  • PKI Security
    • X509 Certificates
    • CA Certificates, Trust Chains
    • Encryption and Digital Signatures
  • Direct terminology such as
    • Organization Certificate
    • Anchors
  • SMTP/ SMTP Server: How to run and host a public Internet mail server.
  • DNS and how it works
    • MX, ANAME, CERT, CNAME, NS, SOA records
You can learn about these concepts by reading the following documents as well as by leveraging the Internet!

Understanding Secure Email

Reference Implementation Architectural Overview


System Requirements Windows 2008

Component
Required
Instructions
Used for
Windows Server 2008
Yes

Everything :-)
Windows HOT FIX
YES
Large Message Hot Fix
To fix an issue with large messages
.Net Framework 3.5+ with SP1
Yes
.Net Framework Download

Smtp Server on Wink28
Yes
Smtp Server Setup Instructions
  • Runs the Direct Smtp Gateway
  • Free
IIS 7x
Yes
Good grasp on IIS Web Site Administration
Direct Middle Tier is WCF with HTTP Activation
WCF with HTTP Activation
Yes
WCF Setup

SQL Server Express, Sql Server 2008 or equivalent Sql Database
Yes
SQL Server 2008 R2 Express Download
  • Used for Storing Domains, Addresses, Certificates, Private Keys, Anchors
SQL Management Studio
No
SQL Server 2016 Management Studio Download
Optional - if NOT already installed
Outbound Firewall Ports:
Yes
  • TCP Port 25
  • TCP Port 53
  • UDP Port 53
  • TCP Port 25 used by Smtp Server to send outbound mail
  • TCP Port 53 used by the Gateway to resolve a mail recipient's CERT records from DNS.
  • UDP Port 53 used to resolve standard MX, NS and ANAME records
Inbound Firewall Ports:
Yes
  • TCP Port 25
  • TCP Port 53
  • UDP Port 53
  • TCP Port 25 used by Smtp Server to receive incoming mail
  • If running DNS Server:
    • TCP Port 53: Primarily used to receive and respond to requests for CERT records
    • UDP Port 53: Respond to requests for DNS records like MX, NS, ANAME and SOA


System Requirements Windows 2012 R2 (document in process)

Component
Required
Instructions
Used for
Windows Server 2012 R2
Yes

Everything :-)
.Net Framework 3.5
No
Install from Windows features
(HSM) Luna SA 5 dependency.
.Net Framework 4.6.1
Yes
.Net Framework Download

Smtp Service
Yes
Smtp Server Setup Instructions
  • Runs the Direct Smtp Gateway
  • Free
Web Server (IIS) Role
Yes
Good grasp on IIS Web Site Administration
Direct Middle Tier is WCF with HTTP Activation
WCF with HTTP Activation
Yes
WCF Setup

SQL Server Express, Sql Server 2012-2016
Yes
SQL Server 2016 Express Download
  • Used for Storing Domains, Addresses, Certificates, Private Keys, Anchors
SQL Management Studio
No
SQL Server 2008 Management Studio Express Download
Optional - if NOT already installed
Outbound Firewall Ports:
Yes
  • TCP Port 25
  • TCP Port 53
  • UDP Port 53
  • HSM Port 1792
  • TCP Port 25 used by Smtp Server to send outbound mail
  • TCP Port 53 used by the Gateway to resolve a mail recipient's CERT records from DNS.
  • UDP Port 53 used to resolve standard MX, NS and ANAME records
  • HSM Port used to resolve hardware token. This is the Luna SA 5 default port. Consult your vendor docs for the port.
Inbound Firewall Ports:
Yes
  • TCP Port 25
  • TCP Port 53
  • UDP Port 53
  • TCP Port 25 used by Smtp Server to receive incoming mail
  • If running DNS Server:
    • TCP Port 53: Primarily used to receive and respond to requests for CERT records
    • UDP Port 53: Respond to requests for DNS records like MX, NS, ANAME and SOA


Setup

To setup the system with test data perform the following steps:
  1. Get installer
  2. Create or ensure SMTP Domains

Installing Binaries

You deploy binaries in one of two ways:
  1. Use the installer and only select the "Developer gateway configuration to SMTP" option to install bits from the the last "good" release
  2. OR sync to the source, build and deploy the binaries yourself
Beginners are best off using the Developer Gateway Installer. In addition to binaries, the Installer will set up most of the system for you. Some steps are deliberately left out as a learning exercise, but can be easily automated for your own private installers.

Once you get it working and understand the data flow, you can:
  • Install bits in directories other than the ones selected by the installer
  • Deploy bits manually
By default, the installer places all binaries and configuration files in C:\Program Files\Direct Project .NET Gateway.
In the rest of the document, we'll refer to the installation directory as $InstallDir.

Sending Test Mail

You can send test mail by placing a mail file in the SMTP Server Pickup folder.
  • The pickup folder gives you a trivial way to send messages.
  • You can also point any SMTP mail client at the Gateway, although you should hold off on it until you've got everything in place.
  • Verify that Smtp Service is actually running
  • Go to $InstallDir\Samples
  • Copy simple.eml to C:\inetpub\mailroot\pickup

Where did the mail go?

Recall that by default, the developer install does not put messages on the wire.

Messages are instead written to Message Folders.

  • Go to C:\inetpub\mailroot\dev\incoming
  • You should see your message
Big deal, you say? What did the Gateway do?

What the Gateway did

Your message went through two entire Direct pipelines - Outgoing and Incoming.

Since both the sender and recipient domains are on the same server, your Gateway executable sees both Outoing and Incoming messages - and consequently, both pipelines.
  • Outgoing
    • Parsed the outgoing test message
    • Resolved the sender's (toby@redmond.hsgincubator.com) private key
    • Resolved the anchors the sender trusts
    • Resolved the recipient's (biff@nhind.hsgincubator.com) public X509 Certificate
    • Verified that the recipient's certificate is both valid and trusted - i.e. issued by an anchor the sender trusts.
    • Created appropriate S/MIME envelopes
    • Signed the message with the sender's private key
    • Encrypted the message with the recipient's certificate
    • Produced a new, secure email message...
  • Incoming
    • Parsed the incoming secure email message
    • Resolved the recipient's (biff@nhind.hsgincubator.com) private key
    • Resolved the anchors the recipient trusts
    • Decrypted the message
    • Verified the sender's signature
    • Verified that the sender's certificate is both valid and trusted - i.e. issued by an anchor the recipient trusts
    • Produced a new, non-encrypted message...

Can I see what it did?

To see what the Direct Gateway did to your message:
  • Go to C:\inetpub\mailroot\dev\Raw
  • Here, you can see copies the messages that passed through the Gateway
  • You will notice 2 messages.
  • Open both in Notepad
  • One message is the test message you sent - the Outgoing message
  • The second message is SECURE - with S/MIME - what your test message was turned into.
  • Since both the sender and recipient domains are on the same server, your Gateway also sees the Incomingmessage
    • The secure message is the Incoming message
  • The incoming message is processed... and dropped into the C:\inetpub\mailroot\dev\incoming
  • Outgoing mail in C:\inetpub\mailroot\dev\outgoing
  • Incoming mail in C:\inetpub\mailroot\dev\incoming
  • A copy of the original message is placed in C:\inetpub\mailroot\dev\raw
  • All rejected mail is placed in C:\inetpub\mailroot\dev\badmail
Mail sent to or between domains managed or owned by this Gateway are automatically placed in the Incoming folder (you won't find it in Outgoing).

The Gateway did all this using the Test Certificates, Anchors and Private Keys it installed. Certificates, anchors and private keys are in the Direct Database fronted by the Middle Tier Config Service.

Test Anchors

  • Open a cmd prompt
  • cd to your $InstallDir
  • start ConfigConsole.exe
  • type anchors_get redmond.hsgincubator.com true
    • Anchors that redmond.hsgincubator.com trusts
  • type anchors_get nhind.hsgincubator.com true
    • Anchors that nhind.hsgincubator.com trusts
Your gateway is configured to resolve anchors using a ServiceResolver.
...
 <Anchors>
 <ServiceResolver>
   <ClientSettings>
     <Url>http://localhost/ConfigService/CertificateService.svc/Anchors</Url>
   </ClientSettings>
 </ServiceResolver>
 </Anchors>

Test Private Keys(PrivateCerts)

Your gateway is configured to resolve Private Keys using a ServiceResolver
....
 <PrivateCerts>
   <ServiceResolver>
     <ClientSettings>
       <Url>http://localhost/ConfigService/CertificateService.svc/Certificates</Url>
     </ClientSettings>
   </ServiceResolver>
 </PrivateCerts>

Test Public Certs

In real deployments, your Gateway dynamically resolves an email recipient's X509 Public Cert using DNS.

Your developer gateway is configured to resolve certs using a MachineResolver. Certificate resolution using DNS is not enabled out of the box because:
  • Firewall configuration may be needed for DNS to work
  • Good way to introduce you to Machine certificate stores.
 <PublicCerts>
 <!--
 DO NOT ENABLE DNS WITHOUT LOOKING AT FIREWALL SETTINGS, BOTH MACHINE & CORPORATE
 Firewalls must ALLOW BOTH UDP & TCP on Port 53.
 -->
 <!--
 <DnsResolver>
     <ServerIP>8.8.8.8</ServerIP>
     <Timeout>5000</Timeout>
 </DnsResolver>
 -->
 <!-- Uses a cert store on Local Computer -->
 <MachineResolver>
     <Name>NHINDExternal</Name>
 </MachineResolver>

Machine Store

  • Open your Windows Certificate Store
  • Expand the left pane and inspect certificates in the following folders:
    • NHINDExternal (Public Certs)
    • NHINDPrivate
    • NHINDAnchors
Your public certs are in the NHINDExternal folder.

Machine stores are useful if you want to run a very simple test installation that does not use:
  • The Direct Database
  • Middle Tier
  • Dns

Creating Your Own Domain

A working knowledge of SMTP Server Management is assumed.

When you installed the binaries, you created 2 test domains. Follow those instructions.
  • Add yourdomain to SMTP Server
  • Add yourdomain to the Gateway config file
  • Register your domain with the Config Service
  • Restart Smtp Server
Of course, for this to be a real email domain, you'll need to set up DNS with MX records etc.

Create Your Email Addresses

You must create email addresses. The Gateway rejects messages sent to unknown addresses.

Creating PrivateKeys and Certificates for your domain

You create private keys and their associated X509 Certificates using the venerable makecert too.
Bing for help on makecert.

Create your CA Certificate (Trust Anchor!)

Sample batch file you can use to generate CA certs:
Notepad the file to see what it does.
Example:
 
genca "Your Domain CA" C:\junk\
  • This CA certificate is your Trust Anchor!
  • Any Direct node that installs your Trust Anchor will trust certificates issued by this anchor.
  • The batch file places the Private Key for your CA in your machine's Windows certificate Store.
  • It places the CA's X509 Certificate in C:\junk\
    • You can hand this certificate to your partners

Create your Secure Mail Certificate

Use this batch file to create a secure email certificate issued by your the CA you made above:
Notepad the file to see what it does.
Example:
 
gencert yourdomain.xyz "Your Domain CA" C:\junk\
  • The batch file places the private key in your your machine's Windows certificate Store.
  • It places a public X509 Certificate in C:\junk\
Your Gateway will use this certificate/private key for S/MIME cryptography.

Export your Certificate & Private Key to a File

To use your private key with the Direct Gateway, you must store it in the Config Service. Because makecert puts your private key in the Windows Certificate Store, you must first export the Cert + private key out of the Windows store into a PFX file.

CertStore.PNG
  • In the right pane, Right Click on your Certificate and choose AllTasks\Export
ExportCert.png
  • Make sure you export the private key
PrivateKey.PNG

Store PrivateKey + Certificate in Config Service

You must store your new private key in the Config Service.

certificate_add C:\junk\yourdomain.pfx foopassword Enabled
    • Make sure you use the Enabled option
  • type certificate_get yourDomainName true true
certificate_get yourdomain.xyz true true
 
certificate_get redmond.hsgincubator.com true true

Publishing your Secure Mail Certificate

To send an encrypted message to an addresses, a sender must first resolve the certificate for that address. The sender then extracts a public key from the certificate certificate and uses it to encrypt the message.

The sender can resolve certificates in several ways:
  • Using DNS
  • From the Config Service
  • From a local Machine certificate store
Note: these resolvers can be chained in a pipeline...so the gateway will try them in order until it finds what it needs.

When you run the Direct Dns Responder Service, and store your certificates in the Config Service (see above), your certificates are automatically made available for resolution by both internal and external senders.

Dns resolution is disabled by default - since you will typically need to make sure Dns subdomains and the like are appropriately owned and delegated.

Add your Config Service to the Resolver Chain

Add the ServiceResolver section below to your gateway configuration file. See example below. The Url should be identical to the one ServiceResolver you define in your PrivateCerts section.

 <PublicCerts>
 
 <!--
 DO NOT ENABLE DNS WITHOUT LOOKING AT FIREWALL SETTINGS, BOTH MACHINE & CORPORATE
 Firewalls must ALLOW BOTH UDP & TCP on Port 53.
 -->
 <!--
 <DnsResolver>
   <ServerIP>ENTER DNS SERVER IP addresses to use to resolve certs</ServerIP>
   <Timeout>5000</Timeout>
 </DnsResolver>
 -->
 <!-- Look in the config service -->
 <ServiceResolver>
   <ClientSettings>
     <Url>http://localhost/ConfigService/CertificateService.svc/Certificates</Url>
   </ClientSettings>
 </ServiceResolver>
 <!-- Finally, look in the machine store-->
 <MachineResolver>
   <Name>NHINDExternal</Name>
 </MachineResolver>
 </PublicCerts>
Details

Trusting An Anchor

To exchange messages with another Direct node, install their trust anchor.
anchor_add yourdomain.xyz C:\junk\domainCA.cer null Enabled
anchor_list yourdomain.xyz true
anchor_list nhind.hsgincubator.com true

Internal Messages

For messages to flow within your domain, you must trust your own anchor. To send messages from bob@yourdomain.com to sue@yourdomain.com, you must install your own Trust Anchor.
anchor_add yourdomain.xyz "C:\junk\Your Domain CA.cer" null Enabled

Generating ACKS (MDN)

The Gateway has built in support for auto-generating MDN ACKs for accepted messages. ACKS are disabled in the Developer Gateway configuration - for obvious reasons.

To enable acks, alter Notification settings in your Gateway configuration file.

By default, the Gateway obeys the MDN standard and generates ACKs only if they were requested. But as per the Direct standard, Acks should be always generated, so be sure to set AlwaysAck to true.
 <Notifications>
 <AutoResponse>true</AutoResponse>
 <AlwaysAck>true</AlwaysAck>
 <ProductName>Your Direct Product or HISP</ProductName>
 <Text>Your message was processed.</Text>
 </Notifications>

DNS

To run the DNS Server:
  • Ensure you have UDP and TCP Port 53 open on your machine (both Inbound/Outbound)
  • Your server is listening on the right IP addresses

IP Addresses and Ports

  • The developer answers requests only on the localhost (127.0.0.1) IP address.
<DnsServerSettings Address="127.0.0.1" Port="53" DefaultTTL="3600">
You can change this configuration (DirectDnsResponderSvc.exe.config) to listen on any IP addresses on your machine. 0.0.0.0 is a special reserved IP address called Any.
 <DnsServerSettings Address="0.0.0.0" Port="53" DefaultTTL="3600">
 ....
 </DnsServerSettings>

Local Firewall

  • Start Menu
  • In the Search Box, type "windows firewall advanced"
  • Select the listing
Firewall_Search.png

Inbound Rules

In the left pane, validate that you have local TCP Port 53 and UDP Port 53 Open. It is unlikely that you do.

To open inbound ports:
  • In the left pane, click on Inbound Rules
  • Select New Rule
  • Set Rule Type = "Port", click Next
RuleType_Port.PNG

  • Set Protocol to TCP, Port 53, click Next
  • NOTE: You will need to repeat these steps for UDP.
Port_53.PNG

  • Set "Allow the Connection", click Next
Connection_Allow.PNG

  • Check all options
Firewall_appliesto.PNG

REPEAT STEPS, for UDP.

Outbound Rules

You should typically have UDP Port 53 already open. But do verify.

To open outbound TCP Port 53:
  • In the left pane, click on Outbound Rules
  • Select New Rule
  • The User Interface is similar to Inbound Rules (above)
  • Make sure you allow the inbound connection
For your Dns server to respond to request from

Adding DNS Records

You manage Dns records using the Config Console.
Adding Dns Records

You will minimally need need to create records of this type:
  • SOA
    • This is your "Root" record
  • MX
    • Provides the domain name of of the Mail Server/s (if they are load balanced) for your domain.
    • You can create multiple MX records for your domain
You don't need to explicitly create CERT records, because the Direct Dns Server automatically serves up your certificates.

A set of sample commands:
DNS_SOA_ENSURE yourdomain.com nameserver.yourdomain.com. foo.example.domain 20100602 3600
DNS_MX_ENSURE yourdomain.com mailserver.yourdomain.com 3600

Lookup Records

You can use nslookup. However, nslookup does not understand CERT records, and neither does the default Windows Dns stack. You instead use a simple Direct command line tool.

  • Open a command prompt
  • Make sure the Direct Dns Server is running
    • type net start directdnsrespondersvc
  • Go to $InstallDir
  • type AgentConsole.exe
  • type commands dns
    • Shows a list of dns commands
  • type dns_setserver 127.0.0.1
    • This directs all Dns queries to your local Dns server
    • By default, the resolver sends queries to Google's public DNS (8.8.8.8)
    • Note that by setting 127.0.0.1 as your server, you will only resolve local Dns records.
  • type dns_resolve domain CERT
dns_setserver 127.0.0.1
dns_resolve redmond.hsgincubator.com CERT
dns_resolve redmond.hsgincubator.com MX
//
// Use like nslookup
//
dns_setserver 8.8.8.8
dns_resolve www.microsoft.com
dns_resolve www.google.com

Troubleshooting PARTNER DNS Servers

You can use AgentConsole.exe to resolve your partner certificates from the Internet. You can also use it to resolve DNS records in general. For example, the dns_resolvecert command will let you resolve a partner certificate and save it to a file.

See docs on: AgentConsole.exe


Corporate/Data Center Firewall

Most corporate firewalls block TCP Port 53 - mostly from ignorance - even though the DNS standard allows for both UDP & TCP based domain lookups.
  • You will be unable to resolve external CERT records - which are resolved over TCP.
  • External callers will be unable to resolve yours.
You will almost certainly need to speak with your Ops team.

Configuration Settings

Configuration Settings

Direct components support a number of configuration settings and extensibility points. They are documented in detail below:

Using a mail client

Once you have everything working, you can also setup up a mail client to point at your SMTP Server Windows Live Essentials Mail Client