Exchange Server 2007 Event ID 15002 and “4.3.1 Insufficient system resources” Errors

An Exchange Server 2007 server may stop processing mail and begin logging errors when the disk volume used for the Exchange Transport queue database becomes low on free disk space.

There are two key indicators of the issue:

Event ID 15002 or 15003 logged to the Application event log

Event Type: Warning
Event Source: MSExchangeTransport
Event Category: ResourceManager
Event ID: 15002
Date: 24/07/2007
Time: 1:40:03 PM
User: N/A
Computer: TESTEXC1

Description:

The resource pressure is constant at MediumHigh. Statistics:

Queue database and disk space (“C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue\mail.que”) = 64% [MediumHigh] [Normal=61% MediumHigh=63% High=65%]

Queue database logging disk space (“C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue\”) = 64% [Normal] [Normal=91% MediumHigh=93% High=95%]

Version buckets = 1 [Normal] [Normal=40 MediumHigh=60 High=100]

Private bytes = 21% [Normal] [Normal=71% MediumHigh=73% High=75%]

Physical memory load = 64% [limit is 94% to start dehydrating messages.]

Inbound mail submission from the Internet, the Pickup directory, and the Replay directory has stopped. Loading of e-mail from the queuing database, if available, continues.

Inbound internet mail is not received and external messaging systems report SMTP error “4.3.1 Insufficient system resources” when connecting to your Edge Transport or Hub Transport servers

220 TESTEXC1.testing.local Microsoft ESMTP MAIL Service ready at
Tue, 24 Jul 2007 13:47:21 +1000
helo
250 TESTEXC1.testing.local Hello [10.1.1.11]
mail from: test@test.com
452 4.3.1 Insufficient system resources

The cause of this issue is free disk space falling below the normal threshold for the disk volume used by the Exchange Transport queue database.  Free disk space for the Exchange Transport queue database is one of the resources monitored by the Exchange Server 2007 “Back pressure” feature.  You can read a very detailed description of this feature here.

The “medium” pressure threshold is calculated by a formula as follows:

100*(hard disk drive size – 4 GB) / hard disk drive size

So for example, a 16Gb volume will have a “medium” pressure threshold of:

100*(16 – 4) / 16 = 75%

At “medium” pressure internal mail flow will still occur, however inbound internet email will not be received and any applications that use non-MAPI methods to send email (eg they use the Pickup queue) will also cease being able to send email.  Anything under 4Gb of free space is automatically considered “high” pressure, which ceases all mail flow.  These thresholds are configurable so that instead of relying on the formula you can hardcode the percentage values that you would like, however I don’t recommend this and I am leaving it outside the scope of this particular blog post.

However a solution I do recommend is to relocate the Exchange Transport queue database to another volume.  By default the database is installed to the same location as the Exchange Server 2007 program files.  For some deployments this means the C: drive, along with the operating system and other application files.  With Exchange Server 2003 it was a simple enough task to relocate the MTA queue to a different volume (often a dedicated volume).  The MTA would also operate quite happily as long as it had at least 10mb of free disk space.  The new Exchange Transport queue database will by default require more than 4Gb of disk space, and you would size this volume according to your organisation’s requirements.

Exchange Server 2007 has a limited administrative GUI.  Most simple tasks are enabled in the GUI, but more advanced tasks are only available via the Exchange Management Shell (PowerShell).  Moving the Exchange Transport queue database is not even available in the shell – you need to directly edit a configuration file to make this change.

The config file is the EdgeTransport.exe.config file, located in the \bin folder of your Exchange Server 2007 program files installation (eg C:\Program Files\Microsoft\Exchange Server\bin).  By default this would mean the location of the queue database is “C:\Program Files\Microsoft\Exchange Server\TransportRoles\data\Queue”.

Open up the config file and look for these entries:

<add key=”QueueDatabasePath” value = “…” />
<add key=”QueueDatabaseLoggingPath” value = “…” />

Change the path to the new location where you would like to store the database.  Exchange will automatically create the new folder if it does not already exist, provided that the drive or parent folder has the correct permissions already.  These are:

• Administrators – Full Control
• System – Full Control
• Network Service – Full Control

You can also manually create the new location yourself.

If you are planning to just relocate the database without copying the existing database, then simply edit and save the file, and then restart the Microsoft Exchange Transport service.  If you want to retain the existing queue database, then you need to manually copy or move the files yourself, edit and save the file, and then restart the service.

If you have performed these tasks correctly and the new volume has adequate free disk space then the Exchange Transport agent will resume normal email processing.