Exchange Server 2007: Managing And Filtering Anti-Spam Agent Logs
by Bharat Suneja
Exchange 2007 includes a number of anti-spam agents to filter spam. The anti-spam agents log their actions in (anti-spam) agent logs. The default agent log locations:
- Exchange 2010: \Exchange Server\V14\TransportRoles\Logs\AgentLog
- Exchange 2007: \Exchange Server\TransportRoles\Logs\AgentLog
Agent Log Configuration
You can t change the agent log location. Here are the available config options:
- Enable/Disable agent log: On transport servers with anti-spam agents installed, agent logging is enabled by default. You can disable it by adding the AgentLogEnabled key, and setting it to FALSE.
- Max file size: By default, the maximum file size of each agent log is 10 Mb (10485760 bytes). You can modify it by adding AgentLogMaxFileSize key and providing the desired value (in bytes). Parsing very large log files can be slower the default max file size works for most deployments.
- Max directory size. By default, the size of the directory is capped at 250 Mb (262144000 bytes). You can modify it by adding the AgentLogMaxDirectorySize key and providing the desired value (in bytes). In a high volume environment, you must consider the volume of log files generated in a day, and determine the space required based on how many days worth of log files you want to retain for troubleshooting or to meet your organization s retention requirements.
- Agent log age: The age of a log is capped at 30 days. You can modify it by adding the AgentLogMaxAge key and providing the desired value. Logs are flushed when they re older than the AgentLogMaxAge. or if the directory size reaches the AgentLogMaxDirectorySize (250 Mb by default), whichever happens first. If it s important to maintain agent logs for a certain number of days, you must consider the volume of logs generated daily, and raise the AgentLogMaxDirectorySize if required.
Exchange 2007 RTM has only one configuration option for the agent log that to enable or disable it. Exchange Server 2007 SP1 includes the additional configuration parameters listed above to control the max directory size, file size and age of agent logs added.
The agent log configuration parameters can be controlled by editing the EdgeTransport.exe.config file, located in \Exchange Server\Bin folder on Edge and Hub Transport servers.
Note, the key names in EdgeTransport.exe.config are case-sensitive.
Disable agent logging
To disable agent logging, insert the following key under appsettings /appsettings in the config file:
add key= AgentLogEnabled value= FALSE /
Modify agent log configuration
In the following example, we modify max directory size to 500 Mb, file size to 20 Mb, and age to 60 days, by creating new keys in EdgeTransport.exe.config :
add key= AgentLogMaxDirectorySize value= 524288000 /
add key= AgentLogMaxFileSize value= 20971520 /
add key= AgentLogMaxAge value= 60.00:00:00 /
Parsing the agent log
You can parse the agent log using the Get-AgentLog command from the shell.
By default, the Get-AgentLog cmdlet parses agent logs in the default location ( \Exchange Server\TransportRoles\Logs\AgentLog ). Depending on your troubleshooting and log retention requirements, you can move older files to an another location. To parse agent logs in an alternate location, you must specify the path. In this example, the agent logs have been copied to Z:\Antispam Agent Logs directory:
Get-AgentLog -location Z:\AntiSpam Agent Logs
Agent log format and fields
Agent log file names are named AGENTLOGyyyymmdd-nnnn.log. It s easy to decipher yyyy stands for the year, mm is the month, and dd is the date. The nnnn is a serial number, starting at 0001, rolled over when more than one log is generated in a day. The date is UTC.
Each agent log starts with the following header fields:
#Software: Microsoft Exchange Server
#Log-type: Agent Log
Here s what an entry in the agent log looks like, as output by the Get-AgentLog cmdlet. Note, not all fields are displayed by default when you use the cmdlet:
Timestamp. 4/16/2007 12:39:49 AM
P1FromAddress. [email protected]
Recipients. [email protected] >
Agent. Connection Filtering Agent
SmtpResponse. 550 5.7.1 Recipient not authorized, your IP has been found on a block list
ReasonData. Spamhaus SBL-XBL
As seen in the above output, the logs provide adequate information for reporting on anti-spam activity, as well as for troubleshooting anti-spam issues like messages not being received/wrongly filtered out.
Getting to know the agent logs will make troubleshooting such issues much easier.
Searching agent logs
By default, the Get-AgentLog command returns all the entries in the agent logs. This can take a long time and results in all entries scrolling by quickly in your shell window. The only filtering options available with the Get-AgentLog cmdlet are start and end dates.
Filter by date and time
Agent logs can be constrained to a particular date and time the recommended way to perform most agent log searches, unless you want to immerse yourself in 30 days (or 250 Mb) of anti-spam goodness! This is done using the -StartDate and -EndDate parameters, as shown in this example:
Get-AgentLog -StartDate 4/16/2007 -EndDate 4/17/2007
You can also constrain it further by adding time of the day :
Get-AgentLog -StartDate 4/17/2007 8:00 AM -EndDate 4/17/2007 2:00 PM
Search messages by recipients and senders
Although the Get-AgentLog command only takes these 3 parameters location, StartDate, and EndDate, you can further filter the logs using most of its logged fields by using pipelining.
To filter the log to show messages to a particular recipient :
To search for messages from a particular sender :
Search messages from a sender domain
To search for messages from a particular domain :
Search messages filtered by a specified anti-spam agent
To filter by the anti-spam agent that acted on a message, e.g. Connection Filtering Agent:
Similarly, you can filter by other agents that write to the agent logs: 1) Content Filter Agent 2) SenderID agent 3) Sender Filter agent 4) Recipient Filter agent and 5) Edge Rules agent.
Searching by IP address
To filter agent logs by the sending host s IP address. use the following command:
Get-AgentLog -StartDate 4/15/2007 -EndDate 4/17/2007 | where <$_.IPAddress -eq 126.96.36.199 >
Search messages blocked by RBLs
The reason field in each log entry specifies the reason supplied by the anti-spam agent that takes the action. For instance, as seen in the agent log entry shown earlier in this article, the agent that acted on the message is the Connection Filtering Agent. the reason is BlockListProvider (i.e. RBL or Real-time Block List , known as IP Block Lists in Exchange Server 2007). The ReasonData field gives you the name of the IP Block List Provider. as configured in Exchange. In the above agent log entry, it is Spamhaus SBL-XBL . To constrain the search for messages blocked by IP Block List Providers :
Get-AgentLog -StartDate 4/15/2007 -EndDate 4/17/2007 | where <$_.Reason -eq BlockListProvider >
To get a list of all IP addresses blocked by IP Block List Providers :
Get-AgentLog -StartDate 12/21/2007 | where <$_.Reason -eq BlockListProvider > | ft Timestamp,IPAddress,ReasonData
You can also look for messages blocked by a particular IP Block List Povider :
Search by SCL thresholds
For messages scanned by the Content Filter Agent, the Reason field contains details like SCLAtOrAboveDeleteThreshold, SCLAtOrAboveRejectThreshold. etc. The ReasonData field contains the SCL value assigned to the message. To get a list of messages above the SCLDeleteThreshold. use the following command:
As shown in the above examples, you can use the Get-AgentLog command and pipe the data to filter it based on the fields logged. You can get more details about agent logs including the fields logged, from the Managing Agent Logging section in Exchange Server 2007 documentation.
2/9/2008: Added sidebar about additional configuration parameters available in SP1.