The anti-spam agent installation process with Exchange 2013 is similar to previous versions of Exchange. When you install anti-spam agents on Exchange 2013 servers, most agents will be installed on the mailbox role but not the Connection filtering agent, also known as RBL, DNS Block List, etc.
The PowerShell script install-AntispamAgents.ps1 will look for which server role is installed and will not install Connection filtering if the server holds the mailbox role. This is understandable since SMTP connection should come in from the CAS server and then the original sending IP will not be show since CAS do Source-NAT. So the logic would be to install the connection filtering agent on CAS. However the install script will not let you do that either. Connection Filtering will only install on Edge role.
I can only speculate why this is the case. Either Microsoft wants it to be like this or they have found some trouble with the Connection Filtering Agent running on CAS.
I figured I will give this a try anyway, and here is how you get it to work:
Start Exchange Management Shell as administrator.
Change Directory to scripts folder.
cd $exscripts
Install the agent.
Install-TransportAgent -Name "Connection Filtering Agent" -TransportService FrontEnd -TransportAgentFactory "Microsoft.Exchange.Transport.Agent.ConnectionFiltering.ConnectionFilteringAgentFactory" -AssemblyPath "C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\agents\Hygiene\Microsoft.Exchange.Transport.Agent.Hygiene.dll"
If you have multiple agents running on the frontend transport you must set them in the correct order with the priority parameter
Add a IPBlocklistprovider of your choice
Add-IPBlockListProvider -Name zen.spamhaus.org -LookupDomain zen.spamhaus.org -AnyMatch $true -Enabled $true
You can add more than one provider if you like. If you don’t provide a custom response it will be “Recipient not authorized, your IP has been found on a block list”
Enable the agent
Enable-TransportAgent -TransportService FrontEnd -Identity "Connection Filtering Agent"
Restart FrontEnd transport service
Restart-Service MSExchangeFrontEndTransport
Now the agent should be live and kicking. Logging for the frontend agent is here
“C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\FrontEnd\AgentLog”
instead of the directory for the backend transport
C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\AgentLog”
Since the script doesn't install the Connection filtering agent on CAS it is probably unsupported to install the agent manually, but I had it running for months without any problem so make your own judgment.