Enforcing a Custom 404 Page in ASP.NET

Chris Love

Last Updated - Thu Nov 23 2017

Yesterday Google started hijacking 404 status codes through the Google Toolbar from unknowing Web sites to show pages with ads. TechCrunch even has a poll asking if you think the Google hijacking is a good thing or not. While Google publicly states they do not like Made For Adsense sites (MFAs) they have gone out of their way to force their own where unknowing Web site owners have missed a basic task in designing their site. While I think the stated purpose sounds good, helping users have a better online experience, is a good one.

I don't want to single Google out as the only bad guy. There are many other toolbars and ISPs that are taking advantage of every little mistake developers and designers make with their public sites. I noticed Road Runner is doing the exact same thing. Shame on them all, but it is a wake up call for us to do our due diligence as web developers.

According to Matt Cutts Blog entry about their custom 404 pages, Google will not display their ads if the error page you serve is larger than 512 bytes, which means this should be a fringe occurrence for now. (Just as an aside, if you are not a part of the Search Engine World, Matt Cutts is the equivalent to Scott Guthrie). But like all liberal legislators I am sure this policy will eventually creep to cover more of our lives.

This type of practice has long been the territory of predatory hosting companies for years. In fact many used to inject their own promotions into your site's content, but that has long past away as useless. So I do not fault Google for bringing back an old practice. Regardless we should be responsible Web property owners and provide a good user experience on our own.

In the ASP.NET/IIS world there are three ways to create a custom 404 page, in the IIS configuration, in the web.config file or with a custom httpModule. The first two are preferable and simple. A custom httpModule is an advanced method.

Setting up IIS 404 Custom Error Page

First you can configure this in IIS itself. In IIS 6.0 this is done through the Custom Errors page. Scroll through the list of error codes and point the error response to either the file or URL you want to display.

Click for full image

In IIS 7 you open the Management Console and move to the site that needs customized. You will see an Errors Icon in the list of items in the center of the console. Double clicking it reveals a basic list of HTTP Error Status Codes, select the 404 code. Then click Edit in the top right of the console to access the edit dialog. Here you can set the file or URL you want to have displayed.

Defining HTTP Status Code Error Responses in the web.config

We could simply use the custom error code mapping built into ASP.NET to map our 404 status code to a custom page, which is the simple thing to do. Just about every ASP.NET programmer will have access to do this.

<

customErrorsmode="RemoteOnly"defaultRedirect="GenericErrorPage.htm">

<

errorstatusCode="403"redirect="NoAccess.htm" />

<

errorstatusCode="404"redirect="FileNotFound.htm" />

</

customErrors>

 

Creating a Custom 404 Error Handler Custom Module

Finally is you do not own your server and do not have administrative access to your server you can programatically designate the 404 response page. This can be done with a custom httpModule. A custom module can be used to perform URL Rewriting and custom error handling. By using both we can now control what happens when a missing resource is requested.

Another thing we can control that will throw Google off the scent is to send a 302 or temporarily moved status code to the client. Since we are programmatically controlling our site's response to a bad URL we can do this as well. I think this would be even more beneficial, but only if we plan on being a good steward and eventually fixing the issue altogether with a valid resource, or eventually creating a permanent or 301 redirect for this missing item. Using a custom httpModule is a great option for an ASP.NET site to perform these tasks.

To do this you need to create a custom httpModule class, which is simply a class that implements the IhttpModule interface. In the module's Init method you should register a custom Error event handler.

PublicSub Init(ByVal context As System.Web.HttpApplication) Implements System.Web.IHttpModule.Init

AddHandler context.Error, AddressOf ErrorHandler

EndSub

In this handler you should catch any 404 requests and change the response to custom content. This routine can be simple, like the one shown, or as complex as you like. Unfortunately at this point in the processing pipeline we cannot rewrite the request to another page, so we have to clear the text and inject our custom message.

Dim

request As HttpRequest = DirectCast(sender, HttpApplication).Request

Dim context As HttpContext = DirectCast(sender, HttpApplication).Context

'Create a regular expression to look for 404 errors.

'Since the status code at this point is 200 we need to

'examine the error message to see what we have exactly.

'Regular expressions give us a strong programtic way

'to do this, and can be expanded for many scenarios.

Dim rx AsNew Regex("The file '[^\']*' does not exist.")

Dim mc As MatchCollection = rx.Matches(context.Error.Message.ToString)

If mc.Count > 0 Then

'First Clear the Error

context.Server.ClearError()

'We cannot redirect the request to another resource at this point.

'So Clear the text being sent to the browser.

context.Response.Clear()

Dim root AsString = context.Request.PhysicalApplicationPath

'Read a standard error page we have in our site

'display that to the user by writing it to the response stream.

context.Response.Write( _

My.Computer.FileSystem.ReadAllText( _

Path.Combine(root, "notfound.htm")))

End If

Also realize in IIS 7 you can set up custom modules inside of the server, not just the site. So this custom handler could offer a global solution for you as well.

Summary

Toolbars are a very sneaky way for Google and many other companies to track usage on your site and in come cases even control how your site behaves. It is important for you as a Web site administrator to control every aspect of your site and the way it handles every scenario.