404 errors

["Jason" <judgej (at) xaraya.com>]

Just a note, in case it can be fed into any design decisions.

When pages in Xaraya are accessible, perhaps an article is been pulled or 
expired, the user is normally presented with an error page listing the 
problem. That is fine for users, but makes the search engines' work much 
harder.

I think we need better HTTP code support somewhere in the core. If a module 
believes an item being displayed is no longer available, then it should be 
able to return a 404. Similarly, permissions errors, moved items, unexpected 
errors etc. should all return the appropriate HTTP code. This does no mean 
the user error should not be displayed, as that can be displayed as well.

I'm discovering this as I'm setting up a search engine for a large site 
(mnoGoSearch), and modifying various modules to return the appropriate 
codes. The search engine can quickly determine that a page has now gone, and 
remove it from its database, so long as a 404 is returned.

This would also explain why I keep getting MSNbot constantly coming around 
to pages that have been removed from the site years ago. The bots just don't 
know the pages have actually gone, and happily index the error page.

There are similar issues with the HTTP dates in the headers, which need to 
be more closely coupled with the documents that are being displayed in the 
pages.

Anyway - I've no quick fix, but just thought I'd highlight this area for 
consideration.

-- JJ


_______________________________________________
Xaraya_devel mailing list
Xaraya_devel (at) xaraya.com
http://xaraya.com/mailman/listinfo/xaraya_devel