I recently noticed that I was having an issue with WordPress creating duplicate url’s that were being indexed by Google. Duplicate url’s in search engine results is what is known as duplicate content, which can penalize you in search engines. The url’s were being formatted like this.
Here is the original url.
Both of these url’s were being indexed by Google. I immediately recognized that this could cause a problem with duplicate content, since the content of those pages are exactly the same. Searching my site, I noticed these url’s were being linked to from the Recent Comment widget in the sidebar.
A new feature in WordPress 2.7 is the pagination of comments. I’m not sure if this is turned on by default or not. Checking my options, I saw that this was turned on. Turning it off solved the problem. The widget now links to the normal url’s. The remaining issue is that duplicate url’s are now in the search results. To get rid of them, I added a line to my robots.txt file preventing them from being indexed. Within a few weeks, those duplicate url’s should be gone.
So if you are using WordPress 2.7, you should check your search engine results for these duplicate url’s. If it’s a problem, turning off comment pagination should fix it.