Last modified: 2011-03-13 18:04:27 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T4798, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 2798 - robots are instructed to index redirect pages
robots are instructed to index redirect pages
Status: RESOLVED WONTFIX
Product: MediaWiki
Classification: Unclassified
Redirects (Other open bugs)
unspecified
All All
: Lowest enhancement (vote)
: ---
Assigned To: Nobody - You can work on this!
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-07-11 13:35 UTC by Warren Stuart
Modified: 2011-03-13 18:04 UTC (History)
0 users

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Warren Stuart 2005-07-11 13:35:12 UTC
When I use the search engine (mnoGoSearch) to index my site I get lots of
duplicate pages caused by redirects. What I propose is that redirect pages have
the meta tag for robots changed to noindex. This might also help with search
engines indexing wikipedia and make them get a better index of the site.

current on all pages
<meta name="robots" content="index,follow" />
proposed for redirect pages
<meta name="robots" content="noindex,follow" />
Comment 1 Warren Stuart 2005-07-12 08:42:12 UTC
an quick example of this is:

http://en.wikipedia.org/wiki/Firefox

Which gives 

Mozilla Firefox
From Wikipedia, the free encyclopedia.
(Redirected from Firefox)

but the robots meta is
<meta name="robots" content="index,follow" />
Comment 2 Niklas Laxström 2005-07-23 23:56:40 UTC
Changing summary to reflect the problem and upping severity to normal
Comment 3 Ævar Arnfjörð Bjarmason 2005-07-24 00:16:16 UTC
Wouldn't this hurt rankings for alternate spellings of the title?
Comment 4 Christian Schmidt 2005-10-05 19:21:53 UTC
In order to allow alternate spellings to be indexed by search engines, the name
of redirect articles could be mentioned in a meta tag of the target article,
i.e. <meta name="keywords" content="...">.

I don't know whether meta tags are honoured by search engines anymore, though.
Comment 5 ABCD 2005-10-05 20:06:16 UTC
I think that maybe, if the User-Agent is that of a major search engine (e.g.
Google), then when requesting a redirect page, instead of the standard redirect,
the bot should recieve an HTTP 301, so that it will consider it to be the same
page as the target - reducing dups in the search results.
Comment 6 lɛʁi לערי ריינהארט 2005-10-05 20:25:30 UTC
(In reply to comment #4)
> In order to allow alternate spellings to be indexed by search engines, the name
> of redirect articles could be mentioned in a meta tag of the target article,
> i.e. <meta name="keywords" content="...">.
> 
> I don't know whether meta tags are honoured by search engines anymore, though.

this is the requirement of
bug 846: feature request: control of meta name="KEYWORDS" content="..."
Comment 7 Rowan Collins [IMSoP] 2005-10-06 00:17:53 UTC
(In reply to comment #5)
> I think that maybe, if the User-Agent is that of a major search engine (e.g.
> Google), then when requesting a redirect page, instead of the standard redirect,
> the bot should recieve an HTTP 301

I may be wrong about this, but I remember hearing that Google make spot checks
of some kind with a user-agent of something like IE, to penalise sites that send
completely different "optimised" content to the main crawler. 

Comment 8 Brion Vibber 2006-01-22 04:25:04 UTC
Restored bug from flood attack.
Comment 9 Brion Vibber 2006-01-22 04:25:36 UTC
Closing this WONTFIX; this behavior is deliberate, so titles can be searched on.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links