Last modified: 2011-03-13 18:05:23 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 5707 - tell spiders not to index Wikipedia footer and navigation
tell spiders not to index Wikipedia footer and navigation
Status: RESOLVED WONTFIX
Product: MediaWiki
Classification: Unclassified
Parser (Other open bugs)
unspecified
All All
: Lowest normal with 1 vote (vote)
: ---
Assigned To: Nobody - You can work on this!
http://www.mediawiki.org/wiki/How_bes...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2006-04-24 21:49 UTC by S Page
Modified: 2011-03-13 18:05 UTC (History)
0 users

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description S Page 2006-04-24 21:49:24 UTC
1.  In Google, search en.wikipedia.org for 'privacy':
http://www.google.com/search?hl=en&q=site%3Aen.wikipedia.org%20privacy

Results:
153,000,000 results!

This is because "Privacy" is in the footer, so Google matches every page.

Expected:
Turn off indexing of common areas.  I added some notes on how to do this to
http://www.mediawiki.org/wiki/How_best_to_search_or_spider_mediawiki_systems and
http://en.wikipedia.org/wiki/Robots_Exclusion_Standard#Directives_within_a_page ,
for Google the key is <!--googleoff: index--> ... <tt><!--googleon: index--> and
old spiders use <NOINDEX>.

You could counter-argue that if a word appears on a page and the user pastes it
into a search engine, then the engine MUST find that page.  But I think the
value of eliminating all those search results outweighs this.
Comment 1 Antoine "hashar" Musso (WMF) 2006-05-01 20:41:39 UTC
* NOINDEX is not valid in XHTML.
* the google only comment is ... only for google. That would not really fix the
issue.
Comment 2 Mark Clements (HappyDog) 2006-05-02 02:24:06 UTC
Hmmm.... I think there are several million people who use Google for their
searches (call me naive...)
Comment 3 Rob Church 2006-05-14 04:06:08 UTC
(In reply to comment #2)
> Hmmm.... I think there are several million people who use Google for their
> searches (call me naive...)

The point as raised was that Google isn't the be-all and end-all; people can and
do use alternative search engines, which that particular special case wouldn't
affect. So yes, it would fix the problem for a lot of cases, but not all.
Comment 4 Antoine "hashar" Musso (WMF) 2007-05-01 20:50:43 UTC
Marking as WONTFIX, looks like google is smart enough to give
us back pages related to "privacy".

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links