Last modified: 2010-05-15 15:33:07 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 1191 - Database hot issues tracking, December 2004
Database hot issues tracking, December 2004
Product: MediaWiki
Classification: Unclassified
General/Unknown (Other open bugs)
All All
: Normal normal with 4 votes (vote)
: ---
Assigned To: Nobody - You can work on this!
Depends on:
  Show dependency treegraph
Reported: 2004-12-24 18:28 UTC by Jamesday
Modified: 2010-05-15 15:33 UTC (History)
2 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Description Jamesday 2004-12-24 18:28:10 UTC
Here are the current most significant issues for the Wikimedia database servers,
in approximate importance order. To qualify for this list it must be an issue
which I've noticed as a significant issue on the Wikimedia servers. If you're
after smallish tasks which can help performance, you'll probaby find some here... 

1. Making queries at the master instead of the slave when it is not an update
operation and it is not essential to have current data (getting the most recent
history or watchlist or recent changes is not essential). It is is not an update
it probably doesn't belong on the master. Few exceptions, like showing the
article history to the person who just edited it or showing an article from the
master after a brief master_pos_wait at the slave still failed to find it at
all. Anything but a complete failure should stay at the slave, because the usual
slave failure scenario makes _all_ slaves have out of date data and the master
can't handle the whole load of the site: going to the master would take the site
down. Tim Starling has been doing some work to help selectivity.

2. Search is using an inefficient query. A more efficient one (which sends data
in the query to avoid a join which makes it far, far slower) is in one of the
bug reports.

3. Watchlists and any other query which needs the title of an article, because
the current schema includes the text in cur. This is being changed, so these
aren't worth wondering about now.

4. Use of limit for paging where there may be tens of thousands of results to be
skipped to get to the right place. Applies to article histories, user
contributions, lists of what is in a category, any other place. If an item can
have tens of thousands of members, it must have efficient paging of some sort.

5. Not using reporting summaries. The query cache can't cache tables like cur
which are changed regularly for much time, because the first change removes the
query from the cache. Bbut many semi-static queries use that live data even when
the source is a report which has data produced in batch runs. Things like the
allpages special report. If those used a static reporting table for article
titles the query cache could be used, because it would only be updated
peridically and wouldn't be flushed from the cache regularly. For allpages,
using the live data is actually a bug, because new entris can be added which
cause missing entries when the number of articles in a range exceeds 500 because
of new articles. Often those who are not logged in just don't care about the
latest data and don't need to be shown, say, the latest list of members of a
category, when one from an hour ago is available and could be served from a
cachable reporting table.

6. Including fields which are changed far more often than the rest of the record
in the main tables. Things like cur_touhed in cur or the new patrolled field in
watchlist. This decreases the cachability of the data and increases the disk
write load, one of the medium term (6 months+) performance limits. MySQL stores
data in rows so forcing a larger row update than necessary is a usually a bad
idea. Task-specific tables and joins are good...:)

7, Updating fields in "hot" tables long after the original record was written.
The rc_old_id field is an example of this. It increases the lock contention rate
for the table. Try to put data updated later into a different table. This also
unncessarily increases the disk write load and decreases the number of records
which can be cached.
Comment 1 Antoine "hashar" Musso (WMF) 2005-12-09 19:45:50 UTC
Jamesday, is that one fixed ? :o)
Comment 2 MZMcBride 2008-10-20 06:39:43 UTC
Closing this as INVALID. It really seems that this would've been best to post to a mailing list originally. It claims to be a tracking bug but lists no dependencies or blocks. :-)

Additionally, at this point, the information here is outdated, inaccurate, and not serving any particular purpose that I can see.

Note You need to log in before you can comment on or make changes to this bug.