Last modified: 2013-06-26 14:39:58 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T51604, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 49604 - Run an A/B split test for VisualEditor enabled by default on enwiki for new accounts
Run an A/B split test for VisualEditor enabled by default on enwiki for new a...
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
unspecified
All All
: Normal enhancement (vote)
: ---
Assigned To: James Forrester
http://thread.gmane.org/gmane.science...
: shell
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-15 01:58 UTC by James Forrester
Modified: 2013-06-26 14:39 UTC (History)
9 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description James Forrester 2013-06-15 01:58:12 UTC
This is to run from 19 June (end on with the Notifications split test, so we don't much their data up).
Comment 1 Gerrit Notification Bot 2013-06-15 02:09:13 UTC
Related URL: https://gerrit.wikimedia.org/r/68845 (Gerrit Change I5c603ece309d61641d32ccc9eff5ea2890d5b816)
Comment 2 p858snake 2013-06-15 02:10:31 UTC
Has en.wikipedia been notified?
Comment 3 Gerrit Notification Bot 2013-06-15 02:12:47 UTC
Related URL: https://gerrit.wikimedia.org/r/68846 (Gerrit Change I94d74fd8abeba465f1f9463890541e8c8f786a43)
Comment 4 MZMcBride 2013-06-15 02:21:04 UTC
(In reply to comment #2)
> Has en.wikipedia been notified?

This would be good.

From what I can tell, this test only applies to new users, though it's difficult to figure that out based on the current Bugzilla subject text or the related commit. It currently reads as though this will be an A/B on all users, not just new users.
Comment 5 MZMcBride 2013-06-15 02:27:11 UTC
It seems like auto-opting in a user also requires an auto-opt out mechanism. Otherwise, this is a dramatic departure from user expectations. As I understand it, anonymous users currently get the wikitext editor. Nearly every MediaWiki installation, including the one the user was just using use the standard wikitext editor, so suddenly changing that needs to be done cautiously.

Is there currently an auto-opt mechanism? If not, I consider that a blocker to this bug being resolved.

While I think it's ethically questionable to run experiments on users in this manner, one safeguard is ensuring that it's as easy as possible for users to _escape_ such experiments. Within the VisualEditor interface (and possibly elsewhere in the user interface), there should be one-click "I didn't ask for this" option to disable VisualEditor. In addition to an option on the editor itself, other areas of the site (such as the post-login screen) could be modified. "Welcome to Wikipedia! You've been selected to test our beta software. [opt out]" or similar.
Comment 6 MZMcBride 2013-06-15 03:23:46 UTC
(In reply to comment #0)
> This is to run from 19 June (end on with the Notifications split test, so we
> don't much their data up).

June 19 is a few days away. Given some of the recent feedback at [[Wikipedia:VisualEditor/Feedback]] regarding severe bugs in VisualEditor, it doesn't seem like VisualEditor is ready to be deployed more widely.
Comment 7 NicoV 2013-06-15 08:52:33 UTC
I fully agree with MZMcBride :
* I think it's ethically questionable to run experiments on users in this manner
* There are still severe bugs in VisualEditor at this time, so it shouldn't be deployed more widely

And I've more concerns about this kind of experiments :
* There are still a lot of minor/major bugs in VisualEditor at this time, I don't expect new users to have to handle them when discovering Wikipedia edition.
* There are still major limitations on what can be done with VisualEditor (templates, references, ...). What will be the experience for new users if they can only edit some parts of articles ?

I believe this kind of experiment should only be done when VisualEditor is fully functional (when users can do almost every modification they want) and without major bugs left.
Comment 8 MZMcBride 2013-06-19 02:53:09 UTC
From <https://en.wikipedia.org/w/index.php?title=Wikipedia:Administrators%27_noticeboard&oldid=560544063#VisualEditor_-_A.2FB_test_launch_on_18_June>:

---
Hey all. We're going to postpone the A/B test for several days; I'll post more details as I get them, but I understand it's largely down to known bugs with the existing software - bugs that you reported, and bugs that were crucial in making a go/no go decision. Thank you to everyone for all your hard work poking at the VE, and for all your reports thus far; it's much appreciated :). Okeyes (WMF) (talk) 19:25, 18 June 2013 (UTC)
---

Thanks to Legoktm for pointing this out to me, otherwise I likely never would've seen it.
Comment 9 James Forrester 2013-06-24 17:11:32 UTC
This is now deployed and will run for a week or so to help advise on whether the main switch should be thrown for all users. Data to be published as we get it.
Comment 10 MZMcBride 2013-06-24 18:24:13 UTC
https://en.wikipedia.org/w/index.php?title=Minnesota&curid=19590&diff=561394244&oldid=561292962

Dirty diff, HTML comments being stripped, etc.

Will the VisualEditor team be cleaning up these edits?

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links