Last modified: 2014-11-17 09:21:24 UTC

Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 61876 - Attach all broken autocreated local accounts to their global account
Attach all broken autocreated local accounts to their global account
Status: RESOLVED FIXED
Product: Wikimedia
Classification: Unclassified
Site requests (Other open bugs)
unspecified
All All
: High normal with 1 vote (vote)
: ---
Assigned To: Kunal Mehta (Legoktm)
:
Depends on: 67350 broken-sul
Blocks: 16660 29782 sulfinalization 39817
  Show dependency treegraph
 
Reported: 2014-02-24 21:10 UTC by Nemo
Modified: 2014-11-17 09:21 UTC (History)
13 users (show)

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments
MetaWiki autocreated, detached accounts (1.26 MB, text/plain)
2014-02-24 21:10 UTC, Nemo
Details

Description Nemo 2014-02-24 21:10:59 UTC
Created attachment 14675 [details]
MetaWiki autocreated, detached accounts

See bug 29234, bug 35792, bug 39060, bug 39996, bug 39997 and friends for context.

One example silly query one can run e.g. on toolserver (sql metawiki_p):

SELECT
  log_title,
  log_user,
  log_timestamp
FROM logging
JOIN user ON logging.log_action = 'autocreate' /*
AND log_timestamp > 20130431000000 */
AND logging.log_user = user.user_id
WHERE user.user_name NOT IN (
  SELECT lu_name
  FROM centralauth_p.localuser lu
  WHERE lu.lu_wiki = 'metawiki'
);

15k results on Meta-Wiki only.

An actual query needs to be made slightly faster, I suppose, and actually check for the validity of the local account (it can be attached but nevertheless unusable, with password and email not working) and of its global username (which might have been deleted, or detached on purpose).

Note, at some point James said this would be done as part of the work for bug 35707.
Comment 1 Kunal Mehta (Legoktm) 2014-07-05 00:01:55 UTC
Discussed with hoo and Nemo in #wikimedia-tech just now, if the user is unattached, has no edits, and has an empty user_password & user_email, we should be able to safely attach them.

If the user has a non-empty user_password or user_email, they should be able to either login or force a password reset.
Comment 2 Nemo 2014-07-05 08:44:20 UTC
(In reply to Kunal Mehta (Legoktm) from comment #1)
> if the user is
> unattached, has no edits, and has an empty user_password & user_email, we
> should be able to safely attach them.

Right. So this is the very first thing to do even before bug 39817 comment 10 or all the rest gets too muddy. (Cf. https://www.mediawiki.org/wiki/Admin_tools_development/SUL_Audit/Metrics_20140704 .) Would be worth an "Highest", who wants to add this feature to the script(s)?
Comment 3 Gerrit Notification Bot 2014-07-05 09:15:57 UTC
Change 144257 had a related patch set uploaded by Legoktm:
Add --attachbroken option to migrateAccount.php

https://gerrit.wikimedia.org/r/144257
Comment 4 Kunal Mehta (Legoktm) 2014-07-05 09:52:22 UTC
I made a fork of sul-audit.py (source: <https://gist.github.com/legoktm/85192e897d57c89aad56>) to detect broken accounts, it's running on terbium right now.
Comment 5 Kunal Mehta (Legoktm) 2014-07-05 10:50:16 UTC
Script took ~43m:

legoktm@terbium:~$ cat sul/broken-accounts.tsv | wc -l
18571
Comment 6 Kunal Mehta (Legoktm) 2014-07-05 23:11:19 UTC
(In reply to Kunal Mehta (Legoktm) from comment #5)

> legoktm@terbium:~$ cat sul/broken-accounts.tsv | wc -l
> 18571

For accounts with a user_editcount of 0:

legoktm@terbium:~/sul$ cat broken-no-ec.tsv | wc -l
18347
Comment 7 Gerrit Notification Bot 2014-07-05 23:12:32 UTC
Change 144257 merged by jenkins-bot:
Add --attachbroken option to migrateAccount.php

https://gerrit.wikimedia.org/r/144257
Comment 8 Kunal Mehta (Legoktm) 2014-07-08 20:23:58 UTC
2014-07-08 20:21:23 processed 18347 usernames (29.3/sec), 166 (0.9%) fully migrated, 17957 (97.9%) partially migrated

I'll look through the log to see why some accounts didn't get processed.
Comment 9 Kunal Mehta (Legoktm) 2014-07-12 01:56:40 UTC
(In reply to Kunal Mehta (Legoktm) from comment #8)
> 2014-07-08 20:21:23 processed 18347 usernames (29.3/sec), 166 (0.9%) fully
> migrated, 17957 (97.9%) partially migrated
> 
> I'll look through the log to see why some accounts didn't get processed.

I don't see any errors in the log. I think the count is off because some users had multiple unattached accounts, but as soon as the script processed the first of their usernames, it fixed all of them, so the later instances of those names were already "ok" and had nothing to do.

I'm going to mark this as fixed now. If the 224 users whose accounts we couldn't automatically re-attach would like their accounts back (and have a reasonable way to prove they owned it), I can take care of those manually (just file a bug and cc me). Otherwise they'll just be renamed out of the way during the finalization.
Comment 10 Kunal Mehta (Legoktm) 2014-07-12 07:25:41 UTC
Ok, turns out this isn't fixed yet. There are about 1000 still broken accounts that the script could fix, but CA doesn't think they exist due to bug 67350.
Comment 11 Kunal Mehta (Legoktm) 2014-07-14 09:45:56 UTC
2014-07-14 09:43:06 processed 1014 usernames (24.9/sec), 3 (0.3%) fully migrated, 1011 (99.7%) partially migrated

Running a script to verify there are no more of these left.
Comment 12 Kunal Mehta (Legoktm) 2014-07-14 10:39:30 UTC
(In reply to Kunal Mehta (Legoktm) from comment #11)
> 2014-07-14 09:43:06 processed 1014 usernames (24.9/sec), 3 (0.3%) fully
> migrated, 1011 (99.7%) partially migrated
> 
> Running a script to verify there are no more of these left.

Aargh. I ran migrateAccount.php --attachbroken using a list I generated 2 days ago. I re-generated the list, expecting it to be empty. There are now 7 usernames, all from commonswiki that are broken. They all have user_registration dates in the past 2 days, so it just means new broken accounts are being created.
Comment 13 Andre Klapper 2014-08-05 22:25:19 UTC
(In reply to Kunal Mehta (Legoktm) from comment #12)
> Aargh. I ran migrateAccount.php --attachbroken using a list I generated 2
> days ago. I re-generated the list, expecting it to be empty. There are now 7
> usernames, all from commonswiki that are broken. They all have
> user_registration dates in the past 2 days, so it just means new broken
> accounts are being created.

legoktm: Did you have some time to follow up on this? Is this still an existing problem, or has this been fully tackled?
Comment 14 Kunal Mehta (Legoktm) 2014-08-06 11:59:05 UTC
The problem is that new broken accounts are still being created (bug 39996), so until that is fixed, we can't really fix all of them.
Comment 15 Nemo 2014-09-25 05:28:41 UTC
(In reply to Kunal Mehta (Legoktm) from comment #14)
> The problem is that new broken accounts are still being created (bug 39996),
> so until that is fixed, we can't really fix all of them.

Bug 39996 was closed, so the last maintenance script run should be the last!
Comment 16 Kunal Mehta (Legoktm) 2014-10-11 00:41:23 UTC
List of broken users is generating now...
Comment 17 Kunal Mehta (Legoktm) 2014-10-15 21:28:38 UTC
2014-10-15 21:02:37 processed 771 usernames (22.6/sec), 1 (0.1%) fully migrated, 770 (99.9%) partially migrated
Comment 18 Kunal Mehta (Legoktm) 2014-10-17 22:52:47 UTC
Forgot to close this :)

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links