Last modified: 2014-10-09 19:03:55 UTC
1.18 added a lot of new image metadata properties (Exif and friends) that are now extracted that weren't previously. Please run refreshImageMetadata.php after the 1.18 deployment, which will go through every image and re-extract all the metadata. Also, the script outputs the name of any image that had a larger img_metadata field before the refresh relative to after it. This could indicate a bug in new img_metadata stuff since almost all of the old properties are still extracted, so the img_metadata field should either stay the same or increase in size. (There is one exception to this, but it should be very rare). It'd be great if stdout of that script could be redirected to a file and posted somewhere, so I could investigate any cases of the metadata decreasing in size. Cheers.
Adding Reedy as CC since he has a clue what needs to be done here and says it needs some planning.
With --force ?
(In reply to comment #2) > With --force ? That's not necessary. Various version fields were incremented in 1.18 so the script will recognize the "old" metadata values as needing updating, which are the only one's that need to be updated.
It's in progress.
Still running ? Finished ? Halted ?
(In reply to comment #5) > Still running ? Finished ? Halted ? I believe its halted pending me fixing bug 31740 (which i plan to soon)
Yes, it was stopped for that bug, and I've now backported and deployed the bugfix and restarted the script.
It looks like it just crashed due to OOM, while doing commonswiki.
Core dumping indicates that the bulk of the leaked memory is in image metadata structures, such as XMP. It's possible that it is leaking LocalFile objects.
Has this been run successfully?
(In reply to comment #10) > Has this been run successfully? I imagine not, as no one has addressed the memory leaks
Was scheduled for 1.18 deployment. Adding to 1.20wmf3 deployment now.
(In reply to comment #12) > Was scheduled for 1.18 deployment. Adding to 1.20wmf3 deployment now. There's no point. The script needs fixing first
(In reply to comment #13) > (In reply to comment #12) > > Was scheduled for 1.18 deployment. Adding to 1.20wmf3 deployment now. > > There's no point. The script needs fixing first Aha, I didn't read the previous comment right. Makes sense now.
(In reply to comment #6) > (In reply to comment #5) > > Still running ? Finished ? Halted ? > > I believe its halted pending me fixing bug 31740 (which i plan to soon) That bug has been resolved as fixed in the mean time. (In reply to comment #9) > Core dumping indicates that the bulk of the leaked memory is in image metadata > structures, such as XMP. It's possible that it is leaking LocalFile objects. (In reply to comment #13) > (In reply to comment #12) > > Was scheduled for 1.18 deployment. Adding to 1.20wmf3 deployment now. > > There's no point. The script needs fixing first What has to be fixed still? Can a new bug be opened please as blocker to this one. Other wise, how about another run?
> What has to be fixed still? Can a new bug be opened please as blocker to this > one. Other wise, how about another run? The script appearently has memory leaks. Personally I'm not sure how or why that happens in a garbage collected language like php (furthermore one needs a test wiki with more than 10 files on it to really reproduce that issue), so fixing is a bit beyond me.
(In reply to comment #15) > > The script needs fixing first > > What has to be fixed still? Can a new bug be opened please as blocker to this > one. Other wise, how about another run? Could somebody please specify in a new bug report what's need to fix (any pointers etc) in the script, and make it block this report? High priority for 20 months & no changes for 9 months => resetting to normal.
High priority was probably an exageration, and its especially not now. As people notice a file without metadata, they can purge it if it bothers them. As for fixing the bug. Im not really sure how or why there is a mem leak in the script or how to debug it.
