# Wikimedia Bugzilla is closed!

Wikimedia has migrated from Bugzilla to Phabricator. Bug reports should be created and updated in Wikimedia Phabricator instead. Please create an account in Phabricator and add your Bugzilla email address to it.
Wikimedia Bugzilla is read-only. If you try to edit or create any bug report in Bugzilla you will be shown an intentional error message.
In order to access the Phabricator task corresponding to a Bugzilla report, just remove "static-" from its URL.
You could still run searches in Bugzilla or access your list of votes but bug reports will obviously not be up-to-date in Bugzilla.
Bug 11103 - Lucene search 2 uses GNU-specific cp options when snapshotting indexes
 Summary: Lucene search 2 uses GNU-specific cp options when snapshotting indexes
 Status: Product: RESOLVED FIXED Wikimedia Unclassified Component: lucene-search-2 (Other open bugs) unspecified All All Normal enhancement (vote) --- Nobody - You can work on this! Show dependency tree / graph

 Reported: 2007-08-28 20:49 UTC by Brion Vibber 2013-03-26 11:25 UTC (History) 2 users (show) rainman wikibugs-l --- --- ---

Attachments

 Brion Vibber 2007-08-28 20:49:23 UTC Testing a local copy of the lucene-search-2 module on my MacBook, I noticed this error dropping out while building the indexes: 2383 [main] INFO org.wikimedia.lsearch.index.IndexThread - Making snapshot for trunkwiki 2412 [main] WARN org.wikimedia.lsearch.util.Command - Got exit value 64 while executing /bin/cp -lr /opt/web/lsearch/import/trunkwiki/_8t.cfs /opt/web/lsearch/snapshot/trunkwiki/20070828164041/_8t.cfs 2414 [main] ERROR org.wikimedia.lsearch.index.IndexThread - Error making snapshot /opt/web/lsearch/snapshot/trunkwiki/20070828164041: Error executing command: cp: illegal option -- lusage: cp [-R [-H | -L | -P]] [-f | -i | -n] [-pv] src target cp [-R [-H | -L | -P]] [-f | -i | -n] [-pv] src1 ... srcN directory The -l option to cp seems to be a GNU extension, and isn't present in the Mac OS X version of cp. Needless to say, /bin/cp may also not be present on non-Unix systems. ;) cp -lr is also used in UpdateThread, as is rm -rf and /usr/bin/rsync (if rsync is in use). It might be wise to make the snapshots in a more portable way, using portable file-manipulation functions. The rsync path should be customizable; in theory it might be nice if it runs on Windows as well as name-brand *nixes. Robert Stojnic 2007-08-29 00:28:27 UTC Currently lsearch requires linux. It could work on MacOS since it afaik support hard and soft links, with some minor modification. Windows is more difficult, one would probably need something like info files that would describe the links, or just use a slower copy, instead of linking. So, the above cp would be: cp + ln. Hopefully, in some next release it should have customizable paths, and should be able to detect the OS, and use the most efficient solution for it.  Brion Vibber 2007-08-29 20:08:07 UTC Creating the destination directory, then shelling out to ln to link each file should be compatible with any UNIXy system (Linux, Mac OS X, *BSD, Solaris, etc) and not a very difficult change, so that'd be my recommendation for a short-term change. Windows does support hardlinks of some sort on NTFS, though I couldn't tell you if it's worth the bother. :) I did a little googling yesterday for Java code supporting creation of hardlinks; this util class from an Apache project has a function which allegedly can create hardlinks on either Unix or Windows by shelling out the appropriate low-level command: http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/fs/FileUtil.java?view=markup Haven't tested it meself. :) Rob Church 2007-08-29 20:11:45 UTC (In reply to comment #2) > Windows does support hardlinks of some sort on NTFS, though I couldn't tell you > if it's worth the bother. :) NTFS Junction Points? No, it's not worth the bother, and there's no standard toolkit to create or manage them - Windows uses them internally, and exposes them via Win32, but doesn't ship with any kind of command-line utility comparable to ln. Brion Vibber 2007-08-29 20:18:01 UTC *ahem* C:\Documents and Settings\brion> fsutil hardlink create bar.txt foo.txt Hardlink created for C:\Documents and Settings\brion\bar.txt <<===>> C:\Documents and Settings\brion\foo.txt :) Rob Church 2007-08-29 20:27:34 UTC Well, in my defence, what I said was more or less accurate; that's not an NTFS Junction Point at all. What's actually indefensible here is that I read "hardlink" and thought of the (dodgy) equivalent of a symlink. Points off all round. Robert Stojnic 2007-10-02 23:15:46 UTC Fixed in r26333.  Andre Klapper 2013-03-26 11:25:13 UTC [Merging "MediaWiki extensions/Lucene Search" into "Wikimedia/lucene-search2", see bug 46542. You can filter bugmail for: search-component-merge-20130326 ]

 Note You need to log in before you can comment on or make changes to this bug.