Last modified: 2014-11-18 15:20:25 UTC
* Update term class to not have a numeric entity id * Provide a migration script for wb_terms
Rationale: we are dropping the assumption that ids will always be prefix+number. For the current code and use case, wikidata.org, this works fine, but we need to migrate away from this in order to support things like meta-data storage on commons.
https://gerrit.wikimedia.org/r/#/c/101197/
I fixed compat with sqlite and several other issues. The tests now pass: https://gerrit.wikimedia.org/r/#/c/114490/ The commits are doing some stuff I don't like, though that can be fixed after we got rid of the main issue, the bad assumption in the table, which this commit fixes.
Springle wrote at https://gerrit.wikimedia.org/r/#/c/101197/ This one still seems dangerous to me :-) I understand the reason for the change, however please do also consider: 1. Have we done any real profiling of the new query forms against the production dataset? I'd really like to see how much of an impact this has on data and index disk usage, and more importantly on runtime memory usage. Happy to do this if a Dev can generate a few thousand samples of each query type... 2. Would it be wise to keep a numeric entity id field as an interim step on the wikidata production dataset, so we can fail back if needs be? Ie, treat this as a denormalization step (which is /all it is/ for now) until #1 is assured? That might even make the migration less painful. 3. VARBINARY(255) smells like an arbitrary size choice :-) Variable field widths really start to matter for large datasets as the server must convert it to fixed-width BINARY while working. If the choice /was/ arbitrary, can we arbitrarily choose to make this smaller from the get go?
