Last modified: 2010-10-08 17:01:47 UTC

Wikimedia Bugzilla is closed!

Wikimedia migrated from Bugzilla to Phabricator. Bug reports are handled in Wikimedia Phabricator.
This static website is read-only and for historical purposes. It is not possible to log in and except for displaying bug reports and their history, links might be broken. See T27205, the corresponding Phabricator task for complete and up-to-date bug report information.
Bug 25205 - Automate generation of fundraiser tracking data, long term storage, and data mining
Automate generation of fundraiser tracking data, long term storage, and data ...
Status: RESOLVED WORKSFORME
Product: Wikimedia
Classification: Unclassified
General/Unknown (Other open bugs)
unspecified
All All
: Normal enhancement (vote)
: ---
Assigned To: Nimish Gautam
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2010-09-18 00:17 UTC by Tomasz Finc
Modified: 2010-10-08 17:01 UTC (History)
0 users

See Also:
Web browser: ---
Mobile Platform: ---
Assignee Huggle Beta Tester: ---


Attachments

Description Tomasz Finc 2010-09-18 00:17:20 UTC
Currently we generate stats on locke to one landingpage or banner impressions file. Then someone has to rsync it for long term archive storage on dataset. After, we have to run the appropriate mining script and ship the fundraiser data.

Lets automate this.

We'll need to setup a regular job that pulls a specific set of log files from locke to dataset1. Then a cron job can kick off at a specific point in time (say 30min post fundraiser) that will email the fundraisers with a csv version of the data.
Comment 1 Tomasz Finc 2010-09-28 21:28:21 UTC
The 2step proces is now well automated thanks to Nimish. Documentation to come next.
Comment 2 Nimish Gautam 2010-10-08 17:01:47 UTC
There is now a cron running on locke which has user file_mover automatically compress the data, send it to dataset1, run analysis on it, and email the CSV to tfinc@wikimedia at 55 min past the hour.

Note You need to log in before you can comment on or make changes to this bug.


Navigation
Links