16 Mar

User:HYanWong

‎Automatic credit (attribution) lines
← Older revision
Revision as of 05:23, 16 March 2013
Line 3:
Line 3:

I've written some code to generate [[Commons:Credit_line | credit lines]] for any image on Commons. It's a bit like [[Help:Gadget-Stockphoto | Gadget-Stockphoto]] but is designed for batch processing large numbers of images, so instead of scraping Commons directly, it goes through the [http://dumps.wikimedia.org/commonswiki/latest/ xml and sql data dump files].

I've written some code to generate [[Commons:Credit_line | credit lines]] for any image on Commons. It's a bit like [[Help:Gadget-Stockphoto | Gadget-Stockphoto]] but is designed for batch processing large numbers of images, so instead of scraping Commons directly, it goes through the [http://dumps.wikimedia.org/commonswiki/latest/ xml and sql data dump files].

−
You give the script a list of image names, or a search pattern, and it slurps through the entire commons-pages-articles xml dump looking for matching pages, tries to recognise authors from those pages, then grabs the licences by looking through a (parsed) copy of the commonswiki categorylinks sql table.
+
You give the script a list of image names, or a search pattern, and it slurps through the entire commons-pages-articles xml dump looking for matching pages, tries to recognise authors from those pages, then grabs the licences by looking through a (parsed) copy of the commonswiki categorylinks sql table.

+

+
You first need to download the [http://dumps.wikimedia.org/commonswiki/latest/commonswiki-latest-categorylinks.sql.gz sql file], unzip it, and run [User:HYanWong/categorylinks_parser|this simple C program] to pre-parse it into a text-readable form. You can then download and unpack the [http://dumps.wikimedia.org/commonswiki/latest/commonswiki-latest-pages-articles.xml.bz2 commons xml dump], and run my script from the command line, something like

+

+
The perl script is available at [User:HYanWong/imagenames2credits.pl].

http://commons.wikimedia.org/w/index.php?title=User:HYanWong&diff=92660625&oldid=92660175