Jump to content

Commons:Village pump/Technical

From Wikimedia Commons, the free media repository

Shortcuts: COM:VP/T • COM:VPT

Welcome to the Village pump technical section
Technical discussion
Village pump/Technical
 Bug reports
 Code review
Tools
 Tools/Directory
 Idea Lab



This page is used for technical questions relating to the tools, gadgets, or other technical issues about Commons; it is distinguished from the main Village pump, which handles community-wide discussion of all kinds. The page may also be used to advertise significant discussions taking place elsewhere, such as on the talk page of a Commons policy. Recent sections with no replies for 30 days and sections tagged with {{Section resolved|1=--~~~~}} may be archived; for old discussions, see the archives; recent archives: /Archive/2025/04 /Archive/2025/05.

Please note
 
SpBot archives all sections tagged with {{Section resolved|1=~~~~}} after 1 day and sections whose most recent comment is older than 30 days.

Empty Wikidata Infobox

[edit]

Hi everybody. I have a question about this {{Wikidata Infobox}}. I makes sense to create a category including this code <!-- {{Wikidata Infobox}} -->. or, after, comment this code {{Wikidata Infobox}} when the wikidata is not existing yet or is not yet connect? An example here Category:Companies_based_in_Heemstede,_North_Holland. I don't want to start an editwar, but this kind of edit seem to not very wise, because when the wikidata element will be created it will be more difficult to connect it or, when already present, is not easy to find it, for example Category:Klouwer (surname), it was better to look for it instead of comment it <!-- {{Wikidata Infobox}} -->. Thank you in advance for your answers. P.S. If not necessary, I would not insert at all {{Wikidata Infobox}} MrKeefeJohn (talk) 18:40, 22 March 2025 (UTC)[reply]

Create your category with {{Wikidata Infobox}} but without the HTML comment markup. The infobox is designed for such use. I have just done this on your example category. Note that the infobox there includes a "Create new Wikidata item" link, Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 19:01, 22 March 2025 (UTC)[reply]
@Pigsonthewing: thank you for your answer. I understand what is designed for <!-- {{Wikidata Infobox}} --> and always add it creating new categories. The question was more about if wise to remove it when the wikidata element is not created yet. MrKeefeJohn (talk) 06:58, 3 April 2025 (UTC)[reply]
Usually a Wikidata infobox is not added when there is no Wikidata item to connect it to / that it's connected to. Adding an infobox nevertheless is I think an implicit suggestion/request for others to create a Wikidata item for the subject of the category. For these, I usually just create the Wikidata item but there are exceptions when it's not clear how the item would look like. In regards to your case (example?) I don't think there are Wikidata items for 'Companies based in {region}' but maybe I'm wrong or these could be created – however usually items are created for companies with some location information so there's no Wikidata item for that. They could maybe be created as dynamic list pages somehow though. Prototyperspective (talk) 09:51, 3 April 2025 (UTC)[reply]
That is the context in which I gave my answer. Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 11:29, 3 April 2025 (UTC)[reply]
@Prototyperspective: I do understand your opinion, but in my experience quite often it's just a problem of missing connection. Just todady a couple of example Category:Companies based in Udine wikidata and Category:Companies based in Udine wikidata MrKeefeJohn (talk) 19:30, 11 April 2025 (UTC)[reply]
Don't know what you mean. This isn't what you wrote about earlier. Yes, there are categories that are not connected to their Wikidata items. You could for example always add the infobox and then check if the Wikidata item exists via search and if not create it or if not just leave the empty infobox as is. Prototyperspective (talk) 21:35, 11 April 2025 (UTC)[reply]
I mean that if you don't add a {{Wikidata Infobox}}, or remove it or comment like this <!-- {{Wikidata Infobox}} -->, it will be more difficult to connect with wikidata elements and/or with already existing articles or categories in some wiki. And it seems to me that it makes more difficult the bots' operation. MrKeefeJohn (talk) 06:56, 12 April 2025 (UTC)[reply]
Why would that be the case, I don't think so. That infobox template is added by a bot. Prototyperspective (talk) 10:16, 12 April 2025 (UTC)[reply]

HotCat and Cat-a-lot are adding categories inside template brackets

[edit]

I added a category with HotCat here and one with Cat-a-lot here and for some reason the categories were added inside the brackets of the crop-template that was already there as if the categories were part of the template syntax. As a result, when another user cropped the image with CropTool the categories were (automatically) removed together with the crop-template, see here. That looks like a chain of things going wrong. Nakonana (talk) 18:20, 1 April 2025 (UTC)[reply]

Several images are affected: see File talk:Bundesarchiv Bild 101I-133-0703-38, Polen, Ghetto Litzmannstadt, Kinder.jpg#Categories (and that list might not be comprehensive). Nakonana (talk) 18:27, 1 April 2025 (UTC)[reply]
I suppose the tools' logic is to find the last [[Category:...]] and then open a new line after it and add the new cats.
The logic makes sense, since it seeks to put categories together.
I could imagine a little more could be added to the logic. It can try to find the last \n\s*\[\[Category\:.+\]\]\s* instead. RoyZuo (talk) 18:43, 1 April 2025 (UTC)[reply]
I think that it would work that function find_insertionpoint() in Gadget-Cat-a-lot.js line 916 would contain also following replace when it is cleaning the wikicode:
.replace( /\{\{Crop\|category=(\s|\S)*?\}\}/g, replaceByBlanks );
I think that HotCat is using similar cleanup.
-- Zache (talk) 18:58, 1 April 2025 (UTC)[reply]
Thanks for your ideas. I have one suggestion: there could be one or more blanks between | and category= that needs to be considered. Kind regards, --Sebastian Wallroth (talk) Sebastian Wallroth (talk) 06:32, 2 April 2025 (UTC)[reply]
The much simpler solution would be to move the category link inside the template and only write the category name when using the template. GPSLeo (talk) 08:41, 2 April 2025 (UTC)[reply]
Whom/Where to ask to make that happen? Nakonana (talk) 08:38, 6 April 2025 (UTC)[reply]
I changed the template[1]. GPSLeo (talk) 09:05, 6 April 2025 (UTC)[reply]
Thank you! Nakonana (talk) 09:06, 6 April 2025 (UTC)[reply]
Is this solved then? Prototyperspective (talk) 11:35, 6 April 2025 (UTC)[reply]
I think? I haven't tested how CropTool is handling the change, though. Nakonana (talk) 18:00, 7 April 2025 (UTC)[reply]
@GPSLeo: Did you clean up existing usage? Without that, pages are broken, including those like Commons:Modèles that simply want to suppress the category (there’s no way to achieve that after your edit). I think your edit should be reverted and the tools should be fixed instead, ideally by not reinventing the wheel but rather using an API, which uses the same backend as VisualEditor for example – I fear no such API exists yet, though, so the solution for now is probably
  • fixing the regexes,
  • or simply telling users to put the crop template above the categories, which would not confuse the regexes.
Tacsipacsi (talk) 21:48, 4 May 2025 (UTC)[reply]
or simply telling users to put the crop template above the categories — if a user is using the CropTool through the (tools) menu, the user doesn't get to place the template themselves, the tool "decides" where to place the template. The placement is automated. Something in this automated process would need to be changed to fix the HotCat/Cat-a-lot issue. Nakonana (talk) 17:09, 5 May 2025 (UTC)[reply]

Multiple files uploaded under one name

[edit]

When I use the mobile uploader, usually it succeeds in picking names for my files that don't overwrite each other (for example Category:Views from the Internationale chairlift). With File:Blewett Pass - Tronsen Ridge ski tour.jpg, they were just uploaded on top of each other. Weirdly I had just done a previous multi-upload for the name File:Alpental.jpg that worked perfectly. What's the easiest way to fix this? Buidhe (talk) 03:37, 5 April 2025 (UTC)[reply]

@Buidhe I think this is a serious bug that you should write to https://github.com/commons-app/apps-android-commons , or follow Commons:Mobile app#Issues. RoyZuo (talk) 09:40, 6 April 2025 (UTC)[reply]

Image Rotation

[edit]

Hello! I have submitted a request to rotate File:ATTRITOR GRINDING MACHINE - NARA - 17496809.jpg by 180° via @SteinsplitterBot. Now the thumbnail has the desired orientation, but the actual image still remains upside down. What is the correct procedure to handle such cases? --Aciarium (talk) 10:49, 7 April 2025 (UTC)[reply]

@Aciarium: it looks ok for me. See COM:PURGE. Commander Keane (talk) 12:07, 7 April 2025 (UTC)[reply]
@Commander Keane: My bad, I wasn't aware that I had to purge the site after this action. Now it looks ok also on my side. Thanks for clarifying!! --Aciarium (talk) 13:49, 7 April 2025 (UTC)[reply]

Does someone have the means to extract the image from the British Museum's website? My usual methods are failing. JayCubby (talk) 14:49, 7 April 2025 (UTC)[reply]

@JayCubby https://www.britishmuseum.org/collection/image/1613085647 maybe? RoyZuo (talk) 08:36, 12 April 2025 (UTC)[reply]

Tech News: 2025-15

[edit]

MediaWiki message delivery 18:48, 7 April 2025 (UTC)[reply]

PDF upload broken

[edit]

When I use the upload tool, PDF is not one of the options, and it does not recognize a PDF as a file type when I try to upload. When I fill in the information, the tool fails with nonsensical errors. This is especially problematic as Foundation policy directs that PDFs (which are heavily used on English Wikisource and other language Wikisource projects) which are free by Commons rules should be uploaded here and not locally. TE(æ)A,ea. (talk) 01:30, 9 April 2025 (UTC)[reply]

@TE(æ)A,ea. which tool do you mean? can you plz write down the link? RoyZuo (talk) 02:37, 9 April 2025 (UTC)[reply]
Hello @TE(æ)A,ea., I assumed you have found a solution to your problem since you uploaded File:Japanese.pdf after you posted here. Tvpuppy (talk) 02:47, 9 April 2025 (UTC)[reply]

Mysterious editing of page

[edit]

on Special:Upload, there's a line "Permitted file types: tiff, tif, png, gif..."

after or around the page finishes loading, something deletes ", pdf" from the list of extensions. sorry my skill is too low to find out what does that. RoyZuo (talk) 08:16, 9 April 2025 (UTC)[reply]

The relevant code, and an explanation, is at MediaWiki:Upload.js#L-141--L-148. Omphalographer (talk) 00:14, 10 April 2025 (UTC)[reply]

Overwrite a video file

[edit]

I uploaded a video a while ago File:TTC Line 2 T-1 Subway arrives at Pape station (2023-08-08) 001.webm, it had some frame rate issues and never got around to fixing it, until now. However I am unable to re-upload it as the file exceeds 100MB. How can I overwrite the video file with the new one? PascalHD (talk) 02:02, 10 April 2025 (UTC)[reply]

@PascalHD User talk:Rillke/bigChunkedUpload.js. RoyZuo (talk) 06:36, 10 April 2025 (UTC)[reply]
@RoyZuo Thanks! It worked perfectly! PascalHD (talk) 21:38, 10 April 2025 (UTC)[reply]
This is the reason why this issue exists: phab:T382814 Missing info on how to upload a file larger than 100 MB in the 'Upload a new version of this file' page – You're not the only one who doesn't know that. Prototyperspective (talk) 21:52, 10 April 2025 (UTC)[reply]

Batch uploading of file

[edit]

I have here files I ripped from the SLSA's website (Category:Files_from_the_State_Library_of_South_Australia), which are higher-resolution than the existing files.

The files are named as follows: stitched_John_L_Bonython_12yrs_1887-B7723_43.jpg.jpg corresponds to file:John_L_Bonython_12yrs_1887-B7723_43.jpg

The files are here: https://files.catbox.moe/nmqlbs.zip

Thanks a bunch JayCubby (talk) 18:39, 12 April 2025 (UTC)[reply]

I have uploaded them  REAL 💬   19:31, 12 April 2025 (UTC)[reply]
Thank you very much, 999real! JayCubby (talk) 21:26, 12 April 2025 (UTC)[reply]

Edit protected template

[edit]

Can somebody please take a look at my edit request for {{Created with}}? — Mikhail Ryazanov (talk) 21:09, 13 April 2025 (UTC)[reply]

Tech News: 2025-16

[edit]

MediaWiki message delivery 00:20, 15 April 2025 (UTC)[reply]

Media missing infobox template

[edit]

Does somebody know how Category:Media missing infobox template gets added to files?

Because it seems to be missing for quite a few files, such as this. For various structured access to files such as via the Commons app, the MediaViewer, an API, etc, having a standardized populated information template is very useful. It could (and to some extent) can also be searched / queried and metadata be gathered via the standardized format.

If whatever added that category and its subcats is not running anymore it would be good if it was restarted or if it's running improved so that more files are added to it.

Additionally, is there any effort to add the {{Information}} to files at scale? The cat currently contains as many as 334,529 files. For most of these, some script could add the template and add the source, author and description data from whatever has been added instead if there is anything. Prototyperspective (talk) 19:50, 17 April 2025 (UTC)[reply]

Looking at a few random images, the category was added by User:JarektBot and User:YiFeiBot. @Jarekt and Zhuyifei1999: Why did your bots stop adding it? —Tacsipacsi (talk) 22:20, 4 May 2025 (UTC)[reply]
@Tacsipacsi and Prototyperspective: I created Category:Media missing infobox template category and added it to all the files missing any of the main infoboxes in November 2014. The files were found as files without {{Infobox template tag}}. I used my bot account because I was using AutoWikiBrowser in a "bot" mode. Back then the idea was that once identified we will figure out some way of adding the {{Information}} template to those files. We did add some templates to large batches of similar images but for most files we did not figured out a way to add them. I guess at some point YiFeiBot was tagging new uploads, but to my knowledge there is no effort to use this category for any corrections. Even worse, for many of the files we do not know anything about them, like who is the photographer or why do they have a license they do. Many of those files, uploaded in the early days of Commons, would be deleted under the current rules. --Jarekt (talk) 00:05, 5 May 2025 (UTC)[reply]
Many of those files, uploaded in the early days of Commons, would be deleted under the current rules. Even more reason for why that tagging is useful.
but for most files we did not figured out a way to add them. I guess at some point YiFeiBot was tagging new uploads, but to my knowledge there is no effort to use this category for any corrections. That's an issue but it's somewhat separate from the tagging. Restarting the tagging of files only long after may make it more difficult to tag the files. Moreover, it can also slightly reduce the number of files with that issue because some of the uploaders are made aware of that being an issue / discouraged so they use the standard UploadWizard / Information template for their later uploads. In any case, if any of you can, please restart the bot tagging – that being used to correct those files is a separate issue and it could be many years until something is being done but another thread about that could be made right after this one. Prototyperspective (talk) 12:50, 5 May 2025 (UTC)[reply]

Understanding AbuseFilter code

[edit]

Looking at AbuseFilter 153 - Cross-wiki upload filter the way I understand it is that it prevents uploads of files matching that specific edit summary that are not images, or they are pngs, or jpegs less than 2Mpix or under 60kB.

But the bit (user_age < 3600*24*180 | user_editcount < 50) indicates that uploaders must be at least 180 days old (or have at least 50 edits)? (mw doc page says user_age is in seconds). Unless they are confirmed or autopatrolled. I am guessing that the Commons abuse filter can't see the user_age and user_edits in outside projects and that is why there was a propsal to block new users from uploading (that meant Wikipedia non-autoconfirmed right?). But if that user_age is applied to an uploader's Commons age and edit count, how do ~500 edits per day get through? For example File:Guillotina de 1789.jpg's uploader wasn't 180 days old anywhere and didn't have 50 edits (global account info). I must be misunderstanding something.

I will ping @Billinghurst as they last edited the filter. Commander Keane (talk) 08:47, 19 April 2025 (UTC)[reply]

@Commander Keane So I looked into this and realized that the statement !("confirmed" in user_groups | "autopatrol" in user_rights) also matches autoconfirmed users, see https://phabricator.wikimedia.org/T181024. This means that the filter has de-facto been exempted for user accounts more than 4 days old, which is unfortunate because it means the filter has probably been missing a very large number of files. This is a good catch and we should change this immediately, however given that the filter has been like this since 2016 it might result in a large increase in filter hits (maybe complaints from other wikis). Dylsss (talk) 23:54, 19 April 2025 (UTC)[reply]
Moved to Commons:Help desk#COM:ADVERT

Tech News: 2025-17

[edit]

MediaWiki message delivery 20:56, 21 April 2025 (UTC)[reply]

How is the pixel size your Flickr photo upload to Commons at (4,438 × 6,101 pixels), larger than the maximum size on this Flickr account?

[edit]

User talk:Basque mapping#How is the pixel size your Flickr photo upload to Commons at (4,438 × 6,101 pixels), larger than the maximum size on this Flickr account?

Can someone explain how photo uploads grow in size from a source that has a smaller size. Please, see the link above and it's associated links to help me understand how these changes occur.

Thank you, -- Ooligan (talk) 03:02, 22 April 2025 (UTC)[reply]

@Ooligan: The history of File:Official portrait of Secretary Marco Rubio.jpg says that it came from https://www.state.gov/wp-content/uploads/2025/01/Official-Photo-Secrertary-Marco-Rubio.jpg which now appears to have been moved to https://www.state.gov/wp-content/uploads/2025/01/Marco-Rubio.jpg and which is 4438x6101 and matches the file that was uploaded. So I guess linking it to Flickr was just a convenience, as it's perhaps considered more stable than a WordPress attachment, and is the same file. Sam Wilson 07:44, 22 April 2025 (UTC)[reply]

How to check whether PDF has text layer?

[edit]

given a random pdf here, how to check if it contains a text layer (that has a lot of text)?

i found https://www.reddit.com/r/pdf/comments/npz1ak/how_to_identify_which_pdfs_contain_a_text_layer/ . not sure if something already does that for commons files. RoyZuo (talk) 17:20, 22 April 2025 (UTC)[reply]

the practical use of this for me is, for a certain book category, there're a dozen different pdfs inside.
now i wanna know which one of them might already have the text. if some of them do, i can extract that and feed into wikisource directly, instead of wasting my time to ocr page by page. RoyZuo (talk) 17:23, 22 April 2025 (UTC)[reply]
You can use the API, it will return the text layer contents of a pdf. For example: https://commons.wikimedia.org/w/api.php?titles=File:Serial_Programming.pdf&action=query&prop=imageinfo&iiprop=metadata Bawolff (talk) 20:08, 22 April 2025 (UTC)[reply]

Should we not run a bot, to categorise as, say, "PDFs with text layer" and "PDFs with no text layer"? Andy Mabbett (Pigsonthewing); Talk to Andy; Andy's edits 15:17, 27 April 2025 (UTC)[reply]

That would be much better handled by structured data than categories. Nosferattus (talk) 17:29, 4 May 2025 (UTC)[reply]

New Autowikibrowser (6.4.0.0) is malfunctioning

[edit]

I am using AutoWikiBrowser version 6.4.0.0 at Commons. I've found the software is almost useless because I cannot input the "replace with" field in the advanced settings.
Options --> Find and replace --> Advanced settings --> Find --> Replace with:
Find field can be inputted, but Replace with field cannot be. Please fix it as soon as possible. Thank you. --トトト (talk) 05:01, 24 April 2025 (UTC)[reply]

Tech News: 2025-18

[edit]

MediaWiki message delivery 19:28, 28 April 2025 (UTC)[reply]

File revision url format

[edit]

I'm gonna try making a tool that goes from file revisions or previews back to file description page (in case you land on those direct links). does anyone know of any documentation on this? RoyZuo (talk) 12:02, 30 April 2025 (UTC)[reply]

Two digits from a hexadecimal MD5 hash and prefixed dates. See MW:Manual:MediaWiki file usage.
previews have similar information
The 525px-... states the image size. SVG files may have lang prefixes (this one is made up for German as spoken in Switzerland):
Of course the lang and size would be easy to strip.
Glrx (talk) 18:57, 30 April 2025 (UTC)[reply]

Tech News: 2025-19

[edit]

MediaWiki message delivery 00:11, 6 May 2025 (UTC)[reply]

API for accessing Data: namespace or other .json files

[edit]

Is there a special API or other way to get a raw data table from outside of MediaWiki? Chemistry articles on enwiki (and maybe others) link to an external program to display a graphical representation of chemicals, but the data-set that program uses is limited and not always containing the details that *wiki want. That program can load external data, so I was hoping we could have the data on commons, as we do for map-data and similar back-ends. JSON, CSV, even raw wiki-source would be fine. DMacks (talk) 01:31, 6 May 2025 (UTC)[reply]

You can make a request like [11] and then parse the output as json. GPSLeo (talk) 06:57, 6 May 2025 (UTC)[reply]

We will be enabling the new Charts extension on your wiki soon!

[edit]

(Apologies for posting in English)

Hi all! We have good news to share regarding the ongoing problem with graphs and charts affecting all wikis that use them.

As you probably know, the old Graph extension was disabled in 2023 due to security reasons. We’ve worked in these two years to find a solution that could replace the old extension, and provide a safer and better solution to users who wanted to showcase graphs and charts in their articles. We therefore developed the Charts extension, which will be replacing the old Graph extension and potentially also the EasyTimeline extension.

After successfully deploying the extension on Italian, Swedish, and Hebrew Wikipedia, as well as on MediaWiki.org, as part of a pilot phase, we are now happy to announce that we are moving forward with the next phase of deployment, which will also include your wiki.

The deployment will happen in batches, and will start from May 6. Please, consult our page on MediaWiki.org to discover when the new Charts extension will be deployed on your wiki. You can also consult the documentation about the extension on MediaWiki.org.

If you have questions, need clarifications, or just want to express your opinion about it, please refer to the project’s talk page on Mediawiki.org, or ping me directly under this thread. If you encounter issues using Charts once it gets enabled on your wiki, please report it on the talk page or at Phabricator.

Thank you in advance! -- User:Sannita (WMF) (talk) 15:07, 6 May 2025 (UTC)[reply]