Looking beyond on-wiki solutions for addressing crawlers

Test copy of the proposal on semantic-mediawiki.org, used to test Template:AgendaDay26. Time slot and status are test values from the draft program.

Looking beyond on-wiki solutions for addressing crawlers (Talk, 30 min, Remote, Accepted)

Speakers: Jeffrey Wang · Start: 2026/11/12 16:25:00 · Conference: MediaWiki Users and Developers Conference Fall 2026

Crawler bots have made running public wikis increasingly difficult. This talk looks beyond MediaWiki itself and shares practical techniques that have proven effective against abusive crawling. Crawler bots have raised the table stakes of running a publicly accessible wiki to record levels. The techniques these bots use have shown that successful solutions go beyond changes made to MediaWiki. Some solutions which have worked will be shared.