Datasette plugin that blocks all robots using robots.txt - Simon Willison - simonw |
GitHub | https://github.com/simonw/datasette-block-robots | |
Size | 836K (684K (+152K)) | |
Commits | 17 (12 (+5)) | |
Update Stats | 0s ... 7s [Ø (last 10) 1s (1,1,1,1,1,1,1,1,1,1))] | |
Last Change | 2022-08-31 13:56:48 | |
Last Check | 2023-09-15 10:56:39 | |
Created | 2020-10-06 17:57:40 |
Operation
Verifying url ...
Mail to Andreas Kupries