gitea-mirror/OSINT-Cheat-sheet
1
0
Fork 0
mirror of https://github.com/Jieyab89/OSINT-Cheat-sheet.git synced 2026-06-12 11:01:18 -07:00
Code Issues Packages Projects Releases Wiki Activity

edit robots.txt

Browse Source
This commit is contained in:
Jieyab89
2026-05-27 20:00:38 +07:00
parent 26a7bc960b
commit d75f872244
2 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Web-Based/robots.txt
+1
View File
@@ -10,6 +10,7 @@ Sitemap: https://jieyab89-osint.gitbook.io/jieyab89-osint-cheat-sheet-wiki-tips/
# Block unnecessary paths
Disallow: /Script
Disallow: /mail
# AI / LLM discovery
Allow: /llms.txt
robots.txt
+1
View File
@@ -10,6 +10,7 @@ Sitemap: https://jieyab89-osint.gitbook.io/jieyab89-osint-cheat-sheet-wiki-tips/
# Block unnecessary paths
Disallow: /Script
Disallow: /mail
# AI / LLM discovery
Allow: /llms.txt