🔎 Added browse pages to robots.txt (#2885)

* Added all variants of browse.aspx to robots.txt * 🔎 Added all routes containing BrowseXML.aspx to robots.txt * 🔎 Added lowercase browsexml pages to robots.txt * removed star at the end of /*browsexml.aspx$ exclusion route * removed unnecessary "ends with" routes that were a already excluded by wildcards * 🔎 added Browse.aspx and browse.aspx as wild card routes --------- Co-authored-by: Calinator444 <[email protected]>
SSWConsulting · Aug 5, 2024 · af68027 · af68027
1 parent 82dbf6f
commit af68027
Showing 1 changed file with 4 additions and 0 deletions.
diff --git a/next-sitemap.config.js b/next-sitemap.config.js
@@ -49,6 +49,10 @@ module.exports = {
         disallow: [
           "/404",
           "/500",
+          "/*Browse.aspx*",
+          "/*browse.aspx*",
+          "/*browsexml.aspx*",
+          "/*BrowseXML.aspx*",
           "/ssw/Redirect",
           "/ssw/CodeAuditor",
           "/ssw/Version.aspx",