Technical SEO

Technical SEO files tell crawlers how to treat your site. Each is simple to generate and easy to misunderstand in ways that quietly undo the intent.

robots.txt does not hide pages

This is the most consequential misconception in the group. Disallow asks crawlers not to fetch a URL; it does not keep that URL out of the index. A disallowed page can still be indexed from inbound links, appearing without a description. To keep a page out of results, use a noindex meta tag and leave the page crawlable so the tag can be read.

It is also not a security control. The file is public, so listing your admin paths publishes a map of what you consider sensitive. Protect private areas with authentication. And do not block CSS or JavaScript: Google renders pages to assess them, so blocking your assets means it sees a broken layout.

Sitemaps aid discovery, not indexing

The Sitemap Generator produces the XML. A sitemap helps search engines find URLs; whether they index them depends on crawl budget and perceived value. Google ignores priority and changefreq entirely, and only trusts lastmod when it is consistently accurate. Leave out anything you do not want indexed — noindex pages, redirects, canonicalised duplicates — since including them sends contradictory signals. One file holds up to 50,000 URLs and 50 MB uncompressed.

Structured data buys eligibility

The Schema Markup Generator outputs JSON-LD, which Google explicitly recommends over Microdata. Markup is not a ranking factor; it makes a page eligible for rich results. Critically, it must describe content the user can actually see — marking up prices or reviews that do not appear on the page is a policy violation that can cost you all rich results.