The Disallow directive triggers on partial matches too. Be as distinct as is possible when defining the Disallow directive to stop unintentionally disallowing access to data files. Decrease the probability of certain web pages being crawled, which include indexing and showing up in the search engine results. To put it https://rankingseotoolz.com/robots-txt-generator