Be Mindful, you utilize the robots.txt generator at your personal hazard. It's sensible to validate that your robots file won't exclude pages by oversight. There are lots of tools to check this much like the Google tool. Now, the user is necessary to specify directories that needs to be restricted https://seotoolstube.com/