Step 1: Sign in to your AsteroBuilder account or create one if you're new to access the dashboard.
Step 2: Once successfully logged in, proceed to the 'SEO' section in the dashboard to explore various SEO settings.
Step 3: Look for the 'Common' tab within the SEO section to find essential settings, including robots.txt.
Step 4: Edit Robots.txt: Click on the robots.txt option to view the default configuration. Modify it based on your preferences using the provided demo file.
# robots.txt generated by asterobuilder.com
User-agent: *
Disallow: /admin/
Disallow: /portal/
Step 5: Customise Disallow Directives: Personalise the 'Disallow' directives to control which sections of your website should be excluded from search engine crawlers
Step 6: Save Changes: Ensure to save your changes after customization to apply the updated robots.txt configuration to your live website.
Effortlessly manage your AsteroBuilder website's robots.txt file to refine how search engines interact with your content.