React SPAs need special robots.txt consideration. You want search engines to crawl your rendered pages but not your build artifacts, source maps, or internal routes. This generator creates the right rules.
React apps built with Create React App or Vite serve from /static/ or /assets/. These build files should not be indexed. But your actual page routes must remain accessible for Google to render.
Try It Now — Free, No Sign-up
Open the tool and get started instantly. No sign-up, no installation needed.
Open Robots.txt Generator Now100% browser-based • No upload to server • No sign-up required
How to Generate Robots.txt for React Single Page Apps
- Allow all bots to access / (root) — Google needs to render your JS
- Disallow: /static/js/*.map (source maps)
- Disallow any auth/private routes: /dashboard, /admin, /account
- Add sitemap URL pointing to your pre-rendered sitemap
- Place in public/ folder of your React project
Pro Tips
- Google CAN render JavaScript — do not block JS/CSS files
- Block .map files to prevent source code exposure
- If using SSR (Next.js, Remix), crawling is much easier
- Use Google Search Console URL Inspection to verify rendering
Frequently Asked Questions
Can Google crawl React SPAs?
Yes, Googlebot renders JavaScript. But it takes longer (days vs hours for static HTML). Ensure your JS bundles are not blocked in robots.txt.
Should I block /static/ in React robots.txt?
No! Blocking /static/ blocks your CSS and JS files, preventing Google from rendering your pages. Only block .map source map files.
Where does robots.txt go in Create React App?
In the /public folder. CRA automatically serves files from /public at the root URL.
Related Tools & Guides
Ready to Use Robots.txt Generator?
Free, instant, and 100% private. No sign-up needed.
Open Robots.txt Generator