Audit Crawlability
## Crawlability Analysis
### Robots.txt Review
**URL**: [domain]/robots.txt
**Status**: [Found/Not Found/Error]
**Current Content**:
[robots.txt content]
| Check | Status | Notes |
|-------|--------|-------|
| File exists | ✅/❌ | [notes] |
| Valid syntax | ✅/⚠️/❌ | [errors found] |
| Sitemap declared | ✅/❌ | [sitemap URL] |
| Important pages blocked | ✅/⚠️/❌ | [blocked paths] |
| Assets blocked | ✅/⚠️/❌ | [CSS/JS blocked?] |
| Correct user-agents | ✅/⚠️/❌ | [notes] |
**Issues Found**:
- [Issue 1]
- [Issue 2]
**Recommended robots.txt**:
User-agent: *
Allow: /
Disallow: /admin/
Disallow: /private/
Sitemap: https://example.com/sitemap.xml
---
### XML Sitemap Review
**Sitemap URL**: [URL]
**Status**: [Found/Not Found/Error]
| Check | Status | Notes |
|-------|--------|-------|
| Sitemap exists | ✅/❌ | [notes] |
| Valid XML format | ✅/⚠️/❌ | [errors] |
| In robots.txt | ✅/❌ | [notes] |
| Submitted to GSC | ✅/⚠️/❌ | [notes] |
| URLs count | [X] | [appropriate?] |
| Only indexable URLs | ✅/⚠️/❌ | [notes] |
| Includes priority | ✅/⚠️ | [notes] |
| Includes lastmod | ✅/⚠️ | [accurate?] |
**Issues Found**:
- [Issue 1]
---
### Crawl Budget Analysis
| Factor | Status | Impact |
|--------|--------|--------|
| Crawl errors | [X] errors | [Low/Med/High] |
| Duplicate content | [X] pages | [Low/Med/High] |
| Thin content | [X] pages | [Low/Med/High] |
| Redirect chains | [X] found | [Low/Med/High] |
| Orphan pages | [X] found | [Low/Med/High] |
**Crawlability Score**: [X]/10