User-agent: * Disallow: /settings Disallow: /assets/ Disallow: /javascript/ Disallow: /api/ Disallow: /search/ Disallow: /admin/