Home Explore Help
Register Sign In
alroyso
/
neiyou_user_web
1
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Branch: master
Branches Tags
neiyou_user_web
/
public
/
robots.txt

robots.txt 806 B
Permalink History Raw

12345678910111213141516171819202122232425262728293031323334353637383940414243444546 
  1. User-agent: Baiduspider
    2. 

  2. Disallow: /
    3. 

  3. User-agent: Sosospider
    4. 

  4. Disallow: /
    5. 

  5. User-agent: sogou spider
    6. 

  6. Disallow: /
    7. 

  7. User-agent: YodaoBot
    8. 

  8. Disallow: /
    9. 

  9. User-agent: Googlebot
    10. 

  10. Disallow: /
    11. 

  11. User-agent: Bingbot
    12. 

  12. Disallow: /
    13. 

  13. User-agent: Slurp
    14. 

  14. Disallow: /
    15. 

  15. User-agent: Teoma
    16. 

  16. Disallow: /
    17. 

  17. User-agent: ia_archiver
    18. 

  18. Disallow: /
    19. 

  19. User-agent: twiceler
    20. 

  20. Disallow: /
    21. 

  21. User-agent: MSNBot
    22. 

  22. Disallow: /
    23. 

  23. User-agent: Scrubby
    24. 

  24. Disallow: /
    25. 

  25. User-agent: Robozilla
    26. 

  26. Disallow: /
    27. 

  27. User-agent: Gigabot
    28. 

  28. Disallow: /
    29. 

  29. User-agent: googlebot-image
    30. 

  30. Disallow: /
    31. 

  31. User-agent: googlebot-mobile
    32. 

  32. Disallow: /
    33. 

  33. User-agent: yahoo-mmcrawler
    34. 

  34. Disallow: /
    35. 

  35. User-agent: yahoo-blogs/v3.9
    36. 

  36. Disallow: /
    37. 

  37. User-agent: psbot
    38. 

  38. Disallow: /
    39. 

  39. Disallow: /public/
    40. 

  40. Disallow: /sql/
    41. 

  41. Disallow: /config/
    42. 

  42. Disallow: /app/
    43. 

  43. Disallow: /database/
    44. 

  44. Disallow: /resources/
    45. 

  45. Disallow: /routes/
    46. 

  46. Disallow: /vendor
    47.