User-agent: * Crawl-delay: 10 Sitemap: http://www.24sign.ro/sitemap.xml # No need to index this stuff Disallow: /feed/ Disallow: /html/ Disallow: /scripts/ Disallow: /remote Disallow: /invoice # Avoid duplicate content as it may do more harm than good Disallow: /*?f=* Disallow: /*?fb_xd_fragment Disallow: /*?qty=* Disallow: /*?mobile=* # Disallow bad bots User-agent: 008 User-agent: AhrefsBot User-agent: Amazonbot User-agent: Baiduspider User-agent: BecomeBot User-agent: CCBot User-agent: EtaoSpider User-agent: Exabot User-agent: MJ12bot User-agent: NPBot User-agent: ShopWiki User-agent: SMTBot User-agent: sogouspider User-agent: Sogou web spider User-agent: Stanford User-agent: Stanford Comp Sci User-agent: Synthesio User-agent: YandexBot Disallow: /