### Pages with rec parameter - IAI Recommendation System User-agent: * Disallow: /*?rec=* Disallow: /*&rec=* Sitemap: https://taimen.com/sitemap.xml.gz User-agent: AI2Bot User-agent: Ai2Bot-Dolma User-agent: Amazonbot User-agent: Brightbot 1.0 User-agent: Bytespider User-agent: Cotoyogi User-agent: Crawlspace User-agent: Diffbot User-agent: FacebookBot User-agent: Factset_spyderbot User-agent: img2dataset User-agent: MyCentralAIScraperBot User-agent: TikTokSpider User-agent: Timpibot User-agent: YandexAdditional User-agent: YandexAdditionalBot Disallow: / User-agent: ClaudeBot User-agent: GPTBot User-agent: anthropic-ai User-agent: Meta-ExternalAgent User-agent: meta-externalagent Disallow: / User-agent: AhrefsBot User-agent: MJ12bot User-agent: dotbot User-agent: rogerbot User-agent: SemrushBot User-agent: BLEXBot User-agent: Lumar User-agent: Ezooms User-agent: cognitiveSEO User-agent: Screaming Frog SEO Spider Disallow: / User-agent: DealavoBot User-agent: SzpiegomatBot User-agent: 008 User-agent: ApifyBot User-agent: PriceSentryBot User-agent: Exabot User-agent: MojeekBot User-agent: metajobbot User-agent: fyberspider User-agent: wotbox User-agent: Scrapy Disallow: / User-agent: TwengaBot-Discover User-agent: TwengaBot/2.0 User-agent: TwengaBot User-agent: TwengaBot-2.0 Disallow: / # --- Taimen.com robots.txt (slim & SEO-safe) --- User-agent: * # 1) Renderowanie zasobów (nie blokuj CSS/JS/obrazów/fontów) Allow: /*.css$ Allow: /*.js$ Allow: /*.woff$ Allow: /*.woff2$ Allow: /*.ttf$ Allow: /*.svg$ Allow: /*.png$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.webp$ # 2) Strefy prywatne / checkout / konto # (warianty z prefiksem języka i bez) Disallow: /*/basketchange.html Disallow: /*/basketedit.html Disallow: /*/basket-onchange.html Disallow: /*/client-address.html Disallow: /*/client-addresses.html Disallow: /*/client-new.html Disallow: /*/client-orders.html Disallow: /*/client-save.html Disallow: /*/client-files.html Disallow: /*/download-attachment.html Disallow: /*/login.html Disallow: /*/loginedit.html Disallow: /*/loginonce.html Disallow: /*/membership-card.html Disallow: /*/order1.html Disallow: /*/order2.html Disallow: /*/order3.html Disallow: /*/ordercancel.html Disallow: /*/orderconfirm.html Disallow: /*/orderdetails.html Disallow: /*/order-document.html Disallow: /*/order-newpayment.html Disallow: /*/order-payment.html Disallow: /*/order-postauction.html Disallow: /*/order-wrappers.html Disallow: /*/payments-confirm.html Disallow: /*/products-bought.html Disallow: /*/products-requests.html Disallow: /*/przelew.html Disallow: /*/rebate-code.html Disallow: /*/rma-add.html Disallow: /*/rma-list.html Disallow: /*/rma-open.html Disallow: /*/returns-open.html Disallow: /*/return.html Disallow: /*/settings.html Disallow: /*/checkup.html Disallow: /*/affiliate-resources.html # (bez prefiksu – na wszelki wypadek) Disallow: /basketchange.html Disallow: /basketedit.html Disallow: /basket-onchange.html Disallow: /client-address.html Disallow: /client-addresses.html Disallow: /client-new.html Disallow: /client-orders.html Disallow: /client-save.html Disallow: /client-files.html Disallow: /download-attachment.html Disallow: /login.html Disallow: /loginedit.html Disallow: /loginonce.html Disallow: /membership-card.html Disallow: /order1.html Disallow: /order2.html Disallow: /order3.html Disallow: /ordercancel.html Disallow: /orderconfirm.html Disallow: /orderdetails.html Disallow: /order-document.html Disallow: /order-newpayment.html Disallow: /order-payment.html Disallow: /order-postauction.html Disallow: /order-wrappers.html Disallow: /payments-confirm.html Disallow: /products-bought.html Disallow: /products-requests.html Disallow: /przelew.html Disallow: /rebate-code.html Disallow: /rma-add.html Disallow: /rma-list.html Disallow: /rma-open.html Disallow: /returns-open.html Disallow: /return.html Disallow: /settings.html Disallow: /checkup.html Disallow: /affiliate-resources.html # 3) API/serwisy wewnętrzne Disallow: /edi/ Disallow: /ajax/ # 4) Parametry śmieciowe (tracking) Disallow: /*?*utm_ Disallow: /*?*gclid= Disallow: /*?*fbclid= Disallow: /*?*yclid= Disallow: /*?*mc_eid= # 5) Typowe parametry filtrów/sortowania (duplikaty) # (na szablonie faceted dodaj dodatkowo meta robots "noindex,follow") Disallow: /*?*sort= Disallow: /*?*order= Disallow: /*?*view= Disallow: /*?*dir= Disallow: /*?*filter= Disallow: /*?*brand= Disallow: /*?*size= Disallow: /*?*color= Disallow: /*?*price= Disallow: /*?*availability= Disallow: /*?*in-stock= Disallow: /*?*stock= Disallow: /*?*query_id= # 6) (Opcjonalnie) strefy testowe / admin Disallow: /preview/ Disallow: /test/ Disallow: /tmp/ Disallow: /admin # 7) Sitemapy (podaj realne ścieżki indeksu) # 8) Tempo crawlu (Google ignoruje, Bing/Yandex respektują) Crawl-delay: 1