Intrebari despre fisierul Robots.txt

#1
mihaixmen

Posted 31 July 2015 - 22:43

New Member

Grup: Junior Members
Posts: 4
Înscris: 31.07.2015

Buna ziua ! Vreau sa stiu daca acest code pentru robots.txt este bun pentru a indexa tot continutul site-ului!

code :

Quote

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow:
Allow: /
Sitemap: site/sitemap.xml

Codul original era asa (avea "/search" in plus):

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: site/sitemap.xml

Multumesc !

Edited by mihaixmen, 31 July 2015 - 22:44.

Back to top

#2
Adm

Posted 31 July 2015 - 23:01

Property Caretaker

Grup: Administrators
Posts: 24,136
Înscris: 22.05.2003

Ce doresti tu, mai precis sa faca acel fisier?

User-agent: Mediapartners-Google
Disallow:

==== codul de mai sus permite botului adsense acces la site ===

User-agent: *
Disallow:
Allow: /

==== codul asta e redundant sau mai precis cel de bot adsense e redundant, pt ca asta spune ca permiti oricarui bot indexarea (deci si celui adsense) ===

Avea logica daca ramanea

Disallow: /search

Asta implica ca are acces oriunde, mai putin directorul /search.

===
Asadar, ce urmaresti?

Sitemap: site/sitemap.xml

Back to top

#3
mihaixmen

Posted 31 July 2015 - 23:04

New Member

Grup: Junior Members
Posts: 4
Înscris: 31.07.2015

Multumesc pentru raspuns! Eu vreau in special ca googlebot sa indexeze tot continutul site-ului , si nu vreau sa sara peste /search ! Am observat ca daca sterg /search dispar pagini ale site-ului de pe google!
Sau mai bine zis vreau ca toate motoarele de cautare sa indexeze tot site-ul ! Cum fac acest lucru?

Back to top

#4
Adm

Posted 31 July 2015 - 23:18

Property Caretaker

Grup: Administrators
Posts: 24,136
Înscris: 22.05.2003

User-agent: *
Allow: /

Back to top

#5
mihaixmen

Posted 31 July 2015 - 23:27

New Member

Grup: Junior Members
Posts: 4
Înscris: 31.07.2015

Multumesc !
daca mai pun si acest cod:

User-agent: Mediapartners-Google
Disallow:

ajuta cu ceva ?

Back to top

#6
Adm

Posted 31 July 2015 - 23:28

Property Caretaker

Grup: Administrators
Posts: 24,136
Înscris: 22.05.2003

nu

Back to top

#7
mihaixmen

Posted 31 July 2015 - 23:30

New Member

Grup: Junior Members
Posts: 4
Înscris: 31.07.2015

Ok , multumesc mult!

Back to top

#8
open-your-eyes

Posted 31 October 2017 - 18:18

Member

Grup: Members
Posts: 841
Înscris: 04.01.2017

Adm, on 31 iulie 2015 - 23:18, said:

User-agent: *
Allow: /

Salut, Adm. Stiu ca tu te pricepi. Ce vreau sa te intreb este daca am un sitemap, daca trebuie adaugat si el in robots.txt?
Deci cum ar trebui sa scriu in robots.txt? Asa?

User-agent: *
Allow: /
Sitemap: http://onanisti.ro/sitemap.php

Altii vor sa foloseasca robots.txt pentru a limita accesul botilor. Eu sunt invers, vreau sa imi fie indexat saitul.
Daca am deja sitemap, ma ajuta cu ceva adaugarea unui robots.txt?
Multumesc.

Back to top

#9
OriginalCopy

Posted 31 October 2017 - 19:07

I'm harmful, fear me please! :))

Grup: Senior Members
Posts: 27,268
Înscris: 10.08.2006

https://developers.g...ots_txt#sitemap

Back to top

#10
open-your-eyes

Posted 31 October 2017 - 19:17

Member

Grup: Members
Posts: 841
Înscris: 04.01.2017

^ Nu inteleg.

Back to top

#11
sonamg

Posted 02 August 2018 - 14:46

New Member

Grup: Junior Members
Posts: 5
Înscris: 16.07.2018

Robots.txt it’s a text file, it's instructed to the search engine which pages on your site to crawl.

Robots.txt file allowed
User-agent: *

Robots.txt file not allowed
Disallow: /*?

Back to top

Anunturi

Bun venit pe Forumul Softpedia!

Intrebari despre fisierul Robots.txt

#1 mihaixmen Posted 31 July 2015 - 22:43

#2 Adm Posted 31 July 2015 - 23:01

#3 mihaixmen Posted 31 July 2015 - 23:04

#4 Adm Posted 31 July 2015 - 23:18

#5 mihaixmen Posted 31 July 2015 - 23:27

#6 Adm Posted 31 July 2015 - 23:28

#7 mihaixmen Posted 31 July 2015 - 23:30

#8 open-your-eyes Posted 31 October 2017 - 18:18

#9 OriginalCopy Posted 31 October 2017 - 19:07

#10 open-your-eyes Posted 31 October 2017 - 19:17

#11 sonamg Posted 02 August 2018 - 14:46

Anunturi

▶ 0 user(s) are reading this topic

Sign In

#1
mihaixmen

Posted 31 July 2015 - 22:43

#2
Adm

Posted 31 July 2015 - 23:01

#3
mihaixmen

Posted 31 July 2015 - 23:04

#4
Adm

Posted 31 July 2015 - 23:18

#5
mihaixmen

Posted 31 July 2015 - 23:27

#6
Adm

Posted 31 July 2015 - 23:28

#7
mihaixmen

Posted 31 July 2015 - 23:30

#8
open-your-eyes

Posted 31 October 2017 - 18:18

#9
OriginalCopy

Posted 31 October 2017 - 19:07

#10
open-your-eyes

Posted 31 October 2017 - 19:17

#11
sonamg

Posted 02 August 2018 - 14:46