MENU service case
Website construction website design Beijing website construction high-end website production company Shangpin China
We create by embracing change
360 ° brand value__
simplified Chinese character
Simplified Chinese English

Shangpin China Joins Hands with Beisheng Internet to Create a New Chapter in Website Construction

Type: Shangpin Dynamic Learn more

Common mistakes in the use of SEO website optimization robots. txt

Source: Shangpin China | Type: website encyclopedia | Time: April 28, 2012
Website production Shangpin China (ihucc.com): Myth 1. Allow covers more than Disallow and is in front of Disallow.

For example, many people wrote:

　　User-agent: *

　　Allow: /

　　Disallow: /mulu/

If you want all spiders to crawl all the pages of the site except the url below/mulu/, you are not allowed to crawl all the urls below/mulu/.

　　 SEO website optimization The search engine spider executes the rule from top to bottom, which will cause the failure of the second command. That is to say, the rule prohibiting fetching the/mulu/directory is invalid. The correct thing is that Disallow should be above Allow in order to take effect.

Myth 2. Disallow and Allow commands do not begin with a slash "/".

This is also easy to ignore. For example, someone wrote Disallow: 123.html, which has no effect. The slash "/" indicates the location of the file to the root directory. If it is not, it is impossible to determine the URL, so it is an error.

The correct one is Disallow:/123.html, so that the/123.html file can be blocked and this rule can take effect.

Myth 3. There is no slash "/" behind the masked directory.

For example, if you want to block the directory/mulu/, some people write Disallow:/mulu. Is that right?

First of all, writing (Disallow:/mulu) can indeed block all urls below/mulu/, but it also blocks all pages starting with/mulu, such as/mulu123.html. The effect is the same as Disallow:/mulu *.

The correct way is to use a slash "/" at the end if you only want to mask the directory.
This article was published by Beijing website production company Shangpin China //ihucc.com/
Source Statement: This article is original or edited by Shangpin China's editors. If it needs to be reproduced, please indicate that it is from Shangpin China. The above contents (including pictures and words) are from the Internet. If there is any infringement, please contact us in time (010-60259772).
TAG label:

What if your website can increase the number of conversions and improve customer satisfaction?

Make an appointment with a professional consultant to communicate!

* Shangpin professional consultant will contact you as soon as possible

Disclaimer

Thank you very much for visiting our website. Please read all the terms of this statement carefully before you use this website.

1. Part of the content of this site comes from the network, and the copyright of some articles and pictures involved belongs to the original author. The reprint of this site is for everyone to learn and exchange, and should not be used for any commercial activities.

2. This website does not assume any form of loss or injury caused by users to themselves and others due to the use of these resources.

3. For issues not covered in this statement, please refer to relevant national laws and regulations. In case of conflict between this statement and national laws and regulations, the national laws and regulations shall prevail.

4. If it infringes your legitimate rights and interests, please contact us in time, and we will delete the relevant content at the first time!

Contact: 010-60259772
E-mail: [email protected]

Communicate with professional consultants now!

  • National Service Hotline

    400-700-4979

  • Beijing Service Hotline

    010-60259772

Please be assured to fill in the information protection
Online consultation

Disclaimer

Thank you very much for visiting our website. Please read all the terms of this statement carefully before you use this website.

1. Part of the content of this site comes from the network, and the copyright of some articles and pictures involved belongs to the original author. The reprint of this site is for everyone to learn and exchange, and should not be used for any commercial activities.

2. This website does not assume any form of loss or injury caused by users to themselves and others due to the use of these resources.

3. For issues not covered in this statement, please refer to relevant national laws and regulations. In case of conflict between this statement and national laws and regulations, the national laws and regulations shall prevail.

4. If it infringes your legitimate rights and interests, please contact us in time, and we will delete the relevant content at the first time!

Contact: 010-60259772
E-mail: [email protected]