Adapt robots.txt

Adapt your robots.txt file
 

To pass the validation of the mobile configuration you must adapt your robots.txt file.

This file is not present in the cumulative and non-cumulative patches to not overwrite your possible changes such as blocking unwanted robots.
 

Note :

Sitemap: https://www.monsite.net/sitemaps.xml

This line is present only if you selected the generation of sitemaps in config services, of course www.monsite.net is replaced by the URL of your site.

As against the file exists in the pack GuppY 5.01.00.
 

Content of the robots.txt file on your website:

User-agent: *
Disallow: /compte.php
Disallow: /connect.php
Disallow: /admin/
Disallow: /data/
Disallow: /file/
Disallow: /flash/
Disallow: /inc/
Allow: /inc/csshead/
Allow: /inc/jshead/
Allow: /inc/img/
Allow: /inc/lang/*.gif$
Allow: /inc/*.css$
Allow: /inc/*.js$
Allow: /inc/calendar.php
Disallow: /install/
Disallow: /plugins/
Disallow: /save/
Disallow: */trackback
Disallow: /cgi-bin
Sitemap: https://www.monsite.net/sitemaps.xml

 

 

GuppY 5.02 : Adapt your robots.txt file by Papinou – Licence Libre CeCILL – June 2015 - update November 2018