JCE Blocked by Robots.txt - Google Search Console Error

You need to be logged in to post in the forum - Log In

An active JCE Pro Subscription is required to post in the forum - Buy a Subscription

Support is currently Online

Official support hours

Monday to Friday
09:00 - 17:00 Europe/London (BST)

#111780 JCE Blocked by Robots.txt - Google Search Console Error

Posted in ‘Editor’

This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Latest post by ibweb on Monday, 26 December 2022 22:43 GMT

Saturday, 03 September 2022 16:17 BST

ibweb

We noticed on at least two websites that there is a warning when using the URL inspection tool within Google Search Console.
Specifically this is the warning shown on all pages:
Googlebot blocked by robots.txt Stylesheet https://[DOMAIN].com/plugins/system/jce/css/content.css?badb4208be409b1335b815dde676300e

Version Information:
JCE Editor (used as global editor): 2.9.29
Theme: Yootheme Pro 2.7.22
Joomla: 3.10.10
PHP: 7.4.x

Please advise why JCE resources are being blocked by robots.txt. We see that /plugins/ is blocked, but shouldn't that resource be stored elsewhere to avoid being blocked (e.g. media folder)?

Robots.txt output:
# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

Monday, 05 September 2022 09:41 BST

Ryan

Please advise why JCE resources are being blocked by robots.txt. We see that /plugins/ is blocked, but shouldn't that resource be stored elsewhere to avoid being blocked (e.g. media folder)?

There is no need for the content.css file to be indexed by Google at all, as it is system file, not a displayable media file.

Ryan Demmer

Lead Developer / CEO / CTO

Just because you're not paranoid doesn't mean everybody isn't out to get you.

Thursday, 29 September 2022 19:03 BST

ibweb

Thank you for the clarification. I understand that it is not required to be indexed by Google. But if Google's bot is seeing that as blocked every time and it's referenced on numerous pages, wouldn't it be better to output that file be stored in the media folder so it can be indexed? Or is it actually preferred that the file generated not be indexed.

Friday, 30 September 2022 11:49 BST

Ryan

But if Google's bot is seeing that as blocked every time and it's referenced on numerous pages, wouldn't it be better to output that file be stored in the media folder so it can be indexed? Or is it actually preferred that the file generated not be indexed.

The media folder does not contain content media, ie: media items (images, pdf, video etc.) that is displayed in articles, and the kind of stuff you want Google to index so it is searchable. The media folder generally contains system assets - javascript and css files, image icons etc. - used by Joomla and installed extensions. You don't usually want this media to be indexed as it has no search value in terms of the content of your site. Your robots.txt file should therefore only allow access to stuff you want to be indexed, such as the contents of the images folder, eg:


User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/

Ryan Demmer

Lead Developer / CEO / CTO

Just because you're not paranoid doesn't mean everybody isn't out to get you.

Monday, 26 December 2022 22:43 GMT

ibweb

I apologize for the lengthy delay. Given that you have confirmed that "there is no need for the content.css file to be indexed by Google at all, as it is system file, not a displayable media file", I believe we are good to close this out.