Don’t Panic, This Is Fixable
If you’re an artist, you’ve probably felt a jolt reading about AI companies collecting images and text from websites. Your artwork is original, and it’s easy to feel that someone else is profiting from it without asking you. Many artists are also frustrated with big tech companies that take first and ask later. And when something happens to your website, it’s hard to know how to fix it.
I’ve been a web designer for more than 20 years, and I’m also a working artist. Most artists never needed to learn this side of the web, so I wrote this tutorial to walk you through it in plain language. Start with the first step below, and don’t worry if you need to ask your platform for help.
What Is Meta AI Scraping?
Meta runs automated programs, called crawlers or bots, that visit websites and collect text and images. Some of that material can be used to train AI products. Your website has a small public file called robots.txt that tells these bots which parts of your site they’re allowed to visit. Think of it as a posted request. Well-behaved bots read it and follow it.
Meta publishes the names of its crawlers. These are the ones we’ll ask to stay out:
- meta-externalagent, which Meta uses to collect content that can be used to train its AI
- FacebookBot, which Meta also uses to gather public web content
- meta-webindexer, which Meta uses to index pages for its AI search features
We’re not blocking facebookexternalhit. That’s the one that creates the preview image when you share a link on Facebook or Instagram, so you’ll want to leave it alone.
The Text You’ll Add to Your robots.txt File
Copy these lines exactly as written. If you already have a robots.txt file, add them to the bottom and don’t delete anything that’s there.
- User-agent: meta-externalagent
- Disallow: /
- User-agent: FacebookBot
- Disallow: /
- User-agent: meta-webindexer
- Disallow: /
Each “User-agent” line goes on its own line, followed by its “Disallow” line. Leave a blank line between each pair. Meta may add or rename crawlers over time, so check their crawler documentation now and then for the current list.
Step-by-Step by Platform
WordPress
I run several WordPress sites, so this is the setup I know best. Your easiest route depends on what you have installed.
- If you use Yoast SEO, go to Yoast SEO, then Tools, then File editor. Paste the lines into the robots.txt box and save.
- If you use Rank Math, go to Rank Math, then General Settings, then Edit robots.txt. Paste the lines and save.
- If you don’t use either plugin, log in to your hosting account and open the File Manager. Look in the main folder of your site (often called public_html) for a file named robots.txt. Open it, add the lines, and save. If the file doesn’t exist, create a plain text file with that name and paste the lines in.
- Don’t use the “Discourage search engines from indexing this site” box in WordPress Settings. It’s a different feature, and it can hide your site from Google.
Then visit yourwebsite.com/robots.txt in your browser to confirm the new lines appear.
Adding Cloudflare’s Free Plan to a WordPress Site
A robots.txt file asks bots to stay out. Cloudflare can actually stop them at the door, and the free plan is enough for this. If you’re comfortable with a few settings, here’s the path:
- Create a free account at cloudflare.com and add your website.
- Choose the Free plan.
- Cloudflare will copy your existing website records. Compare them to what your host shows, especially anything related to email, so nothing breaks.
- Cloudflare will give you two nameservers. Log in to wherever you bought your domain and replace the current nameservers with those two. This change can take a few hours to finish.
- Once your site is active, open the Security section and look for the bot settings. Turn on the option to block AI bots or AI crawlers. Cloudflare changes its menu names from time to time, so use the search box in your dashboard if you can’t find it.
- For extra control, create a custom security rule. Set it so that if the User Agent contains “meta-externalagent,” the action is Block. Add a second rule for “FacebookBot” if you like.
If changing nameservers feels like too much, contact your host’s support and ask them to help you add Cloudflare. They do this all the time.
Squarespace
Squarespace doesn’t let you edit the robots.txt file directly. Instead, it has a setting to block known AI crawlers. Log in, open your site settings, and search for “crawlers” or “AI crawlers” in the settings search. Turn on the option to block known artificial intelligence crawlers. If you can’t find it or want to confirm Meta’s bots are included, search the Squarespace help center or contact their support team.
Wix
Wix includes a robots.txt editor in its SEO tools. Open your dashboard, find Marketing and SEO (or SEO Tools), then choose the robots.txt editor. Add the lines above and save. If the menu looks different, search the Wix help center for “robots.txt” or message their support team.
Art Galleria and ArtCloud
Each of these platforms controls its own settings. Search the platform’s help section for “AI crawlers” or “robots.txt.” If you don’t find a clear answer, contact their support team and ask this exact question: “Can I block Meta’s AI crawlers, meta-externalagent and FacebookBot, from my site?”
Any Other Platform
If your website is built on a platform not listed here, contact its technical support team. Ask them how to block AI crawlers or edit your robots.txt file.
What I Do on My Own Sites
Blocking Meta is one step in how I look after my websites. On my WordPress sites, I:
- Keep my robots.txt file updated to block AI training crawlers
- Use the server-side protections offered by my hosting company
- Run technical updates every month
- Check visitor activity in Google Site Kit (a summary of Google Analytics)
- Use plugins that watch for cyber attacks
None of these steps is complicated on its own, and together they give you a clear picture of who is visiting your site.
Will This Stop Every Bot?
Think of this as a preventative measure, the same as locking your studio door. It helps, but it isn’t a guarantee. Technology moves fast, and some companies use practices that are hard to detect or that ignore the rules entirely. I’m not a cybersecurity expert. I do everything I can within my professional skill set to protect my digital work, and when I can’t fix something, I bring in a subject matter expert. I’d encourage you to do the same.
Your Next Step
Choose the path that fits you:
- If you have access to your hosting provider or website settings, follow the tutorial for your platform above and update your robots.txt file today.
- If you don’t, contact your host’s support team and ask for help blocking Meta’s AI crawlers. If you’re planning a new site or a rebuild, our artist website design service is another option.
- If you use Squarespace, Wix, or Art Galleria Website, search their help section for AI crawler recommendations or message their support team.
