Facebook link preview feature used as proxy in website scraping scheme

Image: Facebook

Several data recovery groups have abused the Facebook link preview feature to retrieve data from websites disguised as Facebook’s content crawler.

The technique involved using Facebook developer accounts to make calls to Facebook or Facebook Messenger API servers, requesting link preview for pages a group wanted to grab.

Facebook would grab the data, assemble it into a link preview, and send it back to the data scrappers as an API response, ready to be ingested into the scrapper’s database.

The technique was successful because most website operators allow Facebook servers to crawl their sites, knowing that the data that Facebook collects from their pages is typically used for legitimate purposes, as part of link previews on the social network, Facebook Messenger, WhatsApp or Instagram.

Several groups abused the technique

But in a report published last week by DataDome, a security company that provides bot detection capabilities for online sites, the company said it has discovered several “scratching operators” using the technique to (ab) use Facebook as a proxy for their scratching activities. data.

DataDome said it identified several groups abusing the technique at multiple sites, but the initial detection came on the network of one of its customers, a classifieds portal.

“Our heuristic analysis revealed that certain parameters, unlikely to be used by humans, were overrepresented in the URLs requested by Facebook,” DataDome explained.

This included the URLs of the classifieds site pages that users would not normally share on Facebook frequently, such as search results pages – a dead sign that someone was scratching the classifieds site for recent entries.

Testing by the DataDome team confirmed the effectiveness of the technique and found that data scraping groups could abuse this feature to retrieve link previews up to 10,000 URLs / hr from a only Facebook developer account.

The French security firm said it informed Facebook of the attacks earlier this year.

“Facebook has now improved the rate limiting on Messenger’s preview API. As our testing (and some thread on hacker forums) confirms, this effectively prevents continued abuse of the preview function for scratching purposes, ”the security company said.

A Facebook spokesperson confirmed the scrapes and the API fix, but the company had nothing to add besides the DataDome report.