weixin_39815456
weixin_39815456
2020-12-08 18:43

facebook crawler

Description

I believe there is an issue with the facebook crawler not getting through the bot whitelist.

How to reproduce

I currently use the setup a middleware detector for outdated browsers, similar to that in the documentation:


Rails.configuration.middleware.use Browser::Middleware do
  redirect_to '/406' unless browser.modern?
end

for When I go to test the facebook share dialog here I constantly get redirected implying the bot didn't pass through. When I change it to the following, facebook is able to get through no problem.


Rails.configuration.middleware.use Browser::Middleware do
  next if browser.ua.include? 'facebookexternalhit'
  redirect_to '/406' unless browser.modern?
end

What do you expect

I expected the built in bot.yml file to allow this, but it appears facebook might have changed the user agent? - More info here

What happened instead

[Describe the actual results]

Software:

  • Rails version: 4
  • Browser gem version: 2.0.2

Full backtrace


(No backtrace - just a redirect that shouldn't happen)

Report

Unfortunately the new site isn't live yet so I can't post a url

该提问来源于开源项目:fnando/browser

  • 点赞
  • 回答
  • 收藏
  • 复制链接分享

4条回答

为你推荐