[Content Extraction] not able to show the blog

[santoshyadavdev]

What URL did you save?

https://www.coderabbit.ai/blog/behind-the-curtain-what-it-really-takes-to-bring-a-new-model-online-at-coderabbit?utm_source=breadbox

What kind of problems are you seeing?

• Missing/incorrect favicon
• Missing/incorrect image
• Missing/incorrect title
• Missing/incorrect author(s)
• Missing article content
• Extra HTML artifacts in article content
• Other (please specify below)

Describe the problems you're seeing in more detail. Please provide a screenshot if possible.

The content is not extracted properly

Anything else we should know?

No response

[nzakas]

Thanks for letting me know. It looks like the blog post is actually rendered by JavaScript via Next.js, so just doing a plain fetch() doesn't return the article content. I'll need to dig into this a bit.

[nzakas]

Just wanted to give an update: I tried a couple of approaches to address this but nothing worked. I'm not sure if they're employing some kind of anti-scraping technique but other read-later apps also seem to have trouble with this site. Unsure at the moment if this is a Next.js thing or specific to this site.

For now, I'm going to focus on making Bredbox not try to extract data when it sees this type of pattern.

[nzakas]

After a bit more investigation, I've found that this site is employing anti-scraping measures to prevent bots from scraping. I have an idea about how to work around this, but it will take me a while to get to it.

[santoshyadavdev]

Sounds good, no rush, I am still able to save the link so all good