i wrote a fairly complex script in perl for scraping fanfiction off the internet, extracting data from it, and formatting that data into beautifully and consistently formatted epubs to read on my ereader.
because i’d previously been scraping by hand and i’ve never heard of that tool before right now. i regret nothing; i learned a ton. i’m curious how they got around the cloudflare captcha issue.
Nothing wrong with that of course. There are advantages to making your own things.
FanFicFare isn’t perfect either. For cloudflare it supports using FlareSolverr, but from what I heard it doesn’t always work well.
Alternatively it can open the pages in your browser and grab the content from the browser cache.
I mainly like that it’s available as a Calibre plugin and that it supports just about any fan fiction site.
i wrote a fairly complex script in perl for scraping fanfiction off the internet, extracting data from it, and formatting that data into beautifully and consistently formatted epubs to read on my ereader.
Is there a reason you wrote your own instead of using FanFicFare? Just curious
because i’d previously been scraping by hand and i’ve never heard of that tool before right now. i regret nothing; i learned a ton. i’m curious how they got around the cloudflare captcha issue.
Nothing wrong with that of course. There are advantages to making your own things.
FanFicFare isn’t perfect either. For cloudflare it supports using FlareSolverr, but from what I heard it doesn’t always work well. Alternatively it can open the pages in your browser and grab the content from the browser cache.
I mainly like that it’s available as a Calibre plugin and that it supports just about any fan fiction site.