mirror of
https://github.com/Dvorinka/facr-scraper.git
synced 2026-06-03 12:02:58 +00:00
Tomas Dvorak
ed61d8ab8e
Integrate CloakBrowser to improve success rates against Cloudflare challenges and implement more robust request handling in the Go backend. - Add CloakBrowser integration to Dockerfile and requirements - Implement domain-specific request semaphores in Go to prevent rate-limiting - Add shared HTTP client with cookie jar and header preservation for better session management - Enhance request headers in Go to include modern client hints (Sec-Ch-Ua) - Add benchmarking scripts to compare fetch methods (urllib vs Scrapling vs CloakBrowser) - Update docker-compose to support CloakBrowser environment variables - Optimize Docker image by pre-downloading patched Chromium binaries
3 lines
48 B
Plaintext
3 lines
48 B
Plaintext
scrapling[fetchers]==0.4.1
|
|
cloakbrowser==0.3.28
|