Python Scrapy and Selenium Scraper developer
$2-8 USD /uro
Hi,
We are developing scrapers for some websites that require a combination of fast scraping using Scrapy and Javascript processing using Selenium. This involves extracting information from each page, downloading images or files, and storing the raw data to disk or S3.
We would prefer to work with someone who can help identify the data models and a scraping strategy based on a description of what type of data is desired for a site. We have 4 sites that we’ll be processing in the short term
Skills:
• Creating scraper host environment using docker or ansible
• Interacting with page elements using selenium
• Leveraging HTTP/HTTPs proxies
• Working with sites on TOR network
• Working with AWS S3 and SQS
• Scraping using authenticated account
• Outputting logs or results to Elasticsearch and Splunk
• BS4 for advanced processing of stored HTML
Our general process is the same for all sites:
• Develop model for data
• Initially scrape all pages for site
• Download images or files from pages
• Save results and raw html to s3
• Output results and logs to Elasticsearch and Splunk
ID projekta: #18269470
Več o projektu
Dodeljeno:
Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! I hope to work with you!
14 freelancerjev ponuja povprečno za $11/uro na tem delu
Hello. After reviewing your post, I am very interested in that due to my experience. I’d like to be considered for your project position. I am an expert python selenium web scraper. I will do my best. Regards.
Hello! I have 1.5 years of experience in web scraping using Python including following skills: - using Scrapy (also with Splash engine to load JS-generated content) - using proxies - interacting with page elements u Več
Hey I'm a System Engineer with coding skills. I had done many scraping projects using Selenium and Scrapy. Regards.
Hello. Your project is interesting as scraping with selenium, bs4, lxml and the others is my main profession. I prefer to use minimalistic approach in scraping. That means that I use Selenium only in case that advan Več
Dear As I am mobile and web developer, I have good experience with python, javascript and php. If you are interested with my proposal, please let me know it. Thanks.
Hi, I have experience with Python Scrapy and also AWS S3, Elastic Search. I am really interested in scraping websites. Please let me know URL and access to AWS, Thanks, Swapnil
I have done many scrapy projects in my organisation, so I hope I can deal with this requirement in efficient way.