Python Scrapy and Selenium Scraper developer

Končano Objavljeno pred 5 letoma/leti Plačilo ob prevzemu
Končano

Hi,

We are developing scrapers for some websites that require a combination of fast scraping using Scrapy and Javascript processing using Selenium. This involves extracting information from each page, downloading images or files, and storing the raw data to disk or S3.

We would prefer to work with someone who can help identify the data models and a scraping strategy based on a description of what type of data is desired for a site. We have 4 sites that we’ll be processing in the short term

Skills:

• Creating scraper host environment using docker or ansible

• Interacting with page elements using selenium

• Leveraging HTTP/HTTPs proxies

• Working with sites on TOR network

• Working with AWS S3 and SQS

• Scraping using authenticated account

• Outputting logs or results to Elasticsearch and Splunk

• BS4 for advanced processing of stored HTML

Our general process is the same for all sites:

• Develop model for data

• Initially scrape all pages for site

• Download images or files from pages

• Save results and raw html to s3

• Output results and logs to Elasticsearch and Splunk

Python Pridobivanje spletnih informacij

ID projekta: #18269470

Več o projektu

14 predlogov Oddaljen projekt Aktiven pred 5 letoma/leti

Dodeljeno:

oswaldoodavidal

Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! I hope to work with you!

$5 USD / ura
(4 mnenj)
4.1

14 freelancerjev ponuja povprečno za $11/uro na tem delu

zekovicm

Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within Več

$25 USD / ura
(104 ocen)
7.3
Mickelson

Hi Nice to meet you. I'm python expert and my last project on freelancer.com is also python scraping task. My past works: Youtube comment scrapping Real estate property list to csv Job-site content to csv And sc Več

$11 USD / ura
(72 ocen)
6.3
hunmin888

Hi employer. i am a scrapping expert. so if you give me this project, i can complete it in time. i have a good experience in scrapy, selenium, urllib, requests and aiohttp. if you are interest me, ping me. i made Več

$5 USD / ura
(50 ocen)
6.3
liveexperts123

typing.. ............... .................

$5 USD / ura
(15 ocen)
5.8
shingjin

Hello. After reviewing your post, I am very interested in that due to my experience. I’d like to be considered for your project position. I am an expert python selenium web scraper. I will do my best. Regards.

$11 USD / ura
(47 ocen)
5.8
kunitsynartem

Hello! I have 1.5 years of experience in web scraping using Python including following skills: - using Scrapy (also with Splash engine to load JS-generated content) - using proxies - interacting with page elements u Več

$11 USD / ura
(33 ocen)
5.5
AhmadSameer

Hey I'm a System Engineer with coding skills. I had done many scraping projects using Selenium and Scrapy. Regards.

$16 USD / ura
(12 ocen)
4.5
vhsoftwareBA

Hello. Your project is interesting as scraping with selenium, bs4, lxml and the others is my main profession. I prefer to use minimalistic approach in scraping. That means that I use Selenium only in case that advan Več

$10 USD / ura
(5 ocen)
4.3
jk901

Dear As I am mobile and web developer, I have good experience with python, javascript and php. If you are interested with my proposal, please let me know it. Thanks.

$7 USD / ura
(4 ocen)
3.8
SwapnilChouhan

Hi, I have experience with Python Scrapy and also AWS S3, Elastic Search. I am really interested in scraping websites. Please let me know URL and access to AWS, Thanks, Swapnil

$6 USD / ura
(15 ocen)
4.4
aayush66

I have done many scrapy projects in my organisation, so I hope I can deal with this requirement in efficient way.

$11 USD / ura
(0 ocen)
0.0