[login to view URL] crawler

Končano Objavljeno pred 5 letoma/leti Plačilo ob prevzemu
Končano Plačilo ob prevzemu

We want a crawler to capture data about jobs and companies advertising those jobs from Monster.com. It should be written in c# and use a SQL Server database.

1. Start crawl at [login to view URL]

2. Browse jobs by Category, looping through each category

3. Jobs are returned by Ajax. The program should retrieve all available jobs, including the ones that can be accessed by repeatedly clicking the "Load more jobs" button in the web user interface.

4. For each jobs, capture/save to database: Job title, company name, When posted, city, state. Also, use a regular expression to extract any email addresses from the job ad text and associate those with the job ad in the database.

5. Program should be able to use Proxies

6. Allow proxy IPs to be configured in a database table which includes proxyip:port, username, password. If username and/or password are not null, they should be used as the proxy server credentials.

7. Program should switch to a different proxy IP (the one used least recently) for each job title it searches

8. activity of the crawler should be logged to log files that include the date in the filename. For example, ljs[date].log.

9. at startup, the application should delete any logs that are more than 4 days old

10. crawler should be able to run unattended by the windows task scheduler

11. when crawling before retrieving a job, check to see if it's already in the database. If it is, skip it

12. crawler should pause a configurable amount of time after each HTTP request (default 500 miliseconds)

.NET C# programiranje Microsoft SQL strežnik

ID projekta: #18575829

Več o projektu

13 predlogov Oddaljen projekt Aktiven pred 5 letoma/leti

Dodeljeno:

CreatedAutomated

Hey I'd love to develop you a Windows Desktop Application to crawl Monster and scrape up all the jobs based on the categories and such. This is an interesting project to me and would love to work on it and provide y Več

$183 USD v 4 dneh
(9 mnenj)
3.7

13 freelancerjev ponuja v povprečju za $428 na tem delu

bestit4u

Hi. I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert and s Več

$155 USD v 3 dneh
(24 ocen)
5.9
asin1987

Dear Employer, I will do everything that you need but for that I would need atleast 500usd to configure this. Please message me if fine with you. Thanks and Regards, Ash

$155 USD v 3 dneh
(10 ocen)
5.3
ONYEMATeam

I would like to inform you also that I am highly skilled with : • 10+ years of professional experience in C#/.NET software development; • 5+ years Experience with ASP.NET MVC , LINQ , EF , SQL; • 10+ years Experie Več

$2000 USD v 3 dneh
(16 ocen)
4.7
inthebush

Hi I believe my experience and knowledge for .Net Crawler like Abot makes me a great fit for this job. You can see good experiences and feedback in this type of jobs after having a look at my prior work. $1000 is Več

$1000 USD v 3 dneh
(6 ocen)
4.3
vascodiasferreir

Hello, I believe i can develop this software for you, as i've worked alot with HTML elements in C#. I can gather that information and place it inside the SQL server quite easily. I look forward to working with you. B Več

$150 USD v 7 dneh
(15 ocen)
3.7
minaljunction123

Hi, I have now gone through the requirement in detail. I have completed although not same but similar projects. . Please feel free to contact me to discuss in detail.

$155 USD v 15 dneh
(0 ocen)
0.0