Node save web page and send to s3 with npm website-scraper and aws-sdk packages

$30-250 USD

Opravljeno

Objavljeno

pred več kot 5 leti

$30-250 USD

Plačilo ob dostavi

Write a node script which scrapes a webpage using the npm package website-scraper [login to view URL] you must use your own AWS account, the script will use [login to view URL]() to set credentials Use default options and save just the one page / URL to a tmp folder Then use the AWS SDK ( [login to view URL] ) and send the folder to an s3 bucket. For each file set the correct content-type Here is some code to get you started var recursive = require("recursive-readdir"); const options = { urls: ["[login to view URL]"], directory: "/tmp/yahoo" }; scrape(options, (error, result) => { recursive([login to view URL], function(err, files) { for(i = 0; i< [login to view URL];i++){ file = files[i]; file = [login to view URL](1); var params = { Body: // set file content here Bucket: "bucket_name", Key: file, ContentType: // set content type here ACL: 'public-read', }; [login to view URL](params, function(err, data) { if (err) [login to view URL](err, [login to view URL]); // an error occurred else [login to view URL](data); // successful response }); } }); }); Budget is less than $100

Amazon Web Services

Node.js

ID projekta: 18275015

Več o projektu

3 ponudb

Projekt na daljavo

Aktivno pred 5 leti

Želite zaslužiti?

E-naslov

Prednosti oddajanja ponudb na Freelancerju

Nastavite svoj proračun in časovni okvir

Prejmite plačilo za svoje delo

Povzetek predloga

Registracija in oddajanje ponudb sta brezplačna

Dodeljeno:

@SamuelRMeyer

I am a hobbyist JavaScript developer experienced in NodeJS. I have also worked on a couple personal projects which have involved website scrapping. For this reason, I believe I would be a good fit for this project. For a examples of my past work, I would encourage you to take a look at my other personal projects on my github page [login to view URL] Please message me with further details about the project to see if I can help. Thanks for your consideration, Samuel

$94 USD v 5 dneh

5,0

(2 ocen)

3,2

3 freelancerjev je oddalo ponudbo s povprečno vrednostjo $148 USD za to delo

@abdulbarikmb

Hi there, I am Node.js AWS expert, I can scrap the the website with your required modules and can save it to S3 bucket and can give you demo on my machine before delivering the code base. I have done several projects with AWS serivces like S3, Lambda, Kinesis, SQS,SES, SNS, EC2 etc. you may have look at my profile. I have done several projects in Node.js with AWS services. I have 4+ years of experience in software development. I am Sr. Full Stack Developer having experience in Node.js, REST APIs, JavaScript, AngularJS(1/2/4/5),ExpressJS, Google Cloud Function,Google Firebase function, AWS ,SQL(MySQL, PostgreSQL, MS SQL Server, Sqlite3), NoSQL(MongoDB, CouchDB), Linux, Git,Nginx. I am good to deliver your code base in $100 Looking forward to work with you Thanks...

$111 USD v 1 dnevu

5,0

(29 ocen)

4,7

@yash109

I have worked with node scraped pages using python So it will be an opportunity for me to save webpages via node. I have my AWS account and I am Cloud Engineer by profession ensuring continuous Integration and Delivery of services using AWS and automation tools like Chef, Jenkins, Docker, Kubernetes Specialities: Continuous Integration, Continuous Delivery, Docker Containers, Container orchestration platforms, Infrastructure Automation, Microservices, Monitoring and log management List of Software, Tools used/working on • Automation - Chef, Vagrant, Docker, Kubernetes • CI/CD - Jenkins • Amazon Web Services CloudWatch, S3, EC2, Load Balancers, Auto Scaling, VPC, VPC Peering, VPC - Site VPN, IAM, Cloud Front, Elastic Search, RDS, SES, SQS, SNS, ElastiCache, RDS, EC2 Container Service

$240 USD v 3 dneh