Find Jobs
Hire Freelancers

spark application on google cloud platform for fetching and processing data from hdfs

₹1500-12500 INR

Zaprt
Objavljeno pred približno 6 leti

₹1500-12500 INR

Plačilo ob dostavi
Hello we are looking for a scala developer who has experience working on handling data in .packet form on spark clusters on google cloud platform. Basically the task is to access data from hdfs in .packet form, query through the data for relevant UIDs, fetch some specific fields in those UIDs, process parameters by performing some mathematical computations on those fields for those specific UIDs and store the processed values in a separate .packet file on hdfs. Further aggregation needs to be performed on the computed values, and final summary file needs to be stored into Mongo dB. The technologies you need to be comfortable with : Dataproc on google (cloud native hadoop and spark), airflow (will be used for scheduling), google cloud platform (in general), scala (for scripts), Mongo dB (for data export)
ID projekta: 16298138

Več o projektu

8 ponudb
Projekt na daljavo
Aktivno pred 6 leti

Želite zaslužiti?

Prednosti oddajanja ponudb na Freelancerju

Nastavite svoj proračun in časovni okvir
Prejmite plačilo za svoje delo
Povzetek predloga
Registracija in oddajanje ponudb sta brezplačna
8 freelancerjev je oddalo ponudbo s povprečno vrednostjo ₹12.156 INR za to delo
Avatar uporabnika
I have work experience 5 year in big data technology . I have experience in elasticsearch, java, scala , spark. for more info ping m.e
₹11.111 INR v 7 dneh
3,9 (3 ocen)
4,1
4,1
Avatar uporabnika
Hi, I have more than 3+ years of experience in Hadoop technologies like mapreduce , spark, hdfs etc. I can complete your project contact me for more details
₹10.000 INR v 3 dneh
4,5 (4 ocen)
3,6
3,6
Avatar uporabnika
I am interested to work on this project as I have relevant experience in Big Data,Sqoop, Hadoop, Spark, Hive, Kafka, Spark Streaming, Rdd, Datframe, Dataset , Python, Scala, google cloud, azure, aws. I am well versed in Installation and congratulations of Hadoop ecosystem components. looking forward to work with you.
₹11.111 INR v 6 dneh
5,0 (4 ocen)
0,8
0,8
Avatar uporabnika
hi, I am hadoop, sparkand nosql engineer with 6 years of experience. can do this, comfortable with spark, scala, airflow, Google cloude. data proc I can manage.
₹2.250 INR v 1 dnevu
4,0 (1 ocena)
0,0
0,0
Avatar uporabnika
We have 8 years of experience working in Machine Learning. We have built various recommendation engines, web apps, crawlers, analytical dashboards etc. We have rich experience in Python, Spark, R, Scala, Cassandra, Hive, Hbase etc. Price quoted is ballpark. Please share details to discuss about your project.
₹7.777 INR v 3 dneh
0,0 (0 ocen)
0,0
0,0
Avatar uporabnika
Languages: JAVA. Java/J2EE: Core JAVA,JAVAFX, Advanced JAVA, Servlet, JSP, JSTL, EJB, JDBC, Junit, Web Services, XML, XSD, JAX-RS, DOM, SAX, Multithreading, JTA, Custom Tags, JPA API’s. Web Technologies: Html, DHTML, JavaScript, JQuery, Anguler JS, CSS, AJAX, Web Services ( REST). Big Data : Hadoop,Spark,Hive,Pig,Kafka,Hbase,Sqoop,Flume,Impala,MongoDB. Databases: Oracle, MYSQL. Database Tools: SQL Developer. ORM Mapping: JPA, Hibernate Web/Application Servers: Jetty 8.x/9.x, WebLogic 12.x/11.x/9.x, Apache Tomcat 6.x/7.x. DevOps Tools: Jenkins,BugZilla,Jira Frameworks: Struts 1.x/2.x, Hibernate 3.x/4.x, Spring MVC, Spring 2.x/3.x, Spring Boot. Operating Systems: Mac, Linux and Windows. IDEs / Versioning / Performance /Tools: Eclipse, My Eclipse, Net Beans, git, SVN, CVS, Ant, Maven, Log4j. Testing Tools: Junit, TestNG. Methodologies: Waterfall Model, Agile, OOAD, Test Driven Development (TDD).
₹7.777 INR v 3 dneh
0,0 (0 ocen)
0,0
0,0

O stranki

Zastava INDIA
BANGALORE, India
4,9
80
Plačilna metoda je verificirana
Član(ica) od jul. 28, 2016

Verifikacija stranke

Hvala! Po e-pošti smo vam poslali povezavo za prevzem brezplačnega dobropisa.
Pri pošiljanju vašega e-sporočila je šlo nekaj narobe. Poskusite znova.
Registrirani uporabniki Skupaj objavljenih del
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Nalaganje predogleda
Geolociranje je bilo dovoljeno.
Vaša prijavna seja je potekla, zato ste bili odjavljeni. Prosimo, da se znova prijavite.