Simple ETL process using Java/Spark

Končano Objavljeno pred 6 letoma/leti Plačilo ob prevzemu

$30-250 USD

Plačilo ob prevzemu

Končano Plačilo ob prevzemu

Write a ETL process using Java, Spark & HDFS.

Copy the input file to HDFS

Read the input file from HDFS using Java & Spark

Perform below function on the dataset

Average_Calculation()

For each stock , calculate the average trading volume for each month, average trading price for each month.

so for each stock , for each month , calculate the avg of volumen and average of stock close price

STOCK, AVG_VOLUME, AVG_CLOSING_PRICE, MONTH, YEAR

AAPL, 4343434 , 85, JULY, 2007

Write the output file back to HDFS

Run the ETL process using Spark in Cluster mode and client mode

Document all errors encountered and error resolution

------------------------------------------------------------------------------------------------------------------------------

Source input file:

STOCK, ASK_PRICE, BID_PRICE,OPEN_PRICE,CLOSE_PRICE, VOLUME,DATE

AAPL, 100.01, 100.02, 99.5, 99.7, 343434000, 12/7/2001

Destination file

STOCK, AVG_VOLUME, AVG_CLOSING_PRICE, MONTH, YEAR

AAPL, 4343434 , 85, JULY, 2007

Big Data Sales Hadoop Java Oracle Arhitektura porgramske opreme

ID projekta: #16258187

Več o projektu

10 predlogov Oddaljen projekt Aktiven pred 6 letoma/leti

Želiš zaslužiti nekaj denarja?

Prednosti oddajanja ponudb na Freelancerju

Nastavite svoj proračun in časovni okvir
Bodi plačan/a za svoje delo
Oriši svojo ponudbo
Vpis in prijave na dela so brezplačni

Dodeljeno:

Profilna slika deytps86
deytps86

Hello, I didn't notice another calculation below in second line so increasing a little amount of 20$ extra. "so for each stock , for each month , calculate the avg of volumen and average of stock close price" If pro Več

$60 USD v 2 dneh
(42 mnenj)
5.4

10 freelancerjev ponuja v povprečju za $153 na tem delu

Profilna slika chrisvwn
chrisvwn

Hi, I have experience using Java, Spark and HDFS in a Hadoop cluster and can implement your task for you for both Spark client and cluster modes. As an IT specialist I am able to setup, configure and troubleshoot th Več

$155 USD v 3 dneh
(11 ocen)
5.1
Profilna slika dineshrajputit
dineshrajputit

hi, I have expertise on spark,scala, java, hadoop.... done production scripts, scala job which process hdfs data and write back to hdfs. have read JSON, XML, CSV, tab, avro, parquet, orc file format. have read hive Več

$133 USD v 2 dneh
(7 ocen)
4.0
Profilna slika amitkumar0327
amitkumar0327

Hi, I am Amit. I have experience in Spark and Java. I can write the code as per the requirement you have given. Please share the input file for testing. And can provide you with documentation as well. Looking forward Več

$100 USD v 3 dneh
(12 ocen)
4.1
Profilna slika farrukhcheema23
farrukhcheema23

Hi, I am a professional Big Data Consultant with over 5 years of experience. I have read your request and interested to work for you as I am expert of Spark with Scala, and HDFS and can write a spark script for this pr Več

$111 USD v 3 dneh
(3 ocen)
2.7
Profilna slika VirtualBrainInc
VirtualBrainInc

I have briefly read the description on java development, and I can deliver as per the requirements. .................

$200 USD v 4 dneh
(4 ocen)
2.2
Profilna slika haadfreelancing
haadfreelancing

I am interested to work on this project as I have relevant experience in Big Data,Sqoop, Hadoop, Spark, Hive, Kafka, Spark Streaming, Rdd, Datframe, Dataset , Python, Scala and Java. I am well versed in Installation an Več

$55 USD v 3 dneh
(0 ocen)
0.0
Objavi podoben projekt