Scala Jobs

I n I need to develop a spark job streaming and analysis specific words Read kafka topic and hdfs storage Stream data Analyse data Storage results in mongo database Langage is Java or Scala

I have a data frame with a column in the format user_id1, user_id2 and tags {key=value}. I want to do some complicated dataframe column parsing I want to find the hashtags that are common between every user_id1. For example if the user_id1 has two rows 44392 12551 {hello=1, bye=2} 12551 67288 {hello=4, test=2}, I need a new column corresponding to each row with the following values 5 for the fir...

We need a single dedicated part time resource on hive,spark, Scala,shell scripting and hbase to give support on weekdays morning around 90 minutes IST 6 00 am to 8 00 am will provide 22000 per month and minimum 4+ years of experienced resource only eligible for the bid.

We need a single dedicated part time resource on hive,spark, Scala,shell scripting and hbase to give support on weekdays morning around 90 minutes IST 6 00 am to 8 00 am will provide 22000 per month and minimum 4+ years of experienced resource only eligible for the bid.

Parsing a JSON file and having duplicate rows due to multiple array items. Need to flatten the data and make it a single record. Using Spark or Spark SQL.