Redshiftdela
~~~~~~~~~Duties~~~~~~~~~ - Architect a series of microservices based on business logic. - Code them in js/node.js as microservices - Help set up AWS and/all related services/ Database ETC/ REDSHIFT - Upload them to AWS - Test the microservices thoroughly - Aid in integration ~~~~~~~~~~~~~~~ TO QUALIFY ~~~~~~~~~~~ - Must speak good english - Must have clear understanding of MOST AWS services and experience using them in production - Must be extremely proficient in Javascript 5+ years - Experience in production level database architecture in large scale applications is a +
We require a developer to create a Data Export Utility as a component of our Data Processing System. The solution will be a turnkey configurable system which allows n clients to configure daily, weekly and monthly exports in csv format plus adhoc data requests. Data is currently stored on AWS Redshift and S3. The utility requires an Interface to configure exports and a scheduler based system to run each of the required exports. The first stage of our project will be to create file export utility using Dropbox API V2 followed by export options for FTP, email and via API endpoints. The successful applicant will work with our internal Project Manager to agree deliverables and timelines. They must be able to communicate in spoken and written English. Additional Skills: AWS S3, s...
This project is to analyze historic stock data using various parameters and filters. Data includes historical stock prices (intraday by minute) for all stocks since 1998. Help is needed to: 1. Identify the best fit technical solution (AWS, Redshift, Quicksight, Google Big Query, Google Data Studio, Tableau, Pyramid Analytics, etc) to handle data set and analysis at a low cost. 2. Prepare data (data cleanse, load, validation, create calculated fields, etc) 3. Build analysis and reporting solution based on design requirements See attached document for additional details and report example.
I would like to build Windows version of application (). To achieve this I would prefer to build it with some open source. In this link , I found three windows version similar following applications. -Redshift -Turn Off the Lights for Desktop I downloaded windows version VC++ source code from When tried to build getting some compilation error. So please let me know how you would go for this? Have you worked in similar application before? What would be your fixed price quotation for this?
Write a python boto3 script to get a list of aws environment such as EC2,S3,RDS,Lambda,RedShift,Vpc's,Autoscaling,ELB,SNS,SQS,Cloud Formation along with launch time and owner of the resource
- Interface with data from clients, with upwards of a terabyte of data (SQL technologies). - Build out our data pipe, aggregating data from both structured and unstructured sources (Hadoop, Spark, HDFS, EMR, Redshift). - Scale and optimize our client facing data warehouse that provides real-time interactivity across millions of rows of data (Redshift, Elasticsearch). - Lead development of statistical models and predictive technology to improve our experimentation and segmentation tooling (Machine learning, D3). - Develop critical features to in our python software stack.
Create a copy command for a file which is in JSON in S3 by using the JSONPATH manifest file and also the DDL to load the file which needs to be loaded in Amazon Redshift. Sample JSON file will be shared separately to the shortlisted developers. Need to be done in the next 24 hours. Experts ONLY.
Need to populate a table in an web app with data from AWS Redshift. We need to turn any requested data into json and deliver that on the web app's table through websockets. Not too complicated if you're familiar with node.js or alternative messaging approaches. Let me know if you have any questions.
I need you to develop nodejs code to use amazon aws cloud sdk - specifically amazing aws lambda cloud service and other aws services like s3, dynamodb, redshift with nodejs sdk software from official documentation. I would like this software to be developed .
I need you to develop some software for me. I would like this software to be developed . Experience on Amazon EMR and Redshift.
We are looking for help to load data from Staging tables to Star schema inside Amazon Redshift. Currently data is sitting redshift tables .We need a flexible process to automate dimension and fact build. Current Stack : EC2 Windows, S3 ,Amazon Redshift , SQL Server Integration Services . Working Data size: 200GB; Please Bid on this ONLY if you have work experience in building Data warehouse in Amazon Redshift.
...records, regulatory agencies reports, and social media posts. Further, all this data is processed by the cutting-edge Natural Language Processing (NLP) algorithms which are able to produce structured insights which can be then used for advanced analysis. Requirements: Master degree in Mathematics, Physics, Computer Science or similar field(s) Strong Analytical skills Google Big Query/Amazon Redshift, Hadoop/Mongo, Hive/Pig/SQL Familiarity with OOP and Java 4+ Years database programming/engineering experience 2+ Years Big Data engineering experience Experience with strong machine learning component. Experience with multinational organization Online advertising experience — preferred Understanding of mobile, tablet, smart TV platforms Written and oral English ...
Is there any way to load data from redshift to salesforce using python,please provide some sample code am new to python scripts.
Python Scrip for Data ingestion from S3 to redshift in AWS
Hello, we need to properly setup server side for Snowplow Analytics to be ready for being used to track website events. The analytics tool requires multiple services by Amazon. Snowplow - the open-source, web-scale analytics platform powered by Hadoop, Hive and Redshift. ---------------------------------------------------------- Technical specification ---------------------------------------------------------- Find helpful information about the architecture and neccessary setup here: ---------------------------------------------------------- What I need from you if we hire
Looking for an experienced software engineer who can help me in creating a Full Stack Web Application for my online learning platform start-up. Previous start-up experience would be highly desirable. You will work with a team of international experts. Great opportunity to be a part of highly p...years Linux/unix, minimum 2 years Testing methods, including back-end and front-end unit and integration tests Ability to learn, work with, and migrate existing code bases, up and down the software stack. Strong technical skills. Willingness to "go the extra mile" when needed. Preferred Skills: Python, Django Javascript, JQuery, AngularJS MySQL, RedShift (PostgreSQL), Redis, Memcached ElasticSearch AWS Lambda AWS S...
ARQUITECTO DE SOLUCIONES JAVA – SECTOR BANCA(Envía tu CV ó recomienda a un amigo)Tú puedes ser la persona que estamos buscando, esta es tu oportunidad, revisa nuestro perfil: Edad: Indistinta Sexo: IndistintoIdiomas: Inglés AvanzadoSolo debes cumplir con los siguientes requisitos:Escolaridad: Licenciatura o Ingeniería en Sistemas Computacionales, Informática, Actuaria o carr...en alguna de las siguientes tecnologías:? JAVA (Certificación necesaria)? Integración Continua (Jenkins, Nexus, Sonar, Git)? MongoDB (Certificación deseable)? CassandraDB? Elasticsearch? OpenStack (Certificación deseable)? AWS Solution Architect(Certificación necesaria)? Docker? Apache Hadoop (Certificación deseable)? Ap...
...on data pipelining (e.g. Spark/Storm) in both structured and unstructured data environments. • Design optimal framework and architecture for capturing and analyzing data. • Aware of basic statistical models/machine learning algorithms to solve client specific business problems. • Extensive hands on UNIX/LINUX environment. • Good hands on cloud environment (AWS, AWS Console, AWS IOT Gateway, Redshift, Rule Engine, SDK, Node JS Proxy, Kinesis Streams, AWS Lambda functions, S3 bucket) • Experience on any of the hadoop distribution – Cloudera, Hortonworks, MapR etc. along with Pig, Hive/Hawq, Sqoop, Oozie, Rabbit MQ, Kafka, Spark/Storm etc. • Experience on any one of the NoSQL databases – Hbase, Cassandra, Neo4j, MongoDB etc. • ...
1) Text scraping of numbers from 1-2 sites in python 2) setting up a redshift or sql database on amazon web services. Explain me the code 3) Sending and receiving numbers to the database
Decide on the best database to use and design the most cost-effective and appropriate database schema based on application needs. We use AWS to host all of our servers and prefer to continue using them for this database but ultimately want to use the best database for the job. This database will be used to store all compromised user information found on the internet with the key identifier being a users email address. Each hack data inserted into the database may contain different information associated with a user. Similar services include and The following link described how one service did it with Azure. <b>Database Requirements:</b> - Ability to scale to billions of rows - Quickly return results for an email address. (results generated in milliseconds) - Quick...
MOBILE APPLICATION ThAT allows the GPS to look up the current ZIP code, then search to find the nearest clubs, and the application to retrieve venue details, etc, fr...(multipart form post?), and image is added to pictures taken by people at this event. NEED this to be cloud deployable, load balanced, and linearly scalable so he can run 1million users on it or more. Eventual consistency with data is okay. This isn't flight guidance software. He wants this to run on AWS, with load balancing, multi-AZ deploy, CDN static content, S3 storage for data, RDS for DB, RedShift for big-data, maybe MLS for the recommendation engine. Payment system integration (paypal, authorize, ), recommendation engine, user statistics, building out the Venue Owner, Event Promoter, and Administra...
Create a python based server script to generate PDF reports from data in a AWS Redshift database. PDF to be built ideally using ReportLab PDF toolkit to include tables of data, line charts, and colored maps.
Need help with Integrate with AWS Redshift.
I have a load of data in a redshift db. I want to gain access to it from PHP from another server. I need someone to write the PHP connection and to a certain extent work through AWS to allow the connection.
I just want someone to set up my first simple redshift cluster that I can access from home. I would like to watch so I can learn to do it myself next time.
Curre...see Normalize data in Redshift in the following Dimensions andFact Table Dim_Customer, Dim_Market, Dim_Product and Fact_Sales We are open for any of the following approaches. Approach 1: a)Using RDS (Either MS SQL or MySQL Flavor) as Staging Database b)Moving Data From S3 to RDS (don't know this yet, looks like no copy :( command for RDS) and Developing SSIS ETL Packages or SQL Code to model the data and export as separate files (,, dim_product_csv and ) to S3 c)Scheduling a AWS Data Pipeline to load files from S3 to redshift Approach 2: A)Using Cloud Data integration tool such (FlyData, Snaplogic or Quoble ) or any other open source Cloud ETL tools. Please respond only if you have worked on Amazon Redshift Thank you.
I am looking for someone to help me learn programming 2 hours /week. I am an analyst with a lot of experience in Analytics and can do pretty much anything in SPSS and Excel but am not a programmer. I would like to be able to learn to do what I can do in SPSS and Excel using R and/or Python and get some basic SQL skills particularly as it relates to Redshift. English language proficient a "Must have" - doesn't need to be fluent but does need to be functional as I would like to be able to converse via web chat.
I would like to optimize a query that requires joins across 3 tables and reports count() and sum(). I have a working query but would like someone to access my test Redshift cluster (with 20 million records loaded) and see if they can speed up this query using DTE/WITH statements or suggest schema changes, etc.
...Worked on the development of Dashboard reports for the Key Performance Indicators for the top management. End to end experience in designing and deploying data visualizations using Tableau Design, develop, deploy and support Tableau dashboards on Tableau server deployed enterprise-wide Translate business requirements into sustainable and efficient visualizations and reporting Extract data from RedShift and other platforms using SQL and other platforms/languages and put into a form conducive to answering business questions in Tableau, automating this process as much as possible Closely monitor and check the accuracy and completeness of data in Tableau. Perform optimization of Tableau reports Design, update and administer the metadata of the Tableau workbooks, so that descri...
Looking for a backend developer to: 1. Take .txt email files available here: ~./enron/, 2. store them in redshift database then 3. parse each .txt file to identify 'from', 'to' and 'subject' and create three columns in the database. After the data is stored in redshift: 1. Connect to Mechanical Turk API 2. Automatically create an Mturk HIT to have workers review email subjects and categorize words into categories (i.e. verb, action item, due date) 3. Store HIT results in a new column in the redshift database. PLEASE: write a word SERFCOMPANY before your response.
Actualmente buscamos una persona con conocimientos en Business Intelligence y Data Mining para que nos apoye en picos de trabajo como freelance. Si te interesa simplemente apúntate y si conoces a alguien que podría encajar te agradecemos mucho si lo compartes. Necesitamos a una persona con conocimientos en: - Amazon Redshift - Amazon DynamoDB o MongoDB - Amazon Kinesis - Herramientas de ETL y Reporting - Javascript - PHP, MySQL y POSTGRESQL Buscamos a una persona flexible, con capacidad de aprendizaje y capaz de mantenerse al día en las últimas tendencias en desarrollo, tecnología y marketing digital. Valoramos: - Multidisciplinaridad y visión global: que pueda abstraerse del ámbito de la programación para e...
Looking for help to load 100 GB dataset from SQL Server to Amazon Redshift. Please respond only if you have worked on Amazon Redshift .Thank you.
I'm new to Linux, and have sorted out inital issues with two exceptions. One is that my ThinkPad doesn't detect my Benq external monitor, either via VGA or miniDSP-to-HDMI. System settings > Displays > Detect Displays does nothing. The other is that neither nor Redshift are able to affect the laptop screen's brightness. I'd like one of them to work on my laptop as well as on the external monitor. Laptop is a ThinkPad T440p running Ubuntu 14.04. The command lspci gives: 00:02.0 VGA compatible controller: Intel Corporation 4th Gen Core Processor Integrated Graphics Controller (rev 06). No drivers are suggested in "Additional drivers". I've read extensively online about both issues with no success. I would like to solve them without rein...
...improve the existing systems, by making them more efficient, improving uptime, decreasing latency, making deployment/testing easier and more robust, and general troubleshooting. Restrictions No telecommuting No Agencies Please Nice to have: understanding how search engines work C/C++ experience, and how to write fast code experience with AWS technologies like EC2, DynamoDb, RedShift, S3 understanding large distributed systems ("CAP Theorem" and all that). Requirements Responsibilities Writing reusable, testable, and efficient code Design and implementation of low-latency, high-availability, and performant applications Integration of user-facing elements developed by front-end developers with server side logic Implementation ...
...everyday from all of our users. However, as you know, raw data alone isn't information. This is where you come in!! We need to convert this data into actionable information to better understand our customers and give them an even better service every day.<br /><br />Your Main Responsabilities Will Be<br /><br />Transform data from different data sources to integrate with our Data Warehouse in Redshift<br />Create ETLs to transform that data into usable formats.<br />Design and develop reports to help make business decisions<br />Design infrastructure components<br />Integrate to other external systems like Stripe, Zendesk and others<br /><br />What We Need From You<br /><br />Experience as a Data enginee...
Hello I am doing a Entrepreneur class. I have created a Business plan for an idea. I have to present in class in another 2-3 days. I need a simple Video Ad of 30 sec that can explain my idea to common audience . For example using this example If you look at the video "Introduction to Amazon Redshift Data warehousing". I want my ad to be on similar lines. A narration would be awesome If interested, I can give more details
Looking for training in AWS, Redshift, Talend, Pentaho
...Tools to perform the automated process, which we want to give preferences for tools that create workflows on the cloud and not only scripts (for example, , zapier, Microsoft azure integration, etc) However, this is not mandatory to use. --Free Cloud Database Tiers: As our main goal is to store data, we want to leverage the free tiers of the popular Database Cloud Services (Azure, Redshift, Etc). We are not looking to store huge amounts of data. --Final connectivity with Tableau: Once the data is consolidated, we want to be able this database to be connected with Tableau for analysis, we would be doing this task. Samples attached of the 2 type of files that require this task, one is an Excel File and other one is an EDI File Both Files require to be merged with their resp...
We currently have a full text search feature in our web app, based on SQL Server full text indexes. As we're doing quite simple queries and we've moved to AWS platform, we would like to benefit from AWS native full text service, rather than expensive SQL Server setup. We hav...today: 1) Bulk insert text from our engines. I guess we could easily bulk insert into an AWS service? 2) When a user search for something, he search through only the documents he has access to, so we're doing cross server joining from a main SQL Server to the full text SQL Server. I wonder how we would filter out the documents he must not search through, when moved to AWS? I'm not sure if RedShift or CloudSearch (or a third option) would be the choice for us, so I look forward heari...
...have large amount of log file data that I need to analyse. This data is stored on AWS S3 in .gz txt files that are tab delimited . It contains the following fields (some optional) TIMESTAMP UID GEO URL CATEGORIES USERAGENT META_KEYWORDS KEY_TERMS ENTITIES Sample file is attached - File sizes are from KB to 10 MB. Requirement: 1: To load and analyse the data (Via EMR or Redshift on AWS), this choice is based on keeping costs lowest. Performance is not the main criteria. 2: Calculate high level metrics (By time period) including: A: Domain Name based counts B: Domain to Key Terms frequency C: Useragent frequencies D: Entities Frequencies E: Categories Frequencies F: List of Domains based on Categories G: list of Domains based on Key Terms Please...
I need someone to build a wordpress website for our startup company. We are "RedShift Healthcare Management, LLC" We provide management and consulting services for hospitals. I am attaching our logos to this post. I am also adding a few links to similar type companies websites.
WE have data in Redshift and we are looking for immediate help in running some SQL queries and dumping data for analysis by R. You would need to work intimately with me over the next 48 hours to complete the work.
Hi, can you support me for my new project on Hadoop , mainly on Redshift SQL , Pig , Hive and Amazon Web services.
Decide on the best database to use and design the most cost-effective and appropriate database schema based on application needs. We use AWS to host all of our servers and prefer to continue using them for this database but ultimately want to use the best database for the job. This database will be used to store all compromised user information found on the internet with the key identifier being a users email address. Each hack data inserted into the database may contain different information associated with a user. Similar services include and The following link described how one service did it with Azure. <b>Database Requirements:</b> - Ability to scale to billions of rows - Quickly return results for an email address. (results generated in milliseconds) - Quick...
Prosimo, vpišite se ali se prijavite za več podrobnosti.
We are looking for a number of people to support us in the delivery of building a new multi-client reporting tool. ROLE TYPES will include Data Engineers, ETL Developers, Business Intelligence Developers and Data Analysts. The team being put together should have proven success...building a new multi-client reporting tool. ROLE TYPES will include Data Engineers, ETL Developers, Business Intelligence Developers and Data Analysts. The team being put together should have proven successful experience across the following: TECHNOLOGIES: Cloud Platforms - AWS, Azure. ETL - Pentaho, Informatica, Talend. Coding Languages - Python, PHP Databases - mySQL, SQL Server, Oracle, RedShift Analytical Tools - SAS, SPSS Reporting Tools - Tableau, IBM Cognos, SAP Business Objects, Birst,...
I need help to setup now two pipelines in AWS ( RDS to redshift) everything is setup and I'm trying but I need help sharing remote connection with the task. Most be available to work now. Thank you for your support
