I have a lot of csv files (about 18000) containing data that I need to parse as fast as possible . The project consist of read a csv file using pypy and put it in a list. Treat the entire list (about 5000 lines) with GPU using PyOpenCl. Each line get about 40 comma separated financial data (some floats, some integer, and some strings). First, each line have to be tokenized and fast-atof strings data. At the end, a c-structure with all of the data have to be return. For sure, I need to be able to easily recover data from the c-structure using cffi for example.
Find a example of a simplified csv file with only 6 data (Date (have to be convert in integer),Open, High, Low, Close, Volume).
I wrote a single core program in pypy and cffi that works and take about 9s to load 18000 csv files. I hope to considerably decrease time using GPU.
12 freelancerjev ponuja v povprečju za $522 na tem delu
i am a lead .NET software engineer Relevant Skills and Experience responsible for creating a web and desktop applications using different languages and technologies Proposed Milestones $500 CAD - FULL