Android and iOS MRZ (TD1,TD2,TD3) passport reader (opencv and tesseract)

Goals To be able to create any number of android and iOS applications using one software component to scan and read passports MRZ codes using phone camera. Summary We created a project using opencv and tesseract using C++, Objective-C and Swift, got good results, but not good enough to read the MRZ text lines within a reasonable time frame and with the desired accuracy. We are able to find the text lines of the MRZ code, crop each line from a camera frame image and recognize characters with tesseract. We believe to improve the results what should be done is: - Pre-process the images to improve tesseract results (deskew, 1-bit, unsharp masking) - Train tesseract using the correct font (OCR-B); - Use multiple results to successfully read all the fields and verify each check digit; Objectives Develop a software component that allows an app developer to create an app with passport reading capability without any knowledge of computer vision and OCR; Write a sample application using swift or objective-c and compatible with iOS 8+ and android that uses the specified component to start the passport scan and receives the result of the processing; Deliverables 1) Sample application with source code; 2) Framework/library used to scan passports, Identity cards, Visas built for and available to be used on iOS 8+ and android projects; Basic architecture that should be followed Method to start the scanning; Input: - timeout: (int) // in seconds; Output: - lines // read strings of MRZ code; - success: boolean; // true if all the information was read and all the check digits verified; - firstRowSuccess; // true if the first row complies with expected structure - passportType; - country; - surName; - givenNames; - passportNumber; - passportNumberSuccess; // true if the passportNumber check digit is verified; - nationality; - birth; - birthSuccess; // true if the birth check digit was verified; - sex; - expirationDate; - expirationDateSuccess; // true if the expirationDate check digit is verified; - personalNumber; - personalNumberSuccess; // true if the personalNumber check digit is verified; - secondRowSuccess; // true if the last check digit is verified; - thirdRowSuccess; // true if the last check digit is verified; Details - When the scan method is called the user will see the live camera; - User will not be requested to do anything different from filming the passport (no button to take picture, for example); - As soon as a successful result is available or the timeout is reached the user will be redirected to the screen he was before and the method will return its results; - Even if the result is not successful the method shall return what was read. The boolean outputs (success, firstRowSuccess, passportNumberSuccess, birthSuccess, expirationDateSuccess, personalNumberSuccess, secondRowSuccess) will indicate success; Refs [login to view URL] [login to view URL] [login to view URL] [login to view URL]