I have 30+ years of experience and have written thousands of these kinds of interfaces. These kinds of interfaces can be written as single, “blind” interfaces. These are quick, cheap, difficult to maintain, and difficult to extend. Or they can be written robust. This kind of interface will be well-documented, easy to maintain, and easy to extend. I propose writing the more robust version. This will be more expensive initially, but far less expensive should your data set change. My proposal will include:
1. Technical documentation, including process and data flow.
2. Internal documentation, for ease of update.
3. Error and process logging. Error logging will include three levels of logging (low, medium, and high {for debugging})
4. Modular, structured data import. This will allow ease of change should the import data change. Additional modules will be easy to add, should additional files for import be required.
5. Modular, structured data export to table. This will allow ease of change should changes to tables be required. Additional modules will be easy to add, should additional tables be required.
6. Data checking to ensure database normalization is maintained, if required.
7. Date/time stamped Output file for data that could not be processed.