summaryrefslogtreecommitdiff
path: root/pipelinepreprocess/main.go
AgeCommit message (Collapse)Author
2019-08-20Handle errors correctly in main parts of programNick White
2019-08-20Substantially improve problematic object listing part of APINick White
Switch to regular non-concurrent stuff, concurrency is better handled by the main program anyway. Now we handle errors properly, and things are way simpler.
2019-08-20Add basic OCR support, and reorganise codeNick White
The previously committed thing didn't work, as listobjects was sending to a channel synchronously, so it was never being received. The current API isn't great, mixing synchronous and non-synchronous things, not handling errors consistently, and generally is over complicated. That will be fixed soon.
2019-08-20Split aws implementation from main.go in pipelinepreprocessNick White
2019-08-20Export qmsg typeNick White
2019-08-19Fix pipelinepreprocess segfaultsNick White
These were caused by using non-pointer methods, which meant that the values set in Init() were not saved.
2019-08-19Work in progress rearchitecture to use interfaces; currently pointers are ↵Nick White
screwy causing segfaults
2019-08-13Various improvements to pipelinepreprocessNick White
- Ensure temporary directory already being present isn't an issue - Remove temporary directory when done with it - Ensure any already preprocessed files aren't preprocessed themselves (this could happen in the case of a run stopping half way through)
2019-08-13Correct typo in bucket name for pipelinepreprocess; tested and seems to ↵Nick White
work, remarkably
2019-08-13Add bonus verbose log pointsNick White
2019-08-13Reduce SQS WaitTime to something in-spec, and add bonus verbose log pointsNick White
2019-08-13Add basic verbose logging capabilities to pipelinepreprocessNick White
2019-07-25Add first draft of pipelinepreprocess - completely untested, will contain bugsNick White