Age | Commit message (Collapse) | Author | |
---|---|---|---|
2019-10-02 | Add -prebinarised flag to booktopipeline | Nick White | |
2019-10-02 | Add wipeonly queue and functionality | Nick White | |
This is useful for prebinarised images, which don't need full preprocessing, but do require wiping, albeit with a more conservative threshold. | |||
2019-09-27 | Hardcode to ignore "workhorse" from logs | Nick White | |
2019-09-24 | Improve ssh logs; ensure only fully operational servers are tried, and ↵ | Nick White | |
ensure connections to new ips not in known_hosts still succeed | |||
2019-09-24 | Do ssh log collection concurrently | Nick White | |
2019-09-24 | Get ssh logs from all running servers | Nick White | |
2019-09-24 | Add list of books done and in progress to lspipeline | Nick White | |
2019-09-24 | Move ec2 stuff out of lspipeline and into aws.go | Nick White | |
2019-09-23 | Move the sqs stuff out to aws.go | Nick White | |
2019-09-19 | Add queue listing to lspipeline | Nick White | |
2019-09-19 | Switch to using a goroutine for ec2 instance info, so can do all aws ↵ | Nick White | |
requests concurrently in due course | |||
2019-09-18 | Add start of lspipeline | Nick White | |
2019-09-17 | gofmt | Nick White | |
2019-09-16 | Be more careful to try to grab the message after a heartbeat failure more ↵ | Nick White | |
quickly Rather than waiting for the whole length of a visibility timeout, in which time another process may grab the message, instead wait a short amount of time, each time the message is searched for. Also add a bit more logging. | |||
2019-09-14 | Ensure enough time has elapsed before looking for the message to reget in ↵ | Nick White | |
the case of heartbeat running out | |||
2019-09-12 | Don't prefix date/time to logs, as logger will store that anyway | Nick White | |
2019-09-11 | Work around the SQS limit of 12 hours of visibility timeout | Nick White | |
This is done by checking for the error that is emitted in such a case, and if it's found trying several times to find the message back in the queue, and returning the message with an updated handle back to the caller to use in the future. | |||
2019-09-06 | Add flags to disable checking various queues | Nick White | |
2019-09-05 | Handle no words found error in a better way so any page that is actually 0 ↵ | Nick White | |
confidence is recognised | |||
2019-09-05 | Don't abort analysis if we encounter a hocr with no words, just skip it | Nick White | |
2019-09-05 | gofmt | Nick White | |
2019-09-05 | Update Pipeliner interface in getpipelinebook, and update some comments | Nick White | |
2019-09-04 | Rewrite heartbeat so errors during it will be reported, and the aws api ↵ | Nick White | |
doesn't rely on channels | |||
2019-09-04 | Ensure any channels that need to be consumed before goroutine is finished ↵ | Nick White | |
are done in the case of an error | |||
2019-09-03 | Improve debug logging | Nick White | |
2019-09-02 | Log upload and download events | Nick White | |
2019-09-02 | Add initial getpipelinebook cmd (untested) | Nick White | |
2019-08-28 | Add standalone graph tool; confgraph | Nick White | |
2019-08-28 | Move booktopipeline and mkpipeline into bookpipeline/cmd | Nick White | |
2019-08-28 | Split out bookpipeline to cmd/ | Nick White | |