summaryrefslogtreecommitdiff
path: root/aws.go
AgeCommit message (Collapse)Author
2019-12-11Use aws.go with mkpipeline too, plus fix one log.Fatal call in aws.go which ↵Nick White
should have been handled by caller
2019-12-05Remove (the generally empty) files in the case of a failed downloadNick White
2019-12-03Rewrite lspipeline book listing part to be much faster by taking advantage ↵Nick White
of the aws CommonPrefixes output
2019-11-19Add ocrpage queue for processing individual pagesNick White
This should be a good way to get around the ongoing heartbeat issue, as individual page jobs will never come close to a the 12 hour mark that can cause the bug. The OCR page processing is done and working now, still to do is to populate the queue (rather than the ocr queue) after preprocessing / wiping.
2019-11-12Add spotme command to start appropriate spot instancesNick White
2019-10-29Another attempt to fix the ongoing heartbeat issueNick White
This time wait up to 1 second between attempts, reduce long polling time significantly, and attempt for longer before giving up.
2019-10-28Try to fix heartbeat renew issue more fullyNick White
This approach first sets the remaining visibility timeout to zero. This should ensure that the message is available to re-find as soon as the process looks for it. Correspondingly the delay between checks is much shorter, as there shouldn't be a reason for much delay.
2019-10-17Adjust the heartbeat searching function to hopefully have better luck at ↵Nick White
finding it and not letting another process steal it.
2019-10-16Sort book list in lspipeline by modified dateNick White
2019-10-08Separate out bookpipeline from catch-all go.git repo, and rename to ↵Nick White
rescribe.xyz/bookpipeline The dependencies from the go.git repo will follow in due course.