summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-07-20Fix typoNick White
2020-07-20Merge branch 'master' of https://git.rescribe.xyz/bookpipelineNick White
2020-07-20Update preproc to v0.1.4 to take advantage of vertical wiping parameters, ↵v0.2.5Nick White
and change WipeFile() to take advantage of them
2020-06-16[getallhocrs] Skip files which have already been downloadedNick White
2020-06-15Add getallhocrs toolNick White
2020-06-03Hopefully fix last bug in analyse step of bookpipelineNick White
2020-06-03Fix bug in analyse step of bookpipelineNick White
2020-06-02Fix race condition that could cause errors to be silently discardedNick White
This was a nasty one. By closing the up channel, the up() function would finish and send to the done channel. This means that the select between err and done would be random as to which was picked, whereas of course if there has been an error that path must be taken.
2020-06-02Proper full fix for local queue handling (hopefully)Nick White
2020-06-02Fix bug with local queue deletionNick White
2020-06-01Mention documentation URLNick White
2020-05-29[bookpipeline] Remove local copy of original page image once preprocessedNick White
2020-05-29Merge branch 'minimisedisk'v0.2.4Nick White
2020-05-26Merge branch 'local'Nick White
2020-05-26Add -c conntype for necessary tools to allow local connection to be usedNick White
2020-05-26Fix DelFromQueue and Upload for local connectionsNick White
2020-05-22Fix CheckQueue for LocalConnlocalNick White
2020-05-22Fix bookpipeline failing if shutdown option isnt usedNick White
2020-05-22Fix bookpipeline failing if shutdown option isnt usedNick White
2020-05-22Add experimental local connection typeNick White
2020-05-22[untested] Use less disk spaceminimisediskNick White
There are several ways that disk usage is reduced with this patch: - Files are deleted as soon as they have been uploaded - Once a page image has been added to a PDF, immediately delete it This should allow much larger books to be processed without needing bigger disks.
2020-05-19Add getandpurgequeue debugging toolNick White
2020-05-06Update spot image againNick White
2020-05-04Update spot image idNick White
2020-05-04Update preproc dependency so that vwipe is enabledv0.2.3Nick White
2020-04-21Simplify spotmeNick White
2020-04-14Update preproc and utils dependenciesv0.2.2Nick White
2020-04-14Add getbests tool that was previously in the utils repoNick White
2020-04-14Adjust documentation a tadNick White
2020-04-14Remove unused PreprocPattern, allow sensible defaults with aws setup, and ↵Nick White
add a little more documentation
2020-04-14Briefly document each of the commands in a godoc friendly way, and improve ↵Nick White
the cloudsettings documentation slightly
2020-04-07Update dependenciesv0.2.1Nick White
2020-04-07More documentation updatesv0.2.0Nick White
2020-04-07Improve documentationNick White
2020-04-07Remove unused OCR queue (was superceded by the ocrpage queue some time ago)Nick White
2020-04-07gofmtNick White
2020-04-07Separate out cloud settings into a separate file; cloudsettings.goNick White
2020-03-31Disable autoshutdown by default for bookpipeline, and update to ami 0.11 ↵Nick White
(which reenables it for spot instances)
2020-03-31[bookpipeline] Fix typo in previous commit and rename HeartbeatTime to ↵Nick White
HeartbeatSeconds, as it is not a Time
2020-03-31[bookpipeline] Stop using filepath.Join for storage keys, as we want to ↵Nick White
ensure it is always a / delimeter
2020-03-31[bookpipeline] Improve logging outputNick White
2020-03-31[bookpipeline] Add (experimental) log saving functionalityNick White
2020-03-30[bookpipeline] Clean up autoshutdownNick White
2020-03-30[bookpipeline] Enable real shutdown when bookpipeline has been idle for 5 ↵Nick White
minutes
2020-03-30[bookpipeline] Neaten shutdown fixNick White
2020-03-30[bookpipeline] Fix hang bug when restarting shutdown timerNick White
2020-03-30Rewrite autoshutdown to do things right [bugs excluded] (wip)Nick White
2020-03-24[bookpipeline] Improve autoshutdown wipNick White
2020-03-24[bookpipeline] Add experimental (dummy) shutdown partNick White
2020-03-23[getpipelinebook] Switch to MinimalInit() so that it can be run without SQS ↵Nick White
permissions
generated by cgit v1.2.1-24-ge1ad (git 2.21.0) at 2024-11-09 22:59:50 +0000