summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-06-02Fix bug with local queue deletionNick White
2020-06-01Mention documentation URLNick White
2020-05-29[bookpipeline] Remove local copy of original page image once preprocessedNick White
2020-05-29Merge branch 'minimisedisk'v0.2.4Nick White
2020-05-26Merge branch 'local'Nick White
2020-05-26Add -c conntype for necessary tools to allow local connection to be usedNick White
2020-05-26Fix DelFromQueue and Upload for local connectionsNick White
2020-05-22Fix CheckQueue for LocalConnlocalNick White
2020-05-22Fix bookpipeline failing if shutdown option isnt usedNick White
2020-05-22Fix bookpipeline failing if shutdown option isnt usedNick White
2020-05-22Add experimental local connection typeNick White
2020-05-22[untested] Use less disk spaceminimisediskNick White
There are several ways that disk usage is reduced with this patch: - Files are deleted as soon as they have been uploaded - Once a page image has been added to a PDF, immediately delete it This should allow much larger books to be processed without needing bigger disks.
2020-05-19Add getandpurgequeue debugging toolNick White
2020-05-06Update spot image againNick White
2020-05-04Update spot image idNick White
2020-05-04Update preproc dependency so that vwipe is enabledv0.2.3Nick White
2020-04-21Simplify spotmeNick White
2020-04-14Update preproc and utils dependenciesv0.2.2Nick White
2020-04-14Add getbests tool that was previously in the utils repoNick White
2020-04-14Adjust documentation a tadNick White
2020-04-14Remove unused PreprocPattern, allow sensible defaults with aws setup, and ↵Nick White
add a little more documentation
2020-04-14Briefly document each of the commands in a godoc friendly way, and improve ↵Nick White
the cloudsettings documentation slightly
2020-04-07Update dependenciesv0.2.1Nick White
2020-04-07More documentation updatesv0.2.0Nick White
2020-04-07Improve documentationNick White
2020-04-07Remove unused OCR queue (was superceded by the ocrpage queue some time ago)Nick White
2020-04-07gofmtNick White
2020-04-07Separate out cloud settings into a separate file; cloudsettings.goNick White
2020-03-31Disable autoshutdown by default for bookpipeline, and update to ami 0.11 ↵Nick White
(which reenables it for spot instances)
2020-03-31[bookpipeline] Fix typo in previous commit and rename HeartbeatTime to ↵Nick White
HeartbeatSeconds, as it is not a Time
2020-03-31[bookpipeline] Stop using filepath.Join for storage keys, as we want to ↵Nick White
ensure it is always a / delimeter
2020-03-31[bookpipeline] Improve logging outputNick White
2020-03-31[bookpipeline] Add (experimental) log saving functionalityNick White
2020-03-30[bookpipeline] Clean up autoshutdownNick White
2020-03-30[bookpipeline] Enable real shutdown when bookpipeline has been idle for 5 ↵Nick White
minutes
2020-03-30[bookpipeline] Neaten shutdown fixNick White
2020-03-30[bookpipeline] Fix hang bug when restarting shutdown timerNick White
2020-03-30Rewrite autoshutdown to do things right [bugs excluded] (wip)Nick White
2020-03-24[bookpipeline] Improve autoshutdown wipNick White
2020-03-24[bookpipeline] Add experimental (dummy) shutdown partNick White
2020-03-23[getpipelinebook] Switch to MinimalInit() so that it can be run without SQS ↵Nick White
permissions
2020-03-23Add Log() function to Pipeliner interfaceNick White
This simplifies things nicely from using conn.GetLogger().Println() to conn.Log()
2020-03-23Replace errors.New(fmt.Sprintf with fmt.ErrorfNick White
Embarassing I hadn't noticed the fmt.Errorf function before, but better late than never.
2020-03-23Don't try to make a graph with one line (it will fail), and don't mark ↵Nick White
analysis as failed if graph isn't made for that reason
2020-03-23[getpipelinebook] Add -binarisedpdf and -colourpdf flagsNick White
2020-03-23[getpipelinebook] Add -graph flag to download just graphsNick White
2020-03-10Update image used for spotsNick White
2020-03-09Add nobooks flag to lspipeline so it has a faster modeNick White
2020-02-28Tidy go.sumNick White
2020-02-28Update go-chart to a working version in go.modv0.1.2Nick White
generated by cgit v1.2.1-24-ge1ad (git 2.21.0) at 2024-11-15 11:55:41 +0000