summaryrefslogtreecommitdiff
path: root/cmd/booktopipeline/main.go
AgeCommit message (Collapse)Author
2022-02-28Add PreNoWipe queue, that just does binarisation but no wipingNick White
2022-01-31Make pipeline context-aware, so the rescribe tool can cancel jobsNick White
2021-08-17pipeline: use regular storage for tests, rather than a separate oneNick White
2021-08-02internal/pipeline: Add test (incomplete but working) for UploadImagesNick White
2020-11-24[booktopipeline] Add a check to disallow adding a book that already existsNick White
This is important as if a book is added which has already been done, then an analyse job will be added every time a page is OCRed, which will clog up the pipeline with unnecessary work. Also if a book was added with the same name but differently named files, or a different number of pages, the results would almost certainly not be as intended. In the case of a book really wanting to be added with a particular name, either the original directory can be removed on S3, or "v2" or similar can be appended to the book name before calling booktopipeline.
2020-11-10Switch booktopipeline to use internal pipeline functionsNick White
2020-11-09Add a couple of things that should not be forgottenNick White
2020-10-20Improve logging by using Println, which ensures there is a space between ↵Nick White
arguments, even if all are strings
2020-09-22[booktopipeline] Check that all images are valid before adding to pipelineNick White
2020-09-01Fix confusing usage message for booktopipelineNick White
2020-07-28Allow override of autodetected queues for booktopipelineNick White
2020-07-28Autodetect queue for booktopipeline based on file extensionAntonia Karaisl
2020-05-26Add -c conntype for necessary tools to allow local connection to be usedNick White
2020-04-14Briefly document each of the commands in a godoc friendly way, and improve ↵Nick White
the cloudsettings documentation slightly
2020-02-27Add documentation, license notices, and licenseNick White
2019-12-11Clarify use of -training in toolsNick White
2019-12-11Add ability to set a different training for the ocr jobNick White
2019-12-11Use aws.go with mkpipeline too, plus fix one log.Fatal call in aws.go which ↵Nick White
should have been handled by caller
2019-10-16Rewrite booktopipeline to use bookpipeline aws interfaceNick White
2019-10-16Ensure booktopipeline complains if given too many argumentsNick White
2019-10-16Another attempted fix to "too many open files" issueNick White
2019-10-16Ensure files are promptly closed by booktopipelineNick White
2019-10-08Separate out bookpipeline from catch-all go.git repo, and rename to ↵Nick White
rescribe.xyz/bookpipeline The dependencies from the go.git repo will follow in due course.