summaryrefslogtreecommitdiff
path: root/cmd/rescribe/main.go
AgeCommit message (Collapse)Author
2022-03-21Only generate full-size PDF if requestedNick White
This avoids the issue that large PDFs require a lot of RAM, so there are chances of running out of memory. Plus it's a waste of space and time.
2022-03-11Add initial support for full-size PDF generationNick White
Some issues: 1) The PDF generation stores every page in memory while it constructs it. That means that there's a higher chance of failure due to running out of memory with these. There's no getting around this except by improving the PDF generation library, which is not easy. 2) Currently I've just changed the pipeline to always generate these full size PDFs, and then the rescribe tool will just delete them if they weren't requested. This is bad in particular because of point 1, and would probably cause issues of failures in the server pipeline as a result Therefore the plan is to add a tag to queue messages so that full size generation can be selectively enabled. Also, it should be split from the loop with colour pdf generation, as holding them both in RAM at the same time is unnecessary.
2022-03-11Name PDF extracted images so they sort correctlyNick White
2022-02-28rescribe: Add " searchable" to file name for saved PDFNick White
2022-02-28Add PreNoWipe queue, that just does binarisation but no wipingNick White
2022-02-23rescribe: fix typo with embedded getgbook runningNick White
2022-02-23rescribe: Add embedded support for getgbook, for linux only so farNick White
2022-02-21Ensure that no new console windows are opened on Windows when executing ↵Nick White
Tesseract
2022-01-31rescribe: Add context cancelling to extractPdfImgs(), so it's no longer ↵Nick White
possible to get the gui into a bad state by cancelling before startProcess began (hopefully)
2022-01-31rescribe: Ensure status isnt overwritten after an abort, when wipe-only ↵Nick White
preprocessing
2022-01-31Make pipeline context-aware, so the rescribe tool can cancel jobsNick White
2022-01-10rescribe: Rename PDFs taking into account that in some cases one or the ↵Nick White
other of binarised or colour may not exist
2022-01-10rescribe: handle PDF errors much more gracefullyNick White
2021-12-20rescribe: Ensure temporary tesseract data is only removed when the program ↵Nick White
ends, so multiple books can be processed by the gui one after the other
2021-12-20rescribe: Ensure temporary tesseract dir is removed in gui mode tooNick White
2021-12-20whitespace and error clarity changesNick White
2021-12-20fixed -png flag and changed rescribe tool to save binarized png in separate ↵Antonia Rescribe
folder
2021-12-06pipeline: process jpg or png regardless of whether in wipe or preprocess queueNick White
2021-11-23rescribe: Remove debugging printfs related to PDF parsingNick White
2021-11-23rescribe: Improve pdf consumption by ensuring only jpg or png are saved to ↵Nick White
upload
2021-11-22rescribe: Add support for reading images directly from PDFsNick White
There are several TODO items before this can be considered "good enough", let alone complete. See the comments in the code for details. On a good day, with a fair wind, though, this works.
2021-11-22rescribe: replace errors.New with fmt.ErrorfNick White
2021-11-02rescribe: handle directories with spaces correctlyNick White
2021-10-26rescribe: Separate gui code, and organise it better (should be no functional ↵Nick White
change)
2021-10-25rescribe: wip gui using fyneNick White
2021-10-12rescribe: fix lookup of external training filev0.5.3Nick White
2021-10-01rescribe: Add embedded lat.traineddataNick White
2021-10-01rescribe: Add both original training path and embedded version on error ↵Nick White
output for training file not found, so that its clear that the file specified may not exist
2021-08-17pipeline: use regular storage for tests, rather than a separate oneNick White
2021-08-02internal/pipeline: Add test (incomplete but working) for UploadImagesNick White
2021-07-20Cleanup thanks to go vetNick White
2021-07-13gofmtNick White
2021-07-08rescribe: Exit with an error if directory doesn't existNick White
2021-06-29rescribe: Add embed target for darwin (osx) tooNick White
2021-06-22rescribe: Remove erroneous unnecessary mkdirNick White
2021-06-22rescribe: Make it clearer that embedded training files are available to useNick White
2021-06-22rescribe: add embedded tesseract for linuxNick White
2021-06-22rescribe: allow use of embedded training even if -systess is usedNick White
2021-06-22rescribe: Add go generate command to download the needed files to embedNick White
2021-06-22rescribe: Add an embedded tessdataNick White
2021-06-21rescribe: Set up so only Tesseract needed for the build platform is embeddedNick White
2021-06-21rescribe: Embed Tesseract into binary so that no Tesseract install is necessaryNick White
2021-05-31Fix bug after changing pipeliner for tests, to ensure DeleteObjects is ↵Nick White
available to Pipeliner
2021-03-16rescribe: change default training directory to trainings/v0.3.3Nick White
2020-12-07[rescribe] Fix up *.hocr glob, which ensures that using a savedir that ↵v0.3.2Nick White
already has a hocr directory in it will work
2020-12-07[rescribe] Allow saving of results to somewhere other than a directory named ↵Nick White
after the book being processed
2020-12-03[rescribe] Fix portability issue where hocrs may not be correctly moved and ↵Nick White
txt-ified on windows
2020-11-17Remove _bin0.x from txt filenamesv0.3.0Nick White
2020-11-16[rescribe] Default to an appropriate tesscmd for WindowsNick White
2020-11-16[rescribe] Add txt output, only keep colour pdf, and reorganise files so ↵Nick White
they're more user-friendly