| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2022-03-21 | Only generate full-size PDF if requested | Nick White | |
| This avoids the issue that large PDFs require a lot of RAM, so there are chances of running out of memory. Plus it's a waste of space and time. | |||
| 2022-03-11 | Separate out fullsize pdf creation from colour pdf creation, so less memory ↵fullsizepdf | Nick White | |
| is needed | |||
| 2022-03-11 | Add initial support for full-size PDF generation | Nick White | |
| Some issues: 1) The PDF generation stores every page in memory while it constructs it. That means that there's a higher chance of failure due to running out of memory with these. There's no getting around this except by improving the PDF generation library, which is not easy. 2) Currently I've just changed the pipeline to always generate these full size PDFs, and then the rescribe tool will just delete them if they weren't requested. This is bad in particular because of point 1, and would probably cause issues of failures in the server pipeline as a result Therefore the plan is to add a tag to queue messages so that full size generation can be selectively enabled. Also, it should be split from the loop with colour pdf generation, as holding them both in RAM at the same time is unnecessary. | |||
| 2022-02-28 | adjusted file renaming to make suffixes of png and jpg files lowercase and ↵ | Antonia Rescribe | |
| change jpeg to jpg | |||
| 2022-02-28 | Add PreNoWipe queue, that just does binarisation but no wiping | Nick White | |
| 2022-02-21 | Ensure that no new console windows are opened on Windows when executing ↵ | Nick White | |
| Tesseract | |||
| 2022-01-31 | pipeline: Fail if no images are present | Nick White | |
| 2022-01-31 | Make pipeline context-aware, so the rescribe tool can cancel jobs | Nick White | |
| 2022-01-17 | internal/pipeline: if a graph cannot be created, don't leave an empty ↵ | Nick White | |
| graph.png file, and allow failure to download that as it won't be created in the case of a 1 page book, which is fine | |||
| 2022-01-10 | internal/pipeline: Have DownloadPdfs() try to download all PDFs, but only ↵ | Nick White | |
| return an error if none downloaded, as there are times when the colour PDF will not exist, which is fine | |||
| 2021-12-20 | whitespace and error clarity changes | Nick White | |
| 2021-12-20 | fixed -png flag and changed rescribe tool to save binarized png in separate ↵ | Antonia Rescribe | |
| folder | |||
| 2021-12-06 | pipeline: ignore any files with a non-image suffix, rather than erroring on them | Nick White | |
| 2021-11-23 | gofmt, plus update documentation of recently changed pipeline.UploadImages | Nick White | |
| 2021-11-22 | internal/pipeline: remove old and broken requirement for TestStorageId() | Nick White | |
| 2021-11-22 | changed put.go so that a 4-digit number is appended to the end of each ↵ | Antonia Rescribe | |
| filename when images are uploaded to the pipeline | |||
 |
index : bookpipeline | |
| Tools to process books in a cloud based pipeline system |
| summaryrefslogtreecommitdiff |