Daniel Reetz, the founder of the DIY Book Scanner community, has recently started making videos of prototyping and shop tips. If you are tinkering with a book scanner (or any other project) in your home shop, these tips will come in handy. https://www.youtube.com/channel/UCn0gq8 ... g_8K1nfInQ

A task for volunteers: handling out of memory situations

Scan Tailor specific announcements, releases, workflows, tips, etc. NO FEATURE REQUESTS IN THIS FORUM, please.
StevePoling
Posts: 290
Joined: 20 Jun 2009, 12:19
E-book readers owned: SONY PRS-505, Kindle DX
Number of books owned: 9999
Location: Grand Rapids, MI
Contact:

Re: A task for volunteers: handling out of memory situations

Post by StevePoling » 24 Nov 2010, 23:02

Tulon wrote:2. When an out-of-memory situation happens inside the QImage class (part of Qt), it doesn't throw an exception, instead turning the image into a null image. That's very mean if you ask me. This means we would need to insert explicit checks every time we construct a QImage. That's currently done in some places but not in all of them. Fortunately, QImage is rarely used as is. Most of processing inside ST works with either imageproc::BinaryImage or imageproc::GrayImage classes that don't have this problem, even though GrayImage is a wrapper around QImage.
One of the things I've learnt from studying Microsoft code is to check everything outside yourself for indications of failure. in this case, every new should be followed by an is-valid check. The sooner after a failure, the better it is to understand the problem. I liken it to a railroad derailment: all of the train's twisted wreckage is not as interesting as the bad spot of track that goes bad at the very start of problems.
Tulon wrote:These were still good news, if you are wondering. The bad news is that ST is a multi-threaded application, making handling out-of-memory situations difficult. It's not a problem to catch the std::bad_alloc exception in a background thread. The real question is what to do next. You can't inform the user from a non-GUI thread and you can't just eat the exception, because the task that failed probably has someone in another thread expecting its completion.
Have you considered an observer pattern? Exception-throwing threads can send bad news to someplace special. The someone in another thread who is expecting something from its exception-throwing threads can either register for notifications, or poll the bad news repository on a timeout.

I know this is superficial and does not reflect meaningful reflection upon the details you just provided. I've not yet gotten into your code so I let the best parts of what you said in the rest of your post fly over my head. I hope to apply deep thought to them when I'm less ignorant.

Tulon
Posts: 687
Joined: 03 Oct 2009, 06:13
Number of books owned: 0
Location: London, UK
Contact:

Re: A task for volunteers: handling out of memory situations

Post by Tulon » 25 Nov 2010, 05:18

StevePoling wrote:One of the things I've learnt from studying Microsoft code is to check everything outside yourself for indications of failure. in this case, every new should be followed by an is-valid check.
That's not the way to go. You use that strategy only if you don't have better options. Exceptions + centralized error handling is the way to go.
StevePoling wrote:Have you considered an observer pattern? Exception-throwing threads can send bad news to someplace special. The someone in another thread who is expecting something from its exception-throwing threads can either register for notifications, or poll the bad news repository on a timeout.
We should use existing inter-thread communication facilities rather than inventing new ones.
Scan Tailor experimental doesn't output 96 DPI images. It's just what your software shows when DPI information is missing. Usually what you get is input DPI times the resolution enhancement factor.

StevePoling
Posts: 290
Joined: 20 Jun 2009, 12:19
E-book readers owned: SONY PRS-505, Kindle DX
Number of books owned: 9999
Location: Grand Rapids, MI
Contact:

Re: A task for volunteers: handling out of memory situations

Post by StevePoling » 25 Nov 2010, 14:04

Tulon wrote:
StevePoling wrote:Have you considered an observer pattern? Exception-throwing threads can send bad news to someplace special. The someone in another thread who is expecting something from its exception-throwing threads can either register for notifications, or poll the bad news repository on a timeout.
We should use existing inter-thread communication facilities rather than inventing new ones.
I'm sorry I was not more clear. i wasn't suggesting another inter-thread communication mechanism as much as building on it a solution to your out-of-memory exception in thread A communicating the fault to thread B.

Post Reply