Application transferred too few scanlines

[krambriw]

Hello, been using Motion many years now. Running an instance on 4 RPi's v3 w 2 USB cameras each. From one of them Motion from time to time reports the error message below and exits.

The other 3 RPi's never reports this error, all cameres are of the same make and brand. I have changed RPi but it did not help. Error is also reported in 4.7.0. Any hint what could cause this problem?

Best regards, Walter

[1:ml1:CAM31] [ERR] [ALL] [Nov 05 12:19:24] jpgutl_error_exit: Application transferred too few scanlines

[tosiara]

Can you upload the full debug log?

[krambriw]

Here is a log demonstrating the problem. Motion 4.7.0 starts up fine with 2 cameras and at line 518 and 3453 the error happens. I do monitor this and when the error is detected, Motion is automatically restarted

Best regards, Walter

motion.log

[tosiara]

Ok, thanks. Would it be possible to leave only one camera in the config, reproduce again and upload the new log?

[krambriw]

Hello, thank you!
Started Motion now with just one camera. I'll wait until the error shows up again (if so). With two cameras it happens 1-2 times per day. Strange is that the other 3 RPi's with the same setup works fine. Maybe one of the cameras that very rare has some problems to deliver proper frames??

What else you see regularly in the logfile is

• every 10 minute I do set a param (detection pause or detection start to synch with my home alarm system state)
• every 4th I do http request to Motion (http://192.x.x.x:8080) just to check that Motion is still running and responding

[Mr-Dave]

I am actually surprised that Motion is able to start up with two v4l2 cameras at such a high resolution. The typical problem is that the USB2 bus has a capacity limit of 480 Mb/s and that is typically only sufficient to run one camera.

I do see in the log that the palette is not specified. You could possibly try to use the MJPG format for the camera since it is a more compressed format than the one currently selected (YUYV) and that may provide a more consistent set of image data for the JPG compression function. Other options could be to lower the resolution or possibly the frame rate. Again, this isn't really a Motion limitation but instead a limit on the hardware bandwidth. (I've never been able to run two cameras on a single USB2 bus)

[krambriw]

Thanks for the hint about MJPG. Now running w 2 cameras using palette 8 since 10 pm yesterday, so far so good!
Thanks a lot for your help!
Best regards, Walter

[krambriw]

Hello, the MJPG trick did not help. After 9 hours I got the error below. I think I will replace the RPi3 with a RPi4 thas has USB 3 and see if that works better
Best regards, Walter

[1:ml1:CAM31] [ERR] [ALL] [Nov 07 09:30:26] jpgutl_error_exit: Application transferred too few scanlines
[0:motion] [NTC] [ALL] [Nov 07 09:33:06] motion_startup: Motion 4.7.0 Started
[0:motion] [NTC] [ALL] [Nov 07 09:33:06] motion_startup: Using log type (ALL) log level (ERR)
[2:ml2:CAM32] [CRT] [VID] [Nov 07 13:07:43] vid_mjpegtoyuv420p: Corrupt image ... continue

[krambriw]

Hello again, sorry for opening the conversation again
After trying to make it run fine with a RPi3, I have finally swapped it to a RPi4 and with the USB3 on board I have not seen the issue happening again

Just for my better understanding of the underlying problem, I would like to ask why is it necessary that Motion terminates when that error occure? And if so, why could not Motion just restart itself?

My workaround for the problem, when Motion was still running on the RPi3, was to watch the logfile for that error and then restart Motion. This worked actually fully acceptable, just a bit annoying. Most of the time it worked for days until the error happened so I suspect it could very well be the camera itself that failed. I have 7 cameras of the same type & brand it is only one specific that causes the fail

Now it will be interesting to see if everything continuous to work flawless with the RPi4

Best regards, Walter

[Mr-Dave]

I would like to ask why is it necessary that Motion terminates when that error occure?

Well, Motion isn't supposed to terminate at that error.

The error looks to be happening inside of the jpeg library somewhere. Motion provides a pointer to the error exit function provided by the library. As seen in your Motion log, this Motion provided error function is being called but at the end it is doing a longjmp to get back to the place where the setjmp was initiated but I'm guessing that this is where the segfault occurs.

From what I've seen searching github, this longjmp / setjmp method is how others are coding it for the jpeg library but I'll admit that this part of the code and the method goes way back. (I can not even recall a time that the library returned an error to use the longjmp.).

We could write it up / note it as a bug but it would first need to be re-validated using version 5 of Motion. Although the code is pretty similar in version 5 for this section, version 5 is C++ which could handle the longjmp differently than C.

[krambriw]

Dear Mr-Dave, ok understood.

I think, if this is not frequently or even never reported before, we can consider & close it as an academic issue. It also just happens with one of my USB cameras, is very difficult to replicate (takes days), so it might be the camera causing this. I have 4xRPi3, with 2, 2, 2, 1 USB Cams connected, running 24/7 since 2017 and it is just one camera that started causing this (the video from the camera is otherwise perfect). The proper solution in my case was most certainly to upgrade to an RPi4 or better (haven't had the error yet since I upgraded that RPi)

(From an academic point of view, I could also rather easily swap a copy of the sd card back into the RPi3, upgrade to version 5 and see how it behaves...)

With kind regards, Walter

[krambriw]

Just to verify, now built v5 on the RPi3, was easy to do, works fine. Started and running v5 some 30 mins ago and I will now see & watch if the issue is handled differently

Best regards, Walter

[Mr-Dave]

Note that v5 does still have some outstanding development needed. (But it is what I use to run my system). As I thought about this further last night, I strongly suspect that the underlying problem is the longjmp and multi-threading. i.e. There is a setjmp on each camera thread running and when the error occurs it possibly tries to go back to a different thread than where it originated. As such, the probable fix is to re-work the entire error handling of the jpeg utils module...

[krambriw]

Well, after a while the video from the usb cameras (v412 types) suddenly looked overexposed and I could not figure out why & how to adjust (was running config files from version 4.7) and I think config params have changed too much for me to dive into. I have therefore reverted back to running 4.7 on a RPi4, I'm happy with that if it proofs working

[krambriw]

Just an update of previous tests & current status:

• after running version 4.7 on a RPi4, the same problem occured after a few days
• currently running version 5 on the same RPi4 without any problem so far