Interesting video on the state of image processing

[DAP]

It requires raw sensor data, but it might be able to pull license plates out of the mud.

 

[Nigel]

There are a few problems with that from a dashcam point of view, main one being that the result is a video of an imaginary scene, a 3D model created by computer artificial intelligence, and then rendered as a video. It should never be used as evidence of reality!

Second, it does not work when you have motion blur, as we always do in the dark with dashcams at times of interest.

Third, they are creating a 3D model of a scene from multiple raw photographs taken at various points around the scene. This is not possible using dashcam images because a dashcam does not move around a scene in 3D, it only moves linearly, in a straight line through the scene. A lot of the information required for creating their 3D model is missing.

Google Research, like Microsoft Research, comes up with some fantastic stuff, and this certainly has some uses, but dashcams is not going to be one of them.

 

[DAP]

With raw images, and knowing the direction & magnitude of motion, a lot of motion blur can be removed. Also, with multiple raw images of the same subject, they can be stacked to improve resolution. This method can be used to improve license plates, but currently requires manual operations to line up the license plates and remove distortion (angle of plate to camera) from multiple frames of video. The main problem is that image compression deletes the information needed to do either of these things.

To get a full 3D model of the scene as done in the video, yes you would need a bunch of different angles, but one should be able to get a partial 3D scene from a video as you drive through. It will be a long time before this could be done real time in camera, but for some critical event, it might be worth attempting as a post processing job.

 

[Dash cams Serbia]

Looks fake too me.