BlackVue DR750S-2CH GPS data quirk

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
hi guys,

i already posted this on another site, but haven't gotten a response, maybe i will get one here:

this ultimately boring nerdy stuff and probably doesnt really matter beyond data integrity considerations but i figured i'd bring it up anyway.
i've only had the above dashcam for a few weeks (running fw v.1.011_2018.09.12), but yesterday i started writing some code to grab the gps data out of the video files, so i can whack it in a DB and have a record where the car's gone etc... and noticed that the gps data stored in the video files is sometimes checksummed wrong.
the dashcam stores what appear to be timestamped raw nmea 0183 sentences directly inside the video files, approximately 80KB in.
an example:
[1538216648173]
[1538216648173]$GPRMC,002406.00,A,3753.63717,S,14518.49970,E,0.074,,290918,,,D*6C
[1538216648173]$GPVTG,,T,,M,0.074,N,0.138,K,D*2F
[1538216648173]$GPGGA,002406.00,3753.63717,S,14518.49970,E,2,07,1.12,112.1,M,-1.5,M,,0000*51
[1538216648173]$GPGSA,A,3,27,07,09,28,30,08,23,,,,,,1.87,1.12,1.50*01
[1538216648173]$GPGSV,3,1,11,03,04,009,,07,65,227,41,08,25,068,30,09,76,076,41*7D
[1538216648173]$GPGSV,3,2,11,16,18,139,24,23,46,059,38,27,23,103,28,28,13,332,31*7B
[1538216648173]$GPGSV,3,3,11,30,37,266,24,42,46,360,34,50,46,360,35*48
[1538216648173]$GPGLL,3753.63717,S,14518.49970,E,002406.00,A,D*7D[1538216649174]

i found that on the GPRMC sentences, sometimes the checksum is incorrect. here is an example of an incorrect checksum:
$GPRMC,205424.00,V,3754.48531,S,14520.62279,E,0.165,,260918,,,A*6A
the checksum is the *6A part. the correct checksum is actually 7D
i had a look at all the GPRMC sentences i could find in my dashcam footage and found that the incorrect checksum sentences often had the warning flag set for the "Navigation receiver warning" part of the message, while the ones with correct checksums often had the ok flag set.
changing the V flag to the A flag in the incorrect checksum messages results in the checksum actually being correct:
$GPRMC,205424.00,V,3754.48531,S,14520.62279,E,0.165,,260918,,,A*6A
checksum is actually 7D
$GPRMC,205424.00,A,3754.48531,S,14520.62279,E,0.165,,260918,,,A*6A
checksum is 6A, matching the *6A found in the sentence

i also noticed that sometimes A needed to be changed to V for the checksum to be correct. i wrote some code that only substituted A for V or V for A upon checksum error, and found they all subsequently validated.

anyhow,

does anyone know what gps receiver chipset is used in this dashcam? i find it unlikely that the gps receiver itself is making this error, and suspect that blackvue is probably causing the issue, but if i knew what the chipset was i could probably do some googling and find out where the fault lies before i report the issue to the relevant party.
also let me know if you're interested in the code for dumping gps data to db/file. will probably make it export maps by whatever time period to show vehicle path etc. its extracting the data from 110GB of footage in about 5 seconds as it just seeks directly to the relevant crap without reading the entire file.
happy to share.
 

jokiin

Well-Known Member
Manufacturer
Joined
Jan 27, 2013
Messages
51,519
Reaction score
29,518
Location
Sydney, Australia ~ Shenzhen, China
Country
Australia
Dash Cam
Too many ¯\_(ツ)_/¯
no idea what GPS they're using but the A and V flag is normally used to indicate if the data is valid or not, when it's invalid data I'd expect the checksum to be incorrect
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
no idea what GPS they're using but the A and V flag is normally used to indicate if the data is valid or not, when it's invalid data I'd expect the checksum to be incorrect

i wouldn't, as they should calculate the checksum AFTER setting the invalid flag. some marked valid have incorrect checksum, some marked invalid have incorrect checksum. inverting that flag corrects all of them. it's not treated as a single bit difference either, as its the ordinal of 'A' and 'V' that's xorred.

it's appears that sometimes the checksum is calculated before flipping the flag.
 

jokiin

Well-Known Member
Manufacturer
Joined
Jan 27, 2013
Messages
51,519
Reaction score
29,518
Location
Sydney, Australia ~ Shenzhen, China
Country
Australia
Dash Cam
Too many ¯\_(ツ)_/¯
as I said I'm not familiar with what GPS they use, wouldn't expect a valid checksum when there's invalid data though, if there's a lot of invalid data it might be a faulty module
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
the checksum is calculated by the receiver after it assembles the nmea sentence to deliver down the wire. the checksum is only a check on the integrity of the sentence itself, not the validity of any data inside the sentence.

the data all looks good and correct to me, it's just that for some reason either the receiver changes the validity flag post checksum calculation or (far more likely) the blackvue device changes it for some reason. perhaps they have coopted that flag for their own use to record something else.

as to whether or not the module is faulty, i find it very unlikely due to the nature of the quirk. knowing which specific module it is would allow me to see if its a known quirk of the module (although i can't find any that appear to have this quirk). if it's not, then i guess it's blackvue.
 

bpsmicro

Active Member
Joined
Jul 13, 2017
Messages
225
Reaction score
108
Country
Canada
So the 750 also stores the GPS data as a separate .gps file. Or, at least, my own app to suck all the footage down across WiFi collects those files. I did read that the 750 also embeds the GPS data in the video footage.
I looked at some of the .gps files I'd extracted recently, and I'm seeing the same thing. It appears that the 'V' records come in clumps, and then revert back & forth to 'A' records in clumps. But using online decoders, I see the same checksum variance. Very peculiar.

Using the decoder at http://freenmea.net/decoder I plugged in a file segment including a "bad" record and subsequent "good" record, and indeed I'm not seeing any reason for that flag to be set at all.

I can't imagine BV manipulating the statements out of the GPS chip, but who knows. I think your hack will do the job, but is unfortunate. I assume their own Viewer either does the same, or ignores the checksum entirely.
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
thanks for letting me know that you see the same thing bpsmicro, it is appreciated.

my 750 does not store the gps data in separate files. my Record directory only contains mp4 files and an event_upload_list.bin file, though i have seen that some cameras have thm, 3gf and gps files in addition.

i'll post here again if i figure out what is going on with the validity flag.
 

bpsmicro

Active Member
Joined
Jul 13, 2017
Messages
225
Reaction score
108
Country
Canada
I'll have to remember to pull the SD card out of my camera and look. I have an app of my own design that connects to the camera via WiFi and downloads all files using the commonly-known, but unsupported http interface. It generates the separate .gps file, which is curious indeed.
When I wrote the app, I had a 650S, which is why I knew what they were.
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
i'm intending to do the same regarding pulling the data down over wifi, though assuming the http daemon on the thing supports range requests i was just going to yank out the segments that contain gps and accelerometer data

if the http interface carves it out for me and presents them as files though, all the better
 

bpsmicro

Active Member
Joined
Jul 13, 2017
Messages
225
Reaction score
108
Country
Canada
Mine is in C#/.NET. I'd be happy to share knowledge with you. Just PM me.
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
so at this point it looks like the blackvue device might overwrite the fix validity field in the rmc sentence if there are either a low number of satellites visible or a high dilution of precision.

reviewing the footage in which the checksum errors are present (because of the overwrite of the validity field), the kmh indicator displays --- during these periods (and presumably the gps light is off on the unit). after the checksum errors disappear, the kmh indicator starts displaying speed (even if its just 0kmh).

similarly, when viewed in the desktop app, the map position is not updated until the checksum errors have elapsed.

where the checksum errors are present, there is actually data that COULD be used to indicate speed and position, but the camera and apps do not do so.

whether or not this is because they reject the message due to the checksum error, or the fix validity field is being used by bv to selectively not show the data due to low satellites or high dilution of precision i do not yet know.
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
No wonder no one has answered you, you are talking complete 'Gibberish' You must work in 'IT' as that's the way they speak. But 'Hey' normal people on Planet Earth speak differently. How about you tell me what your problem is with the camera i.e.
whats wrong with it, and what is not working. Then maybe I can help you. If you stopped talking like 'Cpo4' from 'Star Wars' this might be possible.
Regards
Mike

thanks for your input mike. i don't think you'd be able to help mate, the problem is highly technical, as you so keenly recognised. i expect your skills probably lie elsewhere. you seem like a real people-person, i can tell by how warm and friendly your tone is, as well as your seemingly effortless ability to relate to people and get them on your side.

i appreciate your offer to help though. have a good one.
 

M8TJT

Well-Known Member
Joined
Feb 14, 2018
Messages
1,376
Reaction score
720
Location
East Sussex
Country
United Kingdom
Dash Cam
2x750 2CH
Although Mike does not fully understand the technicalities of your post, at least he seems to have a shift key on his device to correctly capitalise his posts.:ROFLMAO:
where the checksum errors are present, there is actually data that COULD be used to indicate speed and position, but the camera and apps do not do so.
Presumably they ignore the data from low sat count or high DoP so as not to display or log possibly grossly inaccurate data which could arise under those conditions?
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
unsure, i started reversing the firmware but also bought a house and have mostly been focussing on home improvement. will get back on it soon. will also be adding additional features to the firmware as it appears to be trivial to do so. mostly i just want to add functionality so i can make an http call to the device and have it shutdown.

i'm allergic to capital letters, acute case sensitivity

edit: actually i have all the data in a db and can query the DoP, i did take a look and saw many examples of high dilution without invalidating, and also low dilution with invalidation
 

M8TJT

Well-Known Member
Joined
Feb 14, 2018
Messages
1,376
Reaction score
720
Location
East Sussex
Country
United Kingdom
Dash Cam
2x750 2CH
i'm allergic to capital letters, acute case sensitivity
:ROFLMAO:
actually i have all the data in a db and can query the DoP, i did take a look and saw many examples of high dilution without invalidating, and also low dilution with invalidation
Do you think that they overwrite the 'V flag' deliberately under the aforementioned sat conditions then?
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
i think they do it deliberately, but i am unsure why. there seems to be no DoP related pattern. the min-max bounds for DoP in valid and deliberate invalid overlap quite a lot
 

sigtwo

New Member
Joined
Mar 22, 2019
Messages
1
Reaction score
2
Country
United States
I actually did what you describe. You can check it out at SigTwo.

I take gps data from another receiver in my car and pair it with blackvue files and put it all on a map. Store everything in cloud so it's always there. Can share videos on facebook, twitter. I am looking to develop this further and allow other users to signup. Looking for strong software developers to help. PM me if interested.
 
OP
L

lo_pan

New Member
Joined
Sep 3, 2018
Messages
16
Reaction score
7
Country
Australia
I actually did what you describe. You can check it out at SigTwo.

I take gps data from another receiver in my car and pair it with blackvue files and put it all on a map. Store everything in cloud so it's always there. Can share videos on facebook, twitter. I am looking to develop this further and allow other users to signup. Looking for strong software developers to help. PM me if interested.

looks interesting mate, good work.

i finished my version ages ago, but i’ve basically only built it for my own use. im storing all the gps data from the video files as spatial data in a db, then rendering it to google maps shapes based on whatever query i run.
 
Top