Script to extract GPS Data from Novatek MP4

lacibaci

Well-Known Member
Joined
Mar 5, 2015
Messages
1,341
Reaction score
595
Country
United States
I think you'll find on some of those models they are also embedded in the file as well as the standalone file
You're joking :\ So they only give an illusion of control.
 
OP
S

SergeiF

Active Member
Joined
Dec 20, 2015
Messages
121
Reaction score
97
Country
New Zealand
I have updated my script to allow multiple files to be used.
To do that I had to change the way I handle command line arguments, thus excluding Centos 6 (or less) people out (as it relies on minimum python 2.7).

The script will take globs (*), list of files, and directories as input. If directories are used as input, it will use the files in those directories (but will not recurse). It will output into single gpx file.
I probably introduced new bugs, let me know if you find them ;).
A note of warning, including multiple files from different trips will lead to scrambled track (ie: all over the place, with direct lines connecting discontinuations).

Good luck :).
 

Curtis54

Member
Joined
Oct 4, 2016
Messages
97
Reaction score
36
Country
France
It is fairly straight forward: the gist of it that each atom contains size (4byte long int) and type (4byte long string) at the beginning of atom. The file starts with an atom. I iterate through atoms intil I hit 'moov' type of atom.
The 'moov' contains a special non standard 'gps ' sub-atom. To get to sub-atom 'gps ' I simply iterate through sub-atoms inside of 'moov' (with offset of 8bytes for header).
The 'gps ' contains simple look up table made up of 8byte rows, that point to the 'free' atoms that contains the actual GPS data.
The first row is version/metadata/notsure, I skip that.
The following rows consist of 4byte address (absolute) and 4byte size (0x1000), these point to the GPS data in the file.

The structure of the GPS data atom (the 'free' atoms mentioned above) is following:
hour,minute,second,year,month,day,active,latitude_b,longitude_b,unknown2,latitude,longitude,speed = struct.unpack_from('<IIIIIIssssfff',data, 48)
For those unfamiliar with python struct:
I = int
s = is string (size 1, in this case)
f = float

I hope that helps ;).

Sergei.
thanks for the help,

I will try to develop a GUI version of your script, the language will be java,

the features I want to develop :

- extract GPS data, in gpx format, but also in other formats like csv ( in order to open the file with a spreadsheet like calc or excel )

- display the route on a map, and possibility to save the map in jpeg/png file, I think I will use the api "openstreetmap" :
http://wiki.openstreetmap.org/wiki/Frameworks

- integrate a video player in order to play the video file, or if I don't want to reinvent the wheel : I can launch in parallel vlc with the video, as an external player
 

Mtz

Well-Known Member
Joined
Nov 15, 2012
Messages
3,686
Reaction score
2,066
Location
Nice place
Country
Romania
Dash Cam
Viofo A119. WR1. SGGCX2. GitUp.
I recommend to look also at RV how is working, for example it can use Leaflet.

Leaflet.jpg

enjoy,
Mtz
 
OP
S

SergeiF

Active Member
Joined
Dec 20, 2015
Messages
121
Reaction score
97
Country
New Zealand
thanks for the help,

I will try to develop a GUI version of your script, the language will be java,

the features I want to develop :

- extract GPS data, in gpx format, but also in other formats like csv ( in order to open the file with a spreadsheet like calc or excel )

- display the route on a map, and possibility to save the map in jpeg/png file, I think I will use the api "openstreetmap" :
http://wiki.openstreetmap.org/wiki/Frameworks

- integrate a video player in order to play the video file, or if I don't want to reinvent the wheel : I can launch in parallel vlc with the video, as an external player
I am not a Java fan, but I am glad someone takes this seriously.
Although I enjoy writing little python scripts, I really despise the messy GUI programming (too much OO for my liking).

Regarding CSV format it is really easy to modify my script to generate CSV (perhaps I can add a flag for CSV output mode?).

If you look in my script you can get a hint on how to handle oddball GPS coordinate format that Novatek uses:

Code:
def fix_coordinates(hemisphere,coordinate):
    # Novatek stores coordinates in odd DDDmm.mmmm format
    minutes = coordinate % 100.0
    degrees = coordinate - minutes
    coordinate = degrees / 100.0 + (minutes / 60.0)
    if hemisphere == 'S' or hemisphere == 'W':
        return -1*float(coordinate)
    else:
        return float(coordinate)
 
  • Like
Reactions: Mtz

Gibson99

Well-Known Member
Joined
Jun 20, 2014
Messages
9,193
Reaction score
7,841
Location
Houston, we've had a problem, Texas
Country
United States
Dash Cam
Yes
What about instead of stringing all the GPS data from multiple videos together into one gpx file, you could just make one gpx per mp4, with the same filename, just with .gpx extension. So if you had 4 mp4 files named 001.mp4 thru 004.mp4, the script would create 001.gpx thru 004.gpx. I have never tried using rv or other software that uses gpx files, but that seems like a simple way to approach it, and wouldn't have the problem of GPS tracks jumping all over between files. should be a simple for/next loop when you specify a directory rather than a single file.
 

Curtis54

Member
Joined
Oct 4, 2016
Messages
97
Reaction score
36
Country
France
we can give the choice to the user, in the GUI with two radio buttons "put GPS data in a separate gpx file for each video file", "regroup GPS data from multiple video files in a one gpx file"

same thing can be done in the python script, a switch "--single-gpx"
 

Mtz

Well-Known Member
Joined
Nov 15, 2012
Messages
3,686
Reaction score
2,066
Location
Nice place
Country
Romania
Dash Cam
Viofo A119. WR1. SGGCX2. GitUp.
I think @SergeiF created one GPX file just because he could not create individual files like you described.
All video players are looking for a subtitle file if it have some extension and exactly the same name of video file.
In this way should be created the GPX files because if someday we will have some plugin for some player the start will be that the player to look also for GPX extension.

RV can extract .srt files too which means many people wanted to watch the video with speed and coordinates on the screen like a subtitle file or like it is already embedded in the A119 video file.

enjoy ,
Mtz
 
OP
S

SergeiF

Active Member
Joined
Dec 20, 2015
Messages
121
Reaction score
97
Country
New Zealand
I think @SergeiF created one GPX file just because he could not create individual files like you described.
Actually I can ;).
Not now though, job and all :).
I will look into it at later stage.

For now if you need that functionality you can do it from bash (quick and dirty):

Code:
for file in *.MP4; do ./nvtk_mp42gpx.py -i $file -o $file.GPX; done
Or if you want to get rid of .MP4 for resulting filename (requires bash not sh, ksh, csh etc):

Code:
for file in *.MP4; do ./nvtk_mp42gpx.py -i $file -o ${file%.MP4}.gpx; done
 

Mtz

Well-Known Member
Joined
Nov 15, 2012
Messages
3,686
Reaction score
2,066
Location
Nice place
Country
Romania
Dash Cam
Viofo A119. WR1. SGGCX2. GitUp.
Thank you ! I will wait for your solution and for the GUI.

enjoy ,
Mtz
 

Mtz

Well-Known Member
Joined
Nov 15, 2012
Messages
3,686
Reaction score
2,066
Location
Nice place
Country
Romania
Dash Cam
Viofo A119. WR1. SGGCX2. GitUp.
Hey, @SergeiF are you OK in New Zealand?
 

nuxator

Member
Joined
Sep 5, 2013
Messages
85
Reaction score
26
Country
France
Hi.
It seems that viofo changed GPS format with firmware 2.0
Wan you/will you make a v2 version of your script?
 
  • Like
Reactions: Mtz
OP
S

SergeiF

Active Member
Joined
Dec 20, 2015
Messages
121
Reaction score
97
Country
New Zealand
Hey, @SergeiF are you OK in New Zealand?
Yes we are fine thank you :). As far as I know there were only 2 casualties from the earthquake.
I am about to embark on a road trip across New Zealand so will be out of the action for few weeks. The trip has been planned prior to earthquake, so I will have now a period of 8hours drive, 5 hours ferry and another 8-10 hour drive due to State Highway 1 is no more in upper South Island.

Hi.
It seems that viofo changed GPS format with firmware 2.0
Wan you/will you make a v2 version of your script?
Great :(. I will quickly check what has changed and see if it is a 5 min fix.
 

nuxator

Member
Joined
Sep 5, 2013
Messages
85
Reaction score
26
Country
France
I spend 2 hours to understand your script but in fact it was a 2seconds fix.:confused: The header to ignore is not 48 anymore but 16.
You only have to edit line 99 : struct.unpack_from('<IIIIIIssssfff',data, 16)
 
Last edited:
Thread starter Similar threads Forum Replies Date
RobTrehy A119 138
Top