Finally succeeded to recover the video footage losing just about 0.5 sec until the crash. As far as I know there is no single tool available that can do it automatically, but I hope the instructions below might help to someone in a similar situation.
Recovering video data from the file system
The good news is that video and audio is constantly flushed to the data region of the file system (memory card). The bad news is that Panorama links the 0 byte file entry to the actual data only after the file has been fully written. In case the fragmentation was not needed to store the video data (i.e., there were X contiguous free clusters available at the time of writing), the recovery of video data should be relatively easier.
Before running any repair tools on the memory card first create the "disk image" of the card so that it would be possible to revert to the original state. You can create the image using:
In Linux (assuming /dev/sdb is the device of the connected card reader):
Code:
$ sudo dd if=/dev/sdb of=~/memcard.img bs=1M
Later if you need to revert the memory card to the original state use:
Code:
$ sudo dd if=~/memcard.img of=/dev/sdb bs=1M
In case of FAT32 (<= 32GB)
If the size of the memory card is 32GB or less, it is formated by Panorama built-in formater using FAT32 file system and 32KB cluster size. The camera starts writing the video file by first creating a file entry in the directory. Only after the file is fully written the file entry is updated by specifying the file size and pointer to the first cluster of the data region where the file is stored. The video data is constantly written to the data region, however, the FAT table is updated only time to time.
The easiest solution is to salvage unused cluster chains (FAT chains which are allocated, but which are not used by any directory or file entry) to files. This can be done in Windows using DSKCHK or in Linux using fsck.fat:
In Windows:
Run "Check this drive for errors" from the "Tools" tab on the disk:
To see the hidden directory "FOUND" created by DSKCHK, the directory permissions have to be changed running these commands as "administrator":
Code:
E:\>dir
Volume in drive E is GLG320
Volume Serial Number is 201D-1EEC
Directory of E:\
12/29/2014 11:14 PM <DIR> JPEG
12/29/2014 11:14 PM <DIR> EVENT
05/30/2015 01:48 PM <DIR> VIDEO
0 File(s) 0 bytes
4 Dir(s) 1,620,180,992 bytes free
E:\>attrib -h -r -s /s /d *.*
E:\>dir
Volume in drive E is GLG320
Volume Serial Number is 201D-1EEC
Directory of E:\
12/29/2014 11:14 PM <DIR> JPEG
12/29/2014 11:14 PM <DIR> EVENT
05/30/2015 01:48 PM <DIR> VIDEO
06/01/2015 07:06 PM <DIR> FOUND.000
0 File(s) 0 bytes
5 Dir(s) 1,620,180,992 bytes free
In FOUND.000 you will see files in the format "FILE0000.CHK" which contain clusters of the unassigned FAT chains.
In Linux (assuming /dev/sdb is the device of the connected card reader):
Code:
$ sudo fsck.fat -a /dev/sdb
fsck.fat 3.0.26 (2014-03-07)
0x41: Dirty bit is set. Fs was not properly unmounted and some data may be
corrupt.
Automatically removing dirty bit.
FATs differ but appear to be intact. Using first FAT.
Reclaimed 1402 unused clusters (5742592 bytes) in 1 chain.
Performing changes.
/dev/sdd1: 6 files, 1406/479298 clusters
$ ls -alh /media/user/4C94-FC0C/
total 5.5M
drwx------ 5 user user 4.0K Jan 1 1970 .
drwxr-x---+ 5 user user 4.0K Jun 9 21:37 ..
drwx------ 2 user user 4.0K Jun 8 20:49 EVENT
-rw-r--r-- 1 user user 5.5M Jan 1 1980 FSCK0000.REC
drwx------ 2 user user 4.0K Jun 8 20:49 JPEG
drwx------ 2 user user 4.0K Jun 8 20:49 VIDEO
The root folder of the memory card will contain files in the format "FSCK0000.REC" which contain clusters of the unassigned FAT chains.
The problem with this approach is that since the FAT chain is updated by Panorama only time to time, there will be more video available in the data region than referenced in the unused FAT chain. One approach is to recover X unallocated clusters following right after the last cluster referenced in the unused FAT chain (custom tool needed).
Alternative approach is to release the unused FAT chains as free space (by running fsck.fat interactively) and look for video data in the unallocated space. This approach should be used also when fsck.fat and DSKCHK does not recover any unused FAT chains (in case the camera did not update FAT at all). There are several tools that can extract unallocated clusters from the data region (FTK,Autopsy/Sleuthkit).
Then in the unallocated data look for clusters starting with this MP4 header and extract the following X clusters to the recovered_candidate.mp4 file:
Code:
00000000 00 00 00 1c 66 74 79 70 4d 53 4e 56 01 29 00 46 |....ftypMSNV.).F|
00000010 4d 53 4e 56 6d 70 34 32 69 73 6f 6d 00 00 00 94 |MSNVmp42isom....|
00000020 75 75 69 64 50 52 4f 46 21 d2 4f ce bb 88 69 5c |uuidPROF!.O...i\|
00000030 fa c9 c7 40 00 00 00 00 00 00 00 03 00 00 00 14 |...@............|
00000040 46 50 52 46 00 00 00 00 00 00 00 00 00 00 00 00 |FPRF............|
00000050 00 00 00 2c 41 50 52 46 00 00 00 00 00 00 00 02 |...,APRF........|
00000060 6d 70 34 61 00 00 02 0f 00 00 00 00 00 00 00 80 |mp4a............|
00000070 00 00 00 80 00 00 bb 80 00 00 00 01 00 00 00 34 |...............4|
00000080 56 50 52 46 00 00 00 00 00 00 00 01 61 76 63 31 |VPRF........avc1|
00000090 01 4d 00 28 00 02 00 02 00 00 3a c0 00 00 3e 80 |.M.(......:...>.|
000000a0 00 1e 00 00 00 1e 00 00 07 80 04 38 00 01 00 01 |...........8....|
000000b0 00 00 00 08 66 72 65 65 00 00 00 00 6d 64 61 74 |....free....mdat|
In case of exFAT (> 32GB)
If the memory card is larger than 32GB it will be formated in the Panorama using exFAT file system and 128KB cluster size. The same recovery logic applies also to exFAT with minor differences:
- Linux fsck.fat does not support exFAT file system (use Window's DSKCHK instead).
- In case fragmentation is not needed Panorama will not update FAT at all. In this case there are good chances that the file starts from the first unallocated cluster and follows in X consecutive unallocated clusters.
Repairing unfinalized video file
Panorama uses QuickTime media container to store the video and audio. The video and audio data is dumped into MP4 "mdat" atom, and after video is fully captured, the size of the "mdat" atom is updated and at the end of file is appended MP4 "moov" atom which contains information about codecs used and the information about all the frames contained in "mdat" atom.
The recovered unfinalized video file will have no "moov" atom at the end of file and thus will not be playable by any video player. There are tools out there which will reindex video and audio stored in "mdat" atom and write a proper "moov" atom if given a working reference file taken by the same camera and using the same video settings.
On Linux:
Code:
$ sudo apt-get install libavformat-dev libavcodec-dev
$ wget https://github.com/ponchio/untrunc/archive/master.zip
$ unzip master.zip
$ cd untrunc-master/
$ g++ -o untrunc file.cpp main.cpp track.cpp atom.cpp mp4.cpp -L/usr/local/lib -lavformat -lavcodec -lavutil
$ ./untrunc
Usage: untrunc [options] <ok.mp4> [<corrupt.mp4>]
$ ./untrunc ./good.mp4 ./recovered.mp4
Warning: Panorama uses different H264 codec PPS profile for every file (I guess based on the amount of light at the time of video capture). Most likely the file captured before the crash will contain the same H264 PPS profile and can be used as a correct reference file, but this is not guaranteed. In the worst case there are only 15 different H264 PPS profiles which can simply be bruteforced (see untrunc-master/track.cpp line 393). If reference file with a wrong PPS profile is used the fixed video will have sound, but the video will be all green.
There is also a commercial online service (
http://mp4repair.org) which will fix the unfinalized file by correctly guessing codecs and H264 PPS profile used.