The Definitive [Xiaomi Yi RAW] Guide

Discussion in 'Yi Action Camera' started by schnurbi, Jul 31, 2015.

  1. willk

    willk New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    United States
    Just got home from my first vacation with the Yi. Love it so far, but I did a lot of time lapses with the app, so got a lot of ~13mb compressed raws. Any progress on getting them into usable raws? Anywhere else I should be looking for info on this? The jpgs will work okay, but not great. I've got 6 weeks worth of stuff to dig into, if I can get access to it. Any help would be much appreciated!
     
  2. RiTCHiE007

    RiTCHiE007 Member

    Messages:
    41
    Likes Received:
    2
    Country:
    Netherlands
    Any update bout the 13mp raws?
     
  3. RiTCHiE007

    RiTCHiE007 Member

    Messages:
    41
    Likes Received:
    2
    Country:
    Netherlands
    Everything working fine on my z23l :) Got 30mb raws and this is just what i needed so thanks for all the work.
     
  4. nCaine

    nCaine Member

    Messages:
    73
    Likes Received:
    37
    Country:
    Poland
    I have one problem. Can`t get the lens profile to show in CAMERARAW.
    Dont know why but my files for camera raw are located in C:\ProgramData\Adobe\CameraRaw\LensProfiles\1.0 . AND YES IT IS THE CORRECT folder. Becos if i remove any profiles from there , they stop showing in the CameraRaw selection menu.


    I dont have appdata folder in the user folder. Like shown below. Im using Windows 8.1 64 bit.
    C:\Users\[YOU]\AppData\Roaming\Adobe\CameraRaw\LensProfiles\1.0

    Also the file for the Yi is just 12 kb , while the other profiles are around 150 kb. Is that correct ?

    Any sugestion ?

    And sorry for my english, im self-taught.

    UPDATE:
    Dunno why the appdata folder was NOT showing thru file explorer. I did search for "roaming" and it did find the appdata folder. Placed the file ther and it work perfect.

    I realy dont know why those folders arent showing in normal folder browsing,
     
    Last edited: Mar 26, 2016
  5. gretl4

    gretl4 Member

    Messages:
    57
    Likes Received:
    14
    Country:
    United Kingdom
    Does option 9 work? HDR video?
     
  6. AndreaPaco

    AndreaPaco New Member

    Messages:
    6
    Likes Received:
    4
    Country:
    Italy
    I'm investigating about using Xiaomi Yi RAW files under GNU/Linux. I noticed they are RAW RGGB Bayer so cannot be open directly with dcraw/ufraw/rawtherapee.

    I found the website of raw2dng and the source code of the last available version is public.

    http://e2500.narod.ru/raw2dng_e.htm

    It isn't portable so I made some minor changes to compile with g++ under GNU/Linux, obtaining a working raw2dng and raw2nef command line tools.

    Now I can convert single shot RAW files with this configuration:

    31850496, 0, 4608, 3456, 7, 9216, 16, 4, XIAOMI-YI

    Then I can open the DNG with rawtherapee. I noticed that colors are a bit flat and without auto white balance pictures are "reddish" and dark. I have to investigate further (and any suggestion is welcome!).

    The problem is that I wanna use time lapse. In this case the Xiaomi Yi produces compressed RAW files of ~13MiB (and I don't want to use script for time lapse because I want to be free to change shooting mode with the app). With this configuration I can convert RAW files to DNG:

    13492224, 0, 4608, 3456, 7, 3904, 16, 4, XIAOMI-YI-COMP

    I can open DNG in rawtherapee and I can glimpse the picture (sort of... compressed of the left, using 1/3 of the lenght of the image and with altered colors), but in fact they are unusable... I'm close but I think there is a bits order/depth problem (or there is A-law or DPCM compression?)... and it cannot be solved simply with the configuration parameters...

    I wrote to paul69 asking him to collaborate, but it seems a project abandoned in 2005~2006. I cannot find Copyright on the website or in the comments of the source code and there is not a LICENSE file or so. I could submit the project on my github account, but it would be a bit abusive... I would prefer collaborate with paul69. In case he will not reply me, I'll submit the code to github by myself (as a fork, with credits).

    I'm looking for the IMX206 datasheet or any other tech spect that can help me to understand compressed RAW format.
     
    Last edited: May 24, 2016
    Droscy, willk and kikislater like this.
  7. krashekspress

    krashekspress New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    Croatia
    I'm getting weird reddish blurry images, how to fix that?
    [​IMG]

    EXIF fix wasn't working, fixed now, sry :(
     
    Last edited: May 29, 2016
  8. hanzodeceptor

    hanzodeceptor New Member

    Messages:
    2
    Likes Received:
    0
    Country:
    Indonesia

    Hi AndreaPaco. Can you tell me how to use the 13mb RAW? How to decompress them n convert to dng. I have bunch of 13mb because of timelapse. Many thanks
     
  9. AndreaPaco

    AndreaPaco New Member

    Messages:
    6
    Likes Received:
    4
    Country:
    Italy
    Hi hanzodeceptor, I'm in the same condition. I'm still trying to understand... it's really difficult bacause datasheet of CCD sensors, bus and compression formats are confidential and often released under NDA. The only one documentation I found (and somewhat usable?) is a draft of CSI-2 specification, probably a leaked document:

    DRAFT MIPI Alliance Specification for Camera Serial Interface 2 (CSI-2)

    The interesting part might be Annex E Data Compression for RAW Data Types.

    Anyway, paul69 didn't reply me, so I'll take the initiative to publish the code on github so it will be simple to track changes and collaborate with other developers.

    https://github.com/wildstray/raw2dng

    All credits to paul69 (http://e2500.narod.ru/). For now, respect the original code, it will only compile under GNU/Linux (theoretically the code is now portable and have to compile under MS VC++ or MinGW, but I cannot try... there is some Windows developer here?) but it will just convert 30M RAW files only. To decompress 13M RAWs the way is still long...
     
    kikislater likes this.
  10. AndreaPaco

    AndreaPaco New Member

    Messages:
    6
    Likes Received:
    4
    Country:
    Italy
    I found another document about DPCM image compression... and this is a confirmation that this compression method it is patented :(
    https://www.google.com/patents/US20040202375

    I could try (and probably I will try) to code the algorithm and test decompression in raw2dng, but I fear I cannot publish the code as open source under GPL...
     
  11. hanzodeceptor

    hanzodeceptor New Member

    Messages:
    2
    Likes Received:
    0
    Country:
    Indonesia
    This is so sad. Even Xiaomi Yi team can't help us :(
     
  12. Tanckom

    Tanckom Active Member

    Messages:
    178
    Likes Received:
    61
    Country:
    Germany
    How did you find this code? Was looking in the command and found nothing about t cal -raw
     
  13. AndreaPaco

    AndreaPaco New Member

    Messages:
    6
    Likes Received:
    4
    Country:
    Italy
    I tried to decompress 13M RAW files with DPCM 8-10, DPCM 8-12, RLE and A-Law... no go :(
    I'm struck because of some oddities I'm noticing. The IMX206 sensors send RAW10 or RAW12 (or DPCM 10-8) datas to the A7LS. So is the compression done by the image processing DSP and not by the sensor? And why in the TXT of pictures took with "t cal -raw" I can read "system.raw_resolution 14"? Why 14? And why a row lenght of 3904? (moreover, I suspect that the last 20 bytes of every line contains some information and not RGGB values). The compression is surely row by row.
     
    Last edited: Jun 5, 2016
  14. massud

    massud New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    Poland
    Hi guys. I'm new on the forum.
    I read this forum frequently, use your great ideas to force Yi camera works better.

    But now I have a problem.
    A few weeks ago script below worked properly.

    #RAWLAPSE v2 by nutsey
    #Howto: Put this file into root of SD
    #Steady fix your Yi
    #Turn on, camera will start timelapse shooting loop
    #capturing RAW+JPEG sets with 8s exposure and 8s interval at ISO 100
    #Remove/rename the script to disable it
    sleep 7
    #SET ISO TO 100
    t ia2 -ae exp 100 0 0
    #
    #SET JPEG QUALITY TO 80
    writeb 0xC0BC205B 0x50
    t ia2 -ae still_shutter 1
    sleep 1
    #RAW+JPG
    t app test debug_dump 14
    sleep 1
    #MAIN LOOP thanx VOTerra
    while true
    do
    t app key shutter
    sleep 8
    done



    But I had to reformat the card, and after installing a new script Yi generate compressed RAW files .
    Anyone has script t that works correctly?
    ISO 100, exposure time ~ 8 seconds.
    Have no idea where I made a mistake....

    some suggestion?
    maybe camera settings ?( default mode on the start is camera, not timelapse)
     
    Last edited: Aug 26, 2016
  15. willk

    willk New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    United States
    Hey Massud,

    I've had the same problem. From what I can tell, it makes compressed raws if it doesn't have time to make the full raws in between shots. When I have an 8 second interval, I get compressed, but when I have a 30 second interval I get full. I haven't played around with it enough to find the exact cut off, but try lengthening the time between shots and see if that works. Hope that helps.
     
  16. massud

    massud New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    Poland
    Willk, thanks for quick reply.
    I'm trying to change the interval for longer, but with no effect (still 13 MB RAWs).
    Which one should I change for 30 ? sleep 7, sleep 1, sleep 1 or sleep 8 ( I"ve change the 8's)
     
  17. willk

    willk New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    United States
  18. massud

    massud New Member

    Messages:
    3
    Likes Received:
    0
    Country:
    Poland
    Guys, a little success....

    with this script, Yi gives a 30MB RAW files.

    #RAWLAPSE v2 by nutsey
    #Howto: Put this file into root of SD
    #Steady fix your Yi
    #Turn on, camera will start timelapse shooting loop
    #capturing RAW+JPEG sets with 8s exposure and 8s interval at ISO 100
    #Remove/rename the script to disable it
    sleep 7
    #SET ISO TO 200
    t ia2 -ae exp 200 0 0
    #
    #SET JPEG QUALITY TO 80
    writeb 0xC0BC205B 0x50
    t ia2 -ae still_shutter 1
    sleep 1
    #RAW+JPG
    t app test debug_dump 14
    sleep 1
    #MAIN LOOP thanx VOTerra
    while true
    do
    t app key shutter
    sleep 9
    done



    I have also change camera default settings:

    default camera mode on start : picture
    picture mode : one shot
    resolution : 16M 4:3
    SD card : Sandisk ultra Class 10 16GB
    Kingston class U1, 16GB


    For now, everything works properly.

    Willk, I have already read the link you gave.
    Will try soon ....
    thanks,
     
  19. kimi

    kimi New Member

    Messages:
    1
    Likes Received:
    0
    Country:
    Malaysia
    Hello everyone..
    I have question, why raw2dng only support convert raw still photo with aspect ratio 4:3, and when i take raw still photo with aspect ratio 16:9, it say "unsupported raw format"?
    size of 4:3 = 31 Mb, size of 16:9 =23 Mb.
    I have several raw photo with aspect ratio 16:9. Can anyone help me how to convert it?
    Thank u in advance.

    Ps : firmware 1.2.13 Z23L
     
  20. Chuckrom

    Chuckrom New Member

    Messages:
    1
    Likes Received:
    0
    Country:
    Mexico
    Hello to everyone. So far I am getting the 30mb raw files, but when I convert them to DNG using the raw2dng they end up being of 8mb after y drop them on the Yi-Correct-Raw and import them to Lightroom they are definitely underexposed by at least 2 stops and have this yellowish tint which even dialing the wb settings cannot totally disappear. All of this makes tha raw image unusable.
    Anyone else knows a workaround or are having the same issue?

    Thanks
     

    Attached Files:

Loading...

Share This Page