Xiaomi Yi - Genlock Multiple Cameras

[Armdromeda]

1600x1200 30fps 40mbps and you'll need to update the wifi config too.

 

[Oliver Puschatzki]

Hi Armdromeda,
many thanks for the files! I think, I understood the concept and I also got the scripts to work. Therefore, the script connects to the network, snychronizes the internal clock of the Yi with the ntp server and as soon as this is done, starts to record at the very beginning of the next second. It does not -in any way- communicate with other cameras which might be part of a rig and it does not wait for a record trigger, coming from the "xiaomi 360" app, for example.

The way I use the 360° rig, this automated recording start is a problem. I would rather like to start up all cameras, check whether all of them have enough battery capacity for the recording and start recording via pressing one single button. Because I am not able to change anything in your scripts: Is this possible to achieve?

As far as I understand the synchronisation problem with the Yi rigs, though genlocked recordings will not be possible with them, it will surely help to make them start at exactly the same moment in order to reduce the gap between (the beginning of) the recordings to a minimum. In this case, it might be enough to sync the clocks inside the Yi's to the clock of the phone which runs the "xiaomi 360" app (instead of the ntp server) and make the record at the next "rounded" second or even one or two seconds later in case the wifi start signal for the recording takes more than a few milliseconds to reach all cameras and "the next rounded second" is too early for some of the cameras. Correct?

I'd be happy if some expert could check whether there is a chance to enhance the recordings of the Yi rigs using this synchronisation method in conjunction with the "xiaomi 360" app.

Bye,
Oliver

 

[vadimkozhin]

I would rather like to start up all cameras, check whether all of them have enough battery capacity for the recording and start recording via pressing one single button.

Oliver, but this is what exactly Brahma app can do. It's not millisecond correct as «ntp sync» method, but assume, someone will write the software, which utilize this trick and pull the shutter, and I think it's possible – you need to put a ntp loop in config and execute on multiple cameras to sync the clock and «preparing» to start a record. Cameras need sometime to get this info, so, lets say, a second. So, basically, you schedule every recording. And after recording starts, you can stop it from app too, coz you can understand that camera is starting to recording after receiving signal from it. Ok, it's works. but what problem does it solves? It's still will be out of sync but for milliseconds than the second. And anyway you need to tweak a bit those video files for «perfect» sync in post. So, whats the difference, tweak a second between them or millisecond.

In my practice with Yi (first model for now only) and GoPro 2, 3 and 4 also, there is no way to get a perfect sync even on post, because they have no software synced shutter. When you put everything on a timeline, you will see, that there is a tiny gaps between the audio peaks. And it's really become critical when youe film have fast motion (and I'm not talking about rolling shutter, coz It's second reason why Yi's and Gopros are not fit in serious 360 production), your object just will be at the different position. Technically, it will be on different position «on screen» anyway, because of parallax, but it's the stitching software job to fix this issue.

So, long story short, in my opinion, implementation of this technique it's not anyhow enhance shooting experience. Anyway, from researching point of view, it will be cool

 

[Armdromeda]

Thanks guys for your feedback. I understand this isn't a perfect or finished solution for genlocking. I think the valuable thing I have provided is a demo of how to multi-sync camera time, how to read that time within factions of a millisecond and execute the recording. My hope is that a remote solution will be possible in future. If a remote app can deliver a file containing a start record time, and a version of this script can read, sleep and start recording with the millisecond precision, we have our solution which will finally bring us this long desired piece of functionality.

 

[vadimkozhin]

Armdromeda, it's really cool findings, and from what I saw on this forum, I think is the most closest to «perfect sync», which I think is not possible without hardware solution.

I developing a control app, which control all 6 cams, monitor all statuses, uses GPS, control exposure in real-time and many other stuff, which is not related to topic, but is also uses time sync. All cameras have a static IP's and connects to small portable router which powers from little powerbank. So, my solution is totally portable and 'internet free'.

I use JSON time sync. Every time when I connect to the cameras for the first time I sent current time from control device, which in my case is Windows tablet, to all cams using those JSON packets. This packet as far as I know allows to send 1 sec accurate time to cameras. This was done in order to get correct capture time metadata on photos and videos, but not as shutter trigger. Even this time sync is useful, because second software later copy all files to disk using 6 small card readers, and before starting to copy it looks for all file lists from all cameras and finds closest capture time (plus-minus 4 sec) on photos and videos on all SD_card, which give you 'a take' – group of photos which was taken in one time, so you don't need to mess with copy and sorting hundreds of takes and find 'groups' of photos later on post.

I saw your post and I think it's a great technique, and later will try to test ability to sync time using your method. But it uses internet to get correct time from server, which is 'ruin' portability in my case. What I think is may be possible to install some local time server on the phone or on the router. I use openwrt on the router, and I think it will be possible to install some solution to get something similar to 'ntp' service (or may be it's built in in Linux?), and your finding will help to improve those 'take findings' mechanism. And since it will be there, may be it also be possible to develop shutter releasing functionality based on this 'precise' time sync.

p.s.
Also, not related to particular conversation, but related to topic – I read a lot about 360 last years. I have an interest in 360 video particulary since early 2000's. Was developing a rig using gopro 2 model for one of the first 360 music videos in the world, shooting and make post, when there was no video stitching software at all, stitchting it frame-by-frame, was betatestin this KOLOR videostitching software and their 360 web-videoplayer… so, a lot of reading and researching past years

And what I saw is a lot of 360 reviewers and enthusiasts a bit misinterpreted 'genlock' term. Most of the time genlock means just simultaneous control all settings for all cams, which GoPro remote shutter does, for example, but now shutter sync. As far as I know, REAL genlock can do only a couple of 360 cams for now (NOKIA, for example, and couple of cams which is in BETA now or on kickstarter, and couple of cams which is completely 'black horses', coz no demos and tech specs on them at all right now) and it's not possible in Yi first model at least today. Yes, I saw posts about 'big Yi rig' which is also 'genlocked', but I think it just means that you can control all settings, but not shutter sync. Also saw a solution, some blog where guy uses Gopro stereo sync device and mini arduinos to get perfect shutter sync, but couldn't find any demos, unfortunately, and any review who was really tested this solution.

What I really enjoy, is the enthusiasm of the community and 'researcher itch', when people can try make all possible to get 101% from device. It's deserves respect.

 

[JRSim_UIX]

I don't think the cams will communicate to external domain names - well mine can't talk to the external ntp servers - so I setup ntp on my phone the cams are connected too and synced their time with that.
Maybe it would work with IPs just not domain names?

 

[archenroot]

Actually I am in similar situation as the original user. , as I plan to use the setup for outdoor, instead of RGB-D rig I failover to stereo vision and ordered for experiments 2x 4k plus (60fps). I have no problem with building very precise mounting later possibly extended with servos, but original setup will all hard mounted.

I am Gentoo user, there is quite usefull (not Gentoo only possibly) repository:
https://github.com/irungentoo/Xiaomi_Yi_4k_Camera

And finally the repository, where for me the most important is the hardware sync cable:
https://github.com/YITechnology/YIOpenAPI/blob/master/spec/Multi-endpoint Micro USB Cable.pdf

I tried to order the cable via *.com site, but shipping didn't work as soon as I come from Europe. But I can built one.

The biggest trouble is really the calibration. Does anybody has real world experience with this or even theoretical one? As soon as OpenCV 3.2 is out, there were some changes to calibration process -> disruption map happening as well. Is it now on same level as singe shot algorithm to be applied via Matlab?

Also does anybody have experience with vertical sync issues with horizontal camera setup?

Thanks a lot for so far valuable stuff found here

 

[petrini]

Yi 4k may be hard to sync, but there is a solution to sync Yi Discovery action camera. It is not very elegant, but it seems it works. The synchronisation is hardware based, these guys found out how to press the shutter on more cameras at once. More details are in this article here: http://3djournal.com/3d-photo-and-3d-video-cheap-with-yi-discovery-cameras-1-133 . Maybe something similar could work for Xiaomi Yi 4k camera as well. Let's hope this can help someone.

 

[quen]

very interesting. How news???