Odd wifi issue when downloading videos via wget

[dreniarb]

We have been using the Blackvue 650s dashcams on our fleet of vehicles for about 2 years now. Every night when the vehicles are back in the garage the dashcams connect to our internal wifi. A script on a server then downloads the videos via wget. Each dashcam downloads at the same time. This has been working without issue.

Recently we added some 750s dashcams. We have found that during the download process these dashcams will start to show high ms (1000-4000) then eventually drop off the network. If I stop the download script the ping times return to normal. But shortly after starting the download process again the dashcams will drop off.

So I tried downloading just one dashcam at a time. This does work better but the dashcam being downloaded will still eventually drop offline and the other 750s will as well. The 650s stay connected and don't show any problem.

Next I installed 3 additional APs with unique SSIDs. I connected each 750s to it's own AP. However the same problem occurs.

I then tried to throttle the wifi connection for these dashcams via the Unifi controller. But even throttling down to 1mbps the dashcams would still drop offline during the download process.

So I brought all 3 dashcams to my office in a completely different area of our facility. They connected to the AP in my office. Same problem occurred.

Here's the main point to my posting here - I then separated all 3 dashcams by placing them in completely different areas of our facility, out of range of each other. The download script ran properly and the dashcams never dropped offline. So it seems to me that when these dashcams are within range of each other they are somehow interfering with each other.

Has anyone ever experienced something like this? Perhaps with other wifi devices? When just pinging they all work fine, even when sitting literally inches apart. But shortly after we start downloading files we find that either 1 or all 3 will start dropping offline. Unless we have them out of range of each other.

 

[jokiin]

might find some clues in this thread https://dashcamtalk.com/forum/threa...ndows-taskbar-app-latest-version-4-0-2.37448/

 

[GlennNZ]

Hi

Interesting, almost seem something similar when two 900S devices seem to interfere with each other. Did move to a new dedicated SSID/AP and less of an issue - but still seems better when one only needs to download. May be lower wifi channel width helped here as well (see below)

Few posts about formatting SD card (in computer & copying firmware onto) to fix ‘weak wifi signal’ wifi issues - but not sure how old this info is.

Q:
1. Have you tried 2.4 gig wifi as well?
Or 750 are only 2.4 gig ?
Hmmm - may still be 2.4 gig channel width and would drop and see...

OR
2. If 5gig issues - Have you tried a different channel width in UniFi SSD - eg. VHT20 rather than VHT80?

(wider channels marginally faster but interfere more with each other - if dedicated ssid for these cameras would be easy to change)
In UniFi controller, AP, settings, Radio, 5gig or 2.4gig, channel width

One of UniFi APs has the frequency checker etc - will run it without dash cams around and with dash cams downloading and see..

More I read - does seem that perhaps channel width is issue. 5gig 900S is limited to channel 157 SSID broadcasting (different to channel connecting). Channel 157 only available with VHT20 channel width/clearly this must be default.

So if 2 cameras connected to 40mhz AP, but each sending 20mhz wide broadcasts - could conceivably interfere? Admittedly I have not idea whether than really makes sense it a wifi network sense - but should be easy to test!

Or even the reverse - the wider the channel the better...

Will test when I can as well

Glenn


Sent from my iPad using Tapatalk

 

[dreniarb]

interesting thoughts. our trucks don't run on the weekend so i'm hoping to get to the office today or tomorrow to try some of your suggestions.

thanks

 

[GlennNZ]

Ok.
Be interested in results, as there doesn’t seem any info on channel width and blackvues anywhere.

When I look at insights in UniFi controller - it shows the neighbouring wifi points and details, listing the blackvues. For my 900’s which are on 5gig shows bandwidth as 80mhz (which is wide, VHT80)...

It is possible that anything less than less interferes with each other....

Have set my dedicated 5g wifi SSID AP to VHT80 and cameras connect at 433MBps (!) Max throughput still seems to be 6-7 MB/sec only with one camera active, probably limited by SD card reading/writing speed I suspect. Will test more with 2 at once when I have 2 around!

Edit: Tested:

Does seem for 5gigz at least a super wide band VHT80 is ideal.
2 cameras downloading next to each other, now 9-10mb/sec achieved each. (previous 6-7mb/sec)
No slow down thus far noticed.
Wifi network scan - didn’t really yield any useful information.

Not sure whether this will apply to 2.4gigz or not...


Glenn


Sent from my iPad using Tapatalk

 

[jdct]

This suggestion is offered without the benefit of being able to test the blackvue, but based upon my experience with wifi networking I would suggest that you reduce the wireless functionality of each of the three access points with only one connected dashcam to 802.11g mode only, with 20Mhz maximum channel width, roaming disabled, in order to rule out any potential issues that the blackvue 750s might have with the shorter 400ns default guard interval of 802.11n, automatic channel width selection, or roaming protocol. If that that works, then you can reenable 802.11n, but you must be able to set long guard interval, since the default for 802.11g is 800ns. It's worth checking. Google 'guard interval' for a brief description.

Failing that, your next step might be a wireless packet capture/protocol analyzer, or modifying your scripts to download from only one camera at a time to avoid the problem if that even works. (Obviously not the most ideal solution.)

Oh, and be certain to power cycle both sides after reconfiguring in order to ensure that the new settings take full effect.

If UniFi prevents you from accessing those settings, then find an older consumer grade wireless access point/router at the bottom of your junk pile to perform this same test with. To use a nat router as an access point only, leave the wan port empty and connect the ethernet cable only to the lan side, disable dhcp, give it a valid static ip address, leave gateway blank.

 

[jdct]

You may also wish to try working around what you've already got in place.

Disable all throttling via UniFi and try some combination of new options in wget:

‘--limit-rate=amount’
Limit the download speed to amount bytes per second

‘--wait=seconds’
Wait the specified number of seconds between the retrievals

‘--retry-connrefused’
Consider “connection refused” a transient error and try again.

‘--no-http-keep-alive’
Turn off the “keep-alive” feature for HTTP downloads


Start at a very low limit-rate and work up from there if successful.
Introduce a short delay between downloading each file.
Perhaps retry file download automatically if the 750s drops off the network for a moment.
Try disabling http keep-alives to determine if it has any effect.

In this fashion you can rule out the possibility that your script is overwhelming the http server process and/or os built into the 750s -- even though it is working properly with the 650s.
And perhaps the problem you are experiencing was purposefully included in the dashcam, in order to steer you toward the blackvue cloud / fleet management subscriptions? (It's possible, though admittedly unlikely.)


Though I think the 802.11g-only mode on the access point (802.11n disabled) test is the best next step because of the results from your isolation testing. It will force the 750s to negotiate to that wifi standard during connection phase.

 

[dreniarb]

Thank you for the suggestions. Certainly worth a try.

I had tried rate limiting - both on wget itself and the APs, but the cams would still drop out. Sometimes they would drop out even if only one dashcam was being downloaded. Seems that just having the other 750s within range was enough to cause problems. For now i've swapped around the 750s with 650s from other vehicles whose videos we don't necessarily care about. In that I mean we don't need to have access to the videos the next day.

 

[jdct]

The reason I had suggested rate limiting via wget (like really slow rates for testing purposes) instead of rate limiting via the AP was because there is more in play than just wifi networking protocols that have not yet been eliminated as potential contributing factors. It is also possible that there is some issue with the 750s' implementation of the TCP protocol. Consider the case of dropped/corrupted TCP packet(s), for which the camera should eventually receive a NAK from the downloading computer. Is the dashcam properly retransmitting that portion of the file from its TCP send window or has it made sacrifices in order to operate with fewer resources including RAM? Perhaps the dashcam is being confused by a request to retransmit specific packets, causing the TCP or HTTP server process to hang up the rest of the dashcam OS/SoC? That you are experiencing increasing latency in ping requests, prior to non-response suggests a transport protocol issue occurring first but is not any real proof without network protocol analysis.

Try forcing 802.11g first. Then if problem still persists, you can try to find ways to workaround the issue present in the dashcam, because it may be next to impossible to get blackvue to even find or correct it. Or perhaps they have already identified an issue and it led to them making this recent change in their own hosted service. Please take notice as to how the service change does not affect the 650s, and that the language use of "for now" suggests that they discovered a problem which they may hope to fix?
Read: https://www.blackvue.com/blackvue-over-the-cloud-event-file-auto-upload-for-all-cloud-plans/