User Profile

[dzenc]

One thing I see in your second set of logs is that the Loopback adapter has seen 62 packets transferred, which in my experience is a tell-tail sign that the system lost wifi at some point (though seemed to recover).

(I had tcpdump running on the Loopback interface at one point while my...

[dzenc]

This is suspicious to me because I see something somewhat similar on my 7.7.3 radio after it goes red. When the I use the UI to reconnect to wifi (rather than e.g. power cycling), the next disconnect tends to happen quicker than the previous one. Its like something is getting mucked up in its internal...

[dzenc]

Since I last provided an update, I am still waiting for one of my new-driver radios to disconnect so I can capture more data. Unfortunately (or fortunately?) it can take multiple days. Since the new driver has source code, that’s where I’m able to do the additional logging, so that’s where...

[dzenc]

Thanks. I had a typo in my instructions (should have said ifconfig not ipconfig, now fixed) but that data isn’t needed for now anyway. The only thing I can observe in your log is that, towards the end of the log, it notes that it reconnected to wifi. Although it doesn’t explicitly log a disconnect,...

[dzenc]

Thanks. Sounds like a good plan. I had been hopeful that we were close to getting this checked in, but there is clearly still more work to be done...



I think either way is ok. If someone wants to jump in and take a look at getting them built, that would be great. Otherwise...

[dzenc]

Been thinking more about this. The problem with leaving the message enabled in DEBUG mode is that it STILL floods the logs when DEBUG mode is enabled, whereas the rest of DEBUG output is pretty measured and has been very useful. So it makes DEBUG mode less helpful. I think the right approach may...

[dzenc]

Agreed. I'm still hopeful we can do better, but it's possible that this is the best answer we end up with. In the back of my mind I have been wondering if the difference between this wifi firmware - which I've yet to see "assert" - and the newer wifi firmware, which I've seen intermittently...

[dzenc]

😃 Thank you for testing. I look forward to hearing what happens in the new location.

[dzenc]

I just reproduced this (again) on version 7.7.3. This time I had tcpdump capturing both on the Radio and the router. Comparing the two captures, I can confirm that, at the point of failure, tcpdump on the Radio stops seeing incoming packets, indicating that something in the Radio's network stack...

[dzenc]

Yeah, as frustrating as it is, I’m pretty sure I’m still just finding out the same things that have already been learned by others in the past while analyzing this problem. (…many Bothans died to bring us this information...)

Though I’m happy to take suggestions/hints if anyone...

[dzenc]

The fact that the Loopback adapter starts to see unusual traffic when this occurs makes me suspect it’s some kind of routing issue, either a mis-configuration or bug.

I‘m trying some things out…

[dzenc]

For sure. The catch is that I’m seeing both the dhcp request and successful response when monitoring with tcpdump on the router. So it sure seems like it’s getting a successful response, it’s just that for whatever reason it’s not acting like it is.

(And this repeats multiple...

[dzenc]

Interestingly, I went and looked at my one Radio still running 7.7.3-r16676, the Radio that still pretty reliably turns RED every few hours. And, IT TOO, is in a similar DHCP failure state and it too, is continuing to send and receive wifi traffic that has a bad IP address. So this bug seems to have...

[dzenc]

Please also make note of exactly which track it is stuck on.

[dzenc]

Can you ssh into it? Please copy off /var/log/messages and /var/log/messages.0

please type ‘dmesg’ on the radio via ssh and capture and save the output

please type ‘iwconfig’ on the radio and capture and save the output

please type “ifconfig” on the...

[dzenc]

I caught it in the act! And this time both me and the radio were well prepared to debug this type of issue (said issue which feels different than e.g. the kernel crash, assert, or wifi drops I had observed on previous builds).

The good news is that THIS bug, that I am still observing, is...

[dzenc]

What's interesting is that with the additional driver debugging turned on, I can see that the wifi driver was still receiving (at least some, if not all) packets even when the app was displaying RED. But DHCP was simultaneously repeatedly reporting failure...

More threads to pull on here......

[dzenc]

And... I just had one go RED. With no obvious cause.

Sigh... So frustrating.

[dzenc]

Thanks

[dzenc]

Can you try playing again for more than 24 hours and see if it stops at the same time again?