I made an interesting discovery with my group of T3-LB(ARM)'s. When a controller shows up occasionally with a Red X on the T3000 my usual move is simply to power cycle it. That usually clears it up. But I almost always noticed that the heartbeat LED was flashing in its usual 1 a second way. So I tried simply resetting the Ethernet connection by unplugging the cable, or disabling and enabling the port on the Ethernet switch. Either way, the red X would go away and the T3 would start communicating again. This is a little cleaner way to clear up the problem. I wish there was a software option to recycle the port when the T3 sensed that comms were down for more than a given time period.
The latest T3000 front end clears this up, there were a lot of false negatives with the online/offline state of the devices. There’s still a few false positives, the team is working on that.
I took the following steps to kill the red X.
- Installed new firmware 50.4
- Made sure there were no duplicate variable, input or output names across all 4 T3’s
- Attached each T3 to ground.
For the last week, the dreaded Red X has disappeared! 
I still have some T3’s that occasionally reboot themselves, but I will post that under a different topic
Good news then, thanks for the update.
Call me crazy but the dreaded “Red X” has reappeared. This time it’s a slightly less intense red X. I call it a pink X. It is as if a real Red X is in bold and a “pink” X is just normal intensity. A real red X is a unit that is physically disconnected or certainly un-ping-able. The T3 that has a pink X is still ping-able but it seems to strain when asked to display the Variable list. It eventually shows up but just seems to take longer.
OK, here’s my question for you and Chelsea. Is this all just a part of my imagination or is there something under the covers that would make one red x look different than another??
Hi tdawgtoo , Can you tell me which firmware are you using? Our newest version is 52.2
https://temcocontrols.com/ftp/firmware/T3_BB_LB_TB/mini_arm_rev52.2.hex
Can you give me some screenshot about your device infomation and your PC’s ethernet adatper settings.
Hi Fandu,
Thanks for your response. Yes, I am using firmware 52.2. I have attached a screenshot of the low-intensity Red X, or what I have named the Pink X. It also shows the version of T3000 software I am using. See the blue pointer for the slightly lower intensity Red X. I am just inquiring about the meaning of the different intensity because I can ping this node but within the software the response is a bit slower. The other nodes that have the red X’s (with normal intensity) are not in direct use by any programs and have always been there. I guess they’re discovered automatically. I access those via RS-485 connections.
The other screenshot shows my PC’s IP info that you have requested.
This whole inquiry is just about documenting the meaning of the two types of red X’s. The T3 is functioning normally.
Ted
Yes, I am running firmware 52.2. I have been away for 10 days and the communication seems worse when I returned home. I had some nodes that were allowing me to see them on the T3000 software but a little slow. Now they will not display and I get this error: 
But, if I right-click an active node, it will ping the T3 with no problem. What is causing this? Also, my panel #'s start at 2 instead of 1. Does that cause any kind of comm problem?
I had a test board that ran for 78 days, and I found a similar problem on it, often timeout when ping.The test board did not return to normal until the soft reset.In addition, there are several test boards, which have been running for more than ten days, and have been running for more than a month, all of which seem to be normal.The board with the problem has been reading Network Points, maybe this is one of the reasons, I will update to the latest code to continue to observe.
One of my boards has been running for over a year before it developed this problem. The other T3 is only six months and has the same problem. They both ping perfectly but show no TCP server. Internally they continue to run programs as normal.
However, I have 2 other T3’s that have been running hard for a year any they work very well. Those two units no comm problem.
not sure whether you update all boards to latest firmware. In fact I fixed a bug about the newtork issue which caused some trouble when there are lots of packets after rev51.5. We still worried complitacted network traffic cause some trouble, we was always testing it. But we did not find what you met now, we found ping bad, coummincation slowly. We will continue to focus on it.
It’s likely that your computer’s network adapter has changed, such as its IP address.
Another possibility is that the DEVICE’s IP address has changed, such as if your device is using dynamic IP.
It is recommended that you re-click the Scan of T3000.If the T3000 can be scanned back to the device, the communication between the T3000 and the device will be restored to normal.
Fan,
Thanks for your response. It is quite logical based on our circumstances. But the T3 was totally off-line. A re-scan of devices did not pick it up.The problem was confined only to Ethernet communications. All the programs in the T3 continued to run properly.
I did power-cycle it and it came back very nicely. Even the Network Points came back on line. The fixed IP address never changed.
Unfortunately, that only lasted about 24 hours and the red X has now returned.
I had a drastic idea to run an RS485 cable up to the unit from our basement chiller room. And then extend our existing RS485 MSTP network to the T3 on the roof. It has been my observation the RS485 communications seem slower, but much more reliable.
What do you think of that idea??
T
It is difficult to repeat the phenomenon when the network does not communicate. Fixed a bug in REV51.5 that could cause poor communication.
If you encounter this problem again, can you send a Teamview by email? I can help you to analyze the specific problem remotely.
Another thing that might help is if your T3 RS485 Main Port and RS485 Sub port are not in use, it is recommended that you set it to Unused.Because any subprotocol setup will consume a little CPU.
RS485 is definitely more reliable over longer distances. Sorry to labor the question but how far is this between the T3 controller and the nearest hub again please.
Its about 300 feet, 80 of which is straight up. To keep from interrupting my existing RS485 network wire, I thought I would simply use the secondary RS485 port and connect that from my primary T3 up to the roof and to the T3 that has had all the comm trouble. That primary T3 is physically right in the middle of the RS485 wire between the chillers on one of the VFD’s.
As long as all the panel ID’s and Bacnet ID’s continue to be unique it ought to work right?
The primary T3 continues to work well, especially since Chelsea fixed the time problem.
Ted
Well after a bit of internet sleuthing I see that 100 meters is a hard limit for the length of Ethernet cables, any longer than that will cause problems. The quality of the connectors and termination can reduce that further. One way to test it will be to ping the device from your PC, here’s a test from my PC to a Tstat10 over wifi which looks solid.
A more in depth test will be to use a tool called iperf, you can temporarily set up a PC on each end of the cable and run iperf to test the network bandwidth. You can download iperf for windows here:
Here are the commands to run Iperf on port 5201, run this on the ‘server’ side, it doesn’t matter which side is client and which is the server. Initially I was following an example using port 9000 and not getting a connection so try other port numbers if you dont get a connection right away.
iperf3 -s -p 5201
And run this command on the client side:
iperf3 -c 192.168.0.7 -p 5201
Here’s the view from my PC acting as the client and connecting to the Iperf server running on the local lan at 192.168.0.7, the connection seems solid enough:
In summary: 300 feet is about at the limit of an Ethernet cable connection, iffy cables and terminations would reduce that further. Test the connection throughput using iperf. If the throughput is poor you can always connect over RS485 though the speed would be reduced.
I was able to eliminate the Red X by removing a program that reached out to 3 other T3’s for over 20 different bacnet variables. I needed those variables to create a nested graphic of each area of the building that the T3’s either controlled or monitored. Instead, I put just one graphic screen on each T3 that shows the things that are controlled there.
Once I stopped trying to get so many bacnet variables, the Red X went away and the T3 can be accessed.
2 Likes
While checking on this for you I see that we have some work to do on displaying remote points in a display. There’s no need to poll from other controllers. Once this is working again I’ll post an update.