+ Reply to Thread
Page 1 of 3 123 LastLast
Results 1 to 10 of 30
  1. #1
    Join Date
    Oct 2002
    Location
    Upstate New York think mts and trees
    Posts
    1,915
    Rep Power
    17

    Lightbulb Older hardware and firmware feedback on versions tried and results.

    Anyone want to offer safe approaches to go from one FW to a newer one?
    ----------------
    Lets cover WRAPS to WP-188's
    -------------
    Lets start with I got away with this (Or it worked for me)
    Hardware near end (version)
    Hardware far end (version)
    Results and or problems.
    =============

    Last edited by David L. Vrablic; 03-01-2011 at 05:02 PM.
    Dave
    <
    @)
    (Cooter>>
    ^^^^^^^^^^
    Vrablic

    "I have no excuses, Just reasons"


  2. #2
    Join Date
    Mar 2006
    Location
    Srbobran, Serbia
    Posts
    4,084
    Rep Power
    16

    Default

    For now, I will move my comments and replies from release thread:
    Quote Originally Posted by DrLove73 View Post
    @Dave
    When I upgrade firmware, I upload and apply new firmware in parallel, and then reboot them all at once. That way, all units are rebooted in same time period so there is no multiple data losses.
    I know you have 10 times more units, but maybe you can brake them in several sections/legs and reboot them by sections. Just to remind you, you can upload and apply new firmware now and then wait as long as you want to reboot them. You can even wait for then to lost power and/or reboot themselves.

    If you use Batch scripts on windows (or application that can perform reboot and firmware upload) you can, using starutil, automatically send reboot commands to designated section from furthest one back to closest to you in a matter of seconds. There would practically be one data loss about 40-60 seconds long (should not be longer). They upload and apply new firmware to those units, and when you are satisfied everything is in place you would once again issue reboot command to all units on that section/leg. So you would have another 40-60 seconds data loss, but that should be it, excluding possible failures to boot that are always there, what ever you do.
    Quote Originally Posted by David L. Vrablic View Post
    Whats up Doc? I have always wanted to do that!
    I should have answered this in discussion section but Oh well, I'll play it where it landed.
    I have my favorite sections or groups that I use for testing new firmware.
    --------------
    One such cluster has 7 units that are fed from a AP one hop from the BH.
    It makes a very nice controlled environment for testing.
    I use the Star-Gazer program or sometimes command line to load the upgrade starting out on the end unit and working my way inward.
    With command line all I have to do is up arrow and back arrow to the IP of the next unit enter that info and hit enter.
    Next I Pop a putty session and upgrade.
    It works very neat and I can see the reassociation and results on the previous session as soon as it reboots.
    I am lucky enough to have a 32 inch screen and a 24 inch screen so I have room to keep many things open without switching screens.
    -----------------
    I have had disasters that took truck rolls to get everything back up when I used batch scripts, so I rather do them one by one and check for improvements as I go.
    My first test after a UG is to do a traceroute and -l 1472 ping test back to the server as most traffic flows in that direction.
    I compare with what I had before and if it looks better than I go on to the next unit in the string.
    ---------
    Outages during a maint window are not all that traumatic as I usually do them at "0 dark thirty" in the early AM but I do try to keep the systems up as much as possible.
    I believe that aproach where are using can produce problems where they do not have to exist. If older version has some incompatibility with newer version, then your upgrade can result in no connection, where approach I suggested would work. Since every version has it's quirks and bugs, or designed behavior different then previous

    One (1.4.x? ) versions quirk was to fail to connect to the other side if SuperAG checkbox was enabled on radio without SuperA/G capability.
    Some versions do not use same frequencies for channel numbers and others do not have some frequencies previosly used, like channel 0 on 2.4Ghz on some of the versions.
    Some versions use DFS and some don't, 1.5.x versions (can remember from which revision) have D1, D2,... country codes for w/wo DFS usage, older versions do not.

    That is why I suggest "full assault" approach. Rehears and train in the camp (laboratory/office) to see if units will link on used settings, and/or read on posted problems and solutions to avoid mistakes. But then upgrade all units at once and look for possible issues. So far, only issue I had with upgrading were when unit would forget license (renewed old V2 licences on early 1.3.x version I think), and SuperAG problem on cheap non-SAG radios.
    Ljubomir Ljubojevic - Love is in the Air
    Google is the Mother, Google is the Father, and traceroute is your trusty Spiderman...
    StarOS and CentOS/RHEL/Linux consultant
    Powerful Starv3 manipulation tool - StarV3 Multipractik for Linux

  3. #3
    Join Date
    Jan 2008
    Location
    Texas
    Posts
    545
    Rep Power
    10

    Default

    Dave, we have been flashing and reflashing, we only have 1 2 port war, 2 4 port wars, metros, x4000, mips clients and siam clients,

    we only use on our access points g only, 2x cloaking, super ag, short preamble, hide ssid
    we only use on our backhauls a mode, turbo and 1x for the over 16 mile links, super ag, short preamble, hide ssid

    we have swapped flawlessly between 1.3.23b, 1.4.22r, and 1.5.15.3b on the access points (g only) and clients
    we have swapped flawlessly between 1.4.22r and 1.5.15.3b on the backhauls (a mode)

    we turn of all thresholds when flashing also, and i am sorry but the one wrap we had was replaced long ago because we only had one of em so there is no experience here for you on those platforms

    Hope this helps,
    Kevin Gerrard

    "Anyman who thinks he can be happy and prosperous by letting the
    government take care of him--- better take a closer look at the
    American Indian."

  4. #4
    Join Date
    May 2004
    Location
    Wyoming
    Posts
    885
    Rep Power
    14

    Default

    Our system is SOS from end to end. There are still some alien CPE on some b only AP's. We don't dhcp anywhere, all static routes and we don't bridge CPE. The core server is a x4 PC and does BGP, NAT, CBQ, FW and VDS. It is still on 1.4.22. Most of our clients are on private IP's and are natted at their radio and at the core server. Hardware consists of war1, 2 & 4, Wraps, WP188's, couple PC's, war1a's, no metros and mostly SOS cpe. We use WLM and CM9 radios just about everywhere. No 900 stuff and nothing high powered radio wise other than an occasional 26db wlm. Our longest p2p link is 23 miles and the average link is 6 miles in length. We had 1.3.23 on our AP's and associated CPE and 1.4.22 on the backbone links previously. I upgraded just about every radio to 1.5.15 (after removing all the CPE bridges) and have no issues whether AP is cloaked or not. Doesn't seem to matter whether clients are war1 or war1a, both are working well. We have a FD link to our upstream and are pulling 25meg+ during the busiest time. Most of the backbone hw are wp188's but we have some war2 and 4's at 4 solar powered sites. The solar sites are in very remote and unfriendly terrain. They are usually the last to get upgraded. They are no fun to get to during warm, dry weather let alone in the dead of winter. I haven't bricked anything nor have I had a board go back to factory default. I haven't upgraded much past 1.5.15 yet as I don't see much enhancement in more recent releases. We don't run dns, rip, ospf, mesh, hotspot, dhcp anything, pppoe, wpa, wep Turbo or radius.

    Our typical CPE has connection tracking enabled for natting purposes. Power is set to -2 below rated db level as a max. Rate is auto, distance is 15, cloaking is either x1 (B) x2 or x4, G (or A) only, Super enabled with a hi Sig threshold of -45 and we do hide ssid. NTP is enabled on every board and we have dns entries to resolve the NTP server entry. I do customize the thresholds on each CPE in very noisy areas. Every customer has a router which we program in a static IP of 192.168.22.2 snm of 255.255.255.252 and gw of 192.168.22.1 We very recently started to dabble with dhcp from client CPE to their router in case of factory defaults to the router.

    BB configs are pretty much the same except that we usually set a rate rather than auto. Connection tracking is turned off.

    I upgraded many radios from 1.3.23 directly to 1.5.x. Same with 1.4.22. No issues that I can recall. I don't have any horror stories and the system is as stable as I've ever seen. I have concerns with the Q always being 100% and would love to have that tool back in the box to help troubleshoot. I attribute the smooth, stable performance to removing the bridges out of CPE. After we got that accomplished, we can put whatever version we want on the hardware and let 'er fly!
    If you don't get caught, did you really do it?

  5. #5
    Join Date
    Dec 2002
    Location
    Frozen Tundra of Canada
    Posts
    2,822
    Rep Power
    17

    Default

    This is similar to our network and our experience.

    We have 61 broadcast locations, just about 2000 CPE's. We do have a couple hundred Canopy 900 in a few select areas, and a small handful of old TR's (left over from just after the Great Flood). We have a private 10.x.x.x network, and we NAT in the CPE, and have Connection Tracking on on the CPEs, with ZERO BRIDGES anywhere. We had pseudobridged many, many years ago, and when that started to fall apart, we either needed to learn to route or die. We learned to route, and have all static routes, so we've never had any issues with routing protocals gone rougue.

    We DHCP numbers out from our AP's, using ISC to DHCP out statics, and we DHCP a 192.168.100.x number out from the CPE to the customer's router/computer. VERY IMPORTANT, we CBQ on the CPE's Ethernet, keeping the excess outgoing traffic off the air and away from the Access Point. THAT IS CRITICAL. It's also why I've been asking for a CBQ Update (for more flexibility in our marketing) for so many years.

    We have very, very few fatal horror stories from firmware upgrades. We usually apply new firmware to my home CPE & it's AP first overnight, and/or test it in the shop, bit we've really had very few problems. Sure, some upgrades haven't preformed as well as we'd like, and there has been mild panics from time to time. But only a handful of cases when we've had critical, major panics.

    We LOVED the rate agressivness setting that 1.4.22r had. We'd had experience with that type of setting in some router firmware (agressive/normal/conservative) and found that it was a very good idea - let it really do it's 'auto' deal, but then step up or down a bit - we thought that worked really well for tricky CPE's.

    We find that the major issues come with mix-matched firmwares, and since the SIAM's came out, we had to either stop deploying or we had to deploy 1.5 CPE's. That meant that we had to upgrade the AP's to 1.5, and that whole process has been trying during this last year, but still not a lot of fatal, killer, gotchas during upgrades, and now it's getting pretty good again. We do still have a curious issue on 15.15.3b backhauls, that show packet loss on our MRTG Graphs, and just don't seem to work as well as 1.4.22r on those same hardware did, but we still don't know why/what that is. However, we are THRILLED with the prospect of finally having a single firmware across the whole network. That is really the Holy Grail for us.

  6. #6
    Join Date
    Oct 2002
    Location
    Upstate New York think mts and trees
    Posts
    1,915
    Rep Power
    17

    Default

    Wonderful reports and thank everyone for being so candid with replies.
    I got stuck in a situation where I have followed advice and upgraded a customer owned WRAP Hot Spot system up to 1.5.14.1. The licenses all expired in Nov of 2010.
    --------
    I have kept moving forward hopeing with each version their connection problems would go away.
    After reading all the postings that were submitted today I think it foolish to go backwards.
    One little kick in the backside is I have to pay for the license extensions out of pocket.
    Also there is no way they are going to purchase new equipment in this economic environment.
    ----------
    Our other systems are almost identical to what you folks have posted.
    I don't have any systems a large as ninedd but the setup is pretty the same.
    ----------
    This information is going to be a great help for me to work my way out of these kind of situations.
    Many thanks for the reference material.
    Dave
    <
    @)
    (Cooter>>
    ^^^^^^^^^^
    Vrablic

    "I have no excuses, Just reasons"


  7. #7
    Join Date
    Dec 2002
    Location
    Frozen Tundra of Canada
    Posts
    2,822
    Rep Power
    17

    Default

    Quote Originally Posted by ninedd View Post
    We have very, very few fatal horror stories from firmware upgrades...
    Knock wood. Did have a ''gotcha'' today. An X86PC on one of our building tops that was doing a PTP Link and was still running 1.3.23b on that backhaul. Upgraded to 1.5.15.3b and it didn't come back from the reboot - crap - truck roll.

    Turned out to be Ethernet re-ordering (which is the Kernels fault I understand) but is a minor gotcha that is hard to anticipate. It's also frankly hard for Tony to test and to put a specific reminder about in the release notes. This particular PC was a Dell Optiplex, and they had a 3Com Ethernet in them, so we had added a second Ethernet (since the 3Com's suck) and that had worked great for years. When we rebooted, the two NIC's and their IP's, swapped, and that made the machine unreachable. We had to drive on site, reprogram the IP's on the NICs, and all is well.

    So, just posting this because a) it's kinda funny, and b) this is about the worst kinda thing that we've seen on firmware upgrades. Generally speaking, they go fairly well, and generally speaking, it's pretty safe to apply an update 2-3 days after it's posted in the download area. By then, any fatal flaws have been reported. We've found, almost always, that the latest versions are better, overall.

  8. #8
    Join Date
    Mar 2005
    Location
    North Middlesex
    Posts
    1,454
    Rep Power
    14

    Default

    I have always matched up same brands of Ethernet in anything we put up so that the re-order thing never happens. We have never had it happen yet anyway. Now that I have said something this is sure to change! Most of our core routers only have one Ethernet port that goes to a switch anyway. In your case I probably would have disabled the 3Com before putting something else in to replace it. Of course if we has as good of foresight as we do hindsight many of these issues would be avoided before they happen. Oh to be perfect, even just for a while....

  9. #9
    Join Date
    Jan 2000
    Location
    Langley, Canada
    Posts
    8,090
    Rep Power
    10

    Default

    With the new device infrastructure used on 1.5.x PC and Server Editions, the device reordering issue has been vanquished from future updates. Now when a new device is supported, it will no longer affect the ordering of your existing cards. There may still be an issue with 1.3.23b -> 1.5.x upgrades, as 1.3.23b still uses an arbitrary ordering scheme. We have tried to match it up as much as possible, however some devices may still be reordered during the transition to 1.5.x.

    There can still be a reordering problem when adding new cards, if using mixed brands which is easy to catch and correct, however issues during upgrading should not occur any longer.

  10. #10
    Join Date
    Dec 2002
    Location
    Frozen Tundra of Canada
    Posts
    2,822
    Rep Power
    17

    Default

    Hi Tony, thanks. Just to be clear, that wasn't really any 'problem report' or any kind of complaint at all. This was more a musing about something that would be very, very, very difficult to test for if any of us were in your position. We didn't think about it, but this is about the biggest gotcha we have had when upgrading, and it was pretty minor.

    <HUMOROUS SARCASM>
    What? You didn't test the "Dell OptiPlex Pentium 3 with onboard 3Com & PCI RealTek NIC re-ordering when upgrading from 1.3.23b to 1.5.15.3b" issue? How did you forget to test for that before you released 1.5.15.3b???
    </HUMOROUS SARCASM>

    Besides, our Network Admin does a REALLY good job. We have all our AP's .dat files automatically backed up every day, and we have spare hardware sitting ready to go, so when this machine didn't come back from it's upgrade/reboot, we have an IBM NetVista sitting as a backup for our working X86-PC machines - he booted it up, uploaded the .dat file, and I drove to the building with that replacement in hand. Having a replacement machine on hand costs what - $200? - which is a no-brainer not to have extra gear on hand. In any case, when I got there, I saw the existing machine was actually online, and discovered the card re-ordering issue. I reprogrammed and rebooted and voila! I think the total downtime for that link was 23 minutes.

Similar Threads

  1. Good service feedback
    By DLNoah in forum General Discussion
    Replies: 3
    Last Post: 08-21-2008, 12:25 AM
  2. Need older V3 with SSH
    By trimaxium2003 in forum StarOS™ Releases
    Replies: 5
    Last Post: 06-26-2007, 01:19 AM
  3. Firmware / Hardware Problem?
    By yama in forum Support
    Replies: 7
    Last Post: 10-25-2004, 10:18 PM
  4. Feedback
    By Steve in forum Steve Dietzel's StarCom & UtiliStar
    Replies: 1
    Last Post: 05-23-2004, 04:30 AM
  5. Squid Feedback!
    By Steve in forum Feature Requests
    Replies: 2
    Last Post: 11-07-2002, 05:19 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts