StarV3 BGP Timeframe

[tony]

Some of you are interested in seeing BGP support in StarV3.

It won't be in 1.3.0 due to a feature freeze, however it is planned for inclusion in an upcoming 1.3.x release.

While the overall demand for this feature is limited, we have assigned it a moderate to high priority.

Once released, it will be available for the X86-PC platform.

[Beebe]

Will this show up in the feature list posted on the forum even though the feature is not available under anything but the PC edition? It's not been included yet has it?

Thanks,
Roger

[Mark]

Any ETA on BGP? This is starting..well, now long past needed for me. Its costing me an estimated 300 - 400 dollars a month in increased bandwidth bills because I can't bring the cheaper provider on yet...

[Beebe]

I rolled my two border StarOS boxes back to version 2.10.1b5 to get BGP working in the meantime. Tried a different v2 but had problems, this one seems to work with BGP.

Thanks,
Roger

[lonnie]

One of us does not understand BGP, so maybe educate me.

As I see it BGP is used when an ISP obtains his own IP assignment and then obtains bandwidth from more than one Upstream provider, all of whom are not in any way connected except through the big cloud known as the Internet. They handle the ISP subnet as well as the subnets that have been assigned to them by their upstream provider.

As I see it, unless you have actually taken the step and applied for and received your own AS, and have two or more providers who will advertise that they can handle traffic to your AS, then BGP is of no use.

If I am viewing this correctly, can I ask how many users actually operate under those conditions? It is pretty high level and typically only used by a very large ISP.

If I am not viewing this correctly, can someone explain where I am missing it?

[tog]

Run your own plain vanilla Linux/BSD router with quagga or OpenBGP?

Even if you have no physical space available, you can still do a nice beefy system in a mini-ITX form factor and attach it to a wall :)

[butchkemper]

One of us does not understand BGP, so maybe educate me.

As I see it BGP is used when an ISP obtains his own IP assignment and then obtains bandwidth from more than one Upstream provider, all of whom are not in any way connected except through the big cloud known as the Internet. They handle the ISP subnet as well as the subnets that have been assigned to them by their upstream provider.

As I see it, unless you have actually taken the step and applied for and received your own AS, and have two or more providers who will advertise that they can handle traffic to your AS, then BGP is of no use.

If I am viewing this correctly, can I ask how many users actually operate under those conditions? It is pretty high level and typically only used by a very large ISP.

If I am not viewing this correctly, can someone explain where I am missing it?

BGP requires you to have an ASN from ARIN and multiple Internet feeds. It does not require you to have IP space assigned directly from ARIN. You can do BGP using IPs assigned from the Internet feed vendor as long as the other Internet feed vendor will advertise your routes. This is not a problem with main stream Internet vendors but may be with some of the smaller players.

Some years ago, I had 4 T1s from Sprint and 1 T1 from UUnet, IP space from Sprint, and an ASN from ARIN. I ran BGP with Sprint and UUnet.

If running BGP is such a critical issue, then I would recommend getting a used Cisco router from EBAY to sit between the Internet feeds and your network. The cost of a well equipped router should not exceed $2k.

Butch

[Beebe]

You don't have to be that big to need BGP... If you've taken over another wisp and they have their own Internet feed, and are under a long contract you can't really consolidate the networks unless you stop using one of the pipes, even though you have to pay for it until the end of the contract.

I was in that kind of situation for quite a while, and I'm still paying over half a grand each month for a T1 which isn't being used much, since I'm migrating users off of it. That expires in January though and I'm going to cancel it. It would have been much more effective to run BGP and to have been using both pipes while I moved the users over to the new IP space before shutting off the second connection.

The reason I'm using BGP now is because I have way to many people on a 3 meg pipe, and bandwidth is relatively expensive here. I've partnered with a wisp about 30 miles away, and we've linked our networks. He has access to a fiber connection to the cable company, which he can upgrade to 20Mb far cheaper than I have access to that kind of bandwidth. He's having trouble getting enough IP addresses for his customers, and is being charged a lot for them from the cable company. We're going to start getting our IP's from arin. When both links are running we'll have redundancy, I'll have cheap bandwidth, he'll have cheap IPs. Until that happens, we're both held up from adding customers.

Only problem is, the cable company are taking their time installing a router on their end which supports BGP... I'm running v2 with BGP on my link to AT&T already. His end is waiting for the cable company.

So there are two situations which could use BGP which both happened to one small wisp.

As has been pointed out, if the need is great enough we can just set up a separate box. I chose to downgrade rather than have a separate computer to keep running.

Thanks,
Roger

[Mark]

One of us does not understand BGP, so maybe educate me.

As I see it BGP is used when an ISP obtains his own IP assignment and then obtains bandwidth from more than one Upstream provider, all of whom are not in any way connected except through the big cloud known as the Internet. They handle the ISP subnet as well as the subnets that have been assigned to them by their upstream provider.

As I see it, unless you have actually taken the step and applied for and received your own AS, and have two or more providers who will advertise that they can handle traffic to your AS, then BGP is of no use.

If I am viewing this correctly, can I ask how many users actually operate under those conditions? It is pretty high level and typically only used by a very large ISP.

If I am not viewing this correctly, can someone explain where I am missing it?


I'm currently consuming around 2-4Mbit steady, and yes, it's time for me to get my own ASN and own IP block. You CAN still get your blocks from your providers, but until you get your AS number, you can't get 3 different providers at 3 physically diverse locations on your network and still manage to remain fully operational if even one of them goes down. I want to add 2 more providers. And I'll not be spending a dollar more (and in fact, less than now) to have all 3.

And you do NOT have to be "very big" to need your own netblock or ASN, you merely need redundancy and diverse pathways to improve your network reliability and improve your connectivity.

Oh, and you do NOT get your ASN first. The rules are that once you apply for it, YOU MUST BE MULTIHOMED within 60 days. I'm not sure they have any issue if it takes 70 or 90 to get everyone on board, but I'm not paying the big bucks if I have no chance of running BGP anytime soon. You do it all at the same time. Once you have your providers worked out, you then apply for, and accomplish all the transitions simultaneously. Or, at least that's the stated rules.

The problem I have here, is that I do NOT have any NOC. No building where everything runs through. There is no "center". The closest thing I have to that is a 14X9 box on a rooftop on an airport building - powered by POE.

No physical location for routers, no power to run them (most of my sites in some form or another run off batteries), and certainly no rooftop space in either of the new locations, either. This is truly the kind of network Star-OS was originally created for. All in the air, everything "in the box".

[Mark]

Run your own plain vanilla Linux/BSD router with quagga or OpenBGP?

Even if you have no physical space available, you can still do a nice beefy system in a mini-ITX form factor and attach it to a wall :)

Yes, I have this mini-itx board (well, multiple boards in multiple places) in a box, but it runs STAR-OS, because these boards all have attached radios (more than one).

I don't understand the point of adding a second board. We HAD BGP, and when the time came for me to get some redundancy, it wasn't there anymore :(.

[DrLove73]

But you do have much better wireless performance than v2, you do agree with that? Because of complete rewrite, some thing are not done yet, but solution exists in form of 1 extra board. That is not so horrible trade, I would say.

[lonnie]

I have gone through the exercise to have another ISP subnet routed through a new provider. They hate doing that and unless they have designed their own IP subnet properly it is a mess. Make sure they know you intend to do multihomed when they assign you a subnet.

Can you imagine the complexity of the world's BGP routers when all the little guys start routing /24 through multiple providers? My feeling is that you will soon find a requirement for a minimum of /22 in order to use BGP, but hey, get in now while the water's fine.

[rkreigh]

I have gone through the exercise to have another ISP subnet routed through a new provider. They hate doing that and unless they have designed their own IP subnet properly it is a mess. Make sure they know you intend to do multihomed when they assign you a subnet.

Can you imagine the complexity of the world's BGP routers when all the little guys start routing /24 through multiple providers? My feeling is that you will soon find a requirement for a minimum of /22 in order to use BGP, but hey, get in now while the water's fine.

A lot of the smaller announcements (/22 or smaller) will not be advertised, or they will be filtered. I seen something recently on the ARIN list to this effect about it becoming policy.

Be warned, BGP is not all fun and games, it can be your worst nightmare.

We had more outages CAUSED by BGP than BGP prevented.

Back in the old days we ran several T1's on BGP, it was a pain. Your traffic will never be balanced right. One T1 is packed to the gills, and another is empty, you change your BGP MED's and then the other one is crammed.

We are now single-homed to our Tier-1 provider on a Gig-E, and haven't had a single outage in over 2 years. We are on a path diverse dual SONET ring too, that helps.

I would get creative with policy routing and NAT out to that second backbone before I did BGP.

Thanks,
-Russ

[Mark]

I have discussed my options with both providers, and both are willing to do a little fudging, and they're willing to work together, even going so far as to not have my own ASN, but still work out the multihomed routing.

However, it remains dependent upon my equipment running bgp - still.

[Beebe]

Also having BGP and being multi-homed is not just about redundancy, but also about being able to drop or add a provider without renumbering my network. I would have the ability to add a different provider in a different market, at a different point on my network wherever I find cheap bandwidth, and drop what I have already without it being such a big deal.

I really like that idea - especially when my provider is my main competitor, and it is sometimes in their best interests to give me trouble. I really like the idea of having a /22 of my very own, and not have to give back IPs to an upstream provider. You have much more control. And then it's more scalable too... if I have a T3 at each end of the network it would be much better than having two T3s at one end. Outbound traffic would flow out of the nearest T3, which means less congestion on the backbone. Inbound traffic is not that controllable, but every little helps.

Thanks,
Roger

[Beebe]

Just thought of yet another reason to run BGP... In order to qualify for your own IP space from arin, you have to be able to justify only a /22 if you're multihomed, but a full /20 if you're single homed. So you can have your own IP space a lot sooner if you have multiple providers.

Thanks,
Roger

[knolan]

We are currently running V2.11 of StarOS at our Edge, We are using it for CBQ traffic shaping and for BGP peering. - If it wasn't for the BGP peering it would already be upgraded to V3

I would love to have this edge machine support OLSR, as well as BGP for routing, so I could allow it to be part of our OLSR network.

At this edge site we have 4 core AP's running OLSR, and I have set each AP to announce 0.0.0.0 HNA statements, with each of them having a static default route to the edge machine.
On the Edge machine I have set static routes for each of our /24's to different AP's so if one of the AP's goes down we will not loose 100% of the traffic. - But we would (and have) loose any traffic staticly routed to that AP.

If the edge machine supported OLSR, then it would be the only machine to have a 0.0.0.0 HNA statement and if a core AP went down, users may not even kow about it as OLSR would route their traffic using a different AP to the edge.

The plan in the future would be to add a second Edge Router, with both routers peering with different upstream transit providers, and both machines having the same CBQ script, with each machine announcing 0.0.0.0 HNA statement, using this method we would remove all single points of failure on our Core and Edge networks, leaving only CPE's and the access points they connect to as single points of failure - If one of these AP's goes down the most they would impact would be 120 customers (4 radios * 30 CPE's per radio) - this would be a lot easier to deal with, than the core or our Edge being down.

I'm sure we aren't the only wisp playing with BGP to provide some resilence on their connection to the internet, if BGP was joined with OLSR then VNC would have a great Edge product for WISP's.



As I'm talking about BGP, there is a bug in v2.11, which is causing BGP not to release the memory used for BGP each time a BGP session is stoped/started, if you recieve a full routing table (as we do) it doesn't take too many stops or starts to cause a problem - and an active changes causes the BGP session to stop/start. (if this could be fixed when the server version of V3 is released with BGP it would be great :))

And as we are on the subject of V3 server, the V2 server only seems to 882 MB of RAM, can this be increased in the V3 server.

Regards,
Keith

[tony]

For all those waiting for BGP, not to worry. It is on our list, and will be included in an upcoming X86-PC release.