Step by Step How To - basic qshape

[kbldawg]

I am by no means a master operator of StarOS, but I thought I would contribute what little I do know, and have learned, to others that might appreciate the help.

This is a very basic How-To on qshaping.

----

qshape

Assuming you have a working StarOS AP setup with associated clients.

1. Using putty, log-in to your StarOS router.
2. Click Advanced > Scripts > class based queuing (cbq / bw management)

First you will need to set some variables...

*Net will equal your router's gateway interface.
*Client will equal the interface to which your clients (customers) associate.
These values could be… ether (Ethernet), wpci (mini pci), or wlan (pcmcia) interfaces; this will depend on your specific configuration.

Let's assume “net” is your ether1 and your clients connect to wpci1.

Indicate which interface is net and which interface is client

3. at the top of the script type...

net = ether1
client = wpci1

*anything preceding the “#” symbol will be ignored. When adding rules, make sure you do not have the “#” symbol at the beginning.

Now you can start limiting your customer’s bandwidth speeds by adding a simple qshape script.

First you should understand some of the commands and what they mean.

StarOS simple qshape example:
qshape user 100 bw 128k 56k 192.168.10.1 on $client

qshape = The qshape command
user = The user name you have specified for the customer. This value can be anything you want.
100 = Bandwidth pipe number. In this instructional each customer will get their own unique pipe.
bw 128k 56k = Bandwidth equals 128kbps download and 56kbps upload. The speeds can be anything you want, but the syntax has to be as displayed in the example.
192.168.10.1 = The IP address of the customer’s PC/Router that you are wanting to qshape.
on = This is self explanatory.
$client = This is the interface that the client is associated. The "$" symbol indicates that this value has been bound.

Now you are ready to start limiting your customer’s download and upload speeds.

4. Go to the next available line and type…

qshape asmith 100 bw 1024k 512k 192.168.10.1 on $client

*this qshape rule indicates that asmith’s pipe (pipe 100) has been limited to 1mbps download / 512kbps upload. His PC/Router’s IP address is 192.168.10.1, and his CPE is associated to your AP’s wpci1 interface.

Let’s do another customer. This customer has two static IP addresses. We need to limit both addresses to the same pipe so that they do not go over their allocated bandwidth.

5. Go to the next available line and type…

qshape bsmith 101 bw 2048k 1024k 192.168.10.2 on $client
qshape bsmith 101 192.168.10.3 on $client

*this qshape rule indicates that Bob Smith’s pipe (pipe 101) has been limited to 2mbps download / 1mbps upload. Both of this customer’s IP address share pipe 101’s total bandwidth.

When we combine what we have done in this instructional, your cbq script should look like this…

net = ether1
client = wpci1
#
#
qshape asmith 100 bw 1024k 512k 192.168.10.1 on $client
qshape bsmith 101 bw 2048k 1024k 192.168.10.2 on $client
qshape bsmith 101 192.168.10.3 on $client


Other variables can be set to make life easier, for example:

Lite = “bw 512k 128k”
Basic = “bw 1024k 512k”
Plus = “bw 1024k 1024k”
Premium = “bw 2048k 1024k”

With this variables set, your rules would now read…

qshape asmith 100 $basic 192.168.10.1 on $client
qshape bsmith 101 $premium 192.168.10.2 on $client
qshape bsmith 101 192.168.10.3 on $client
Or you could qshape an entire subnet to a specific pipe.

qshape csmith 102 $plus 10.128.0.1/24 on $client

Our new script looks like this…

net = ether1
client = wpci1
#
#
Lite = “bw 512k 128k”
Basic = “bw 1024k 512k”
Plus = “bw 1024k 1024k”
Premium = “bw 2048k 1024k”
#
#
qshape asmith 100 $basic 192.168.10.1 on $client
qshape bsmith 101 $premium 192.168.10.2 on $client
qshape bsmith 101 192.168.10.3 on $client
qshape csmith 102 $plus 10.128.0.1/24 on $client

Hopefully this will make qshaping a little more clearly, and give you a head-start with more advanced shaping scripts.

Oh, and never ever activate a script until you have checked your syntax. Just click the button that says "syntax check"
--
Chad Halsted

[lonnie]

Thanks for the very clear example. I'm sure others will find this useful.

[go.fast]

Thanks for the very clear example. I'm sure others will find this useful.

Good example Chad.

Would it be too much if you could go a step further and do the p2p shaping as well?

Good example Chad.

[kbldawg]

I have only shaped one AP for p2p apps, and when I did that I just copied and pasted someone elses file that I got here at the forums.

I haven't had the time to learn how to shape p2p and all the options available for that. I haven't had big issus with p2p on my network, or I assure you I would make the time. <grin>

I will be glad to share the information I have, but I wouldn't be able to explain it. All I can say is that it is layer7 shaping, but it simply drops those packets. I would be interested in some layer7 shaping that redirects to a predefined qshape pipe, so that it simply limits the overall amount.

Perhaps someone else can continue on where I have left off?

[knolan]

OK,

Here is how we're doing traffic shaping for our customers.


I've split the information into 3 sections;

1. limit the number of connections per user
2. Shaping P2P traffic at layer7
3. Shaping users, using parent pipes

1. Limiting the number of connections per user. In the Firewall script add the following line

iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 300 -j DROP

This line is using iptables and is dropping all tcp packets above 300 per user. If you want the limit to be a different value, just change the 300 figure


2. Shaping P2P traffic at layer 7


In order to shape traffic at layer7 you need to edit 2 scripts, the firewall script & the cpq script

In the firewall script you will need to "mark" traffic based on layer7

(a full list of layer7 filters and how accurate they are at the following web site http://l7-filter.sourceforge.net/protocols)

The following is an example of the firewall script - the filters I've included here should cover most P2P applications people are using.

## Traffic Shaping at layer 7

iptables -A PREROUTING -t mangle -m layer7 --l7proto applejuice -j MARK --set-mark 101
iptables -A PREROUTING -t mangle -m layer7 --l7proto ares -j MARK --set-mark 102
iptables -A PREROUTING -t mangle -m layer7 --l7proto audiogalaxy -j MARK --set-mark 103
iptables -A PREROUTING -t mangle -m layer7 --l7proto bittorrent -j MARK --set-mark 104
iptables -A PREROUTING -t mangle -m layer7 --l7proto directconnect -j MARK --set-mark 105
iptables -A PREROUTING -t mangle -m layer7 --l7proto edonkey -j MARK --set-mark 106
iptables -A PREROUTING -t mangle -m layer7 --l7proto fasttrack -j MARK --set-mark 107
iptables -A PREROUTING -t mangle -m layer7 --l7proto freenet -j MARK --set-mark 108
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnucleuslan -j MARK --set-mark 109
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnutella -j MARK --set-mark 110 # Limewire etc.
iptables -A PREROUTING -t mangle -m layer7 --l7proto goboogy -j MARK --set-mark 111
iptables -A PREROUTING -t mangle -m layer7 --l7proto hotline -j MARK --set-mark 112
iptables -A PREROUTING -t mangle -m layer7 --l7proto imesh -j MARK --set-mark 113
iptables -A PREROUTING -t mangle -m layer7 --l7proto kugoo -j MARK --set-mark 114
iptables -A PREROUTING -t mangle -m layer7 --l7proto mute -j MARK --set-mark 115
iptables -A PREROUTING -t mangle -m layer7 --l7proto napster -j MARK --set-mark 116
iptables -A PREROUTING -t mangle -m layer7 --l7proto openft -j MARK --set-mark 117
iptables -A PREROUTING -t mangle -m layer7 --l7proto poco -j MARK --set-mark 118
iptables -A PREROUTING -t mangle -m layer7 --l7proto soribada -j MARK --set-mark 119
iptables -A PREROUTING -t mangle -m layer7 --l7proto soulseek -j MARK --set-mark 120
iptables -A PREROUTING -t mangle -m layer7 --l7proto tesla -j MARK --set-mark 121
iptables -A PREROUTING -t mangle -m layer7 --l7proto thecircle -j MARK --set-mark 122






In order to shape the p2p traffic, you need to take the "marked" traffic and shape it in the cbq script.

The following CBQ example goes with the above firewall script

## Defining the bandwidth for the P2P pipes - These pipes are using the fallback feature.

# “When fallback is enabled, the speed will start being decreased if the user has saturated his link, to a capacity of 70% after 10 seconds. The speed will drop 10% every second until it reaches 50% of it's original rate. After the user drops below the 70% saturation rate for the reduced speed for 10 seconds or more, the pipe speed will start increasing again, 10% every second until it has been restored, or until the user becomes overly active again.”

# I have limited the pipes to 1Mbit/s or 1024K, but these will fallback to 512K as per the above

# P2P Pipes
pipe 100 bw fb 1024k
pipe 101 bw fb 1024k


# Outbound traffic
shape all to pipe 100 from mark 101 out via ether2
shape all to pipe 100 from mark 102 out via ether2
shape all to pipe 100 from mark 103 out via ether2
shape all to pipe 100 from mark 104 out via ether2
shape all to pipe 100 from mark 105 out via ether2
shape all to pipe 100 from mark 106 out via ether2
shape all to pipe 100 from mark 107 out via ether2
shape all to pipe 100 from mark 108 out via ether2
shape all to pipe 100 from mark 109 out via ether2
shape all to pipe 100 from mark 110 out via ether2
shape all to pipe 100 from mark 111 out via ether2
shape all to pipe 100 from mark 112 out via ether2
shape all to pipe 100 from mark 113 out via ether2
shape all to pipe 100 from mark 114 out via ether2
shape all to pipe 100 from mark 115 out via ether2
shape all to pipe 100 from mark 116 out via ether2
shape all to pipe 100 from mark 117 out via ether2
shape all to pipe 100 from mark 118 out via ether2
shape all to pipe 100 from mark 119 out via ether2
shape all to pipe 100 from mark 120 out via ether2
shape all to pipe 100 from mark 121 out via ether2
shape all to pipe 100 from mark 122 out via ether2



# Inbound traffic

shape all to pipe 101 from mark 101 in via ether2
shape all to pipe 101 from mark 102 in via ether2
shape all to pipe 101 from mark 103 in via ether2
shape all to pipe 101 from mark 104 in via ether2
shape all to pipe 101 from mark 105 in via ether2
shape all to pipe 101 from mark 106 in via ether2
shape all to pipe 101 from mark 107 in via ether2
shape all to pipe 101 from mark 108 in via ether2
shape all to pipe 101 from mark 109 in via ether2
shape all to pipe 101 from mark 110 in via ether2
shape all to pipe 101 from mark 111 in via ether2
shape all to pipe 101 from mark 112 in via ether2
shape all to pipe 101 from mark 113 in via ether2
shape all to pipe 101 from mark 114 in via ether2
shape all to pipe 101 from mark 115 in via ether2
shape all to pipe 101 from mark 116 in via ether2
shape all to pipe 101 from mark 117 in via ether2
shape all to pipe 101 from mark 118 in via ether2
shape all to pipe 101 from mark 119 in via ether2
shape all to pipe 101 from mark 120 in via ether2
shape all to pipe 101 from mark 121 in via ether2
shape all to pipe 101 from mark 122 in via ether2


# It is important to use the correct interface for the traffic shaping to work, in the above example interface ether2 is our WAN interface connecting to the Internet


3. Using CBQ to limit users, by setting up a parent pipe so that you can contend groups of users


## Group 1
bi-pipe 1001 bw 1024K 256K
res1-1 = "bw fb 1024k fb 256k parent 1001"

# The above lines are used for setting up a parent pipe, "bi-pipe 1001 bw 1024k 256k" is setting up a pipe with the bandwidth 1024k & 256k assigned to it "res1-1 = "bw fb 1024k fb 256k parent 1001"" is assigning any qshape with the variable res1-1 to the parent bi-pipe 1001, with the fallback feature implemented on the 1024k 256k.
# These two lines are creating a dedicated pipe with 1mb download and 256k upload, and then setting a variable within this pipe to contend each user with fallback


## Unknown users
bi-pipe 9009 bw 5k 5k
unknown = "bw 5K 5K parent 9009"
qshape any 999 $unknown 10.1.0.0/16 on ether3

# The above 3 lines are setting up a parent pipe with only 5k upload and download available. and is setting any IP Address in the range 10.1.0.0 --> 10.1.255.255 which doesn't have its own line in the CBQ script to share 5k (not very much :))

####### Customers Bandwidth rules
## Group 1
qshape user1 201 $res1-1 10.1.0.1 on ether3
qshape user2 202 $res1-1 10.1.0.2 on ether3
qshape user3 203 $res1-1 10.1.0.3 on ether3
qshape user4 204 $res1-1 10.1.0.4 on ether3
qshape user5 205 $res1-1 10.1.0.5 on ether3
qshape user6 206 $res1-1 10.1.0.6 on ether3
qshape user7 207 $res1-1 10.1.0.7 on ether3
qshape user8 208 $res1-1 10.1.0.8 on ether3
qshape user9 209 $res1-1 10.1.0.9 on ether3
qshape user10 210 $res1-1 10.1.0.10 on ether3
qshape user11 211 $res1-1 10.1.0.11 on ether3
qshape user12 212 $res1-1 10.1.0.12 on ether3

# The above lines are setting the users to be part of the res1-1 variable (parent pipe 1001)

# Again its important to set the qshape on the correct interface, in this case ether3 is the interface on our network - the default gateway to the internet for our users.






Regards,
Keith Nolan

[tog]

A small addition to this discussion, it seems Lonnie had a pretty good quip earlier about this being able to be a sort of "distributed computing" thing by putting these rules at the customer CPE rather than at some central location like the AP or at a big x86 PC later on down the line.

It seemed like a pretty cool suggestion to me.

These are pretty hefty rulesets taking a lot of resources, but fine if it's at the customer CPE itself just handling one customer's traffic.

[nickwhite]

iptables -A FORWARD -p tcp -m connlimit --connlimit-above 100 -j DROPI'd like to note that it has been reported that setting --connlimit-above to anything below 20 has caused bad results - often seemingly stopping all TCP traffic, or making the link appear unusable.

[tog]

I assume that's meant to limit the number of concurrent active TCP sessions for any single IP address. You would most certainly want that to be pretty high and you would not want to try to use that do prevent P2P or anything, only to try to limit DoS attacks from worms. If I were to use it I'd probably double it to 200.

[kbldawg]

knolan, great example!! Thanks for contributing.

[DrLove73]

tog, how would you limit P2P connections? They made havoc on my network. Some users downloaded simultaniously 30-50 files at once x No. of connections (20-50?). Other users complained poor gaming in that conditions. I'm small ISP, my uplink so far is only 2 Mbit (~100 users). most users use 64k-128k, P2P used to buy 256k max, before I blocked it.

[knolan]

I agree with Tog, these CBQ & Friewall scripts do require a lot of memory & processing power.


We are running all of our traffic shaping at our Edge, The rules are all running on a P4 3Ghz Server, with 2gb ram.

The server is running StarOS v2.11 and is only used for traffic shaping - it doesn't have any radio cards in it.



It may be a good idea or not a good idea to copy how we have implemented traffic shaping. The other way of doing the shaping would be at each clients CPE. In order to achive this each client would need to be using a StarOS CPE - such as the WarTenna.


To achive this at the CPE
I would

1. Turn on Connection tracking (Required for the Firewall scripts to work)

2. Create a Firewall & CBQ script to limit P2P software

a. Create a firewall rule for the P2P as in the example



iptables -A PREROUTING -t mangle -m layer7 --l7proto applejuice -j MARK --set-mark 101
iptables -A PREROUTING -t mangle -m layer7 --l7proto ares -j MARK --set-mark 102
iptables -A PREROUTING -t mangle -m layer7 --l7proto audiogalaxy -j MARK --set-mark 103
iptables -A PREROUTING -t mangle -m layer7 --l7proto bittorrent -j MARK --set-mark 104
iptables -A PREROUTING -t mangle -m layer7 --l7proto directconnect -j MARK --set-mark 105
iptables -A PREROUTING -t mangle -m layer7 --l7proto edonkey -j MARK --set-mark 106
iptables -A PREROUTING -t mangle -m layer7 --l7proto fasttrack -j MARK --set-mark 107
iptables -A PREROUTING -t mangle -m layer7 --l7proto freenet -j MARK --set-mark 108
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnucleuslan -j MARK --set-mark 109
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnutella -j MARK --set-mark 110 # Limewire etc.
iptables -A PREROUTING -t mangle -m layer7 --l7proto goboogy -j MARK --set-mark 111
iptables -A PREROUTING -t mangle -m layer7 --l7proto hotline -j MARK --set-mark 112
iptables -A PREROUTING -t mangle -m layer7 --l7proto imesh -j MARK --set-mark 113
iptables -A PREROUTING -t mangle -m layer7 --l7proto kugoo -j MARK --set-mark 114
iptables -A PREROUTING -t mangle -m layer7 --l7proto mute -j MARK --set-mark 115
iptables -A PREROUTING -t mangle -m layer7 --l7proto napster -j MARK --set-mark 116
iptables -A PREROUTING -t mangle -m layer7 --l7proto openft -j MARK --set-mark 117
iptables -A PREROUTING -t mangle -m layer7 --l7proto poco -j MARK --set-mark 118
iptables -A PREROUTING -t mangle -m layer7 --l7proto soribada -j MARK --set-mark 119
iptables -A PREROUTING -t mangle -m layer7 --l7proto soulseek -j MARK --set-mark 120
iptables -A PREROUTING -t mangle -m layer7 --l7proto tesla -j MARK --set-mark 121
iptables -A PREROUTING -t mangle -m layer7 --l7proto thecircle -j MARK --set-mark 122



b. Create a CBQ rule for the P2P as in the example



# P2P Pipes
pipe 100 bw fb 1024k
pipe 101 bw fb 1024k


# Outbound traffic
shape all to pipe 100 from mark 101 out via wpci1
shape all to pipe 100 from mark 102 out via wpci1
shape all to pipe 100 from mark 103 out via wpci1
shape all to pipe 100 from mark 104 out via wpci1
shape all to pipe 100 from mark 105 out via wpci1
shape all to pipe 100 from mark 106 out via wpci1
shape all to pipe 100 from mark 107 out via wpci1
shape all to pipe 100 from mark 108 out via wpci1
shape all to pipe 100 from mark 109 out via wpci1
shape all to pipe 100 from mark 110 out via wpci1
shape all to pipe 100 from mark 111 out via wpci1
shape all to pipe 100 from mark 112 out via wpci1
shape all to pipe 100 from mark 113 out via wpci1
shape all to pipe 100 from mark 114 out via wpci1
shape all to pipe 100 from mark 115 out via wpci1
shape all to pipe 100 from mark 116 out via wpci1
shape all to pipe 100 from mark 117 out via wpci1
shape all to pipe 100 from mark 118 out via wpci1
shape all to pipe 100 from mark 119 out via wpci1
shape all to pipe 100 from mark 120 out via wpci1
shape all to pipe 100 from mark 121 out via wpci1
shape all to pipe 100 from mark 122 out via wpci1



# Inbound traffic

shape all to pipe 101 from mark 101 in via wpci1
shape all to pipe 101 from mark 102 in via wpci1
shape all to pipe 101 from mark 103 in via wpci1
shape all to pipe 101 from mark 104 in via wpci1
shape all to pipe 101 from mark 105 in via wpci1
shape all to pipe 101 from mark 106 in via wpci1
shape all to pipe 101 from mark 107 in via wpci1
shape all to pipe 101 from mark 108 in via wpci1
shape all to pipe 101 from mark 109 in via wpci1
shape all to pipe 101 from mark 110 in via wpci1
shape all to pipe 101 from mark 111 in via wpci1
shape all to pipe 101 from mark 112 in via wpci1
shape all to pipe 101 from mark 113 in via wpci1
shape all to pipe 101 from mark 114 in via wpci1
shape all to pipe 101 from mark 115 in via wpci1
shape all to pipe 101 from mark 116 in via wpci1
shape all to pipe 101 from mark 117 in via wpci1
shape all to pipe 101 from mark 118 in via wpci1
shape all to pipe 101 from mark 119 in via wpci1
shape all to pipe 101 from mark 120 in via wpci1
shape all to pipe 101 from mark 121 in via wpci1
shape all to pipe 101 from mark 122 in via wpci1



3. Create a CBQ script for the CPE's IP Address to limit the users bandwidth


qshape user1 201 bw 1024k 1024k 0.0.0.0/0 on ether1



4. Create a connection limit rule in the firewall - Some P2P Software will open hundreds of connections while trying to download, by limiting the number of connections per IP you will stop the number of connections growing out of control. We have implemented connection limits at 100, and we don't have anyone complaining.



iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 300 -j DROP



The advantage I can see in doing the shaping at the CPE, is a powerful machine isn't required at the edge and Users are shaped on the network as well as on the internet


The disadvantage I can see is you can't implement parent pipes to group users together for contenion (where you can sell the same bandwidth to 20 or 40 users) and if you want to modify your P2P firewall rules you would need to modify each CPE on the network.


So maybe if the users total bandwidth was shaped at the CPE, and the parent pipes, P2P & Connection limits were done at the edge it would give the best of all worlds :)

Regards,
Keith

[DrLove73]

Yes knolan, I to think you are right.
I've done similar, but I am currently using Clarkconnect Linux based on RHEL4/CentOS for main firewall/router/gateway. I've placed P2P block on it (easily done via web configuration using ipp2p addition to kernel). Office version (75$/year) has also 1-1 NAT and Multi-WAN, or you can do 1-1 NAT by manual rules. There is option for P2P Bandwith shaping via webconfig, but only by separate protokols, not for combined bandwith.
I also tried limiting P2P with Star-OS, but haven't had so much rules. I'll give it a go, to see how it works extended.
I'am implementing (in following days) FreeRADIUS v1.1.3 server with dinamic CBQ Support for Star-OS AP's, also on the same Linux box, and with nice web interface (dialup admin). If I manage to create RHEL rpm for it, with dial admin integrated, I'll post it.
Only, my CBQ rules are done on the Star-OS Ap's, CPE's are relatively inexpensive Ovislink AP's. I currently have around 40-50 users/rules per Star-OS AP's in charge of CBQ driving, on Celeron 633MHz/64&128 MB RAM, and the usage is 5-10% with 1 Prism PCMCIA, and 1-2 CM-9's.
If I figure a way to safely incoporate rules for faster access to local network servers, it will be way to go.
Also, I to tried connlimit, but with 20-30 conections limit, recomended in this forum, but users constantly complained, so I turned P2P instead. If 100 is right number (I willl try it), I'll be able to let users use limited P2P usage. That would be very nice indeed. :D

[pwmaclean]

Guys I have put the iptables statement in my firewall, and limited the connections to 2, and it doesnt' seem to change anything. I wanted to see if it's working so I tried to cut myself off. It still allowed me to limewire a file with 8 connections. Have I got it set right?

connecting tracking is turned on
net = wpci1
client = ether1

iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 2 -j DROP

[lonnie]

Is limewire tcp or udp? udp does not use connections so there is no way to limit that and the rule you have is for tcp, which can be limited.

[pwmaclean]

Apparently tcp (6346, 6347)
I tried setting it to 0 connections to see if it kills my connection, but it keeps chugging away. I'll try setting it up again, I must have missed something.

Does anyone really use connlimit?

[knolan]

I've only tested the setting on V2.11.0 of StarOS.

You can verify the command is working by looking at System Reports - View System Information

In the Mangle Chains you should see the same info as attached.

Keith

[Michal]

Hi,
for me is important this:
-limit new connections per user in p2p. Little packets testing new connections makes havy load of the net.
-limit upload from clients. Wifi is halfduplex, then big upload is problem. Download is'nt problem for me, client can use full payed CBQ limit.
-I use this limit on last mile on wraps. Wrap can match only a few rules for reasonable proc load. Than no more CBQ rules and only a few FW rules more.

It would be fine to have possibility to use iptables "dstlimit" function (limit packet count for every dest. ip separatelly, and not only tcp). But it do'nt work on v2.10.0 . Why Lonnie?

Iptables "limit" function you can not use, it counts packets of all traffic, than in case of low limit it affect normat traffic or, with high limit, don't limit flood.

I had problems use marking in PREROUTING, for this is made on FORWARD.
I use this:

# ******************* P2P connections FLOOD ********

# extra user chain for p2p. It limit connections for an ip. It work fine for me, but limit tcp new coonections only. Allows 10 new/s.
iptables -N DCFLOOD
iptables -A DCFLOOD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A DCFLOOD -m connlimit ! --connlimit-above 10 -j ACCEPT --connlimit-mask 32
iptables -A DCFLOOD -j DROP

# ********************** FORWARD ********************


# P2P marking for DC flood filter and CBQ. Use one mark-number only, you don't need so mutch rules.
iptables -A FORWARD -t mangle -m layer7 --l7proto directconnect -j MARK --set-mark 100
iptables -A FORWARD -t mangle -m layer7 --l7proto edonkey -j MARK --set-mark 100
iptables -A FORWARD -t mangle -m layer7 --l7proto gnutella -j MARK --set-mark 100
iptables -A FORWARD -t mangle -m layer7 --l7proto fasttrack -j MARK --set-mark 100
iptables -A FORWARD -t mangle -m layer7 --l7proto bittorrent -j MARK --set-mark 100
iptables -A FORWARD -m mark --mark 100 -j DCFLOOD

-------------------- CBQ -----------------------------

pipe 101 bw 300k
shape all to pipe 101 from mark 100 out via $wan

-------------------- from log

---[ Firewall chains ]---
Chain DCFILTER (1 references)
pkts bytes target prot opt in out source destination
59283 5022636 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
33534 1568177 DROP all -- * * 0.0.0.0/0 0.0.0.0/0


Chain FORWARD (policy ACCEPT 8234307 packets, 3869838882 bytes)
pkts bytes target prot opt in out source destination
92817 6590813 DCFILTER all -- * * 0.0.0.0/0 0.0.0.0/0 MARK match 0x64

Would be better make rules for tcp only...
Bye :)

[Beebe]

How would you go about applying a qshape rule to ALL traffic passing through the unit?

qshape joe-user 100 bw 300k 150k <ALL IPs> on $client

In the past I've been doing it for whatever /24 but in the future I may change subnets and I'd like to have my clients set up to traffic shape no matter what IP the customer is using.

Thanks,
Roger

[oscarBravo]

For <ALL IPs> use 0.0.0.0/0.

[Beebe]

Brilliant! Thanks!

[spacemind]

Hello guys.

I have learned how to use cbq for bw shapping with this post, but i need to control the traffic also. Here in Portugal we have traffic limits and i need to setup my star os as the example:

all unknown users: 20m per day and 200mb (maximum)per month

clients: 2gb per month

Happy hour for clients from 3am to 8:30 (down and up traffic not counting )

Is this possible ?

thanks

[lonnie]

You are wanting an accounting function to accumulate traffic counts. We do not support that.

[DrLove73]

You can do that on your "exit" point, on your main linux gateway, becouse all traffic for every client goes thru there. Of course, network has to be fully routed, no NAT-ing is allowed.

[totalaccess]

You are wanting an accounting function to accumulate traffic counts. We do not support that.


Could radius with a few tweaks do this ?

[DrLove73]

It should be possible with RADIUS, or in combination RADIUS+???. I already forgot. I'we prepared my self to do that, read everthing, but never came to it.

[knolan]

iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 300 -j DROP


We have found that using the above connlimit statement was causing packet loss for non P2P traffic on our network.

We have implemented the following instead.

Firewall

iptables -N P2P
iptables -A P2P -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A P2P -m connlimit ! --connlimit-above 5 -j ACCEPT --connlimit-mask 32
iptables -A P2P -j REJECT
iptables -A PREROUTING -t mangle -m ipp2p --ipp2p -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto applejuice -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto ares -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto audiogalaxy -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto bittorrent -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto directconnect -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto edonkey -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto fasttrack -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto freenet -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnucleuslan -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto gnutella -j MARK --set-mark 100 # Limewire etc.
iptables -A PREROUTING -t mangle -m layer7 --l7proto goboogy -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto hotline -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto imesh -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto kugoo -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto mute -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto napster -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto openft -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto poco -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto soribada -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto soulseek -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto tesla -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto thecircle -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto xunlei -j MARK --set-mark 100
iptables -A PREROUTING -t mangle -m layer7 --l7proto 100bao -j MARK --set-mark 100
iptables -A FORWARD -m mark --mark 100 -j P2P



CBQ

pipe 100 bw fb 128k parent 9007
shape all to pipe 100 from mark 100 out via ether2

Regards,
Keith

[pwmaclean]

Have you had any trouble with it yet? I'm thinking about using it.

[knolan]

This config has been working for a few months for us with no problems, however be aware we are running this at our gateway to the internet, on a PC Server with 1Gb ram and a P4 processor, also the server is running V2.11 of staros.



I've seen reports that Connlimit doesn't work in the latest versions of V3.


Keith

[cephlon]

Knolan,
I tried your script, but it doesn't appear to limit any connections. My main problem right now are torrent connections slowing down my network. Not the amount of bandwidth, but the amount of connections. The Connlimit statement doesn't work because it effects everyone.

Is there something I should see in the system log that would tell me your method is dropping connections?

[knolan]

You should have something like the following in the system reports


http://www.corkcommunitybroadband.ie/p2p.gif


The important parts for connlimit to be working is "Chain P2P" where it shows pkts being rejected


Regards,
Keith

[cephlon]

I do have those lines in my system report, so it must be effecting the connections some how. I have this one customer that still has at least 50 open connections.

In your example, would I change the 5 to effect how many open connections?

iptables -A P2P -m connlimit ! --connlimit-above 5 -j ACCEPT --connlimit-mask 32

[cephlon]

Is there a way to limit the connections in V3 at the CPE?

[lonnie]

It should work in the latest release, but it is better to have those sorts of rules at the Internet edge, to keep the connection attempts off your wireless.

[cephlon]

I agree Lonnie, But I am having trouble finding a solution that works on the edge.

[lonnie]

Have you tried the latest X86 release?

[cephlon]

I'm still using V2 (build 4759) on my edge router. I haven't upgraded because my customer management program communicates directly with my Router using starutil. Does the latest x86 release support all the starutil commands?

[lonnie]

I'm not sure. Tell me what features you are using. You could also create another Server using V3 and test it out to see if it does what you want then simply swap it.

[DrLove73]

Associations and link stats do not work via starutil (1.2.9b), but was told to use SNMP for them, because they wont work any more(?):(. All others do.

[DrLove73]

iptables -A P2P -m connlimit ! --connlimit-above 5 -j ACCEPT --connlimit-mask 32


How do you set P2P chain and where? Shouldn't it be FORWARD (or PREROUTING or POSTROUTING) insted of P2P?

EDIT: I mised post with P2P chain, I now understand.

[tony]

That is correct, the starutil association information is not available because it has been superseded by the snmp version, that is much more informative.

[ninedd]

Hi. So to clarify this for my small, tired brain... Is this the correct line for a version 1.3.23b or 1.4.4b server:

iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 100 -j DROP

And would that limit each customer on our network to 100 connection, or would that limit the total on the whole network to 100 collectively? I know that's probably a stupid question, and I understand it to be 100 each - but I'm unclear if on a P4 in our shop, if this firewall line will apply to the whole network collectively (bad) or to each customer IP. Thanx.

[DrLove73]

I am in a hurry, but if I remember correctly, that is for all customers. Foe each single customer it is something with "/32"?
STF.

[ninedd]

I have STF and am still unclear, which is why I ask.

This is the output I get in the System Report

│---[ MANGLE chains ]----------------------------------------
│Chain PREROUTING (policy ACCEPT 33971993 packets, 19316472999 bytes)
│ pkts bytes target prot opt in out source destination
│ 124159 21251847 DROP tcp -- * * 0.0.0.0/0 0.0.0.0/0 #conn/32 > 100

[lonnie]

This is a fairly general rule, and you can probably apply it to many other things -->

If you do not see any sort of number or name that ties a rule to a particular subnet or IP then the rule applies globally. If a rule is not specific then it applies to the machine in general.

Your above rule does not even have a device, so it it is the whole machine with 100 connections.

You can always search with Google for iptables. It is one of the great things about using standard Linux tools.

[ninedd]

OK, I was trying to follow the examples in this thread.cephlon said : In your example, would I change the 5 to effect how many open connections?

iptables -A P2P -m connlimit ! --connlimit-above 5 -j ACCEPT --connlimit-mask 32And he then said later on...cephlon said : Is there a way to limit the connections in V3 at the CPE?Since he was now asking about a way to limit at the CPE, I took this to mean that the previous examples in the thread were about limiting connections at the edge or the AP.
lonnie answered : It should work in the latest release, but it is better to have those sorts of rules at the Internet edge, to keep the connection attempts off your wireless.So, I again understood that the previous discussion was about limiting at the edge - which I agree - that's what I'm trying to do as well. We already CBQ on the AP and at the CPE, but I'm wanting to add rules at the edge to help out. Those rules would have to affect connections each IP individually, and not a shared pool of connections for the whole customer network.

[ninedd]

For me, the line...

iptables -A P2P -m connlimit ! --connlimit-above 100 -j ACCEPT --connlimit-mask 32

doesn't seem to do anything - at least I don't seem to see anything change in the System Report.

The line...

iptables -A PREROUTING -t mangle -p tcp -m connlimit --connlimit-above 100 -j DROP

does output that it's dropping packets & bytes, but I'm not sure that's dropping the right packet's (I would want to limit just those customers who open more than 100 connections at the same time). I don't understand how it would know to consider this 100 connections per client, instead of 100 connections overall.

[DrLove73]

--connlimit-mask 32 is what I remembered.


netfilter/iptables - Patch-o-Matic Listing - external

patch-o-matic external repository

connlimit - iptables connlimit match

Author: Gerd Knorr <kraxel@bytesex.org> (kraxel@bytesex.org)
Status: ItWorksForMe[tm]

This adds an iptables match which allows you to restrict the
number of parallel TCP connections to a server per client IP address
(or address block).

Examples:

# allow 2 telnet connections per client host
iptables -p tcp --syn --dport 23 -m connlimit --connlimit-above 2 -j REJECT

# you can also match the other way around:
iptables -p tcp --syn --dport 23 -m connlimit ! --connlimit-above 2 -j ACCEPT

# limit the nr of parallel http requests to 16 per class C sized
# network (24 bit netmask)
iptables -p tcp --syn --dport 80 -m connlimit --connlimit-above 16 \
--connlimit-mask 24 -j REJECTTake also look at this:

http://www.archivum.info/netfilter/2005-05/msg00333.html

and

http://www.sadikhov.com/forum/lofiversion/index.php?t50239.html

[ninedd]

--connlimit-mask 32
is what I remembered.OK, thank you. That mask is the default I guess, so that would be optional for what I'm trying to accomplish. Maybe at the end of the day I'll just have to remortgage my house and buy a NetEqualizer. Seems a shame to spend $7000 just because I can't figure it out though. :)

[Stratolinks]

The rule we use is as follows:

iptables -A FORWARD -t mangle -s xxx.xxx.xxx.xxx/32 -p tcp -m connlimit --connlimit-above 100 -j MARK --set-mark 110replace the xxx.xxx.xxx.xxx with your IP address. Then there is a CBQ rule that restricts the speed of all packets with that mark.

pipe 20 bw 128k
pipe 21 bw 1024k
shape all to pipe 20 from mark 110 out via ether2
shape all to pipe 21 from mark 110 in via ether2We actually generate a separate rule for every IP address so that we can take a a look at the system report and see who is actually hitting these limits. Typically it is less than 4% of the customers.

OK, thank you. That mask is the default I guess, so that would be optional for what I'm trying to accomplish. Maybe at the end of the day I'll just have to remortgage my house and buy a NetEqualizer. Seems a shame to spend $7000 just because I can't figure it out though. :)

Netequalizer does more than this by a long shot. Netequalizer is a packaged solution using the free Bandwidth Abitrator software. So if you are up to building your own high powered machine, and can do the config your self you should be able to do the same thing as Netequalizer for a fair bit less money.

[go.fast]

free Abitrator software

Thanks for that info.

Guess what it comes down to is, do you want to buy a packaged solution that cost $7k or do you or can you do it for less.

[c.davis]

I grabbed a copy of the GPL'd bandwidth arbitrator software today.. just to see what it was all about. In a nutshell, they provide a patch for the 2.6.5 linux kernel bridge code (their entire system relies on bridging to deal with packets between interfaces).. the rest is a set of perl scripts.

The biggest differences from their commercial offering compared to the free version (aside from the hardware itself) is a couple of nice things like a web front end and some rrd graphs.. the configuration is very well documented and in reality should be actually fairly trivial to setup and get going.

If you'd really like to have your own netequalizer-like system then my advice would be to save yourself a bunch of cash and roll-your-own.. if you and/or your staff aren't up to the task then you can still save a big fistfull by setting up a very basic linux box with a build enviroment and grabbing a 'coder for hire', for a less than a thousand bucks the coder-for-hire guy(s) could do a 2.6.5 kernel build and make a web front-end tailored to your needs rather than you and your staff having to learn what's provided.

Just my 0.02

[ninedd]

OK, so if I read these two rules correctly - they mark only those customers with more than 100 connections, and then shape those customers through their own rule - but only while they are over 100 connections, right? I was hoping not to have to code 1,000 rules for 1,000 customers - but if we need to, then we need to. :) Ideally, a server could see the IP's in active use and dynamically creates 'rules' for each IP. Of course, that's pretty much what NetEqualizer or Arbitrator does - it sees the traffic to each IP and dynamically decides who needs to be slowed, and then slows those hogs by adding a small amount of latency into that IP's connection.
Netequalizer does more than this by a long shot. Netequalizer is a packaged solution using the free Bandwidth Abitrator software. So if you are up to building your own high powered machine, and can do the config your self you should be able to do the same thing as Netequalizer for a fair bit less money.Hi. I was more whining for commedic effect. :D Yes, we looked at Arbitrator option as well. Our Linux skills are limited and for obvious (and understandable) reasons, all our requests from the Bandwidth Arbitrator Open Source guys are answered with "You know, we make a product called NetEqualizer that sounds perfect for your knowledge level..." ;) Very understandable from their standpoint - they don't want to spend the time & money to make a comercial turn-key version, and then help people circumvent their business. We're going to have another stab at it in the new year here, but I was hoping that we might be able to have our StarOS edge router help smooth things out for us. As I mentioned, we already CBQ at the AP, and we CBQ at the CPE - but we also want to do something at the edge that can better shape the whole network when it get's busy, since the distributed CBQing has a drawback in that each AP/CPE doesn't really know what the whole rest of the network is up to at that instant.

[ninedd]

...a 2.6.5 kernel build and make a web front-end tailored to your needs rather than you and your staff having to learn what's provided. Just my 0.02Well, that's about 4c worth of advice I think! ;) I do have a couple coders that I subcontract some website stuff to - they are PHP/MySQL guys, so I'll check on their Perl skills. That's been the way we've been leaning anyway, although it runs the risk of spending the money and having a 'sort of' working solution. :( But, that's probably the way for us to continue moving. Thanx.

[c.davis]

Well, that's about 4c worth of advice I think! ;) I do have a couple coders that I subcontract some website stuff to - they are PHP/MySQL guys, so I'll check on their Perl skills. That's been the way we've been leaning anyway, although it runs the risk of spending the money and having a 'sort of' working solution. :( But, that's probably the way for us to continue moving. Thanx.

Here's your 3rd and 4th cent... :D
I'd leave those perl scripts alone unless you really know what you're doing and have a desire to delve right in there, they are very purpose driven and really need no modification.. in fact there are static calls in some of the bridge patches to those perl scripts.. hand off is the best policy. Your best best is to get those coders-for-hire guys to evaluate the config file and have them rewrite it (the config) as changes are made.. for reporting.. have them parse the output from the perl scripts to your liking and when you're in that comfy spot.. the world is your oyster.... send the results to a sql database and go buck-wild with it.

As a side note, be aware of the cost of some coders-for-hire.. make sure that you own the code when the job is done as there are some that we reselll code that has already been paid for unless instructed otherwise.

Ok, I spent 2 more cents than planned and I'm out of coins :D