mentby.com
Blog | Jobs | Help | Signup | Login

haproxy bug or by design ? acl path_beg overriding stick-table rate limiting ?



Reading up on 2 blog posts regarding DDOS protection via haproxy stick-tbles

http://blog.serverfault.com/2010/08/26/1016491873/ http://blog.exceliance.fr/2011/08/25/protect-apache-against-[..]

Problem

i tried testing this on my local test server with haproxy v1.5 dev7. Butit seems if I have an acl rule in place path_beg /forum/images/ and /imaes/ it bypasses the stick-tables completely. It doesn't seem to matter i the acl rule is placed before or after the stick-table entry on the frotend www.

If I remove the acl rule for path_beg, then stick-table works !

the acl path_beg rule i have is these 3 lines in frontend www

        acl imagepath path_beg /images/

        acl imagepathforums path_beg /forums/images/

        use_backend imagepath_backend if imagepath or imagepathforums

Problem explained

1. If these acl path_beg rules are enabled then phpinfo.php gets rate liited with 503 status error but i.png allows full unrestricted connectionspeed to server bypassing the stick-table completely.

2. If I remove the path_beg 3 lines outlined above from haproxy config fle, then both siege tests urls for phpinfo.php and i.png I test against O NOT allow full unrestricted connection to server and are properly ratelimited with 503 status error.

siege -b -c 100 -r 100  http://192.168.56.120/phpinfo.php

siege -b -c 100 -r 100  http://192.168.56.120/forums/images/i.png

So why does the acl path_beg rule bypass the stick-table ?

Config

full frontend www looks like this

##############

# front end options

        frontend www

        mode http

        bind *:80

        default_backend www_backend

        option contstats

        acl spiderbots hdr_sub(user-agent) -i -f /etc/haproxy/spiderbotlst.lst

        use_backend spider_backend if spiderbots

  ### Setup stick table ###

  stick-table type ip size 10k expire 60s store gpc0

  # Configure the DoS src

  acl MARKED src_get_gpc0(www) gt 0

  # tarpit attackers if src_DoS

  use_backend backend_tarpit if MARKED

  # If not blocked, track the connection

  tcp-request connection track-sc1 src if ! MARKED

  ### Setup stick table ###

        acl imagepath path_beg /images/

        acl imagepathforums path_beg /forums/images/

        use_backend imagepath_backend if imagepath or imagepathforums

the backend looks like this

##############

# backend options

        backend www_backend

        mode http

        balance roundrobin # Load Balancing algorithm

        cookie SERVERID insert indirect nocache

        option httpchk OPTION /health_check.php

        option forwardfor # This sets X-Forwarded-For

  ### Setup stick table ###

  # Table to track connection rate

  stick-table type ip size 10k expire 60s store conn_rate(5s),bytes_out_ate(5s)

  # Track request

  tcp-request content track-sc2 src

  # Mark as abuser if more than 10 connection

  acl ABUSER sc2_conn_rate gt 10

  acl DATARATE_ABUSER sc2_bytes_out_rate gt 200

  acl MARKED_AS_ABUSER sc1_inc_gpc0 gt 0

  # Block connection concidered as abuser

  tcp-request content reject if ABUSER DATARATE_ABUSER MARKED_AS_ABUSER

  ### Setup stick table ###

        ## Define your servers to balance

        server server1 192.168.56.101:80 weight 1 maxconn 4096 cookie sv1 check

        server server2 192.168.56.120:82 weight 1 maxconn 4096 cookie sv2 check

        backend spider_backend

        mode http

        balance roundrobin

        cookie SERVERID insert indirect nocache

        option httpchk OPTION /health_check.php

        option forwardfor

        acl too_fast be_sess_rate gt 10

        acl too_many be_conn gt 10

        tcp-request inspect-delay 3000ms

        tcp-request content accept if ! too_fast or ! too_many

        tcp-request content accept if WAIT_END

        server server1 192.168.56.101:80 weight 1 minconn 1 maxconn 10 cokie svr1bot check

        server server2 192.168.56.120:82 weight 1 minconn 1 maxconn 10 cokie svr2bot check

        backend imagepath_backend

        mode http

        balance roundrobin

        cookie SERVERID insert indirect nocache

        option httpchk OPTION /health_check.php

        option forwardfor

  ### Setup stick table ###

  # Table to track connection rate

  stick-table type ip size 10k expire 60s store conn_rate(5s),bytes_out_ate(5s)

  # Track request

  tcp-request content track-sc2 src

  # Mark as abuser if more than 10 connection

  acl ABUSER sc2_conn_rate gt 10

  acl DATARATE_ABUSER sc2_bytes_out_rate gt 200

  acl MARKED_AS_ABUSER sc1_inc_gpc0

  # Block connection concidered as abuser

  tcp-request content reject if ABUSER DATARATE_ABUSER MARKED_AS_ABUSER

  ### Setup stick table ###

        server server1 192.168.56.101:80 weight 1 maxconn 500 cookie svr check

        server server2 192.168.56.120:82 weight 1 maxconn 500 cookie svr check

        backend backend_tarpit

        mode http

        # hold the connection for 10s before answering

        timeout tarpit 10s

        # Emulate a 503 error

        errorfile 500 /etc/haproxy/503.html

        # slowdown any request coming up to here

        reqitarpit .

---
posted at  http://www.serverphorums.com http://www.serverphorums.com /read.php?10,447870,447870#msg-447870


haproxy Fri, 17 Feb 2012 14:30:28 -0800

Hi,

In both cases, what does the log lines says about the backend used?

I thought it could be related to the functions available depending on
the network level.
I mean that the acl path* are layer 7 functions while the tcp-request
content is layer 4 only.
When you enable the acl you moved up to layer 7, then the layer 4
functions are not available anymore or the result is undeterminated.

But if this flow is correct: http://haproxy.1wt.eu/download/1.3/doc/acl.pdf

then it may be a bug.

Only Willy could answer accurately on this question, but he is a bit
busy this last days.

cheers


Bedis 9 Fri, 17 Feb 2012 23:23:45 -0800

You mean haproxy logs or backend web server (apache/nginx logs) ?

---
posted at  http://www.serverphorums.com http://www.serverphorums.com /read.php?10,447870,447964#msg-447964


haproxy Sat, 18 Feb 2012 02:24:16 -0800

haproxy logs :)


Bedis 9 Sat, 18 Feb 2012 03:14:46 -0800

)éÚnW–ˆ Šx"ž©®Œrnë^½é梷œ¢wî±ç


haproxy Sat, 18 Feb 2012 03:53:40 -0800

Hi,

Le 18/02/2012 12:52, haproxy*******a écrit :

I don't understand.
I've played with the configuration you provided (without any defaults
section btw), and couldn't reproduce your issue : in both cases,
requests are rate limited.

Are you alone during your tests or is there huge traffic with a lot of IPs ?

--
Cyril Bonté


Cyril Bonté Sat, 18 Feb 2012 04:11:19 -0800

alone, it's a private local virtualbox test server

---
posted at  http://www.serverphorums.com http://www.serverphorums.com /read.php?10,447870,447977#msg-447977


haproxy Sat, 18 Feb 2012 04:30:16 -0800

looks like problem was with forward slashes match

acl imagepath path_beg /images/

acl imagepathforums path_beg /forums/images/

use_backend imagepath_backend if imagepath or imagepathforums

doesn't work

acl imagepath path_beg /images

acl imagepathforums path_beg /forums/images

use_backend imagepath_backend if imagepath or imagepathforums

works

---
posted at  http://www.serverphorums.com http://www.serverphorums.com /read.php?10,447870,448080#msg-448080


haproxy Sat, 18 Feb 2012 17:52:45 -0800



Post a Comment