Calculate DDOS on the server
Sometimes you sit like that, you don’t touch anyone, but here they call you and say that services are slow, sites open for 2-3 minutes and they manage to give out 504 errors.
You get upset in cacti, and there it is:
Then you start frantically searching for the cause of all this, and it turns out that this is the usual DDos ...
Below are the commands that will help you understand what happened, and whether this is exactly DDos.
By whichteams and what can we define?
First you can see the number of running Apache processes. If there are more than 20-30, then something is obviously wrong.
Consider the number of Apache processes in Debian:
ps aux | grep apache | wc -l
Consider the number of Apache processes on CentOS:
ps aux | grep httpd | wc -l
With this command we can see the number of server connections:
cat / proc / net / ip_conntrack | wc -l
It also indicates that the number of connections per port 80 or 443 can serve as a connection to the server. Here are the commands that can show this number:
netstat -na | grep: 80 | wc -l
netstat -na | grep: 443 | wc -l
There is still a kind of DDod, like SYN. Below is a command to determine the number of SYN requests for the same 80 and 443 ports:
netstat -na | grep: 80 | grep SYN | sort -u | more
netstat -na | grep: 443 | grep SYN | sort -u | more
And this command shows the number of SYN requests:
netstat -n -t | grep SYN_RECV | wc -l
The following command will allow us to understand which domain most requests go to:
tcpdump -npi eth0 port domain
Now let's see how many requests come from each IP. This command shows all ports:
netstat -ntu | awk '{print $ 5}' | cut -d: -f1 | sort | uniq -c | sort -nr | more
similar commands:
netstat -anp | grep 'tcp \ | udp' | awk '{print $ 5}' | cut -d: -f1 | sort | uniq -c | sort -n
netstat -antu | awk '$ 5 ~ / [0-9]: / {split ($ 5, a, ":"); ips [a [1]] ++} END {for (ip in ips) print ips [ip], ip | "sort -k1 -nr"} '
This command shows the number of requests on port 80 only:
netstat -ntu | grep ": 80 \" | awk '{print $ 5}' | cut -d: -f1 | sort | uniq -c | sort -nr | more
This command shows all requests for port 80, not counting them, i.e. “Simplified” but “most complete” output:
netstat -na | grep: 80 | sort | uniq -c | sort -nr | more
By calculating the most active IP, you can also look at which ports are queried from it. Here, for example, IP 127.0.0.1 is substituted:
netstat -na | grep 127.0.0.1
By the way, if you have not configured server-status on Apache, then the status of this server can be viewed in the CLI:
apachectl status
Llog files
Naturally, it is recommended to watch the log files of your Apache and Nginx server (if any).
Apache's global logs, in Debian, are usually found there:
- /var/log/apache2/error.log
- /var/log/apache2/access.log
In CentOS:
- /var/log/httpd/error.log
- /var/log/httpd/access.log
Nginx global logs are there:
/var/log/nginx/error.log
/var/log/nginx/access.log
/var/log/nginx/access.log
Just do not forget to view the logs of virtual hosts, if you have configured hosts. We will be interested in the largest log that is "growing" before our eyes.
Look for these logs need anomalies, namely the same type of requests without user agents (or with the same), a large number of requests from the same IP, requests without specifying a virtual host, etc.
You can identify specific IP with the number of requests to the site with this command:
cat access.log | awk '{print $ 1}' | sort | uniq -c
You can also get statistics on queries grouped by IP using the logtop utility .
First, install this utility:
apt-get install git libncurses5-dev uthash-dev gcc
# in case you don’t have the packages for
GITgit clone to work correctly https://github.com/JulienPalard/logtop.git
And now we get the statistics:
tail -f access.log | awk {'print $ 1; fflush (); '} | logtop
The following command will help us identify popular user agents:
cat access.log | awk -F \ "'{print $ 6}' | sort | uniq -c | sort -n
KHow to block?
One way or another you should have iptables. Most likely it may not be configured, especially if you do not know what it is. Earlier, I wrote an article about how to use it: “ A brief guide on Iptables ”, so here I will bring only the necessary commands to solve the problem here and now.
Here's how to block tcp requests for port 80 from a specific IP :
iptables -A INPUT -p tcp --dport 80 -s 12.34.56.78 -j DROP
So we block requests for all ports from a specific IP :
iptables -A INPUT -s 12.34.56.78 -j DROP
We can view the list of already blocked data with commands:
iptables -l -n
or
iptables -L -n --line-numbers
If we need to remove a specific IP from the blocking , we can use this command.
iptables -D INPUT -s 1.2.3.4 -j DROP
or you can delete a rule by its number , having previously looked at its number with the command iptables -L -n —line-numbers:
iptables -D INPUT 6
To delete all rules , you can use the command:
iptables -F
HA bit of prevention, in order to protect against DDos ...
There are still some rules that can protect us from mindless bots that load the server.
The next command will set the maximum number of connections from one IP to port 80 :
iptables -A INPUT -p tcp --dport 80 -m connlimit --connlimit-above 128 -j DROPiptables -A INPUT -p tcp --dport 80 -j ACCEPT
The same can be done for DNS :
iptables -A INPUT -p udp --dport 53 -m connlimit --connlimit-above 16 -j DROPiptables -A INPUT -p udp --dport 53 -j ACCEPT
The following rule in iptables will prevent spoofing on our behalf. As a rule, during ddos, we receive a packet with the SYN and ACK flags set on a connection that is not yet open (only the response to the SYN packet has this combination of flags). This suggests that someone sent a SYN packet to another host on our behalf, and the answer came to us.
According to this rule, our host will respond with an RST packet, after receiving which the attacked host closes the connection.
iptables -I INPUT -m conntrack --ctstate NEW, INVALID -p tcp --tcp-flags SYN, ACK SYN, ACK -j REJECT --reject-with tcp-reset
You can save the rules with the following command:
iptables-save> /etc/iptables.rules
QWhat else can you do?
Still, it doesn’t hurt to “tune” the kernel a bit, to fine-tune Apache and Nginx (if you have one), to install additional modules and packages to protect against attacks, such as Fail2Ban, mod_evasive, ModSecurity ..
Commentaires