Neighbour table overflow – sysctl.conf tunning

If you have a big network with the hundreds of hosts you can expect “Neighbour table overflow” error which occurs in large networks when there are two many ARP requests which the server is not able to reply. For example you’re using server as a DHCP server, cable modems provisioning, etc.

Nov 10 03:18:17 myhost Neighbour table overflow.
Nov 10 03:18:23 myhost printk: 12 messages suppressed.

Of curse, this can be fixed. The solution is to increase the threshhold values in /etc/sysctl.conf. Add following lines to /etc/sysctl.conf (RH based distros)

net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 8192
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv4.neigh.default.base_reachable_time = 86400
net.ipv4.neigh.default.gc_stale_time = 86400

Save sysctl.conf and exec sysctl -p. You can also reboot but it isn’t necessary.

The default sysctl.conf file

net.ipv4.ip_forward=0
kernel.shmmax=68719476736
kernel.msgmax=65536
kernel.msgmnb=65536
net.ipv4.conf.default.rp_filter=1
kernel.sysrq=0
net.ipv4.conf.default.accept_source_route=0
kernel.shmall=4294967296
kernel.core_uses_pid=1
net.ipv4.tcp_syncookies=1

“Tuned” systctl.conf

net.ipv4.ip_forward=0
kernel.shmmax=4294967295
kernel.msgmax=65536
kernel.msgmnb=65536
net.ipv4.conf.default.rp_filter=1
kernel.sysrq=0
net.ipv4.conf.default.accept_source_route=0
kernel.shmall=268435456
kernel.core_uses_pid=1
net.ipv4.tcp_syncookies=1
net.ipv4.neigh.default.gc_thresh1 = 4096
net.ipv4.neigh.default.gc_thresh2 = 8192
net.ipv4.neigh.default.gc_thresh3 = 8192
net.ipv4.neigh.default.base_reachable_time = 86400
net.ipv4.neigh.default.gc_stale_time = 86400

Explanation…

The neighbour table is generally known as ARP table and the default value for gc_thresh1 is 128 (Adjust where the gc will leave arp table alone)

[root@myServer ~]# cat /proc/sys/net/ipv4/neigh/default/gc_thresh1
128

which is not enough for large networks (more than 128 hosts). Thats why we need to tune this value. The gc_thresh2 is a soft limit (Tell the gc when to become aggressive with arp table cleaning.) and the gc_thresh3 is a hard limit (Don’t allow the arp table to become bigger than this).

To enlarge the ARP cache table on the live system run:

# sysctl -w net.ipv4.neigh.default.gc_thresh3=8192
# sysctl -w net.ipv4.neigh.default.gc_thresh2=8192
# sysctl -w net.ipv4.neigh.default.gc_thresh1=4096

It is possible that after distro update your systctl.conf will be replaced with the default values. Check this file periodically..

6 thoughts on “Neighbour table overflow – sysctl.conf tunning

  1. The other solution is to enable the reverse path filter (rp_filter)
    echo 1 > /proc/sys/net/ipv4/conf/all/rp_filter

  2. I had the same problem even though the arp cache contained roughly a hundred entries and net.ipv4.neigh.default.gc_thresh1 was set to 1024 and so on.
    net.ipv6.neigh.default.gc_thresh1 (ipv6!!) was still set to 128…
    Don’t forget to set the ipv6 values if your system is configured with both ipv4 and ipv6

Leave a Reply

Your email address will not be published. Required fields are marked *