Memory leak in the NET-SNMP snmpd daemon?

[brthb]

I am currently using a RUT241 router, which is running RUT2M_R_00.07.05.4. I have installed the SNMP package and I’m pulling statistics from the router into Prometheus/Grafana every minute. However, I have noticed that memory usage is slowly increasing over time. After three days, the memory usage has reached 28 MB. But, after restarting the SNMP daemon, it goes back down to around 8MB.
The package contains NET-SNMP version 5.9.1, which is three minor versions behind the current version. Is it possible to run/install a newer version of net-snmp?

Thanks, Brendan

This is a graph of the memory usage over the last three days. The drop at the end is when I restarted the daemon.

Mem usage before: (after restarting the daemon)

root@router:/bin#  cat /proc/6357/status 
Name:	snmpd
Umask:	0022
<cut>
VmPeak:	    8448 kB
VmSize:	    8448 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	    4700 kB
VmRSS:	    4700 kB

Mem usage after 3 days:

root@router:/bin#  cat /proc/6425/status 
Name:	snmpd
<cut>
VmPeak:	   27912 kB
VmSize:	   27912 kB
VmLck:	       0 kB
VmPin:	       0 kB
VmHWM:	   24096 kB
VmRSS:	   24096 kB

[flebourse]

Hello,
Which variables do you poll ?
I have the same 5.9.1 snmpd, it is perfectly stable after 95 days.
Regards,

[brthb]

I use the Prometheus snmp_exporter.

    walk:
    - 1.3.6.1.2.1.2
    - 1.3.6.1.2.1.25.1
    - 1.3.6.1.2.1.25.2
    - 1.3.6.1.2.1.31.1.1
    - 1.3.6.1.4.1.2021.10
    - 1.3.6.1.4.1.2021.11
    - 1.3.6.1.4.1.2021.4
    - 1.3.6.1.4.1.48690

[flebourse]

Looks like one library leaks when used.
One way to determine which one is to perform a snmpwalk loop while checking the memory footprint of snmpd, and try to narrow the walk to identify the culprit. For example:

while :
do
    snmpwalk -v 2c -c public 192.168.1.1 .1.3.6.1.4.1.48690
done

then use .1.3.6.1.4.1.48690.1 and so on.

[flebourse]

Whoa that was easy, looping on .1.3.6.1.4.1.48690.6 and .1.3.6.1.4.1.48690.7 both trigger a serious memory leak.

[brthb]

Thanks for the debugging tip. For now, until this is fixed, I limited the scrapping to the deviceGroup and modemGroup as that’s all I needed for my dashboard.

- 1.3.6.1.4.1.48690.1 #Teltonika deviceGroup
- 1.3.6.1.4.1.48690.2 #Teltonika modemGroup

Again thanks! Brendan

[AndzejJ]

Hello,

Thank you for bringing this issue to our attention. I have forwarded it to our RnD department for further investigation.

Kind Regards,

[brthb]

I used this bash script to replicate the problem:

#!/usr/bin/bash

APP=snmpbulkwalk

OIDS=$(cat <<EOF
.1.3.6.1.4.1.48690.1 
.1.3.6.1.4.1.48690.2 
.1.3.6.1.4.1.48690.6 
.1.3.6.1.4.1.48690.7 
.1.3.6.1.4.1.48690.8 
.1.3.6.1.4.1.48690.10 
EOF
)

echo ${OIDS}

while :
do
	for OID in ${OIDS}
	do
	    ${APP} -v 2c -c public <host> ${OID}
	done

done