Any update on this? Was my debugging information helpful? Any status update at all would be greatly appreciated.<br clear="all"><br>Dan<br>
<br><br><div class="gmail_quote">2009/2/24 Amar (ಅಮರ್ ತುಂಬಳ್ಳಿ) <span dir="ltr"><<a href="mailto:amarts@gmail.com">amarts@gmail.com</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Thanks for this data, Dan and Gorman. <br><br>This should help us a lot.<br><br>Regards,<br>Amar<br><br><div class="gmail_quote">2009/2/24 Gordan Bobic <span dir="ltr"><<a href="mailto:gordan@bobich.net" target="_blank">gordan@bobich.net</a>></span><div>
<div></div><div class="h5"><br>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex">Just after mounting:<br>
<br>
top:<br>
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br>
4605 root 20 0 51348 2368 1296 S 0 0.1 0:00.08 glusterfs<br>
<br>
# kill -s SIGUSR1 4605<br>
Arena 0:<br>
system bytes = 479232<br>
in use bytes = 258832<br>
Total (incl. mmap):<br>
system bytes = 1466368<br>
in use bytes = 1245968<br>
max mmap regions = 5<br>
max mmap bytes = 1122304<br>
<br>
After compiling the kernel:<br>
# make clean; make dep; make -j32 all<br>
<br>
top:<br>
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br>
4605 root 20 0 202m 97m 1316 S 0 2.5 2:42.09 glusterfs<br>
<br>
# kill -s SIGUSR1 4605<br>
Arena 0:<br>
system bytes = 26423296<br>
in use bytes = 25972704<br>
Arena 1:<br>
system bytes = 74113024<br>
in use bytes = 73516560<br>
Total (incl. mmap):<br>
system bytes = 101523456<br>
in use bytes = 100476400<br>
max mmap regions = 5<br>
max mmap bytes = 1122304<br>
<br>
# ./pmap 4605<br>
glusterfs(4605)<br>
00400000 (24 KB) r-xp (09:02 442904105) /usr/sbin/glusterfsd<br>
00606000 (4 KB) rw-p (09:02 442904105) /usr/sbin/glusterfsd<br>
00607000 (25804 KB) rw-p (00:00 0) [heap]<br>
40000000 (4 KB) ---p (00:00 0)<br>
40001000 (8192 KB) rw-p (00:00 0)<br>
40801000 (4 KB) ---p (00:00 0)<br>
40802000 (8192 KB) rw-p (00:00 0)<br>
3a38600000 (176 KB) r-xp (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3862c000 (2048 KB) ---p (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3882c000 (4 KB) rw-p (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3882d000 (72 KB) rw-p (00:00 0)<br>
3d4c800000 (104 KB) r-xp (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4ca1a000 (4 KB) r--p (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4ca1b000 (4 KB) rw-p (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4cc00000 (1320 KB) r-xp (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cd4a000 (2048 KB) ---p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4a000 (16 KB) r--p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4e000 (4 KB) rw-p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4f000 (20 KB) rw-p (00:00 0)<br>
3d4d000000 (8 KB) r-xp (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d002000 (2048 KB) ---p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d202000 (4 KB) r--p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d203000 (4 KB) rw-p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4dc00000 (84 KB) r-xp (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4dc15000 (2044 KB) ---p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de14000 (4 KB) r--p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de15000 (4 KB) rw-p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de16000 (16 KB) rw-p (00:00 0)<br>
3d4e400000 (28 KB) r-xp (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e407000 (2048 KB) ---p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e607000 (4 KB) r--p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e608000 (4 KB) rw-p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
2aaaaaaab000 (28 KB) r-xp (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaaab2000 (2044 KB) ---p (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaacb1000 (8 KB) rw-p (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaacb3000 (576 KB) rw-p (00:00 0)<br>
2aaaaad43000 (8 KB) r-xp (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaad45000 (2044 KB) ---p (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaaf44000 (4 KB) rw-p (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaaf4e000 (40 KB) r-xp (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaaaf58000 (2044 KB) ---p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaab157000 (4 KB) r--p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaab158000 (4 KB) rw-p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaac000000 (72380 KB) rw-p (00:00 0)<br>
2aaab06af000 (58692 KB) ---p (00:00 0)<br>
2b4ac185b000 (8 KB) rw-p (00:00 0)<br>
2b4ac1866000 (404 KB) rw-p (00:00 0)<br>
2b4ac18cb000 (104 KB) r-xp (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac18e5000 (2048 KB) ---p (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac1ae5000 (8 KB) rw-p (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac1ae7000 (64 KB) r-xp (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1af7000 (2044 KB) ---p (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1cf6000 (8 KB) rw-p (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1cf8000 (28 KB) r-xp (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1cff000 (2048 KB) ---p (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1eff000 (4 KB) rw-p (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1f00000 (108 KB) r-xp (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac1f1b000 (2044 KB) ---p (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac211a000 (8 KB) rw-p (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac211c000 (176 KB) r-xp (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac2148000 (2048 KB) ---p (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac2348000 (12 KB) rw-p (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac234b000 (56 KB) r-xp (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac2359000 (2044 KB) ---p (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac2558000 (8 KB) rw-p (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac255a000 (116 KB) r-xp (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
2b4ac2577000 (2048 KB) ---p (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
2b4ac2777000 (8 KB) rw-p (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
7fffe923a000 (84 KB) rw-p (00:00 0) [stack]<br>
7fffe93fe000 (8 KB) r-xp (00:00 0) [vdso]<br>
ffffffffff600000 (4 KB) r-xp (00:00 0) [vsyscall]<br>
mapped: 207760 KB writable/private: 115848 KB shared: 0 KB<br>
<br>
And again:<br>
# make clean; make dep; make -j32 all<br>
<br>
top:<br>
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND <br>
4605 root 20 0 266m 192m 1316 S 0 4.9 5:26.57 glusterfs<br>
<br>
# kill -s SIGUSR1 4605<br>
<br>
Arena 0:<br>
system bytes = 26423296<br>
in use bytes = 25972784<br>
Arena 1:<br>
system bytes = 173174784<br>
in use bytes = 172023232<br>
Total (incl. mmap):<br>
system bytes = 200585216<br>
in use bytes = 198983152<br>
max mmap regions = 5<br>
max mmap bytes = 1122304<br>
<br>
# ./pmap 4605<br>
glusterfs(4605)<br>
00400000 (24 KB) r-xp (09:02 442904105) /usr/sbin/glusterfsd<br>
00606000 (4 KB) rw-p (09:02 442904105) /usr/sbin/glusterfsd<br>
00607000 (25804 KB) rw-p (00:00 0) [heap]<br>
40000000 (4 KB) ---p (00:00 0)<br>
40001000 (8192 KB) rw-p (00:00 0)<br>
40801000 (4 KB) ---p (00:00 0)<br>
40802000 (8192 KB) rw-p (00:00 0)<br>
3a38600000 (176 KB) r-xp (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3862c000 (2048 KB) ---p (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3882c000 (4 KB) rw-p (09:02 442904212) /usr/lib64/libglusterfs.so.0.0.0<br>
3a3882d000 (72 KB) rw-p (00:00 0)<br>
3d4c800000 (104 KB) r-xp (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4ca1a000 (4 KB) r--p (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4ca1b000 (4 KB) rw-p (09:02 1194229765) /lib64/<a href="http://ld-2.5.so" target="_blank">ld-2.5.so</a><br>
3d4cc00000 (1320 KB) r-xp (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cd4a000 (2048 KB) ---p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4a000 (16 KB) r--p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4e000 (4 KB) rw-p (09:02 1194229772) /lib64/<a href="http://libc-2.5.so" target="_blank">libc-2.5.so</a><br>
3d4cf4f000 (20 KB) rw-p (00:00 0)<br>
3d4d000000 (8 KB) r-xp (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d002000 (2048 KB) ---p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d202000 (4 KB) r--p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4d203000 (4 KB) rw-p (09:02 1194229776) /lib64/<a href="http://libdl-2.5.so" target="_blank">libdl-2.5.so</a><br>
3d4dc00000 (84 KB) r-xp (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4dc15000 (2044 KB) ---p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de14000 (4 KB) r--p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de15000 (4 KB) rw-p (09:02 1194229786) /lib64/<a href="http://libpthread-2.5.so" target="_blank">libpthread-2.5.so</a><br>
3d4de16000 (16 KB) rw-p (00:00 0)<br>
3d4e400000 (28 KB) r-xp (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e407000 (2048 KB) ---p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e607000 (4 KB) r--p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
3d4e608000 (4 KB) rw-p (09:02 1194229788) /lib64/<a href="http://librt-2.5.so" target="_blank">librt-2.5.so</a><br>
2aaaaaaab000 (28 KB) r-xp (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaaab2000 (2044 KB) ---p (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaacb1000 (8 KB) rw-p (09:02 443222345) /usr/lib64/glusterfs/2.0.0rc1/transport/socket.so.0.0.0<br>
2aaaaacb3000 (576 KB) rw-p (00:00 0)<br>
2aaaaad43000 (8 KB) r-xp (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaad45000 (2044 KB) ---p (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaaf44000 (4 KB) rw-p (09:02 443222303) /usr/lib64/glusterfs/2.0.0rc1/auth/addr.so.0.0.0<br>
2aaaaaf4e000 (40 KB) r-xp (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaaaf58000 (2044 KB) ---p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaab157000 (4 KB) r--p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaab158000 (4 KB) rw-p (09:02 1194229787) /lib64/<a href="http://libnss_files-2.5.so" target="_blank">libnss_files-2.5.so</a><br>
2aaaac000000 (169120 KB) rw-p (00:00 0)<br>
2aaab6528000 (27488 KB) ---p (00:00 0)<br>
2b4ac185b000 (8 KB) rw-p (00:00 0)<br>
2b4ac1866000 (404 KB) rw-p (00:00 0)<br>
2b4ac18cb000 (104 KB) r-xp (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac18e5000 (2048 KB) ---p (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac1ae5000 (8 KB) rw-p (09:02 443252818) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/client.so.0.0.0<br>
2b4ac1ae7000 (64 KB) r-xp (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1af7000 (2044 KB) ---p (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1cf6000 (8 KB) rw-p (09:02 443252834) /usr/lib64/glusterfs/2.0.0rc1/xlator/storage/posix.so.0.0.0<br>
2b4ac1cf8000 (28 KB) r-xp (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1cff000 (2048 KB) ---p (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1eff000 (4 KB) rw-p (09:02 443252764) /usr/lib64/glusterfs/2.0.0rc1/xlator/features/locks.so.0.0.0<br>
2b4ac1f00000 (108 KB) r-xp (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac1f1b000 (2044 KB) ---p (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac211a000 (8 KB) rw-p (09:02 443252823) /usr/lib64/glusterfs/2.0.0rc1/xlator/protocol/server.so.0.0.0<br>
2b4ac211c000 (176 KB) r-xp (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac2148000 (2048 KB) ---p (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac2348000 (12 KB) rw-p (09:02 443222352) /usr/lib64/glusterfs/2.0.0rc1/xlator/cluster/afr.so.0.0.0<br>
2b4ac234b000 (56 KB) r-xp (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac2359000 (2044 KB) ---p (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac2558000 (8 KB) rw-p (09:02 443252786) /usr/lib64/glusterfs/2.0.0rc1/xlator/mount/fuse.so.0.0.0<br>
2b4ac255a000 (116 KB) r-xp (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
2b4ac2577000 (2048 KB) ---p (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
2b4ac2777000 (8 KB) rw-p (09:02 442904974) /usr/lib64/libfuse.so.2.7.4<br>
7fffe923a000 (84 KB) rw-p (00:00 0) [stack]<br>
7fffe93fe000 (8 KB) r-xp (00:00 0) [vdso]<br>
ffffffffff600000 (4 KB) r-xp (00:00 0) [vsyscall]<br>
mapped: 273296 KB writable/private: 212588 KB shared: 0 KB<br>
<br>
I hope this is useful.<br>
<br>
Gordan<br>
<br>
Amar Tumballi (bulde) wrote:<br>
<blockquote class="gmail_quote" style="border-left:1px solid rgb(204, 204, 204);margin:0pt 0pt 0pt 0.8ex;padding-left:1ex"><div>
Hi Gordan and Dan,<br>
<br>
It would help me a lot if its possible for you to get the info as described below,<br>
<br>
compile glusterfs like<br>
<br>
bash# make clean > /dev/null<br>
bash# make CFLAGS="-g -O0 -DDEBUG" > /dev/null<br>
bash# make install<br>
<br>
run the process which consumes memory (mostly client process) like below:<br>
<br>
bash# glusterfs <any argument you give generally> -N<br>
<this process will run in foreground now><br>
<br>
Open another terminal<br>
<br>
bash# ps aux | grep glusterfs<br>
bash# kill -s SIGUSR1 <pid of glusterfs -N process><br>
<Check in other terminal for memory usage stats><br>
<br>
bash# <run your application over glusterfs as you do till you get high memory usage of glusterfs.. ><br>
bash# kill -s SIGUSR1 <pid of glusterfs -N process><br>
<Check the stat in another terminal><br>
<br>
bash# echo 3 > /proc/sys/vm/drop_caches<br>
bash# kill -s SIGUSR1 <pid of glusterfs -N process><br>
<Check the stat in another terminal><br>
<br>
Even after dropping caches, if 'in use bytes =' in malloc stats shows a high value, then it is a leak. If its showing less, but just 'system bytes = ' is a high value, this means glusterfs is not really consuming high memory, but the problem is really in the memory allocation segments.<br>
<br>
Regards,<br>
Amar<br>
<br>
NOTE: 'malloc_stats' will be printed to 'stdout' if we enable -DDEBUG while compiling glusterfs, as it hits performance badly otherwise.<br>
<br>
<br></div>
2009/2/23 Gordan Bobic <<a href="mailto:gordan@bobich.net" target="_blank">gordan@bobich.net</a> <mailto:<a href="mailto:gordan@bobich.net" target="_blank">gordan@bobich.net</a>>><div><br>
<br>
Dan Parsons wrote:<br>
<br>
I'm having an issue with glusterfs exceeding its cache-size<br>
value. Right now I have it set to 4000MB and I've seen it climb<br>
as high as 4800MB. If I set it to 5000, I've seen it go as high<br>
as 6000MB. This is a problem because it causes me to set the<br>
value very low so that my apps don't get pushed into swap. Is<br>
there any way to fix this? To get it to stick to the limit I set<br>
and not exceed?<br>
<br>
<br>
It's possible you are running into the same memory leak that I'm<br>
seeing, and I'm not using io-cache or any other performance<br>
translators at all. With rootfs on Gluster, doing a kernel compile<br>
(kernel source tree being on NFS, so this won't be contributing to<br>
the bloat, hopefully) makes the glusterfsd bloat by about 80MB per<br>
pass, and never frees it.<br>
<br>
Gordan<br>
<br>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br></div>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a> <mailto:<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a>><div><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br>
<br>
<br>
<br>
-- <br>
Amar Tumballi<br>
Gluster/GlusterFS Hacker<br></div>
[bulde on #gluster/<a href="http://irc.gnu.org" target="_blank">irc.gnu.org</a> <<a href="http://irc.gnu.org" target="_blank">http://irc.gnu.org</a>>]<div><br>
<a href="http://www.zresearch.com" target="_blank">http://www.zresearch.com</a> - Commoditizing Super Storage!<br>
</div></blockquote>
<br>
<br><div><div></div><div>
<br>
_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org" target="_blank">Gluster-devel@nongnu.org</a><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
</div></div></blockquote></div></div></div><div><div></div><div class="h5"><br><br clear="all"><br>-- <br>Amar Tumballi<br>Gluster/GlusterFS Hacker<br>[bulde on #gluster/<a href="http://irc.gnu.org" target="_blank">irc.gnu.org</a>]<br>
</div></div><div><div></div><div class="h5"><a href="http://www.zresearch.com" target="_blank">http://www.zresearch.com</a> - Commoditizing Super Storage!<br>
</div></div><br>_______________________________________________<br>
Gluster-devel mailing list<br>
<a href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a><br>
<a href="http://lists.nongnu.org/mailman/listinfo/gluster-devel" target="_blank">http://lists.nongnu.org/mailman/listinfo/gluster-devel</a><br>
<br></blockquote></div><br>