Hi,<div><br></div><div>We've been experiencing repeated (every other day) oopses on hosts with high load glusterfs accesses. Co-occurrent (not immediately tho, but there is some sort of connection) are hanging nginx processes (doing the accessing), which can not be stopped, killed and also block the shutdown of the respective openvz instance. I think I remember at least one instance where this occurred without a warning-oops. The only way to make things working again is a reboot. A bit of googling led me to believe, that the problem here might be with fuse, not glusterfs. The kernel bug that covers this specific problem however is not available since the <a href="http://kernel.org" target="_blank">kernel.org</a> bugzilla is down. Also, there are no indications in gluster.log as to what this may have caused. We're running Gluster 3.2.1 from debian packages provided. The things I've tried are: using an older version of the kernel (lenny backports) than squeeze, and disabling swap (since the process that oopses is kswapd), the solution currently being tried is using nfs for high load hosts, I hope this doesn't crash the gluster server :). This mail is more of a JFYI than a bug report - perhaps somebody else has seen this problem too and can provide more insight.</div>
<div><br></div><div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.037978] CPU 3 </div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038004] Modules linked in: vzethdev vznetdev simfs vzrst vzcpt vzdquota vzmon vzdev xt_tcpudp xt_lengt</div>
<div>h xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit xt_dscp ipt_REJECT ip_tables x_tables ipmi_devintf ipmi_si ipmi_msghan</div><div>dler nfs lockd fscache nfs_acl auth_rpcgss sunrpc 8021q garp bridge stp fuse loop snd_pcm snd_timer snd soundcore snd_page_alloc psmouse dcdbas pcspkr</div>
<div> serio_raw joydev evdev power_meter button processor ext3 jbd mbcache dm_mirror dm_region_hash dm_log dm_snapshot dm_mod raid1 md_mod sd_mod crc_t10di</div><div>f sg sr_mod cdrom usbhid hid ata_generic ata_piix uhci_hcd ehci_hcd mptsas mptscsih mptbase scsi_transport_sas libata usbcore nls_base scsi_mod bnx2 thermal fan thermal_sys [last unloaded: scsi_wait_scan]</div>
</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038545] Pid: 48, comm: kswapd1 Not tainted 2.6.32-bpo.5-openvz-amd64 #1 feoktistov PowerEdge R410</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038602] RIP: 0010:[<ffffffff81102ade>] [<ffffffff81102ade>] clear_inode+0x1b/0xd0</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038665] RSP: 0018:ffff88083c919c30 EFLAGS: 00010202</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038697] RAX: 0000000000000000 RBX: ffff8805d8f9b000 RCX: ffff88083c919c90</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038734] RDX: 0000000000000000 RSI: ffffea0016f46000 RDI: ffff8805d8f9b000</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038770] RBP: 0000000000000000 R08: ffffffffffffffc0 R09: 0000000000000000</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038806] R10: 0000000000000040 R11: 0000000000000002 R12: dead000000100100</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038842] R13: ffff88083c919c90 R14: ffff8804314d3cf0 R15: ffff88083c919d24</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038879] FS: 0000000000000000(0000) GS:ffff880011a20000(0000) knlGS:0000000000000000</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038933] CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.038966] CR2: 00007fe22466054c CR3: 0000000001001000 CR4: 00000000000006e0</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039003] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039039] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039076] Process kswapd1 (pid: 48, veid=0, threadinfo ffff88083c918000, task ffff88043d082000)</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039155] ffff8805d8f9b000 ffffffff81103568 ffff8805d0213be0 ffff8805d0213bd8</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039199] <0> ffff8805d0213be0 ffffffff810ff58b 0000000000000100 ffff8805d0213bd8</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039263] <0> ffff8804314d3c00 ffffffff810ff820 0000000000000000 0000000000000008</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039375] [<ffffffff81103568>] ? generic_delete_inode+0xec/0x160</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039411] [<ffffffff810ff58b>] ? d_kill+0x40/0x61</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039443] [<ffffffff810ff820>] ? __shrink_dcache_sb+0x274/0x30b</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039479] [<ffffffff810ff9bc>] ? shrink_dcache_memory+0x105/0x216</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039516] [<ffffffff810c21c8>] ? shrink_slab+0x10e/0x189</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039550] [<ffffffff810c2a5d>] ? kswapd+0x4c3/0x659</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039583] [<ffffffff810bffa3>] ? isolate_pages_global+0x0/0x1ff</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039620] [<ffffffff8106680a>] ? autoremove_wake_function+0x0/0x2e</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039655] [<ffffffff810c259a>] ? kswapd+0x0/0x659</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039687] [<ffffffff8106653e>] ? kthread+0xc0/0xca</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039722] [<ffffffff81011c6a>] ? child_rip+0xa/0x20</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039754] [<ffffffff8106647e>] ? kthread+0x0/0xca</div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.039786] [<ffffffff81011c60>] ? child_rip+0x0/0x20</div>
<div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.040100] RSP <ffff88083c919c30></div><div>Dec 2 19:46:50 hn-r410-openvz01 kernel: [ 9903.040496] ---[ end trace ddd95abf24096674 ]---</div><div><br></div>