<div dir="ltr">Try mounting like this:<div><br></div><div><font face="courier new, monospace"><span style="color:rgb(31,73,125);font-size:15px">mount -t nfs </span>-o rw,async,vers=3,rsize=65536,wsize=65536,noatime,nodiratime<span style="color:rgb(31,73,125);font-size:15px"> localhost:/caviar_data11 /media</span><br>
</font></div><div><font face="courier new, monospace"><br></font></div><div>Try to test with fresh files, and not necessarily already existing ones. Then the existing ones.</div><div><br></div><div>Let's see what it tells us.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 3, 2014 at 11:05 AM, VAN CAUSBROECK Wannes <span dir="ltr"><<a href="mailto:Wannes.VANCAUSBROECK@onprvp.fgov.be" target="_blank">Wannes.VANCAUSBROECK@onprvp.fgov.be</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="FR-BE" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hello Carlos,<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I created a new disk formatted xfs with an inode size of 512.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I created a new gluster volume and migrated the data. Again, when I mount it as a gluster volume, everything works fine.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">On nfs (mounted on the same server that’s running the gluster volume) I get the following weirdness:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[root@lpr-nas01 /]# mount -t nfs -o mountproto=tcp,vers=3 lpr-nas01:/caviar_data11 /media<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="NL-BE" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[root@lpr-nas01 /]# ll /media/*/<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="NL-BE" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">/media/<a href="tel:2012%2F201206" value="+12012201206" target="_blank">2012/201206</a>:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ls: /media/2012/201206/20120621: No such file or directory<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">total 0<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 3 960 1003 15 Jun 15 2011 20120621<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 38<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 39<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 138 Jun 6 2011 40<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 42<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 171 Jun 6 2011 43<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 45<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 46<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 47<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">drwxrwsr-x 2 nfsnobody 1003 369 Jun 6 2011 48<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">…<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ls: cannot access /media/2011/201106/20110606/81: Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ls: cannot access /media/2011/201106/20110606/55: Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ls: cannot access /media/2011/201106/20110606/30: Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">ls: cannot access /media/2011/201106/20110606/90: Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">total 12<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 00<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 01<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 02<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 03<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 04<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 05<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 06<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 07<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 08<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 09<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">d????????? ? ? ? ? ? 10<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">In the nfs log, I get the following errors:<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">…<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624517] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:a1acf77c-2b81-4b5f-a113-521c6ab8fd23>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624547] E [acl3.c:334:acl3_getacl_resume] 0-nfs-ACL: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : a1acf77c-2b81-4b5f-a113-521c6ab8fd23<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624562] E [acl3.c:342:acl3_getacl_resume] 0-nfs-ACL: unable to open_and_resume<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624914] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:980a29f7-29f4-4c88-9896-2b1a549370e2>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624960] E [acl3.c:334:acl3_getacl_resume] 0-nfs-ACL: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : 980a29f7-29f4-4c88-9896-2b1a549370e2<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.624970] E [acl3.c:342:acl3_getacl_resume] 0-nfs-ACL: unable to open_and_resume<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.625290] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:b67993d2-c647-4493-aa78-64033614dc33>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.625322] E [nfs3.c:755:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : b67993d2-c647-4493-aa78-64033614dc33<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.625335] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: c838d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.626268] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201108/20110802. holes=1 overlaps=0<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.627701] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201108/20110803. holes=1 overlaps=0<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.628839] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:bb587d95-ffa3-42f3-9b27-b1cf0c5c05cb>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.706106] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 6639d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.706585] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:897bf28d-41a3-4cc2-a120-8374656be858>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.706626] E [nfs3.c:755:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : 897bf28d-41a3-4cc2-a120-8374656be858<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.706647] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 6739d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707113] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:ded9c3fe-171b-4104-bc0e-e64c0d3e18e2>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707158] E [nfs3.c:755:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : ded9c3fe-171b-4104-bc0e-e64c0d3e18e2<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707174] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 6839d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707597] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:c5ebeb07-ee2e-4161-9e65-65d65f614628>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707644] E [nfs3.c:755:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : c5ebeb07-ee2e-4161-9e65-65d65f614628<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.707707] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 6939d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.708357] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:da363aa4-6d43-47c0-90fe-edead3689064>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.708407] E [nfs3.c:755:nfs3_getattr_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : da363aa4-6d43-47c0-90fe-edead3689064<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.708432] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 6a39d715, GETATTR: NFS: 22(Invalid argument for operation),
POSIX: 14(Bad address)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.708796] E [dht-helper.c:429:dht_subvol_get_hashed] (-->/usr/lib64/glusterfs/3.4.2/xlator/debug/io-stats.so(io_stats_lookup+0x157)
[0x7f4465bf52e7] (-->/usr/lib64/libglusterfs.so.0(default_lookup+0x6d) [0x3dfe01c03d] (-->/usr/lib64/glusterfs/3.4.2/xlator/cluster/distribute.so(dht_lookup+0xa7e) [0x7f4466037f2e]))) 0-caviar_data11-dht: invalid argument: loc->parent<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.709066] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-caviar_data11-client-0: remote operation failed: Invalid argument.
Path: <gfid:00000000-0000-0000-0000-000000000000> (00000000-0000-0000-0000-000000000000)<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.709128] E [acl3.c:334:acl3_getacl_resume] 0-nfs-ACL: Unable to resolve FH: (<a href="http://192.168.151.21:954" target="_blank">192.168.151.21:954</a>) caviar_data11 : 00000000-0000-0000-0000-000000000000<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.709159] E [acl3.c:342:acl3_getacl_resume] 0-nfs-ACL: unable to open_and_resume<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">[2014-04-03 08:50:13.709741] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:db0eec80-e122-4751-9695-d903a6e6f29e>:
Invalid argument<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">….<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">So….. any ideas?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Carlos Capriotti [mailto:<a href="mailto:capriotti.carlos@gmail.com" target="_blank">capriotti.carlos@gmail.com</a>]
<br></span></p><div class="">
<b>Sent:</b> maandag 31 maart 2014 18:03<br>
<b>To:</b> VAN CAUSBROECK Wannes<br>
</div><b>Cc:</b> <a href="mailto:gluster-users@gluster.org" target="_blank">gluster-users@gluster.org</a><div class=""><br>
<b>Subject:</b> Re: [Gluster-users] nfs acces denied<u></u><u></u></div><p></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal">maybe it would be nice to see your volume info for affected volumes.<u></u><u></u></p><div><div class="h5">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Also, on the server side, what happens if you mount the share using glusterfs instead of nfs ?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">any change the native nfs server is running on your server ?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Are there any auto-heal processes running ?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">There are a few name resolution messages on your logs, that seem to refer to the nodes themselves. Any DNS conflicts ? Maybe add the names of servers to the hosts file ? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">You MS client seems to be having issues with user/group translation. It seems to create files with gid 1003. (I could be wrong).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Again, is SElinux/ACLs/iptables disabled ?<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">All is very inconclusive os far.<u></u><u></u></p>
</div>
</div></div></div><div><div class="h5">
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Mar 31, 2014 at 5:26 PM, VAN CAUSBROECK Wannes <<a href="mailto:Wannes.VANCAUSBROECK@onprvp.fgov.be" target="_blank">Wannes.VANCAUSBROECK@onprvp.fgov.be</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">Well, with 'client' i do actually mean the server itself.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">i've tried forcing linux and windows to nfs V3 and tcp, and on windows i played around with the uid and gid, but the result is always the same<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On 31 Mar 2014, at 17:22, "Carlos Capriotti" <<a href="mailto:capriotti.carlos@gmail.com" target="_blank">capriotti.carlos@gmail.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">Well, saying your client-side is "linux" does not help much. Distro, flavor, etc helps a lot, but I'll take a wild guess here.
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">First, force your NFS mount (client) to use nfs version 3. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The same for Microsoft. (It is fair to say I have no idea if the MS client supports v4 or not).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Additionally, check that firewalls are disabled on both sides, just for testing. The same goes for SElinux.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Windows and ACL, and user mapping is something that might be in your way too. There is a Technet document that describes how to handle this mapping if I am not wrong.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Just for testing, mount your nfs share you your own server, using localhost:/nfs_share and see how it goes.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">It is a good start.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Kr,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Carlos<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Mon, Mar 31, 2014 at 3:58 PM, VAN CAUSBROECK Wannes <<a href="mailto:Wannes.VANCAUSBROECK@onprvp.fgov.be" target="_blank">Wannes.VANCAUSBROECK@onprvp.fgov.be</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hello all,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I’ve already tried to post this, but i’m unsure it arrived to the mailing list.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I have some issues regarding my nfs mounts. My setup is as follows:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Rhel 6.4, gluster 3.4.2-1 running on a vm (4 cores, 8GB ram) attached to a san. I have one disk on which are all the bricks (formatted ext4 in 64 bit mode) of
25TB.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">On the gluster side of things, everything works without issues. The trouble starts when I mount a volume as an nfs mount.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Lots of volumes work without issues, but others behave strangely. The volumes that act weird generally contain many files (can be accidental?).</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">The volumes in question mount without issues, but when I try to go into any subdirectory sometimes it works, sometimes I get errors.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">On windows with nfs client: access denied</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">In nfslog:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771241] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in <gfid:c8d94120-6851-46ea-9f28-c629a44b1015>. holes=1
overlaps=0</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771348] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:c8d94120-6851-46ea-9f28-c629a44b1015>:
Invalid argument</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771380] E [nfs3.c:1380:nfs3_lookup_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.148.46:984" target="_blank">192.168.148.46:984</a>)
caviar_data11 : c8d94120-6851-46ea-9f28-c629a44b1015</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771819] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 1ec28530, LOOKUP: NFS: 22(Invalid argument for operation), POSIX: 14(Bad
address)</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.798967] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in <gfid:14972193-1039-4d7a-aed5-0d7e7eccf57b>. holes=1
overlaps=0</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799039] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:14972193-1039-4d7a-aed5-0d7e7eccf57b>:
Invalid argument</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799056] E [nfs3.c:1380:nfs3_lookup_resume] 0-nfs-nfsv3: Unable to resolve FH: (<a href="http://192.168.148.46:984" target="_blank">192.168.148.46:984</a>)
caviar_data11 : 14972193-1039-4d7a-aed5-0d7e7eccf57b</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799088] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 1ec28531, LOOKUP: NFS: 22(Invalid argument for operation), POSIX: 14(Bad
address)</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">….</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">On linux:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 brick-xiv2]# ll /media/2011/201105/20110530/</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37: No such file or directory</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">total 332</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 32</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 34</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 35</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 36</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 37</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 brick-xiv2]# ll /media/2011/201105/20110530/37</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37/NN.0000073824357.00001.tif: No such file or directory</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37/NN.0000073824357.00003.tif: No such file or directory</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">total 54</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 0 nfsnobody 1003 9340 Jun 6 2011 NN.0000073824357.00001.tif</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 1 nfsnobody 1003 35312 Jun 6 2011 NN.0000073824357.00002.tif</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 0 nfsnobody 1003 9340 Jun 6 2011 NN.0000073824357.00003.tif</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I see in the nfslog:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.941083] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/55. holes=1 overlaps=0</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.958078] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/30. holes=1 overlaps=0</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.959980] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/90. holes=1 overlaps=0</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961094] E [dht-helper.c:429:dht_subvol_get_hashed] (-->/usr/lib64/glusterfs/3.4.2/xlator/debug/io-stats.so(io_stats_lookup+0x157) [0x7fd6a61282e7]
(-->/usr/lib64/libglusterfs.so.0(default_lookup+0x6d) [0x3dfe01c03d] (-->/usr/lib64/glusterfs/3.4.2/xlator/cluster/distribute.so(dht_lookup+0xa7e) [0x7fd6a656af2e]))) 0-caviar_data11-dht: invalid argument: loc->parent</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961283] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-caviar_data11-client-0: remote operation failed: Invalid argument. Path: <gfid:00000000-0000-0000-0000-000000000000>
(00000000-0000-0000-0000-000000000000)</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961319] E [acl3.c:334:acl3_getacl_resume] 0-nfs-ACL: Unable to resolve FH: (<a href="http://192.168.151.21:740" target="_blank">192.168.151.21:740</a>)
caviar_data11 : 00000000-0000-0000-0000-000000000000</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961338] E [acl3.c:342:acl3_getacl_resume] 0-nfs-ACL: unable to open_and_resume</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">…</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">The weirdest thing is it changes from time to time which files and directories work and which don’t
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Any ideas?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Thanks!</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
</blockquote></div><br></div></div>