<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FR-BE" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hello all,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I’ve already tried to post this, but i’m unsure it arrived to the mailing list.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I have some issues regarding my nfs mounts. My setup is as follows:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Rhel 6.4, gluster 3.4.2-1 running on a vm (4 cores, 8GB ram) attached to a san. I have one disk on which are all the bricks (formatted ext4 in 64 bit mode) of 25TB.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">On the gluster side of things, everything works without issues. The trouble starts when I mount a volume as an nfs mount.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Lots of volumes work without issues, but others behave strangely. The volumes that act weird generally contain many files (can be accidental?).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The volumes in question mount without issues, but when I try to go into any subdirectory sometimes it works, sometimes I get errors.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">On windows with nfs client: access denied<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">In nfslog:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771241] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in <gfid:c8d94120-6851-46ea-9f28-c629a44b1015>. holes=1 overlaps=0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771348] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:c8d94120-6851-46ea-9f28-c629a44b1015>: Invalid argument<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771380] E [nfs3.c:1380:nfs3_lookup_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.148.46:984) caviar_data11 : c8d94120-6851-46ea-9f28-c629a44b1015<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.771819] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 1ec28530, LOOKUP: NFS: 22(Invalid argument for operation), POSIX: 14(Bad address)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.798967] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in <gfid:14972193-1039-4d7a-aed5-0d7e7eccf57b>. holes=1 overlaps=0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799039] E [nfs3-helpers.c:3595:nfs3_fh_resolve_inode_lookup_cbk] 0-nfs-nfsv3: Lookup failed: <gfid:14972193-1039-4d7a-aed5-0d7e7eccf57b>: Invalid argument<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799056] E [nfs3.c:1380:nfs3_lookup_resume] 0-nfs-nfsv3: Unable to resolve FH: (192.168.148.46:984) caviar_data11 : 14972193-1039-4d7a-aed5-0d7e7eccf57b<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 13:57:58.799088] W [nfs3-helpers.c:3380:nfs3_log_common_res] 0-nfs-nfsv3: XID: 1ec28531, LOOKUP: NFS: 22(Invalid argument for operation), POSIX: 14(Bad address)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">….<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">On linux:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 brick-xiv2]# ll /media/2011/201105/20110530/<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37: No such file or directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">total 332<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 32<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 34<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 35<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 36<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">drwxrwsr-x 2 nfsnobody 1003 4096 Jun 6 2011 37<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[root@lpr-nas01 brick-xiv2]# ll /media/2011/201105/20110530/37<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37/NN.0000073824357.00001.tif: No such file or directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">ls: /media/2011/201105/20110530/37/NN.0000073824357.00003.tif: No such file or directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">total 54<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 0 nfsnobody 1003 9340 Jun 6 2011 NN.0000073824357.00001.tif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 1 nfsnobody 1003 35312 Jun 6 2011 NN.0000073824357.00002.tif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">-rwxrwxr-x 0 nfsnobody 1003 9340 Jun 6 2011 NN.0000073824357.00003.tif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">I see in the nfslog:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.941083] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/55. holes=1 overlaps=0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.958078] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/30. holes=1 overlaps=0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.959980] I [dht-layout.c:638:dht_layout_normalize] 0-caviar_data11-dht: found anomalies in /2011/201107/20110716/90. holes=1 overlaps=0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961094] E [dht-helper.c:429:dht_subvol_get_hashed] (-->/usr/lib64/glusterfs/3.4.2/xlator/debug/io-stats.so(io_stats_lookup+0x157) [0x7fd6a61282e7] (-->/usr/lib64/libglusterfs.so.0(default_lookup+0x6d)
[0x3dfe01c03d] (-->/usr/lib64/glusterfs/3.4.2/xlator/cluster/distribute.so(dht_lookup+0xa7e) [0x7fd6a656af2e]))) 0-caviar_data11-dht: invalid argument: loc->parent<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961283] W [client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-caviar_data11-client-0: remote operation failed: Invalid argument. Path: <gfid:00000000-0000-0000-0000-000000000000> (00000000-0000-0000-0000-000000000000)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961319] E [acl3.c:334:acl3_getacl_resume] 0-nfs-ACL: Unable to resolve FH: (192.168.151.21:740) caviar_data11 : 00000000-0000-0000-0000-000000000000<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2014-03-31 12:44:18.961338] E [acl3.c:342:acl3_getacl_resume] 0-nfs-ACL: unable to open_and_resume<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">…<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The weirdest thing is it changes from time to time which files and directories work and which don’t
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Any ideas?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks!<o:p></o:p></span></p>
</div>
</body>
</html>