<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 01/23/2013 08:33 AM, Song wrote:<br>
</div>
<blockquote cite="mid:000f01cdf916$4cbca2c0$e635e840$@com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
/* Page Definitions */
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US">Hi, <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">When application access
a directory of glusterfs volume, glusterfs client spend
about 1 hour, from 19:44:05 to 20:44:25.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The gluster volume is
DHT + AFR, native mount to /xmail/gfs1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Access(/xmail/gfs1/xmail_dedup/gfs1_000/011/204/)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">The following message is
displayed in client log:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849597] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-14: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849636] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-11: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849674] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-17: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849756] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-23: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849825] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-26: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.849993] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-29: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
19:14:04.850039] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-20: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
19:44:05.36300] E [rpc-clnt.c:197:call_bail]
0-gfs1-client-75: bailing out frame type(GlusterFS 3.1)
op(INODELK(29)) xid = 0x1668135x sent = 2013-01-22
19:14:04.843525. timeout = 1800<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
19:44:05.36903] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-75: remote operation failed: Transport
endpoint is not connected<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
20:14:15.229286] E [rpc-clnt.c:197:call_bail]
0-gfs1-client-76: bailing out frame type(GlusterFS 3.1)
op(INODELK(29)) xid = 0x1693471x sent = 2013-01-22
19:44:05.36960. timeout = 1800<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
20:14:15.229508] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-76: remote operation failed: Transport
endpoint is not connected<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
20:44:25.439449] E [rpc-clnt.c:197:call_bail]
0-gfs1-client-77: bailing out frame type(GlusterFS 3.1)
op(INODELK(29)) xid = 0x1683252x sent = 2013-01-22
20:14:15.229559. timeout = 1800<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:red" lang="EN-US">[2013-01-22
20:44:25.440202] E
[client3_1-fops.c:1226:client3_1_inodelk_cbk]
0-gfs1-client-77: remote operation failed: Transport
endpoint is not connected<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.243548] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-18: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.243616] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-20: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.243645] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-19: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.246854] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-18: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.247769] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-19: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.252241] E
[client3_1-fops.c:1303:client3_1_entrylk_cbk]
0-gfs1-client-20: remote operation failed: No such file or
directory<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">[2013-01-22
20:44:26.252269] W [fuse-bridge.c:1684:fuse_create_cbk]
0-glusterfs-fuse: 12541819:
/xmail_dedup/gfs1_000/011/12D/110 => -1 (No such file or
directory)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Please advice.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Thanks!<o:p></o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@nongnu.org">Gluster-devel@nongnu.org</a>
<a class="moz-txt-link-freetext" href="https://lists.nongnu.org/mailman/listinfo/gluster-devel">https://lists.nongnu.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
Song,<br>
Could you attach the statedumps of the bricks 75, 76, 77? <br>
You can create statedumps using kill -USR1 <pid of
brick-process>. They will be created in /tmp<br>
<br>
Pranith.<br>
</body>
</html>