Hi Christopher,<div><br></div><div>The issue of excessive logs (only the 'pure path resolution' logs) is being debugged (and it happens only in case of afr self-heal, or a rebalance at this time). But anyways, just those logs are not harmful, and you can ignore them.</div>
<div><br></div><div>You can contact <meta http-equiv="content-type" content="text/html; charset=utf-8"><a href="http://www.gluster.com/company/contact/">http://www.gluster.com/company/contact/</a> (<meta http-equiv="content-type" content="text/html; charset=utf-8"><span class="Apple-style-span" style="border-collapse: collapse; color: rgb(51, 51, 51); font-family: Arial, Verdana, Helvetica, sans-serif; font-size: 13px; line-height: 19px; "> +1 (800) 805 5215) </span><span class="Apple-style-span" style="border-collapse: collapse; font-family: Arial, Verdana, Helvetica, sans-serif; font-size: 13px; line-height: 19px; ">and somebody should be getting back to you.</span></div>
<div><font class="Apple-style-span" color="#333333" face="Arial, Verdana, Helvetica, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse; line-height: 19px;"><br></span></font></div><div><font class="Apple-style-span" color="#333333" face="Arial, Verdana, Helvetica, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse; line-height: 19px;">Regards,</span></font></div>
<div><font class="Apple-style-span" color="#333333" face="Arial, Verdana, Helvetica, sans-serif"><span class="Apple-style-span" style="border-collapse: collapse; line-height: 19px;">Amar<br></span></font><br><div class="gmail_quote">
On Mon, May 2, 2011 at 11:50 AM, Christopher Anderlik <span dir="ltr"><<a href="mailto:christopher.anderlik@xidras.com">christopher.anderlik@xidras.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
hi.<br>
<br>
hm, so it seems that nobody else have these problems.<br>
is there any possibility to get professional support for glusterfs?<br>
does someone has a link/contact?<br>
<br>
thank you<br>
christopher<br>
<br>
<br>
<br>
<br>
<br>
Am 29.04.2011 08:42, schrieb Christopher Anderlik:<div><div></div><div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
hello.<br>
<br>
we use glusterfs 3.2.0<br>
2 glusterfs-server with SLES 11.1<br>
and several clients which access the gfs-volumes.<br>
<br>
<br>
configuration:<br>
<br>
<br>
<br>
info<br>
----<br>
type=2<br>
count=2<br>
status=1<br>
sub_count=2<br>
version=1<br>
transport-type=0<br>
volume-id=05168b54-6a5c-4aa3-91ee-63d16976c6cd<br>
brick-0=10.0.1.xxx:-glusterstorage-macm03<br>
brick-1=10.0.1.xxy:-glusterstorage-macm03<br>
<br>
<br>
<br>
<br>
<br>
<br>
macm03-fuse.vol<br>
---------------<br>
volume macm03-client-0<br>
type protocol/client<br>
option remote-host 10.0.1.xxx<br>
option remote-subvolume /glusterstorage/macm03<br>
option transport-type tcp<br>
end-volume<br>
<br>
volume macm03-client-1<br>
type protocol/client<br>
option remote-host 10.0.1.xxy<br>
option remote-subvolume /glusterstorage/macm03<br>
option transport-type tcp<br>
end-volume<br>
<br>
volume macm03-replicate-0<br>
type cluster/replicate<br>
subvolumes macm03-client-0 macm03-client-1<br>
end-volume<br>
<br>
volume macm03-write-behind<br>
type performance/write-behind<br>
subvolumes macm03-replicate-0<br>
end-volume<br>
<br>
volume macm03-read-ahead<br>
type performance/read-ahead<br>
subvolumes macm03-write-behind<br>
end-volume<br>
<br>
volume macm03-io-cache<br>
type performance/io-cache<br>
subvolumes macm03-read-ahead<br>
end-volume<br>
<br>
volume macm03-quick-read<br>
type performance/quick-read<br>
subvolumes macm03-io-cache<br>
end-volume<br>
<br>
volume macm03-stat-prefetch<br>
type performance/stat-prefetch<br>
subvolumes macm03-quick-read<br>
end-volume<br>
<br>
volume macm03<br>
type debug/io-stats<br>
subvolumes macm03-stat-prefetch<br>
end-volume<br>
<br>
<br>
<br>
<br>
<br>
macm03.10.0.1.xxx.glusterstorage-macm03.vol<br>
-------------------------------------------<br>
volume macm03-posix<br>
type storage/posix<br>
option directory /glusterstorage/macm03<br>
end-volume<br>
<br>
volume macm03-access-control<br>
type features/access-control<br>
subvolumes macm03-posix<br>
end-volume<br>
<br>
volume macm03-locks<br>
type features/locks<br>
subvolumes macm03-access-control<br>
end-volume<br>
<br>
volume macm03-io-threads<br>
type performance/io-threads<br>
subvolumes macm03-locks<br>
end-volume<br>
<br>
volume /glusterstorage/macm03<br>
type debug/io-stats<br>
subvolumes macm03-io-threads<br>
end-volume<br>
<br>
volume macm03-server<br>
type protocol/server<br>
option transport-type tcp<br>
option auth.addr./glusterstorage/macm03.allow *<br>
subvolumes /glusterstorage/macm03<br>
end-volume<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
macm03.10.0.1.xxy.glusterstorage-macm03.vol<br>
-------------------------------------------<br>
volume macm03-posix<br>
type storage/posix<br>
option directory /glusterstorage/macm03<br>
end-volume<br>
<br>
volume macm03-access-control<br>
type features/access-control<br>
subvolumes macm03-posix<br>
end-volume<br>
<br>
volume macm03-locks<br>
type features/locks<br>
subvolumes macm03-access-control<br>
end-volume<br>
<br>
volume macm03-io-threads<br>
type performance/io-threads<br>
subvolumes macm03-locks<br>
end-volume<br>
<br>
volume /glusterstorage/macm03<br>
type debug/io-stats<br>
subvolumes macm03-io-threads<br>
end-volume<br>
<br>
volume macm03-server<br>
type protocol/server<br>
option transport-type tcp<br>
option auth.addr./glusterstorage/macm03.allow *<br>
subvolumes /glusterstorage/macm03<br>
end-volume<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
client<br>
------<br>
the client has mounted the volume via fstab like this:<br>
server:/macm03 /srv/www/GFS glusterfs defaults,_netdev 0 0<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
now we registered strange behavior and i have some questions:<br>
<br>
<br>
<br>
1) files with size 0<br>
we find many files with size 0. in server-log we only find this. what does this mean?<br>
(most all files in one directory has size 0).<br>
<br>
<br>
[2011-04-28 23:52:00.630869] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (LOOKUP)<br>
[2011-04-28 23:52:00.637384] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (UNLINK)<br>
[2011-04-28 23:52:00.693183] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (LOOKUP)<br>
[2011-04-28 23:52:00.711092] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (MKNOD)<br>
[2011-04-28 23:52:00.746289] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (SETATTR)<br>
[2011-04-28 23:52:16.373532] I [server-resolve.c:580:server_resolve] 0-macm03-server: pure path<br>
resolution for /xxx/preview/4aa76fa541413.jpg (LOOKUP)<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
2) then the client is selfhealing meta-data all the time.... (because the file has has size 0 on one<br>
of the servers???)<br>
but we triggered selfhealing severalt time like this:<br>
<a href="http://europe.gluster.org/community/documentation/index.php/Gluster_3.1:_Triggering_Self-Heal_on_Replicate" target="_blank">http://europe.gluster.org/community/documentation/index.php/Gluster_3.1:_Triggering_Self-Heal_on_Replicate</a><br>
<br>
<br>
<br>
<br>
[2011-04-29 07:55:27.188743] I [afr-common.c:581:afr_lookup_collect_xattr] 0-macm03-replicate-0:<br>
data self-heal is pending for /videos12/29640/preview/4aadf4b757de6.jpg.<br>
[2011-04-29 07:55:27.188829] I [afr-common.c:735:afr_lookup_done] 0-macm03-replicate-0: background<br>
meta-data data self-heal triggered. path: /videos12/29640/preview/4aadf4b757de6.jpg<br>
[2011-04-29 07:55:27.194446] W [dict.c:437:dict_ref]<br>
(-->/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/protocol/client.so(client3_1_fstat_cbk+0x2bb)<br>
[0x2aaaaafe833b]<br>
(-->/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/cluster/replicate.so(afr_sh_data_fstat_cbk+0x17d)<br>
[0x2aaaab11c9ad]<br>
(-->/opt/glusterfs/3.2.0/lib64/glusterfs/3.2.0/xlator/cluster/replicate.so(afr_sh_data_fix+0x1fc)<br>
[0x2aaaab11c64c]))) 0-dict: dict is NULL<br>
<br>
<br>
<br>
<br>
<br>
3)<br>
on some of the clients we then can not access the whole directory:<br>
<br>
# dir xxx/preview/<br>
/bin/ls: reading directory xxx/preview/: File descriptor in bad state<br>
total 0<br>
<br>
<br>
in logs we find this:<br>
<br>
[2011-04-29 08:36:17.224301] W [afr-common.c:634:afr_lookup_self_heal_check] 0-macm03-replicate-0:<br>
/videos12/30181: gfid different on subvolume<br>
[2011-04-29 08:36:17.241330] I [afr-common.c:680:afr_lookup_done] 0-macm03-replicate-0: entries are<br>
missing in lookup of /xxx/preview.<br>
[2011-04-29 08:36:17.241373] I [afr-common.c:735:afr_lookup_done] 0-macm03-replicate-0: background<br>
meta-data data entry self-heal triggered. path: /xxx/preview<br>
[2011-04-29 08:36:17.243160] I [afr-self-heal-metadata.c:595:afr_sh_metadata_lookup_cbk]<br>
0-macm03-replicate-0: path /videos12/30181/preview on subvolume macm03-client-0 => -1 (No such file<br>
or directory)<br>
[2011-04-29 08:36:17.302228] I [afr-dir-read.c:120:afr_examine_dir_readdir_cbk]<br>
0-macm03-replicate-0: /videos12/30181/preview: failed to do opendir on macm03-client-0<br>
[2011-04-29 08:36:17.303836] I [afr-dir-read.c:174:afr_examine_dir_readdir_cbk]<br>
0-macm03-replicate-0: entry self-heal triggered. path: /xxx/preview, reason: checksums of directory<br>
differ, forced merge option set<br>
<br>
<br>
<br>
<br>
<br>
<br>
4)<br>
sometimes when we umount glusterfs-volume on client and remount it again, we can access the dirctory<br>
which was in bad state before -> and then also selfhealing works at it should<br>
but sometimes also a remount does not work.<br>
<br>
<br>
<br>
<br>
<br>
<br>
any help would be appreciated.<br>
thank you very very much!<br>
<br>
<br>
<br>
thx<br>
christopher<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
<br>
</blockquote>
<br></div></div>
-- <br>
<br>
Mag. Christopher Anderlik<br>
Leiter Technik<br>
<br>
________________________________________________________________________________<br>
<br>
<br>
Xidras GmbH<br>
Stockern 47<br>
3744 Stockern<br>
Austria<br>
<br>
Tel: 0043 2983 201 30 5 01<br>
Fax: 0043 2983 201 30 5 01 9<br>
Email: <a href="mailto:christopher.anderlik@xidras.com" target="_blank">christopher.anderlik@xidras.com</a><br>
Web: <a href="http://www.xidras.com" target="_blank">http://www.xidras.com</a><br>
<br>
FN 317036 f | Landesgericht Krems | ATU64485024<br>
<br>
________________________________________________________________________________<br>
<br>
VERTRAULICHE INFORMATIONEN!<br>
Diese eMail enthält vertrauliche Informationen und ist nur für den berechtigten<br>
Empfänger bestimmt. Wenn diese eMail nicht für Sie bestimmt ist, bitten wir Sie,<br>
diese eMail an uns zurückzusenden und anschließend auf Ihrem Computer und<br>
Mail-Server zu löschen. Solche eMails und Anlagen dürfen Sie weder nutzen,<br>
noch verarbeiten oder Dritten zugänglich machen, gleich in welcher Form.<br>
Wir danken für Ihre Kooperation!<br>
<br>
CONFIDENTIAL!<br>
This email contains confidential information and is intended for the authorised<br>
recipient only. If you are not an authorised recipient, please return the email<br>
to us and then delete it from your computer and mail-server. You may neither<br>
use nor edit any such emails including attachments, nor make them accessible<br>
to third parties in any manner whatsoever.<br>
Thank you for your cooperation<br>
<br>
________________________________________________________________________________<div><div></div><div class="h5"><br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
</div></div></blockquote></div><br></div>