Marc,<div> Can you get us a process state dump of the glusterfs client while it is in the hung state? This will be very useful for us to debug the issue.</div><div><br></div><div>Avati<br><br><div class="gmail_quote">On Fri, Jun 17, 2011 at 11:17 PM, Marc Geerlings <span dir="ltr"><<a href="mailto:marc.geerlings@gmail.com">marc.geerlings@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I think I have the same problems as:<br>
<br>
<a href="http://gluster.org/pipermail/gluster-users/2011-June/007980.html" target="_blank">http://gluster.org/pipermail/gluster-users/2011-June/007980.html</a><br>
<br>
and<br>
<br>
<a href="http://gluster.org/pipermail/gluster-users/2011-May/007697.html" target="_blank">http://gluster.org/pipermail/gluster-users/2011-May/007697.html</a><br>
<br>
Recently I installed Glusterfs 3.2.0 on four workstations which share a<br>
total of 28TB of disk-space between them for batch processing of FMRI<br>
and DTI data. Had the same setup between three workstations and 9TB with<br>
Glusterfs 3.0.4 for over a year, but the hard-disk pool was to small.<br>
The three "old" workstations from the old cluster and two more will also<br>
access the 28TB pool.<br>
<br>
OS: Scientific Linux 6<br>
Glusterfs 3.2.0 and 3.2.1 compiled from source rpm<br>
<br>
I've been testing the new setup for over a week now, but it seemed<br>
unstable, Glusterfs clients would stop responding and tasks hung without<br>
being able to kill -9. Only a reboot would stop the tasks.<br>
a kill of the Glusterfs client and a mount afterwards would make the<br>
gluster storage accessible again.<br>
<br>
One of the tasks is converting a medical image format called nifti to<br>
another format called analyze. We use a application included in a<br>
package called mrtrix (2.9.0) for this. The application is mrconvert.<br>
<br>
-A conversion of a large nifti file (256MB) to a analyze file will hang<br>
application and the Gluster fuse client<br>
- A small one (10MB) will work.<br>
- If I convert it reading from the Gluster storage to a local disk<br>
everything works fine.<br>
- Reading from a local disk to the Gluster storage will hang the<br>
application and gluster fuse client<br>
- disabling the quick-read does nothing.<br>
<br>
Errors are the same as a in the second post above, I can post them<br>
Monday. I did a strace on a functioning and a none functioning convert<br>
and the application hangs on the last close like:<br>
<br>
munmap .............<br>
close(4<br>
<br>
The conversion was completed.<br>
I tried 3.2.0, 3.1.4 and 3.2.1 and all of them have the same problem.<br>
I went back to 3.0.8 and this is functioning.<br>
<br>
--<br>
Marc Geerlings<br>
<br>
Application Manager Research<br>
Department of Radiology<br>
<br>
Maastricht University Medical Center<br>
P.O. Box: 5800, 6202 AZ<br>
Maastricht, The Netherlands<br>
<br>
P. Debyelaan 25<br>
6229 HX, Maastricht<br>
Room: 0.C1.037<br>
<br>
Tel. + 31 43 – 387 49 50<br>
Fax. + 31 43 – 387 69 09<br>
<br>
email: <a href="mailto:m.geerlings@mumc.nl">m.geerlings@mumc.nl</a><br>
<a href="mailto:m.geerlings@maastrichtuniversity.nl">m.geerlings@maastrichtuniversity.nl</a><br>
<br>
_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br></div>