pranith - a huge and heartfelt thanks for your super prompt attention. a very scary event turned into a non-event. :)<div><br></div><div>regards,</div><div><br></div><div>-p</div><div><br><br><div class="gmail_quote">On 8 August 2011 18:35, Pranith Kumar K <span dir="ltr"><<a href="mailto:pranithk@gluster.com">pranithk@gluster.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><u></u>
<div bgcolor="#ffffff" text="#000000">
After debugging the problem with paul on IRC, we found that because
his disk had no free space, the subsequent writes on one of the peer
files (used for recovering run-time information) failed so the file
became empty. Because of this glusterd could not restore that peer
so it is not re-starting successfully. We copied the contents of
that file from other peer in the cluster to the problematic one.
Then glusterd started successfully.<br><font color="#888888">
<br>
Pranith.</font><div><div></div><div class="h5"><br>
<br>
On 08/08/2011 10:32 PM, paul simpson wrote:
<blockquote type="cite">
hi pranith,
<div><br>
</div>
<div>many thanks for the super quick reply! i've attached the
files asked for - be keen to hear your thoughts. i'm stumped -
and scared! </div>
<div><br>
</div>
<div>regards,</div>
<div><br>
</div>
<div>paul</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
<br>
<div class="gmail_quote">On 8 August 2011 17:59, Pranith Kumar K
<span dir="ltr"><<a href="mailto:pranithk@gluster.com" target="_blank">pranithk@gluster.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div bgcolor="#ffffff" text="#000000"> zip /etc/glusterd and
send across<br>
<br>
Pranith
<div>
<div><br>
On 08/08/2011 10:15 PM, paul simpson wrote: </div>
</div>
<blockquote type="cite">
<div>
<div> hi gluster gurus,
<div><br>
</div>
<div>i have 4 servers g1,g2,g3 & g4 with 24T
each running gluster 3.1.5 on opensuse 11.3. they
have been running well for the last few months in
a distributed+replicated setup.</div>
<div> <br>
</div>
<div>i just found that the nfs log had filled up my
root disk of g4 (my bad). so, i removed the log
file - and a couple of other large ones and
restored a load of disk space. however, gluster
3.1.5 will not restart on this machine!! it err's
out with <a href="http://pastebin.com/646W8zjg" target="_blank">http://pastebin.com/646W8zjg</a></div>
<div><br>
</div>
<div>i've searched this forum, and searched the
documentation. however, i cant see anything that
mentions this situation. please can anyone help -
i'm quite concerned about my system. this is a
live server with live data. i need to get g4 up
and running and back into sync ASAP.</div>
<div><br>
</div>
<div>many thanks in advance,</div>
<div><br>
</div>
<div>-paul</div>
<div><br>
</div>
<div>ps - the following command just hangs:</div>
<blockquote style="margin:0pt 0pt 0pt 40px;border:medium none;padding:0px">
<div><font face="'courier new', monospace">g4:~ #
gluster peer status </font></div>
</blockquote>
<div><br>
</div>
<div>..however, on g3 it works:</div>
<blockquote style="margin:0pt 0pt 0pt 40px;border:medium none;padding:0px">
<div>
<div><font face="'courier new', monospace">g3:/etc/glusterd/logs
# gluster peer status</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Number
of Peers: 3</font></div>
</div>
<div>
<div><font face="'courier new', monospace"><br>
</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Hostname:
10.0.0.12</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Uuid:
8061196e-a075-42f6-89f5-1f60281485f5</font></div>
</div>
<div>
<div><font face="'courier new', monospace">State:
Peer in Cluster (Connected)</font></div>
</div>
<div>
<div><font face="'courier new', monospace"><br>
</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Hostname:
g2</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Uuid:
154d5c46-f62f-4e9c-a328-443e30cadf4e</font></div>
</div>
<div>
<div><font face="'courier new', monospace">State:
Peer in Cluster (Connected)</font></div>
</div>
<div>
<div><font face="'courier new', monospace"><br>
</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Hostname:
g4</font></div>
</div>
<div>
<div><font face="'courier new', monospace">Uuid:
62365589-61f8-479f-bb50-11519beba045</font></div>
</div>
<div>
<div><font face="'courier new', monospace">State:
Peer in Cluster (Disconnected)</font></div>
</div>
</blockquote>
<div>..i've also tried rebooting the machine - and
nothing changes. </div>
</div>
</div>
<pre><fieldset></fieldset>
_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="http://gluster.org/cgi-bin/mailman/listinfo/gluster-users" target="_blank">http://gluster.org/cgi-bin/mailman/listinfo/gluster-users</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>