<div class="gmail_quote"><div class="HOEnZb"><div class="h5"><div class="gmail_quote"><div>Hi!</div><div><br></div><div>I have errors in my application caused by GlusterFS:</div><div><br></div><div>copy(/storage/2060000/2064000/2064000/2064011/dat): failed to open stream: Input/output error</div>
<div>readfile(/storage/2060000/2064000/2064000/2064011/dat): failed to open stream: Input/output error</div>
<div><br></div><div><div>$ cat /storage/2120000/2125000/2125800/2125844/25/dat</div><div>cat: /storage/2120000/2125000/2125800/2125844/25/dat: Input/output error</div></div><div><br></div><div>On client Instance:</div><div>
<br></div><div><div>$ tail /var/log/gluster.log </div><div>[2012-10-25 07:17:14.904162] W [afr-self-heal-data.c:831:afr_lookup_select_read_child_by_txn_type] 0-storage-replicate-0: /2120000/2125000/2125800/2125844/25/game.dat: Possible split-brain</div>
<div>[2012-10-25 07:17:14.904225] W [afr-common.c:1226:afr_detect_self_heal_by_lookup_status] 0-storage-replicate-0: split brain detected during lookup of /2120000/2125000/2125800/2125844/25/game.dat.</div><div>[2012-10-25 07:17:14.911705] E [afr-self-heal-common.c:2156:afr_self_heal_completion_cbk] 0-storage-replicate-0: background data gfid self-heal failed on /2120000/2125000/2125800/2125844/25/game.dat</div>
<div>[2012-10-25 07:17:14.911885] W [afr-open.c:213:afr_open] 0-storage-replicate-0: failed to open as split brain seen, returning EIO</div><div>[2012-10-25 07:17:14.912047] W [fuse-bridge.c:713:fuse_fd_cbk] 0-glusterfs-fuse: 457439: OPEN() /2120000/2125000/2125800/2125844/25/game.dat => -1 (Input/output error)</div>
<div>[2012-10-25 07:17:22.386426] W [afr-self-heal-data.c:831:afr_lookup_select_read_child_by_txn_type] 0-storage-replicate-0: /2120000/2125000/2125800/2125844/25/game.dat: Possible split-brain</div><div>[2012-10-25 07:17:22.386469] W [afr-common.c:1226:afr_detect_self_heal_by_lookup_status] 0-storage-replicate-0: split brain detected during lookup of /2120000/2125000/2125800/2125844/25/game.dat.</div>
<div>[2012-10-25 07:17:22.393897] E [afr-self-heal-common.c:2156:afr_self_heal_completion_cbk] 0-storage-replicate-0: background data gfid self-heal failed on /2120000/2125000/2125800/2125844/25/game.dat</div><div>[2012-10-25 07:17:22.397818] W [afr-open.c:213:afr_open] 0-storage-replicate-0: failed to open as split brain seen, returning EIO</div>
<div>[2012-10-25 07:17:22.397853] W [fuse-bridge.c:713:fuse_fd_cbk] 0-glusterfs-fuse: 459320: OPEN() /2120000/2125000/2125800/2125844/25/game.dat => -1 (Input/output error)</div></div><div><br></div><div><div>$ glusterd -V</div>
<div>glusterfs 3.3.0 built on Jun 4 2012 11:39:45</div></div><div><br></div><div>$ gluster volume info</div><div><div> Volume Name: storage</div><div>Type: Distributed-Replicate</div><div>Volume ID: a278889e-16fe-4913-9af6-eca4c099c484</div>
<div>Status: Started</div><div>Number of Bricks: 4 x 2 = 8</div><div>Transport-type: tcp</div><div>Bricks:</div><div>Brick1: gluster-1a:/storage/1a</div><div>Brick2: gluster-1b:/storage/1b</div><div>Brick3: gluster-2a:/storage/2a</div>
<div>Brick4: gluster-2b:/storage/2b</div><div>Brick5: gluster-1a:/storage/3a</div><div>Brick6: gluster-1b:/storage/3b</div><div>Brick7: gluster-2a:/storage/4a</div><div>Brick8: gluster-2b:/storage/4b</div><div>Options Reconfigured:</div>
<div>performance.stat-prefetch: off</div><div>auth.allow: 10.*</div></div><div><br></div><div><div>$ gluster volume status</div><div>Status of volume: storage</div><div>Gluster process<span style="white-space:pre-wrap"> </span>Port<span style="white-space:pre-wrap"> </span>Online<span style="white-space:pre-wrap"> </span>Pid</div>
<div>------------------------------------------------------------------------------</div><div>Brick gluster-1a:/storage/1a<span style="white-space:pre-wrap"> </span>24009<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>17302</div>
<div>Brick gluster-1b:/storage/1b<span style="white-space:pre-wrap"> </span>24009<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>12188</div>
<div>Brick gluster-2a:/storage/2a<span style="white-space:pre-wrap"> </span>24009<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>10863</div>
<div>Brick gluster-2b:/storage/2b<span style="white-space:pre-wrap"> </span>24009<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>13486</div>
<div>Brick gluster-1a:/storage/3a<span style="white-space:pre-wrap"> </span>24011<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>17308</div>
<div>Brick gluster-1b:/storage/3b<span style="white-space:pre-wrap"> </span>24011<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>12194</div>
<div>Brick gluster-2a:/storage/4a<span style="white-space:pre-wrap"> </span>24011<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>10869</div>
<div>Brick gluster-2b:/storage/4b<span style="white-space:pre-wrap"> </span>24011<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>13492</div>
<div>NFS Server on localhost<span style="white-space:pre-wrap"> </span>38467<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>17314</div>
<div>Self-heal Daemon on localhost<span style="white-space:pre-wrap"> </span>N/A<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>17320</div>
<div>NFS Server on gluster-2a<span style="white-space:pre-wrap"> </span>38467<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>10879</div>
<div>Self-heal Daemon on gluster-2a<span style="white-space:pre-wrap"> </span>N/A<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>10885</div>
<div>NFS Server on gluster-1b<span style="white-space:pre-wrap"> </span>38467<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>12200</div>
<div>Self-heal Daemon on gluster-1b<span style="white-space:pre-wrap"> </span>N/A<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>12206</div>
<div>NFS Server on gluster-2b<span style="white-space:pre-wrap"> </span>38467<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>13503</div>
<div>Self-heal Daemon on gluster-2b<span style="white-space:pre-wrap"> </span>N/A<span style="white-space:pre-wrap"> </span>Y<span style="white-space:pre-wrap"> </span>13509</div>
</div><div><br></div><div><br></div><div>How can I fix the situation?</div><span><font color="#888888"><div><br></div>-- <br><img src="https://lh6.googleusercontent.com/-RsqMrU0PeDg/To283Rfrw9I/AAAAAAAAFQA/WwK8X09YjD0/s100/4645.jpg"></font></span></div>
</div></div></div><br>