Hi Pranith,<br /><br />I have attached two source files: one for the reader (reader.cpp) and one for the writer (writer.cpp). I run those simultaneously on two different nodes.<br />The test relies on the existence of a small file (<1000 bytes) called "target" under /mnt/gv0. Alternatively you can modify the source files to point to another location.<br />I have four nodes: one primary server, one replicate secondary server, one for the reader (client node) and one for the writer (client node).<br />The gluster file system is physically located on a single disk local to the primary server.<br />Let me know if you need more details.<br /><br />Thanks,<br /><br />Louis<br /><br /><span>Le 06/08/14, <b class="name">Pranith Kumar Karampuri </b> <pkarampu@redhat.com> a écrit :</span><blockquote cite="mid:53E244D7.9050906@redhat.com" class="iwcQuote" style="border-left: 1px solid #00F; padding-left: 13px; margin-left: 0;" type="cite"><div class="mimepart text html"><span><p>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type" />
<table ><tbody><tr><td bgcolor="#FFFFFF" text="#000000"><p>
Louis,<br />
Do you have a small test case or something we can try to
re-create the issue?<br />
<br />
Pranith<br />
</p><div class="moz-cite-prefix">On 08/06/2014 08:33 PM,
<a href="mailto:louisbenoit@videotron.ca">louisbenoit@videotron.ca</a> wrote:<br />
</div>
<blockquote cite="mid:7730be542597c.53e20b7c@videotron.ca" type="cite"> Hi,<br />
<br />
Version: gluster 3.4.2<br />
Use case: on one node a target file is updated while another node
is reading it. <br />
Issue: only about half of the read succeeds.<br />
The strategy elected is to write to a temporary file then rename
the temporary file to the target file.<br />
During file renaming the gluster log from the reading node
complains about stale file handle amongst other things:<br />
<br />
[2014-08-06 13:54:24.653951] W
[fuse-resolve.c:147:fuse_resolve_gfid_cbk] 0-fuse:
f98aa46f-0e37-49bc-b578-8e70bec57a7f: failed to resolve (Invalid
argument)<br />
[2014-08-06 13:54:24.653983] E
[fuse-bridge.c:1996:fuse_open_resume] 0-glusterfs-fuse: 428919:
OPEN f98aa46f-0e37-49bc-b578-8e70bec57a7f resolution failed<br />
[2014-08-06 13:54:24.754741] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.755263] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.755302] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.755347] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428921: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:24.755687] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.756193] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.756232] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.756267] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428922: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:24.756794] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.757292] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.757324] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.757356] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428924: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:24.958465] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.959020] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.959059] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.959100] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428929: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:24.959420] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.959885] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.959912] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.959949] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428930: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:24.960420] W
[client-rpc-fops.c:2697:client3_3_readv_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.960940] W
[client-rpc-fops.c:1179:client3_3_fgetxattr_cbk] 0-gv0-client-0:
remote operation failed: No such file or directory<br />
[2014-08-06 13:54:24.960966] E
[dht-helper.c:725:dht_migration_complete_check_task] 0-gv0-dht:
(null): failed to get the 'linkto' xattr No such file or directory<br />
[2014-08-06 13:54:24.960993] W [fuse-bridge.c:2089:fuse_readv_cbk]
0-glusterfs-fuse: 428932: READ => -1 (No such file or
directory)<br />
[2014-08-06 13:54:25.161921] W
[client-rpc-fops.c:526:client3_3_stat_cbk] 0-gv0-client-0: remote
operation failed: No such file or directory<br />
[2014-08-06 13:54:25.263287] W
[client-rpc-fops.c:526:client3_3_stat_cbk] 0-gv0-client-0: remote
operation failed: No such file or directory<br />
[2014-08-06 13:54:25.364792] W
[client-rpc-fops.c:526:client3_3_stat_cbk] 0-gv0-client-0: remote
operation failed: No such file or directory<br />
[2014-08-06 13:54:25.466796] W
[client-rpc-fops.c:526:client3_3_stat_cbk] 0-gv0-client-0: remote
operation failed: No such file or directory<br />
[2014-08-06 13:54:25.568329] W
[client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-gv0-client-0:
remote operation failed: Stale file handle. Path: /target_file
(f98aa46f-0e37-49bc-b578-8e70bec57a7f)<br />
[2014-08-06 13:54:25.568381] W
[client-rpc-fops.c:2624:client3_3_lookup_cbk] 0-gv0-client-1:
remote operation failed: Stale file handle. Path: /target_file
(f98aa46f-0e37-49bc-b578-8e70bec57a7f)<br />
<br />
I tried using RSYNC_FRIENDLY_NAME so that the file name hashing is
the same for the temporary and target file (all my temporary file
follows the rsync pattern ".<target filename>.tmp") but the
issue remains.<br />
<br />
Is there a way to make the "renaming" atomic? If not what is the
best strategy to use when a node tries to read a file while
another tries to update it?<br />
<br />
Thanks,<br />
<br />
Louis<br />
<br />
<fieldset class="mimeAttachmentHeader"></fieldset>
<br />
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="1">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<br />
</td></tr></tbody></table>
</p></span></div></blockquote>