Hello,<div><br></div><div> Going ahead on my translator attempt, the file read at node "A" and write at node "B" is working OK already with the syncop calls since some days now. I am copying the same gfid to the temporary local cached file, but I am using a different name and path at this file at node B because it wasn't my plan to have all the dir paths replicated at all nodes.</div>
<div><br></div><div> So, at the first read of the file it gets the content from the node "A" where the original file is placed. Although, once this is done I insert some extended metadata there so that further calls coming from node B can know the file is cached there. </div>
<div> This implies on a redirection of the open and readv calls, but I am always getting a "file not found" error at the end because the posix layer is trying to get the original path at the posix level within resolve_gfid(). Instead of getting the hash for the file created at node B /48344277 which is stored at <node B brick>/.glusterfs/20/dd/20dd... it is looking for the <node B brick>/.glusterfs/bc/47/bc47... in this case, which is the hash created by the posix layer to /dir01/f01.txt.</div>
<div><br></div><div><div>[2012-09-17 15:45:23.447620] I [gbfs_t.c:448:gbfs_open] 0-gbfs-test: entering gbfs_open function</div><div>[2012-09-17 15:45:23.448253] W [client-rpc-fops.c:470:client3_3_open_cbk] 0-examplevol-client-1: remote operation failed: No such file or directory. Path: /48344277 (bc477228-42d9-4f90-8806-bd68a5590519)</div>
<div>[2012-09-17 15:45:23.448271] I [gbfs_t.c:435:gbfs_open_cbk] 0-gbfs-test: entering gbfs_open_cbk function</div><div>[2012-09-17 15:45:23.448283] W [fuse-bridge.c:808:fuse_fd_cbk] 0-glusterfs-fuse: 14: OPEN() /dir01/f01.txt => -1 (No such file or directory)</div>
</div><div><br></div><div> I know I am doing it wrong, but I'm not sure how to correctly proceed here. Currently I am trying to follow the lookup/open/readv calls without performing suboperations apart (like syncops I used to write the file), but I'm trying to redirect them to the correct subvolume with the modified loc for the target file.</div>
<div><br>Gustavo Bervian Brand<br>---------------------------------------------------------------------------------<br>
<br><br><div class="gmail_quote">On Mon, Sep 10, 2012 at 9:30 PM, Anand Avati <span dir="ltr"><<a href="mailto:anand.avati@gmail.com" target="_blank">anand.avati@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><br><div class="gmail_quote"><div>On Mon, Sep 10, 2012 at 12:11 PM, Gustavo Bervian Brand <span dir="ltr"><<a href="mailto:gugabrand@gmail.com" target="_blank">gugabrand@gmail.com</a>></span> wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hello,</div><div><br></div><div><div><div> I wasn't using the synctask_new worker, but now I implemented my code using it to trigger the create/write call. Btw, is the usage of syncop calls documented somewhere? I found it through the history of the dev-list.<div>
<br></div><div> But going ahead on my issue, I was able to create the file avoiding the NFS stale handle error. The file is being created at the backend, but the syncop_create call it's still returning an error from the posix translator at the server side. I copied the error from the glusterd log below... I am checking the code trying to figure it out.</div>
<div><br></div><div> To write the file I am generating a new gfid for the local file and setting the pargfid as 1 (root node) with the loc.parent pointing to the inode->table->root inode. Below I copied what is being copied the loc, fd, flag and mode_t structures being used at the create call. </div>
<div><br></div><div>*** From the glusterd log:</div><div><div>[2012-09-10 18:12:34.035940] W [posix-handle.c:590:posix_handle_hard] 0-examplevol1: mismatching ino/dev between file /bricks/exampleDir_local/1ccfca9f (146628/2049) and handle /bricks/exampleDir_local/.glusterfs/00/00/00000000-0000-0000-0000-000000000000 (146617/2049)</div>
<div>[2012-09-10 18:12:34.035977] E [posix.c:1724:posix_create] 0-examplevol1: setting gfid on /bricks/exampleDir_local/1ccfca9f failed</div></div><div><br></div>*** Relevant code before calling the create function</div>
<div>
<div> 107 inode_t *newinode = inode_new (old_loc.inode->table);</div><div> 108 </div><div> 109 loc.path = strdup(hashed_path_str);</div><div> 110 <a href="http://loc.name" target="_blank">loc.name</a> = strdup(hashed_filename_str);</div>
<div> 111 loc.inode = newinode;</div><div> 113 loc.parent = loc.inode->table->root;</div><div> 115 uuid_copy(loc.gfid, gfid);</div><div> 116 </div><div> 118 memset (loc.pargfid, 0, 16);</div>
<div> 119 loc.pargfid[15] = 1;</div><div> 120 </div><div> 121 fd = fd_create (loc.inode, getpid());</div><div>...</div><div> 127 </div><div> 128 uint32_t flags = O_CREAT | O_RDWR | O_EXCL;</div>
<div> 129 mode_t mode = (mode_t)0700;</div><div> 130 fd->flags = flags;</div><div> 131 fd = fd_ref(fd);</div><div><br></div></div></div></div></blockquote><div><br></div>
<div>It is very likely you are just missing setting the "gfid-req" field in the dictionary. You can check how fuse-bridge, nfs, api/src/glfs.c are all setting gfid-req. In your case you would want to copy the GFID of the original file for the cached file as well.</div>
<span><font color="#888888">
<div><br></div><div>Avati</div><div><br></div></font></span></div>
</blockquote></div><br></div>