<div dir="ltr"><div>Hi james: <br><br>Looks like standard Hadoop seems to want to keep the files as permission 700, just like you mention in your email: <br><br><a href="https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/mapred/org/apache/hadoop/mapreduce/JobSubmissionFiles.java">https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1/src/mapred/org/apache/hadoop/mapreduce/JobSubmissionFiles.java</a><br>
</div><div><br></div><div>Just a guess : but -- maybe it will work if you try submitting the job from the same machine that is running your jobtracker? I've seen this error before when submitting jobs from random places. <br>
<br>Again, the above is more of a guess than anything else, until we look further into it.....<br></div><br><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Apr 8, 2013 at 4:01 PM, Jay Vyas <span dir="ltr"><<a href="mailto:jvyas@redhat.com" target="_blank">jvyas@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi james !<br>
<br>
1) Yes, right now, we run as root. Thanks for noticing :) ... We are working on modifying this in the very near future. The problem is that<br>
the plugin attempts to mount a filesystem, but we recently have discussed that auto mount behaviour may be a superfluous feature, since mounting can easily be automated for<br>
nodes in a cluster.<br>
<br>
2) You're right the pervious version of the gluster hadoop filesystem implementation did not deal correctly with privileges.<br>
This is now fixed, however. You can get a "bleeding edge" jar which fixes your permissions error from the<br>
glusterfs-hadoop github repository: <a href="https://github.com/gluster/hadoop-glusterfs" target="_blank">https://github.com/gluster/hadoop-glusterfs</a>, where these fixes have been merged into head.<br>
<br>
Also we can get you this jar prebuilt if you want, just let me know!<br>
<br>
Thanks for trying out the GlusterFileSystem and keep the feedback coming !<br>
<div><div class="h5"><br>
----- Original Message -----<br>
From: "James Gurtowski" <<a href="mailto:gurtowskij@gmail.com">gurtowskij@gmail.com</a>><br>
To: <a href="mailto:jvyas@redhat.com">jvyas@redhat.com</a><br>
Cc: <a href="mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><br>
Sent: Monday, April 8, 2013 2:17:44 PM<br>
Subject: Gluster and Cloudera's Hadoop<br>
<br>
Hello,<br>
<br>
It seems the gluster hadoop plugin assumes all hadoop daemons/commands are<br>
run as root? I was having trouble getting the jobtracker to start because<br>
every time the fs is initialized a system call "mount -t glusterfs ..." is<br>
issued. Cloudera runs all daemons as the mapred user who is not allowed to<br>
run mount, so this is failing. I modified GlusterFileSystem.java (see<br>
attached diff) and set fs.glusterfs.automount to false in core-site.xml so<br>
this wouldn't happen.<br>
That fixed the initial issue of getting daemons to start.<br>
<br>
My next issue is getting hadoop jobs to run. I get an error:<br>
<br>
File /mnt/glusterfs/user/james/.staging/job_201304081221_0013/job.xml does<br>
not exist.<br>
<br>
I believe this to be a permissions issue, I can access this file fine from<br>
my account, but the .staging directory is only accessible by the user who<br>
launches the job :<br>
<br>
drwx------ 8 james james 870 Apr 8 14:10 .staging<br>
<br>
If I change the permissions, they are changed back (by Cloudera's hadoop)<br>
when I launch a job:<br>
Permissions on staging directory<br>
glusterfs://node001:9000/user/james/.staging are incorrect: rwxrwxrwx.<br>
Fixing permissions to correct value rwx------<br>
<br>
Any ideas of a work around would be greatly appreciated.<br>
<br>
Thanks,<br>
James<br>
</div></div>_______________________________________________<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
<a href="http://supercolony.gluster.org/mailman/listinfo/gluster-users" target="_blank">http://supercolony.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>Jay Vyas<br><a href="http://jayunit100.blogspot.com" target="_blank">http://jayunit100.blogspot.com</a>
</div>