Hi list,<br><br>I see the following exception on release-3.3 configured with hadoop compatible storage, <br>The problem was occurring when I run the Terasort benchmarks "hadoop jar hadoop-0.20.2-examples.jar terasort output sorted"<br>
<br><br>12/07/31 01:08:36 INFO terasort.TeraSort: starting<br>Initializing GlusterFS<br>12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to process : 1<br>12/07/31 01:08:36 WARN snappy.LoadSnappy: Snappy native library is available<br>
12/07/31 01:08:36 INFO util.NativeCodeLoader: Loaded the native-hadoop library<br>12/07/31 01:08:36 INFO snappy.LoadSnappy: Snappy native library loaded<br>12/07/31 01:08:36 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library<br>
12/07/31 01:08:36 INFO compress.CodecPool: Got brand-new compressor<br>Making 1 from 100000 records<br>Step size is 100000.0<br>12/07/31 01:08:36 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=<br>
12/07/31 01:08:36 INFO mapred.FileInputFormat: Total input paths to process : 1<br>12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating _partition.lst in /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output-work-2689044872239508516 with rwxr-xr-x<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Cached glusterfs://Namenode1:9000/mnt/output/_partition.lst#_partition.lst as /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst<br>
12/07/31 01:08:37 WARN mapred.LocalJobRunner: LocalJobRunner does not support symlinking into current working dir.<br>12/07/31 01:08:37 INFO mapred.TaskRunner: Creating symlink: /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst <- /data/mapred/local/localRunner/_partition.lst<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.jar <- /data/mapred/local/localRunner/job.jar<br>12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.xml.crc <- /data/mapred/local/localRunner/.job.xml.crc<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.split <- /data/mapred/local/localRunner/job.split<br>12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.splitmetainfo <- /data/mapred/local/localRunner/job.splitmetainfo<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.split.crc <- /data/mapred/local/localRunner/.job.split.crc<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.splitmetainfo.crc <- /data/mapred/local/localRunner/.job.splitmetainfo.crc<br>
12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/job.xml <- /data/mapred/local/localRunner/job.xml<br>12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Creating symlink: /tmp/hadoop-root/mapred/staging/root-434405171/.staging/job_local_0001/.job.jar.crc <- /data/mapred/local/localRunner/.job.jar.crc<br>
12/07/31 01:08:37 INFO mapred.JobClient: Running job: job_local_0001<br>12/07/31 01:08:37 INFO mapred.MapTask: numReduceTasks: 1<br>12/07/31 01:08:37 INFO mapred.MapTask: io.sort.mb = 100<br>12/07/31 01:08:37 INFO mapred.MapTask: data buffer = 79691776/99614720<br>
12/07/31 01:08:37 INFO mapred.MapTask: record buffer = 262144/327680<br>12/07/31 01:08:37 WARN mapred.LocalJobRunner: job_local_0001<br>java.lang.RuntimeException: Error in configuring object<br> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)<br>
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)<br> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)<br> at org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(MapTask.java:481)<br>
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)<br> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)<br> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:210)<br>
Caused by: java.lang.reflect.InvocationTargetException<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br>
at java.lang.reflect.Method.invoke(Method.java:597)<br> at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)<br> ... 6 more<br>Caused by: java.lang.IllegalArgumentException: can't read paritions file<br>
at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:213)<br> ... 11 more<br>Caused by: java.io.FileNotFoundException: File _partition.lst does not exist.<br> at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:383)<br>
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)<br> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:776)<br> at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)<br>
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)<br> at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.readPartitions(TeraSort.java:153)<br> at org.apache.hadoop.examples.terasort.TeraSort$TotalOrderPartitioner.configure(TeraSort.java:210)<br>
... 11 more<br>12/07/31 01:08:37 INFO filecache.TrackerDistributedCacheManager: Deleted path /data/mapred/local/archive/3689403515442034031_-1440303010_1461729625/Namenode1/mnt/output/_partition.lst<br>12/07/31 01:08:38 INFO mapred.JobClient: map 0% reduce 0%<br>
12/07/31 01:08:38 INFO mapred.JobClient: Job complete: job_local_0001<br>12/07/31 01:08:38 INFO mapred.JobClient: Counters: 0<br>12/07/31 01:08:38 INFO mapred.JobClient: Job Failed: NA<br>java.io.IOException: Job failed!<br>
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1246)<br> at org.apache.hadoop.examples.terasort.TeraSort.run(TeraSort.java:248)<br> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)<br> at org.apache.hadoop.examples.terasort.TeraSort.main(TeraSort.java:257)<br>
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)<br> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br>
at java.lang.reflect.Method.invoke(Method.java:597)<br> at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)<br> at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)<br>
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)<br> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)<br>
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)<br> at java.lang.reflect.Method.invoke(Method.java:597)<br> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)<br><br clear="all">
<br>-- <br>Best Regards,<br>NiuZG<br>