java - Apache Pig: unable to run my own pig.jar and pig-withouthadoop.jar -


i have cluster running hadoop 0.20.2 , pig 0.10. i'm interested add logs pig's source code , run own pig version on cluster.

what did:

  1. built project 'ant' command

  2. got pig.jar , pig-withouthadoop.jar

  3. copied jars pig home directory on cluster's namenode

  4. run job

then i've got following std output:

2013-03-25 06:35:05,226 [main] warn  org.apache.pig.backend.hadoop20.pigjobcontrol -   falling default jobcontrol (not using hadoop 0.20 ?) java.lang.nosuchfieldexception: runnerstate     @ java.lang.class.getdeclaredfield(class.java:1882)     @ org.apache.pig.backend.hadoop20.pigjobcontrol.<clinit>(pigjobcontrol.java:51)     @  org.apache.pig.backend.hadoop.executionengine.shims.hadoopshims.newjobcontrol(hadoopshims.java:97)     @ org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler.compile(jobcontrolcompiler.java:287)     @ org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher.launchpig(mapreducelauncher.java:177)     @ org.apache.pig.pigserver.launchplan(pigserver.java:1320)     @ org.apache.pig.pigserver.executecompiledlogicalplan(pigserver.java:1305)     @ org.apache.pig.pigserver.execute(pigserver.java:1295)     @ org.apache.pig.pigserver.executebatch(pigserver.java:375)     @ org.apache.pig.pigserver.executebatch(pigserver.java:353)     @ org.apache.pig.tools.grunt.gruntparser.executebatch(gruntparser.java:137)     @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:198)     @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:170)     @ org.apache.pig.tools.grunt.grunt.exec(grunt.java:84)     @ org.apache.pig.main.run(main.java:480)     @ org.apache.pig.main.main(main.java:157)     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method)     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39)     @  sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25)     @ java.lang.reflect.method.invoke(method.java:597)     @ org.apache.hadoop.util.runjar.main(runjar.java:208) 2013-03-25 06:35:05,229 [main] info  org.apache.pig.tools.pigstats.scriptstate - pig script settings added job 2013-03-25 06:35:05,260 [main] info  org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler - mapred.job.reduce.markreset.buffer.percent not set, set default 0.3 2013-03-25 06:35:05,272 [main] info  org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler - setting parallelism 1 2013-03-25 06:35:06,041 [main] info  org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler - creating jar file job9091543475518322185.jar 2013-03-25 06:35:10,974 [main] info  org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler - jar file job9091543475518322185.jar created 2013-03-25 06:35:10,995 [main] info  org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler - setting single store job 2013-03-25 06:35:11,006 [main] info  org.apache.pig.data.schematuplefrontend - key [pig.schematuple] false, not generate code. 2013-03-25 06:35:11,006 [main] info  org.apache.pig.data.schematuplefrontend - starting process move generated code distributed cacche 2013-03-25 06:35:11,006 [main] info  org.apache.pig.data.schematuplefrontend - setting key [pig.schematuple.classes] classes deserialize [] 2013-03-25 06:35:11,181 [main] error org.apache.pig.tools.grunt.grunt - error 2998: unhandled internal error. org.apache.hadoop.mapred.jobcontrol.jobcontrol.addjob(lorg/apache/hadoop/mapred/jobcontrol/job;)ljava/lang/string; 

pig stack trace:

error 2998: unhandled internal error. org.apache.hadoop.mapred.jobcontrol.jobcontrol.addjob(lorg/apache/hadoop/mapred/jobcontrol/job;)ljava/lang/string;  java.lang.nosuchmethoderror: org.apache.hadoop.mapred.jobcontrol.jobcontrol.addjob(lorg/apache/hadoop/mapred/jobcontrol/job;)ljava/lang/string;     @ org.apache.pig.backend.hadoop.executionengine.mapreducelayer.jobcontrolcompiler.compile(jobcontrolcompiler.java:298)     @ org.apache.pig.backend.hadoop.executionengine.mapreducelayer.mapreducelauncher.launchpig(mapreducelauncher.java:177)     @ org.apache.pig.pigserver.launchplan(pigserver.java:1320)     @ org.apache.pig.pigserver.executecompiledlogicalplan(pigserver.java:1305)     @ org.apache.pig.pigserver.execute(pigserver.java:1295)     @ org.apache.pig.pigserver.executebatch(pigserver.java:375)     @ org.apache.pig.pigserver.executebatch(pigserver.java:353)     @ org.apache.pig.tools.grunt.gruntparser.executebatch(gruntparser.java:137)     @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:198)     @ org.apache.pig.tools.grunt.gruntparser.parsestoponerror(gruntparser.java:170)     @ org.apache.pig.tools.grunt.grunt.exec(grunt.java:84)     @ org.apache.pig.main.run(main.java:480)     @ org.apache.pig.main.main(main.java:157)     @ sun.reflect.nativemethodaccessorimpl.invoke0(native method)     @ sun.reflect.nativemethodaccessorimpl.invoke(nativemethodaccessorimpl.java:39)     @ sun.reflect.delegatingmethodaccessorimpl.invoke(delegatingmethodaccessorimpl.java:25)     @ java.lang.reflect.method.invoke(method.java:597)     @ org.apache.hadoop.util.runjar.main(runjar.java:208) 

what went wrong? should else except replacing pig.jar , pig-withouthadoop.jar in installation directory of namenode?

help...

the point missed was: pig-withouthadoop.jar should compiled specific hadoop version. compiled jar in following way , worked:

% ant clean jar-withouthadoop -dhadoopversion=23 

Comments

Popular posts from this blog

php - Wordpress website dashboard page or post editor content is not showing but front end data is showing properly -

How to get the ip address of VM and use it to configure SSH connection dynamically in Ansible -

javascript - Get parameter of GET request -