Created
November 21, 2013 01:09
-
-
Save imaxxs/7574282 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| STORE cleaned_table_3 INTO 'hdfs://localhost/queryResults/dcdabb39a7f14304996378a825182a16/b29175766a4a58b62cf384fc67e0c7b56d8dd4da/cleaned_table_3.csv' USING PigStorage(','); | |
| 2013-11-21 01:02:38,955 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigRunner - pig -P /tmp/pig-6923067481220530956.properties -l /tmp/pig-9019460696332228699.log -f /tmp/pig-script-3626976431948664845.pig | |
| 2013-11-21 01:02:38,963 [pool-3-thread-2] INFO org.apache.pig.Main - Apache Pig version 0.11.0-cdh4.3.0 (rexported) compiled May 27 2013, 20:40:22 | |
| 2013-11-21 01:02:38,963 [pool-3-thread-2] INFO org.apache.pig.Main - Logging error messages to: /tmp/pig-9019460696332228699.log | |
| 2013-11-21 01:02:38,963 [pool-3-thread-2] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS | |
| 2013-11-21 01:02:38,969 [pool-3-thread-2] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS | |
| 2013-11-21 01:02:38,969 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:02:38,969 [pool-3-thread-2] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/vagrant/.pigbootup not found | |
| 2013-11-21 01:02:39,205 [pool-3-thread-2] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS | |
| 2013-11-21 01:02:39,206 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: localhost:8020 | |
| 2013-11-21 01:02:39,207 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:02:39,207 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021 | |
| 2013-11-21 01:02:39,207 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:02:39,218 [pool-3-thread-2] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS | |
| 2013-11-21 01:02:39,218 [pool-3-thread-2] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum | |
| 2013-11-21 01:02:39,218 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:02:41,999 [Low Memory Detector] INFO org.apache.pig.impl.util.SpillableMemoryManager - first memory handler call - Collection threshold init = 43253760(42240K) used = 409277344(399684K) committed = 577110016(563584K) max = 692060160(675840K) | |
| 2013-11-21 01:02:42,997 [Low Memory Detector] INFO org.apache.pig.impl.util.SpillableMemoryManager - first memory handler call- Usage threshold init = 43253760(42240K) used = 485160992(473790K) committed = 577110016(563584K) max = 692060160(675840K) | |
| 2013-11-21 01:03:03,196 [pool-3-thread-2] WARN org.apache.pig.PigServer - Encountered Warning IMPLICIT_CAST_TO_DOUBLE 1 time(s). | |
| 2013-11-21 01:03:43,098 [pool-3-thread-2] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: FILTER | |
| 2013-11-21 01:03:55,489 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false | |
| 2013-11-21 01:03:55,797 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 3 | |
| 2013-11-21 01:03:55,797 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - Merged 2 map-only splittees. | |
| 2013-11-21 01:03:55,797 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - Merged 2 out of total 3 MR operators. | |
| 2013-11-21 01:03:55,797 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1 | |
| 2013-11-21 01:03:55,914 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigProgressListener - Script '69526e42-2cfa-4976-ab5f-ab3567b74261' for job '528d5ae394a1609d0a000004' launching (1 child MapReduce jobs to launch) | |
| 2013-11-21 01:03:55,914 [pool-3-thread-2] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job | |
| 2013-11-21 01:03:56,046 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 | |
| 2013-11-21 01:03:56,083 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator | |
| 2013-11-21 01:03:56,153 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=22088297 | |
| 2013-11-21 01:03:56,153 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting Parallelism to 1 | |
| 2013-11-21 01:03:56,155 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:03:56,276 [pool-3-thread-2] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:03:56,277 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - creating jar file Job5168084813806921098.jar | |
| 2013-11-21 01:04:00,044 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - jar file Job5168084813806921098.jar created | |
| 2013-11-21 01:04:00,089 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up multi store job | |
| 2013-11-21 01:04:00,247 [pool-3-thread-2] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code. | |
| 2013-11-21 01:04:00,248 [pool-3-thread-2] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cacche | |
| 2013-11-21 01:04:00,248 [pool-3-thread-2] INFO org.apache.pig.data.SchemaTupleFrontend - Setting key [pig.schematuple.classes] with classes to deserialize [] | |
| 2013-11-21 01:04:08,203 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 1 map-reduce job(s) waiting for submission. | |
| 2013-11-21 01:04:08,203 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigProgressListener - Script '69526e42-2cfa-4976-ab5f-ab3567b74261' for job '528d5ae394a1609d0a000004' submitted as 1 child MapReduce jobs | |
| 2013-11-21 01:04:08,214 [JobControl] WARN org.apache.hadoop.mapred.JobClient - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. | |
| 2013-11-21 01:04:08,227 [JobControl] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:04:08,681 [JobControl] WARN org.apache.hadoop.fs.FileSystem - "localhost:8020" is a deprecated filesystem name. Use "hdfs://localhost:8020/" instead. | |
| 2013-11-21 01:04:08,704 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete | |
| 2013-11-21 01:04:08,727 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.client.MonitorClient - Request succeeded to monitor precise64:8001 | |
| 2013-11-21 01:04:08,805 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 | |
| 2013-11-21 01:04:08,808 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 | |
| 2013-11-21 01:04:09,516 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_201311210050_0002 | |
| 2013-11-21 01:04:09,516 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases castOutput,cleaned_table,cleaned_table_1,cleaned_table_2,cleaned_table_3,original_528bd071475a011915000003_1 | |
| 2013-11-21 01:04:09,516 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: original_528bd071475a011915000003_1[2,38],cleaned_table[-1,-1],cleaned_table_2[6,18],castOutput[4,13],cleaned_table_1[5,18],cleaned_table_3[7,18],cleaned_table_3[9,18],cleaned_table_3[10,18] C: R: | |
| 2013-11-21 01:04:09,516 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_201311210050_0002 | |
| 2013-11-21 01:04:09,611 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigProgressListener - Script '69526e42-2cfa-4976-ab5f-ab3567b74261' for job '528d5ae394a1609d0a000004' started (assigned MR job ID job_201311210050_0002) | |
| 2013-11-21 01:05:39,623 [pool-3-thread-2] INFO org.apache.pig.tools.pigstats.JobStats - using output size reader: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader | |
| 2013-11-21 01:05:39,630 [pool-3-thread-2] INFO org.apache.pig.tools.pigstats.JobStats - using output size reader: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.FileBasedOutputSizeReader | |
| 2013-11-21 01:05:39,650 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.client.MonitorClient - Request succeeded to monitor precise64:8001 | |
| 2013-11-21 01:05:39,650 [pool-3-thread-2] WARN org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure. | |
| 2013-11-21 01:05:39,650 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - job job_201311210050_0002 has failed! Stop running all dependent jobs | |
| 2013-11-21 01:05:39,672 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.client.MonitorClient - Request succeeded to monitor precise64:8001 | |
| 2013-11-21 01:05:39,672 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 100% complete | |
| 2013-11-21 01:05:39,674 [pool-3-thread-2] ERROR org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to recreate exception from backed error: Error: GC overhead limit exceeded | |
| 2013-11-21 01:05:39,674 [pool-3-thread-2] ERROR org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! | |
| 2013-11-21 01:05:39,674 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigProgressListener - Script '69526e42-2cfa-4976-ab5f-ab3567b74261' for job '528d5ae394a1609d0a000004' completed (0/1 child MapReduce jobs completed) | |
| 2013-11-21 01:05:39,693 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.client.MonitorClient - Request succeeded to monitor precise64:8001 | |
| 2013-11-21 01:05:39,693 [pool-3-thread-2] INFO org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: | |
| HadoopVersion PigVersion UserId StartedAt FinishedAt Features | |
| 2.0.0-cdh4.3.0 0.11.0-cdh4.3.0 vagrant 2013-11-21 01:03:55 2013-11-21 01:05:39 FILTER | |
| Failed! | |
| Failed Jobs: | |
| JobId Alias Feature Message Outputs | |
| job_201311210050_0002 castOutput,cleaned_table,cleaned_table_1,cleaned_table_2,cleaned_table_3,original_528bd071475a011915000003_1 MULTI_QUERY,MAP_ONLY Message: Job failed! hdfs://localhost/queryResults/dcdabb39a7f14304996378a825182a16/b29175766a4a58b62cf384fc67e0c7b56d8dd4da/cleaned_table_3,hdfs://localhost/queryResults/dcdabb39a7f14304996378a825182a16/b29175766a4a58b62cf384fc67e0c7b56d8dd4da/cleaned_table_3.csv, | |
| Input(s): | |
| Failed to read data from "hdfs://localhost/f77be6ba92214126bdcbaf2bb1290e2f_5000.txt" | |
| Output(s): | |
| Failed to produce result in "hdfs://localhost/queryResults/dcdabb39a7f14304996378a825182a16/b29175766a4a58b62cf384fc67e0c7b56d8dd4da/cleaned_table_3" | |
| Failed to produce result in "hdfs://localhost/queryResults/dcdabb39a7f14304996378a825182a16/b29175766a4a58b62cf384fc67e0c7b56d8dd4da/cleaned_table_3.csv" | |
| Counters: | |
| Total records written : 0 | |
| Total bytes written : 542849 | |
| Spillable Memory Manager spill count : 0 | |
| Total bags proactively spilled: 0 | |
| Total records proactively spilled: 0 | |
| Job DAG: | |
| job_201311210050_0002 | |
| 2013-11-21 01:05:39,693 [pool-3-thread-2] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed! | |
| 2013-11-21 01:05:39,699 [pool-3-thread-2] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Unable to recreate exception from backed error: Error: GC overhead limit exceeded | |
| Details at logfile: /tmp/pig-9019460696332228699.log | |
| 2013-11-21 01:05:39,708 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigRunner - Notifying monitor for job '528d5ae394a1609d0a000004', code 'FAILURE' | |
| 2013-11-21 01:05:39,708 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.pig_runner.BatchPigRunner - Exception: ScriptFailedException: Script execution failed | |
| 2013-11-21 01:05:39,721 [pool-3-thread-2] INFO com.trifacta.hadoopdata.monitor.client.MonitorClient - Request succeeded to monitor precise64:8001 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment