Hi,
I'm facing a problem with the Sqoop import step I'd like to use. When retrieving data from a MySQL database, I encounter the following error:
I've already looked at the apache site for CouldOnlyBeReplicatedTo (http://wiki.apache.org/hadoop/CouldOnlyBeReplicatedTo)
I'm running Hadoop in a Parallels VM on one node. It is the Cloudera CDH distribution (4.3.0) installed trough Cloudera Manager.
I'm stuck on this for the last two days, and I can't seem to figure it out.
If anyone got any pointers, that would be greatly appreciated.
I'm facing a problem with the Sqoop import step I'd like to use. When retrieving data from a MySQL database, I encounter the following error:
Code:
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : 2013/08/08 09:43:47 - PriviledgedActionException as:vincentk (auth:SIMPLE) cause:java.io.IOException: File /user/vincentk/.staging/job_201308080940_0001/libjars/mysql-connector-java-5.1.26-bin.jar could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1331)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2198)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:480)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:299)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44954)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at java.security.AccessController.doPrivileged(Native Method)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at javax.security.auth.Subject.doAs(Subject.java:396)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : 2013/08/08 09:43:47 - Encountered IOException running import job: java.io.IOException: File /user/vincentk/.staging/job_201308080940_0001/libjars/mysql-connector-java-5.1.26-bin.jar could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1331)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2198)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:480)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:299)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44954)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at java.security.AccessController.doPrivileged(Native Method)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at javax.security.auth.Subject.doAs(Subject.java:396)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) :
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Client.call(Client.java:1161)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:184)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at com.sun.proxy.$Proxy25.addBlock(Unknown Source)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at java.lang.reflect.Method.invoke(Method.java:597)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:165)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:84)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at com.sun.proxy.$Proxy25.addBlock(Unknown Source)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:285)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1104)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:979)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:455)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : 2013/08/08 09:43:47 - Got exception running Sqoop: java.lang.RuntimeException: java.io.IOException: File /user/vincentk/.staging/job_201308080940_0001/libjars/mysql-connector-java-5.1.26-bin.jar could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1331)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2198)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:480)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:299)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:44954)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:453)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1002)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1701)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1697)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at java.security.AccessController.doPrivileged(Native Method)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at javax.security.auth.Subject.doAs(Subject.java:396)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
2013/08/08 09:43:47 - mainJob - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1695)- The datanode in my VM is running.
- There are no networking problems, there is nothing in the logs about connectivity problems and the NameNode and DataNode are on the same server (VM). Writing files to HDFS is possible from local vm filesystem.
- There is enough hard disk capacity:
| Configured Capacity | : | 81.41 GB | |||
| DFS Used | : | 59.05 MB | |||
| Non DFS Used | : | 5.20 GB | |||
| DFS Remaining | : | 76.15 GB | |||
| DFS Used% | : | 0.07% | |||
| DFS Remaining% | : | 93.54% | |||
| Block Pool Used | : | 59.05 MB | |||
| Block Pool Used% | : | 0.07% | |||
| DataNodes usages | : | Min % | Median % | Max % | stdev % |
| 0.07% | 0.07% | 0.07% | 0.00% | ||
| Live Nodes | : | 1 (Decommissioned: 0) | |||
| Dead Nodes | : | 0 (Decommissioned: 0) | |||
| Decommissioning Nodes | : | 0 | |||
| Number of Under-Replicated Blocks | : | 103 |
- Reserved space is not bigger than the available space.
- jobtacker.info can be created.
I'm running Hadoop in a Parallels VM on one node. It is the Cloudera CDH distribution (4.3.0) installed trough Cloudera Manager.
I'm stuck on this for the last two days, and I can't seem to figure it out.
If anyone got any pointers, that would be greatly appreciated.