Массовая загрузка данных в Кассандре с помощью Pelops - PullRequest
Новая
       31

Массовая загрузка данных в Кассандре с помощью Pelops

2 голосов
/ 03 апреля 2012

Я использую Пелопс для работы с Кассандрой.Я хочу вставить объемные данные, используя мутатор.Когда я вставляю около 20000 записей, он работает нормально, но когда я превышаю число записей, он показывает «Сброс соединения по пиру: ошибка записи сокета».Я пишу следующий код. 

Pelops.addPool(CASSANDRA_POOL, new Cluster(CASSANDRA_HOST,
            CASSANDRA_PORT), CASSANDRA_KEYSPACE);
Mutator mutator = Pelops.createMutator(CASSANDRA_POOL);

//writing cols in mutator
mutator.execute(CL_ONE)

Но он работает для ограниченного числа записей.

Я получаю следующую ошибку: 

2757 17:42:01,211 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Attempting to borrow free connection for node 'localhost'

2757 17:42:01,211 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Borrowing connection 'Connection[MyKeyspace][localhost:9160][44937684]'

2977 17:42:01,431 WARN  org.scale7.cassandra.pelops.Operand - Operation failed as result of network exception. Connection to node localhost is being marked as corrupt (and will probably be be destroyed). Cause of failure is org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset by peer: socket write error

2977 17:42:01,431 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Returned connection 'Connection[MyKeyspace][localhost:9160][44937684]' has been closed or is marked as corrupt

2977 17:42:01,431 DEBUG org.scale7.cassandra.pelops.pool.CommonsBackedPool - Destroying connection 'Connection[MyKeyspace][localhost:9160][44937684]'

Exception in thread "main" org.scale7.cassandra.pelops.exceptions.TransportException: java.net.SocketException: Connection reset by peer: socket write error
    at org.scale7.cassandra.pelops.exceptions.IExceptionTranslator$ExceptionTranslator.translate(IExceptionTranslator.java:57)
    at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:115)
    at org.scale7.cassandra.pelops.Mutator.execute(Mutator.java:102)
    at org.scale7.cassandra.pelops.Mutator.execute(Mutator.java:71)
    at org.brickred.Cassandra.insertData(Cassandra.java:182)
    at org.brickred.Test.main(Test.java:7)
Caused by: org.apache.thrift.transport.TTransportException: java.net.SocketException: Connection reset by peer: socket write error
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:147)
    at org.apache.thrift.transport.TFramedTransport.flush(TFramedTransport.java:157)
    at org.apache.cassandra.thrift.Cassandra$Client.send_batch_mutate(Cassandra.java:1020)
    at org.apache.cassandra.thrift.Cassandra$Client.batch_mutate(Cassandra.java:1008)
    at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:97)
    at org.scale7.cassandra.pelops.Mutator$1.execute(Mutator.java:93)
    at org.scale7.cassandra.pelops.Operand.tryOperation(Operand.java:81)
    ... 4 more
Caused by: java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(Unknown Source)
    at java.net.SocketOutputStream.write(Unknown Source)
    at org.apache.thrift.transport.TIOStreamTransport.write(TIOStreamTransport.java:145)
    ... 10 more
...