Узлы Кассандры постоянно выдают эту ошибку - PullRequest
0 голосов
/ 24 февраля 2020

У меня есть 6 узлов Cassandra cluster, где 3 - это узлы данных, а 3 - начальные узлы

они все используют Cassandra версии 3.7

ниже приведены исключения, которые я получаю на узлах данных:

java.lang.IllegalStateException: Unable to compute ceiling for max when histogram overflowed
        at org.apache.cassandra.utils.EstimatedHistogram.rawMean(EstimatedHistogram.java:231) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.utils.EstimatedHistogram.mean(EstimatedHistogram.java:220) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.io.sstable.metadata.StatsMetadata.getEstimatedDroppableTombstoneRatio(StatsMetadata.java:115) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.io.sstable.format.SSTableReader.getEstimatedDroppableTombstoneRatio(SSTableReader.java:1829) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:387) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundSSTables(SizeTieredCompactionStrategy.java:99) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy.getNextBackgroundTask(SizeTieredCompactionStrategy.java:183) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:119) ~[apache-cassandra-3.7.jar:3.7]
        at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:258) ~[apache-cassandra-3.7.jar:3.7]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_222]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_222]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_222]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]

ниже - вывод для cfhistograms nodetool


I have one keyspace with 4 tables below is the output of nodetool cfhistorgrams

ubuntu@ankur:~$ nodetool cfhistograms keyspace.tabelname
Column counts are larger than 1996099046, unable to calculate percentiles
keyspace.tabelenamehistograms
Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                              (micros)          (micros)           (bytes)                  
50%            20.00             29.52          14530.76              9887               NaN
75%            24.00             35.43          25109.16             88148               NaN
95%            24.00             42.51          36157.19           4866323               NaN
98%            24.00             51.01          52066.35          36157190               NaN
99%            24.00             61.21          52066.35         155469300               NaN
Min             2.00              6.87             61.22                61                 0
Max            24.00       10299432.64         268650.95       765256466109223372036854776000

ubuntu@ankur:~$ nodetool cfhistogramskeyspace.tablename
keyspace.tabelename histograms
Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                              (micros)          (micros)           (bytes)                  
50%             0.00             20.50              0.00               215                 8
75%             0.00             24.60              0.00               310                14
95%             0.00             29.52              0.00               446                24
98%             0.00             29.52              0.00               535                29
99%             0.00             35.43              0.00               535                29
Min             0.00              3.31              0.00                73                 0
Max             0.00       10299432.64              0.00           4055269            219342

ubuntu@ankur:~$ nodetool cfhistograms keyspace.tabelename
keyspace.tabelename histograms
Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                              (micros)          (micros)           (bytes)                  
50%            24.00             24.60           1629.72               446                24
75%            29.00             29.52           2816.16               535                29
95%            29.00             35.43           7007.51               924                50
98%            29.00             42.51          12108.97              1109                60
99%            29.00             42.51          14530.76              1331                86
Min             2.00              5.72             20.50                51                 0
Max            35.00       10299432.64         557074.61           4866323            654949

ubuntu@ankur:~$ nodetool cfhistograms keyspace.tabelename
keyspace.tabelename histograms
Percentile  SSTables     Write Latency      Read Latency    Partition Size        Cell Count
                              (micros)          (micros)           (bytes)                  
50%             0.00             14.24              0.00               124                 2
75%             0.00             17.08              0.00               149                 2
95%             0.00             24.60              0.00               215                 4
98%             0.00             24.60              0.00               372                 7
99%             0.00             29.52              0.00               535                12
Min             0.00              1.33              0.00                36                 0
Max             0.00       10299432.64              0.00           8409007            182785

Ниже приведена структура таблицы, которой мы следуем для всех таблиц

CREATE TABLE keyspace.tablename (
    xxxx uuid,
    xxxx timeuuid,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx uuid,
    xxxx text,
    city text,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx timeuuid,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx text,
    xxxx text,
    convtype text,
    xxxx float,
    xxxx float,
    xxxx float,
    xxxx float,
    xxxx text,
    xxxx text,
    xxxx ascii,
    xxxx text,
    device text,
    eventxxxxaction text,
    xxxx text,
    xxxx text,
    xxxx int,
    xxxx int,
    xxxx text,
    xxxx uuid,
    xxxx int,
    ip inet,
    xxxx float,
    xxxx int,
    xxxx uuid,
    xxxx float,
    xxxx boolean,
    xxxx uuid,
    xxxx text,
    params map<text, text>,
    referrer text,
    servicelist set<frozen<service_item>>,
    xxxx text,
    xxxx text,
    uxxxxseragent text,
    xxxx text,
    xxxx text,
    PRIMARY KEY (xxx)
) WITH CLUSTERING ORDER BY (created DESC)
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.1
    AND default_time_to_live = 0
    AND gc_grace_seconds = 86400
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = '99PERCENTILE';

Любые предложения или предложения будут цениться

Спасибо, Анкур

Добро пожаловать на сайт PullRequest, где вы можете задавать вопросы и получать ответы от других членов сообщества.
...