This page lists common issues encountered by users of OpenTSDB along with various troubleshooting steps. If you run into an issue, please check the OpenTSDB Google Group or the Github Issues. If you can't find an answer, please include your operating system, TSD version and HBase version in your question.
This can be caused if you use millisecond timestamps and write thousands of data points for a single metric in a single hour. In this case, the column qualifier and row key can grow larger than the configured
hfile.index.block.max.size. In this situation we recommend that you disable TSD compaction code. In the future we will support appends which will allow for compacted columns with small qualifiers.
During region splits or region migrations, OpenTSDB's AsyncHBase client will buffer RPCs in memory and attempt to flush them once the regions are back online. Each region has a 10,000 RPC buffer by default and if many regions are down then the RPCs can eventually fill up the TSD heap and cause long garbage collection pauses. If this happens, you can either increase your heap to accommodate more region splits or decrease the NSRE queue size by modifying the
hbase.nsre.high_watermark config parameter in AsyncHBase 1.7 and OpenTSDB 2.2.
There are a number of potential causes for this problem including:
© 2010–2016 The OpenTSDB Authors
Licensed under the GNU LGPLv2.1+ and GPLv3+ licenses.