皮皮网

1.如何解决 hbase regiontoobusyexception

如何解决 hbase regiontoobusyexception

       Hbase扫描时抛出ScannerTimeoutException处理

       åœ¨å¯¹Hbase进行Scan的时候有时候会抛出ScannerTimeoutException，场景如下：

        :: (mins,类源通达信主力动向公式源码 sec)

       org.apache.hadoop.hbase.client.ScannerTimeoutException: ms passed since the last invocation, timeout is currently set to

        at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:)

        at org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:)

        at org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:)

        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)

        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:)

        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)

        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)

        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)

        at org.apache.hadoop.mapred.Child$4.run(Child.java:)

        at java.security.AccessController.doPrivileged(Native Method)

        at javax.security.auth.Subject.doAs(Subject.java:)

        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)

        at org.apache.hadoop.mapred.Child.main(Child.java:)

       åŽŸå› æŸ¥çœ‹æ­¥éª¤ï¼š

       æŸ¥æ‰¾æºç å¦‚下：

       long timeout = lastNext + scannerTimeout;

        // If we are over the timeout, throw this exception to the client

        // Else, it's because the region moved and we used the old id

        // against the new region server; reset the scanner.

        if (timeout < System.currentTimeMillis()) {

        long elapsed = System.currentTimeMillis() - lastNext;

        ScannerTimeoutException ex = new ScannerTimeoutException(

        elapsed + "ms passed since the last invocation, " +

        "timeout is currently set to " + scannerTimeout);

        ex.initCause(e);

        throw ex;

        }

       æ ¸å®žä¸‹å¦‚下代码

       long timeout = lastNext + scannerTimeout;

       å¾—出：ClientScanner.lastNext和HTable.scannerTimeout

       è·Ÿè¸ªï¼šHTable.scannerTimeout

       this.scannerTimeout =

        (int) conf.getLong(HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY, HConstants.DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD);

       æŸ¥çœ‹HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY和HConstants.DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD

       æˆ‘们可以得知如果没有设置HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY那么他们采用默认值即

       public static long DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD = ;

        跟踪：ClientScanner.lastNext发现此为上次访问时间

       è§£å†³æ–¹æ¡ˆï¼š

       å¯ä»¥é€šè¿‡è®¾ç½®HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY解决

       ä»£ç å¦‚下：

       config.setLong(HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY, );