How to use
trackSendPacketToLastNodeInPipeline
method
in
org.apache.hadoop.hdfs.server.datanode.BlockReceiver

Best Java code snippets using org.apache.hadoop.hdfs.server.datanode.BlockReceiver.trackSendPacketToLastNodeInPipeline (Showing top 1 results out of 315)

  mirrorAddr,
  duration);
trackSendPacketToLastNodeInPipeline(duration);
if (duration > datanodeSlowLogThresholdMs && LOG.isWarnEnabled()) {
 LOG.warn("Slow BlockReceiver write packet to mirror took " + duration

Javadoc

Only tracks the latency of sending packet to the last node in pipeline. This is a conscious design choice.

In the case of pipeline [dn0, dn1, dn2], 5ms latency from dn0 to dn1, 100ms from dn1 to dn2, NameNode claims dn2 is slow since it sees 100ms latency to dn2. Note that NameNode is not ware of pipeline structure in this context and only sees latency between two DataNodes.

In another case of the same pipeline, 100ms latency from dn0 to dn1, 5ms from dn1 to dn2, NameNode will miss detecting dn1 being slow since it's not the last node. However the assumption is that in a busy enough cluster there are many other pipelines where dn1 is the last node, e.g. [dn3, dn4, dn1]. Also our tracking interval is relatively long enough (at least an hour) to improve the chances of the bad DataNodes being the last nodes in multiple pipelines.

Popular methods of BlockReceiver

<init>
cleanupBlock
Cleanup a partial block if this write is for a replication request (and not from a client)
close
close files and release volume reference.
computePartialChunkCrc
reads in the partial crc chunk and computes checksum of pre-existing data in partial chunk.
handleMirrorOutError
While writing to mirrorOut, failure to write to mirror should not affect this datanode unless it is
receiveBlock
receivePacket
Receives and processes a packet. It can contain many chunks. returns the number of data bytes that t
verifyChunks
Verify multiple CRC chunks.
adjustCrcFilePosition
Adjust the file pointer in the local meta file so that the last checksum will be overwritten.
checksum2long
Convert a checksum byte array to a long
claimReplicaHandler
The caller claims the ownership of the replica handler.
copyLastChunkChecksum

Popular in Java

Updating database using SQL prepared statement
getResourceAsStream (ClassLoader)
orElseThrow (Optional)
Return the contained value, if present, otherwise throw an exception to be created by the provided s
setContentView (Activity)
VirtualMachine (com.sun.tools.attach)
A Java virtual machine. A VirtualMachine represents a Java virtual machine to which this Java vir
Color (java.awt)
The Color class is used to encapsulate colors in the default sRGB color space or colors in arbitrary
Component (java.awt)
A component is an object having a graphical representation that can be displayed on the screen and t
DecimalFormat (java.text)
A concrete subclass of NumberFormat that formats decimal numbers. It has a variety of features desig
ZipFile (java.util.zip)
This class provides random read access to a zip file. You pay more to read the zip file's central di
Filter (javax.servlet)
A filter is an object that performs filtering tasks on either the request to a resource (a servlet o
Top plugins for WebStorm

How to use trackSendPacketToLastNodeInPipelinemethodin org.apache.hadoop.hdfs.server.datanode.BlockReceiver

Best Java code snippets using org.apache.hadoop.hdfs.server.datanode.BlockReceiver.trackSendPacketToLastNodeInPipeline (Showing top 1 results out of 315)

How to use
trackSendPacketToLastNodeInPipeline
method
in
org.apache.hadoop.hdfs.server.datanode.BlockReceiver