Metrics Reference¶
Orderer Metrics¶
Prometheus¶
The following orderer metrics are exported for consumption by Prometheus.
Name | Type | Description | Labels | |
---|---|---|---|---|
blockcutter_block_fill_duration | histogram | The time from first transaction enqueing to the block being cut in seconds. | channel | |
broadcast_enqueue_duration | histogram | The time to enqueue a transaction in seconds. | channel | |
type | ||||
status | ||||
broadcast_processed_count | counter | The number of transactions processed. | channel | |
type | ||||
status | ||||
broadcast_validate_duration | histogram | The time to validate a transaction in seconds. | channel | |
type | ||||
status | ||||
cluster_comm_egress_queue_capacity | gauge | Capacity of the egress queue. | host | |
msg_type | ||||
channel | ||||
cluster_comm_egress_queue_length | gauge | Length of the egress queue. | host | |
msg_type | ||||
channel | ||||
cluster_comm_egress_queue_workers | gauge | Count of egress queue workers. | channel | |
cluster_comm_egress_stream_count | gauge | Count of streams to other nodes. | channel | |
cluster_comm_egress_tls_connection_count | gauge | Count of TLS connections to other nodes. | ||
cluster_comm_ingress_stream_count | gauge | Count of streams from other nodes. | ||
cluster_comm_msg_dropped_count | counter | Count of messages dropped. | host | |
channel | ||||
cluster_comm_msg_send_time | histogram | The time it takes to send a message in seconds. | host | |
channel | ||||
consensus_etcdraft_active_nodes | gauge | Number of active nodes in this channel. | channel | |
consensus_etcdraft_cluster_size | gauge | Number of nodes in this channel. | channel | |
consensus_etcdraft_committed_block_number | gauge | The block number of the latest block committed. | channel | |
consensus_etcdraft_config_proposals_received | counter | The total number of proposals received for config type transactions. | channel | |
consensus_etcdraft_data_persist_duration | histogram | The time taken for etcd/raft data to be persisted in storage (in seconds). | channel | |
consensus_etcdraft_is_leader | gauge | The leadership status of the current node: 1 if it is the leader else 0. | channel | |
consensus_etcdraft_leader_changes | counter | The number of leader changes since process start. | channel | |
consensus_etcdraft_normal_proposals_received | counter | The total number of proposals received for normal type transactions. | channel | |
consensus_etcdraft_proposal_failures | counter | The number of proposal failures. | channel | |
consensus_etcdraft_snapshot_block_number | gauge | The block number of the latest snapshot. | channel | |
consensus_kafka_batch_size | gauge | The mean batch size in bytes sent to topics. | topic | |
consensus_kafka_compression_ratio | gauge | The mean compression ratio (as percentage) for topics. | topic | |
consensus_kafka_incoming_byte_rate | gauge | Bytes/second read off brokers. | broker_id | |
consensus_kafka_last_offset_persisted | gauge | The offset specified in the block metadata of the most recently committed block. | channel | |
consensus_kafka_outgoing_byte_rate | gauge | Bytes/second written to brokers. | broker_id | |
consensus_kafka_record_send_rate | gauge | The number of records per second sent to topics. | topic | |
consensus_kafka_records_per_request | gauge | The mean number of records sent per request to topics. | topic | |
consensus_kafka_request_latency | gauge | The mean request latency in ms to brokers. | broker_id | |
consensus_kafka_request_rate | gauge | Requests/second sent to brokers. | broker_id | |
consensus_kafka_request_size | gauge | The mean request size in bytes to brokers. | broker_id | |
consensus_kafka_response_rate | gauge | Requests/second sent to brokers. | broker_id | |
consensus_kafka_response_size | gauge | The mean response size in bytes from brokers. | broker_id | |
deliver_blocks_sent | counter | The number of blocks sent by the deliver service. | channel | |
filtered | ||||
data_type | ||||
deliver_requests_completed | counter | The number of deliver requests that have been completed. | channel | |
filtered | ||||
data_type | ||||
success | ||||
deliver_requests_received | counter | The number of deliver requests that have been received. | channel | |
filtered | ||||
data_type | ||||
deliver_streams_closed | counter | The number of GRPC streams that have been closed for the deliver service. | ||
deliver_streams_opened | counter | The number of GRPC streams that have been opened for the deliver service. | ||
fabric_version | gauge | The active version of Fabric. | version | |
grpc_comm_conn_closed | counter | gRPC connections closed. Open minus closed is the active number of connections. | ||
grpc_comm_conn_opened | counter | gRPC connections opened. Open minus closed is the active number of connections. | ||
grpc_server_stream_messages_received | counter | The number of stream messages received. | service | |
method | ||||
grpc_server_stream_messages_sent | counter | The number of stream messages sent. | service | |
method | ||||
grpc_server_stream_request_duration | histogram | The time to complete a stream request. | service | |
method | ||||
code | ||||
grpc_server_stream_requests_completed | counter | The number of stream requests completed. | service | |
method | ||||
code | ||||
grpc_server_stream_requests_received | counter | The number of stream requests received. | service | |
method | ||||
grpc_server_unary_request_duration | histogram | The time to complete a unary request. | service | |
method | ||||
code | ||||
grpc_server_unary_requests_completed | counter | The number of unary requests completed. | service | |
method | ||||
code | ||||
grpc_server_unary_requests_received | counter | The number of unary requests received. | service | |
method | ||||
ledger_blockchain_height | gauge | Height of the chain in blocks. | channel | |
ledger_blockstorage_commit_time | histogram | Time taken in seconds for committing the block to storage. | channel | |
logging_entries_checked | counter | Number of log entries checked against the active logging level | level | |
logging_entries_written | counter | Number of log entries that are written | level | |
participation_consensus_relation | gauge | The channel participation consensus relation of the node: 0 if other, 1 if consenter, 2 if follower, 3 if config-tracker. | channel | |
participation_status | gauge | The channel participation status of the node: 0 if inactive, 1 if active, 2 if onboarding, 3 if failed. | channel |
StatsD¶
The following orderer metrics are emitted for consumption by StatsD. The
%{variable_name}
nomenclature represents segments that vary based on
context.
For example, %{channel}
will be replaced with the name of the channel
associated with the metric.
Bucket | Type | Description |
---|---|---|
blockcutter.block_fill_duration.%{channel} | histogram | The time from first transaction enqueing to the block being cut in seconds. |
broadcast.enqueue_duration.%{channel}.%{type}.%{status} | histogram | The time to enqueue a transaction in seconds. |
broadcast.processed_count.%{channel}.%{type}.%{status} | counter | The number of transactions processed. |
broadcast.validate_duration.%{channel}.%{type}.%{status} | histogram | The time to validate a transaction in seconds. |
cluster.comm.egress_queue_capacity.%{host}.%{msg_type}.%{channel} | gauge | Capacity of the egress queue. |
cluster.comm.egress_queue_length.%{host}.%{msg_type}.%{channel} | gauge | Length of the egress queue. |
cluster.comm.egress_queue_workers.%{channel} | gauge | Count of egress queue workers. |
cluster.comm.egress_stream_count.%{channel} | gauge | Count of streams to other nodes. |
cluster.comm.egress_tls_connection_count | gauge | Count of TLS connections to other nodes. |
cluster.comm.ingress_stream_count | gauge | Count of streams from other nodes. |
cluster.comm.msg_dropped_count.%{host}.%{channel} | counter | Count of messages dropped. |
cluster.comm.msg_send_time.%{host}.%{channel} | histogram | The time it takes to send a message in seconds. |
consensus.etcdraft.active_nodes.%{channel} | gauge | Number of active nodes in this channel. |
consensus.etcdraft.cluster_size.%{channel} | gauge | Number of nodes in this channel. |
consensus.etcdraft.committed_block_number.%{channel} | gauge | The block number of the latest block committed. |
consensus.etcdraft.config_proposals_received.%{channel} | counter | The total number of proposals received for config type transactions. |
consensus.etcdraft.data_persist_duration.%{channel} | histogram | The time taken for etcd/raft data to be persisted in storage (in seconds). |
consensus.etcdraft.is_leader.%{channel} | gauge | The leadership status of the current node: 1 if it is the leader else 0. |
consensus.etcdraft.leader_changes.%{channel} | counter | The number of leader changes since process start. |
consensus.etcdraft.normal_proposals_received.%{channel} | counter | The total number of proposals received for normal type transactions. |
consensus.etcdraft.proposal_failures.%{channel} | counter | The number of proposal failures. |
consensus.etcdraft.snapshot_block_number.%{channel} | gauge | The block number of the latest snapshot. |
consensus.kafka.batch_size.%{topic} | gauge | The mean batch size in bytes sent to topics. |
consensus.kafka.compression_ratio.%{topic} | gauge | The mean compression ratio (as percentage) for topics. |
consensus.kafka.incoming_byte_rate.%{broker_id} | gauge | Bytes/second read off brokers. |
consensus.kafka.last_offset_persisted.%{channel} | gauge | The offset specified in the block metadata of the most recently committed block. |
consensus.kafka.outgoing_byte_rate.%{broker_id} | gauge | Bytes/second written to brokers. |
consensus.kafka.record_send_rate.%{topic} | gauge | The number of records per second sent to topics. |
consensus.kafka.records_per_request.%{topic} | gauge | The mean number of records sent per request to topics. |
consensus.kafka.request_latency.%{broker_id} | gauge | The mean request latency in ms to brokers. |
consensus.kafka.request_rate.%{broker_id} | gauge | Requests/second sent to brokers. |
consensus.kafka.request_size.%{broker_id} | gauge | The mean request size in bytes to brokers. |
consensus.kafka.response_rate.%{broker_id} | gauge | Requests/second sent to brokers. |
consensus.kafka.response_size.%{broker_id} | gauge | The mean response size in bytes from brokers. |
deliver.blocks_sent.%{channel}.%{filtered}.%{data_type} | counter | The number of blocks sent by the deliver service. |
deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success} | counter | The number of deliver requests that have been completed. |
deliver.requests_received.%{channel}.%{filtered}.%{data_type} | counter | The number of deliver requests that have been received. |
deliver.streams_closed | counter | The number of GRPC streams that have been closed for the deliver service. |
deliver.streams_opened | counter | The number of GRPC streams that have been opened for the deliver service. |
fabric_version.%{version} | gauge | The active version of Fabric. |
grpc.comm.conn_closed | counter | gRPC connections closed. Open minus closed is the active number of connections. |
grpc.comm.conn_opened | counter | gRPC connections opened. Open minus closed is the active number of connections. |
grpc.server.stream_messages_received.%{service}.%{method} | counter | The number of stream messages received. |
grpc.server.stream_messages_sent.%{service}.%{method} | counter | The number of stream messages sent. |
grpc.server.stream_request_duration.%{service}.%{method}.%{code} | histogram | The time to complete a stream request. |
grpc.server.stream_requests_completed.%{service}.%{method}.%{code} | counter | The number of stream requests completed. |
grpc.server.stream_requests_received.%{service}.%{method} | counter | The number of stream requests received. |
grpc.server.unary_request_duration.%{service}.%{method}.%{code} | histogram | The time to complete a unary request. |
grpc.server.unary_requests_completed.%{service}.%{method}.%{code} | counter | The number of unary requests completed. |
grpc.server.unary_requests_received.%{service}.%{method} | counter | The number of unary requests received. |
ledger.blockchain_height.%{channel} | gauge | Height of the chain in blocks. |
ledger.blockstorage_commit_time.%{channel} | histogram | Time taken in seconds for committing the block to storage. |
logging.entries_checked.%{level} | counter | Number of log entries checked against the active logging level |
logging.entries_written.%{level} | counter | Number of log entries that are written |
participation.consensus_relation.%{channel} | gauge | The channel participation consensus relation of the node: 0 if other, 1 if consenter, 2 if follower, 3 if config-tracker. |
participation.status.%{channel} | gauge | The channel participation status of the node: 0 if inactive, 1 if active, 2 if onboarding, 3 if failed. |
Peer Metrics¶
Prometheus¶
The following peer metrics are exported for consumption by Prometheus.
Name | Type | Description | Labels | |
---|---|---|---|---|
chaincode_execute_timeouts | counter | The number of chaincode executions (Init or Invoke) that have timed out. | chaincode | |
chaincode_launch_duration | histogram | The time to launch a chaincode. | chaincode | |
success | ||||
chaincode_launch_failures | counter | The number of chaincode launches that have failed. | chaincode | |
chaincode_launch_timeouts | counter | The number of chaincode launches that have timed out. | chaincode | |
chaincode_shim_request_duration | histogram | The time to complete chaincode shim requests. | type | |
channel | ||||
chaincode | ||||
success | ||||
chaincode_shim_requests_completed | counter | The number of chaincode shim requests completed. | type | |
channel | ||||
chaincode | ||||
success | ||||
chaincode_shim_requests_received | counter | The number of chaincode shim requests received. | type | |
channel | ||||
chaincode | ||||
couchdb_processing_time | histogram | Time taken in seconds for the function to complete request to CouchDB | database | |
function_name | ||||
result | ||||
deliver_blocks_sent | counter | The number of blocks sent by the deliver service. | channel | |
filtered | ||||
data_type | ||||
deliver_requests_completed | counter | The number of deliver requests that have been completed. | channel | |
filtered | ||||
data_type | ||||
success | ||||
deliver_requests_received | counter | The number of deliver requests that have been received. | channel | |
filtered | ||||
data_type | ||||
deliver_streams_closed | counter | The number of GRPC streams that have been closed for the deliver service. | ||
deliver_streams_opened | counter | The number of GRPC streams that have been opened for the deliver service. | ||
dockercontroller_chaincode_container_build_duration | histogram | The time to build a chaincode image in seconds. | chaincode | |
success | ||||
endorser_chaincode_instantiation_failures | counter | The number of chaincode instantiations or upgrade that have failed. | channel | |
chaincode | ||||
endorser_duplicate_transaction_failures | counter | The number of failed proposals due to duplicate transaction ID. | channel | |
chaincode | ||||
endorser_endorsement_failures | counter | The number of failed endorsements. | channel | |
chaincode | ||||
chaincodeerror | ||||
endorser_proposal_acl_failures | counter | The number of proposals that failed ACL checks. | channel | |
chaincode | ||||
endorser_proposal_duration | histogram | The time to complete a proposal. | channel | |
chaincode | ||||
success | ||||
endorser_proposal_simulation_failures | counter | The number of failed proposal simulations | channel | |
chaincode | ||||
endorser_proposal_validation_failures | counter | The number of proposals that have failed initial validation. | ||
endorser_proposals_received | counter | The number of proposals received. | ||
endorser_successful_proposals | counter | The number of successful proposals. | ||
fabric_version | gauge | The active version of Fabric. | version | |
gossip_comm_messages_received | counter | Number of messages received | ||
gossip_comm_messages_sent | counter | Number of messages sent | ||
gossip_comm_overflow_count | counter | Number of outgoing queue buffer overflows | ||
gossip_leader_election_leader | gauge | Peer is leader (1) or follower (0) | channel | |
gossip_membership_total_peers_known | gauge | Total known peers | channel | |
gossip_payload_buffer_size | gauge | Size of the payload buffer | channel | |
gossip_privdata_commit_block_duration | histogram | Time it takes to commit private data and the corresponding block (in seconds) | channel | |
gossip_privdata_fetch_duration | histogram | Time it takes to fetch missing private data from peers (in seconds) | channel | |
gossip_privdata_list_missing_duration | histogram | Time it takes to list the missing private data (in seconds) | channel | |
gossip_privdata_pull_duration | histogram | Time it takes to pull a missing private data element (in seconds) | channel | |
gossip_privdata_purge_duration | histogram | Time it takes to purge private data (in seconds) | channel | |
gossip_privdata_reconciliation_duration | histogram | Time it takes for reconciliation to complete (in seconds) | channel | |
gossip_privdata_retrieve_duration | histogram | Time it takes to retrieve missing private data elements from the ledger (in seconds) | channel | |
gossip_privdata_send_duration | histogram | Time it takes to send a missing private data element (in seconds) | channel | |
gossip_privdata_validation_duration | histogram | Time it takes to validate a block (in seconds) | channel | |
gossip_state_commit_duration | histogram | Time it takes to commit a block in seconds | channel | |
gossip_state_height | gauge | Current ledger height | channel | |
grpc_comm_conn_closed | counter | gRPC connections closed. Open minus closed is the active number of connections. | ||
grpc_comm_conn_opened | counter | gRPC connections opened. Open minus closed is the active number of connections. | ||
grpc_server_stream_messages_received | counter | The number of stream messages received. | service | |
method | ||||
grpc_server_stream_messages_sent | counter | The number of stream messages sent. | service | |
method | ||||
grpc_server_stream_request_duration | histogram | The time to complete a stream request. | service | |
method | ||||
code | ||||
grpc_server_stream_requests_completed | counter | The number of stream requests completed. | service | |
method | ||||
code | ||||
grpc_server_stream_requests_received | counter | The number of stream requests received. | service | |
method | ||||
grpc_server_unary_request_duration | histogram | The time to complete a unary request. | service | |
method | ||||
code | ||||
grpc_server_unary_requests_completed | counter | The number of unary requests completed. | service | |
method | ||||
code | ||||
grpc_server_unary_requests_received | counter | The number of unary requests received. | service | |
method | ||||
ledger_block_processing_time | histogram | Time taken in seconds for ledger block processing. | channel | |
ledger_blockchain_height | gauge | Height of the chain in blocks. | channel | |
ledger_blockstorage_and_pvtdata_commit_time | histogram | Time taken in seconds for committing the block and private data to storage. | channel | |
ledger_blockstorage_commit_time | histogram | Time taken in seconds for committing the block to storage. | channel | |
ledger_statedb_commit_time | histogram | Time taken in seconds for committing block changes to state db. | channel | |
ledger_transaction_count | counter | Number of transactions processed. | channel | |
transaction_type | ||||
chaincode | ||||
validation_code | ||||
logging_entries_checked | counter | Number of log entries checked against the active logging level | level | |
logging_entries_written | counter | Number of log entries that are written | level |
StatsD¶
The following peer metrics are emitted for consumption by StatsD. The
%{variable_name}
nomenclature represents segments that vary based on
context.
For example, %{channel}
will be replaced with the name of the channel
associated with the metric.
Bucket | Type | Description |
---|---|---|
chaincode.execute_timeouts.%{chaincode} | counter | The number of chaincode executions (Init or Invoke) that have timed out. |
chaincode.launch_duration.%{chaincode}.%{success} | histogram | The time to launch a chaincode. |
chaincode.launch_failures.%{chaincode} | counter | The number of chaincode launches that have failed. |
chaincode.launch_timeouts.%{chaincode} | counter | The number of chaincode launches that have timed out. |
chaincode.shim_request_duration.%{type}.%{channel}.%{chaincode}.%{success} | histogram | The time to complete chaincode shim requests. |
chaincode.shim_requests_completed.%{type}.%{channel}.%{chaincode}.%{success} | counter | The number of chaincode shim requests completed. |
chaincode.shim_requests_received.%{type}.%{channel}.%{chaincode} | counter | The number of chaincode shim requests received. |
couchdb.processing_time.%{database}.%{function_name}.%{result} | histogram | Time taken in seconds for the function to complete request to CouchDB |
deliver.blocks_sent.%{channel}.%{filtered}.%{data_type} | counter | The number of blocks sent by the deliver service. |
deliver.requests_completed.%{channel}.%{filtered}.%{data_type}.%{success} | counter | The number of deliver requests that have been completed. |
deliver.requests_received.%{channel}.%{filtered}.%{data_type} | counter | The number of deliver requests that have been received. |
deliver.streams_closed | counter | The number of GRPC streams that have been closed for the deliver service. |
deliver.streams_opened | counter | The number of GRPC streams that have been opened for the deliver service. |
dockercontroller.chaincode_container_build_duration.%{chaincode}.%{success} | histogram | The time to build a chaincode image in seconds. |
endorser.chaincode_instantiation_failures.%{channel}.%{chaincode} | counter | The number of chaincode instantiations or upgrade that have failed. |
endorser.duplicate_transaction_failures.%{channel}.%{chaincode} | counter | The number of failed proposals due to duplicate transaction ID. |
endorser.endorsement_failures.%{channel}.%{chaincode}.%{chaincodeerror} | counter | The number of failed endorsements. |
endorser.proposal_acl_failures.%{channel}.%{chaincode} | counter | The number of proposals that failed ACL checks. |
endorser.proposal_duration.%{channel}.%{chaincode}.%{success} | histogram | The time to complete a proposal. |
endorser.proposal_simulation_failures.%{channel}.%{chaincode} | counter | The number of failed proposal simulations |
endorser.proposal_validation_failures | counter | The number of proposals that have failed initial validation. |
endorser.proposals_received | counter | The number of proposals received. |
endorser.successful_proposals | counter | The number of successful proposals. |
fabric_version.%{version} | gauge | The active version of Fabric. |
gossip.comm.messages_received | counter | Number of messages received |
gossip.comm.messages_sent | counter | Number of messages sent |
gossip.comm.overflow_count | counter | Number of outgoing queue buffer overflows |
gossip.leader_election.leader.%{channel} | gauge | Peer is leader (1) or follower (0) |
gossip.membership.total_peers_known.%{channel} | gauge | Total known peers |
gossip.payload_buffer.size.%{channel} | gauge | Size of the payload buffer |
gossip.privdata.commit_block_duration.%{channel} | histogram | Time it takes to commit private data and the corresponding block (in seconds) |
gossip.privdata.fetch_duration.%{channel} | histogram | Time it takes to fetch missing private data from peers (in seconds) |
gossip.privdata.list_missing_duration.%{channel} | histogram | Time it takes to list the missing private data (in seconds) |
gossip.privdata.pull_duration.%{channel} | histogram | Time it takes to pull a missing private data element (in seconds) |
gossip.privdata.purge_duration.%{channel} | histogram | Time it takes to purge private data (in seconds) |
gossip.privdata.reconciliation_duration.%{channel} | histogram | Time it takes for reconciliation to complete (in seconds) |
gossip.privdata.retrieve_duration.%{channel} | histogram | Time it takes to retrieve missing private data elements from the ledger (in seconds) |
gossip.privdata.send_duration.%{channel} | histogram | Time it takes to send a missing private data element (in seconds) |
gossip.privdata.validation_duration.%{channel} | histogram | Time it takes to validate a block (in seconds) |
gossip.state.commit_duration.%{channel} | histogram | Time it takes to commit a block in seconds |
gossip.state.height.%{channel} | gauge | Current ledger height |
grpc.comm.conn_closed | counter | gRPC connections closed. Open minus closed is the active number of connections. |
grpc.comm.conn_opened | counter | gRPC connections opened. Open minus closed is the active number of connections. |
grpc.server.stream_messages_received.%{service}.%{method} | counter | The number of stream messages received. |
grpc.server.stream_messages_sent.%{service}.%{method} | counter | The number of stream messages sent. |
grpc.server.stream_request_duration.%{service}.%{method}.%{code} | histogram | The time to complete a stream request. |
grpc.server.stream_requests_completed.%{service}.%{method}.%{code} | counter | The number of stream requests completed. |
grpc.server.stream_requests_received.%{service}.%{method} | counter | The number of stream requests received. |
grpc.server.unary_request_duration.%{service}.%{method}.%{code} | histogram | The time to complete a unary request. |
grpc.server.unary_requests_completed.%{service}.%{method}.%{code} | counter | The number of unary requests completed. |
grpc.server.unary_requests_received.%{service}.%{method} | counter | The number of unary requests received. |
ledger.block_processing_time.%{channel} | histogram | Time taken in seconds for ledger block processing. |
ledger.blockchain_height.%{channel} | gauge | Height of the chain in blocks. |
ledger.blockstorage_and_pvtdata_commit_time.%{channel} | histogram | Time taken in seconds for committing the block and private data to storage. |
ledger.blockstorage_commit_time.%{channel} | histogram | Time taken in seconds for committing the block to storage. |
ledger.statedb_commit_time.%{channel} | histogram | Time taken in seconds for committing block changes to state db. |
ledger.transaction_count.%{channel}.%{transaction_type}.%{chaincode}.%{validation_code} | counter | Number of transactions processed. |
logging.entries_checked.%{level} | counter | Number of log entries checked against the active logging level |
logging.entries_written.%{level} | counter | Number of log entries that are written |