Kafka Metrics

Prometheus plugin: kafka_exporter (opens in a new tab), jmx_exporter (opens in a new tab)
Prometheus Built-in Chart: Grafana Kafka Overvie (opens in a new tab)

Note: The sign of behind Metric indicates that the platform provides Built-in Chart.

Description	Metrics
Number of active controllers in the cluster	MegaEase`kafka-controller-activecontrollercount-value-metric` Datadog`kafka.replication.active_controller_count` Prometheus `kafka_controller_kafkacontroller_activecontrollercount` Telegraf `kafka_controller_activecontrollercount_value` MegaEase Dashboard - `kafka-controller-activecontrollercount-value-max-metric`
Number of partitions across all topics in the cluster	MegaEase`kafka-controller-globalpartitioncount-value-metric` Datadog`kafka.replication.partition_count` Prometheus `kafka_controller_kafkacontroller_globalpartitioncount` Telegraf `kafka_controller_globalpartitioncount_value` MegaEase Dashboard - `kafka-controller-globalpartitioncount-value-max-metric`
Total number of topics across all brokers in the cluster	MegaEase`kafka-controller-globaltopiccount-value-metric` Datadog`NotSupport` Prometheus `kafka_controller_kafkacontroller_globaltopiccount` Telegraf `kafka_controller_globaltopiccount_value` MegaEase Dashboard - `kafka-controller-globaltopiccount-value-max-metric`
the max of Leader election rate and latency	MegaEase`kafka-controller-leaderelectionrateandtimems-max-metric` Datadog`kafka.replication.leader_elections.rate` Prometheus `kafka_controller_controllerstats_leaderelectionrateandtimems` Telegraf `kafka_controller_leaderelectionrateandtimems_max` MegaEase Dashboard - `kafka-controller-leaderelectionrateandtimems-max-max-metric`
m1 rate of unclean leader elections	MegaEase`kafka-controller-uncleanleaderelectionspersec-oneminuterate-metric` Datadog`kafka.replication.unclean_leader_elections.rate` Prometheus `kafka_controller_controllerstats_uncleanleaderelectionspersec` Telegraf `kafka_controller_uncleanleaderelectionspersec_oneminuterate` MegaEase Dashboard - `kafka-controller-uncleanleaderelectionspersec-oneminuterate-max-metric`
Controller requests queue size (per broker)	MegaEase`kafka-controller-eventqueuesize-value-metric` Datadog`NotSupport` Prometheus `kafka_controller_controllereventmanager_eventqueuesize` Telegraf `kafka_controller_eventqueuesize_value` MegaEase Dashboard - `kafka-controller-eventqueuesize-value-max-metric`
Number of partitions that don't have an active leader and are hence not writable or readable	MegaEase`kafka-controller-offlinepartitionscount-value-metric` Datadog`kafka.replication.offline_partitions_count` Prometheus `kafka_controller_kafkacontroller_offlinepartitionscount` Telegraf `kafka_controller_offlinepartitionscount_value` MegaEase Dashboard - `kafka-controller-offlinepartitionscount-value-max-metric`
Total number of controller requests to be sent out to brokers	MegaEase`kafka-controller-totalqueuesize-value-metric` Datadog`NotSupport` Prometheus `kafka_controller_controllerchannelmanager_totalqueuesize` Telegraf `kafka_controller_totalqueuesize_value` MegaEase Dashboard - `kafka-controller-totalqueuesize-value-max-metric`
the total number of collections that have occurred	MegaEase`kafka-java-garbage-collector-collectioncount-metric` Datadog`jmx.java.lang.collectioncount` Prometheus `java_lang_garbagecollector_collectioncount` Telegraf `kafka_java_garbage_collector_collectioncount` MegaEase Dashboard - `kafka-java-garbage-collector-collectioncount-ratio-metric`
the approximate accumulated collection elapsed time in milliseconds	MegaEase`kafka-java-garbage-collector-collectiontime-metric` Datadog`jmx.java.lang.collectiontime` Prometheus `java_lang_garbagecollector_collectiontime` Telegraf `kafka_java_garbage_collector_collectiontime` MegaEase Dashboard - `kafka-java-garbage-collector-collectiontime-ratio-metric`
the amount of used memory in bytes of the memory usage	MegaEase`kafka-java-memory-pool-usage-used-metric` Datadog`jmx.java.lang.usage.used` Prometheus `jvm_memory_pool_bytes_used` Telegraf `kafka_java_memory_pool_usage_used` MegaEase Dashboard - `kafka-java-memory-pool-usage-used-max-by-tag-name-metric`
the current number of live daemon threads	MegaEase`kafka-java-threading-daemonthreadcount-metric` Datadog`jmx.java.lang.daemon_thread_count` Prometheus `java_lang_threading_daemonthreadcount` Telegraf `kafka_java_threading_daemonthreadcount` MegaEase Dashboard - `kafka-java-threading-daemonthreadcount-max-metric`
the peak live thread count	MegaEase`kafka-java-threading-peakthreadcount-metric` Datadog`jmx.java.lang.daemon_thread_count` Prometheus `java_lang_threading_peakthreadcount` Telegraf `kafka_java_threading_peakthreadcount` MegaEase Dashboard - `kafka-java-threading-peakthreadcount-max-metric`
the current number of live threads	MegaEase`kafka-java-threading-threadcount-metric` Datadog`jmx.java.lang.thread_count` Prometheus `java_lang_threading_threadcount` Telegraf `kafka_java_threading_threadcount` MegaEase Dashboard - `kafka-java-threading-threadcount-max-metric`
the maximum amount of memory in bytes that can be used of the peak memory usage	MegaEase`kafka-java-memory-pool-peakusage-max-metric` Datadog`jmx.java.lang.peak_usage.max` Prometheus `java_lang_memorypool_peakusage_max` Telegraf `kafka_java_memory_pool_peakusage_max` MegaEase Dashboard - `kafka-java-memory-pool-peakusage-max-max-metric`
the amount of used memory in bytes of the peak memory usage	MegaEase`kafka-java-memory-pool-peakusage-used-metric` Datadog`jmx.java.lang.peak_usage.used` Prometheus `java_lang_memorypool_peakusage_used` Telegraf `kafka_java_memory_pool_peakusage_used` MegaEase Dashboard - `kafka-java-memory-pool-peakusage-used-max-metric` - `kafka-java-memory-pool-peakusage-used-avg-metric` - `kafka-java-memory-pool-peakusage-used-min-metric`
the maximum amount of memory in bytes that can be used of the memory usage	MegaEase`kafka-java-memory-pool-usage-max-metric` Datadog`jmx.java.lang.usage.max` Prometheus `java_lang_memorypool_usage_max` Telegraf `kafka_java_memory_pool_usage_max` MegaEase Dashboard - `kafka-java-memory-pool-usage-max-max-metric` - `kafka-java-memory-pool-usage-max-avg-metric` - `kafka-java-memory-pool-usage-max-min-metric`
the total number of threads started	MegaEase`kafka-java-threading-totalstartedthreadcount-metric` Datadog`jmx.java.lang.total_started_thread_count` Prometheus `java_lang_threading_totalstartedthreadcount` Telegraf `kafka_java_threading_totalstartedthreadcount` MegaEase Dashboard - `kafka-java-threading-totalstartedthreadcount-ratio-metric` - `kafka-java-threading-totalstartedthreadcount-max-metric`
Number of leaders on this broker	MegaEase`kafka-replica-manager-leadercount-value-metric` Datadog`kafka.replication.leader_count` Prometheus `kafka_server_replicamanager_leadercount` Telegraf `kafka_replica_manager_leadercount_value` MegaEase Dashboard - `kafka-replica-manager-leadercount-value-max-metric`
Offline Replica counts	MegaEase`kafka-replica-manager-offlinereplicacount-value-metric` Datadog`NotSupport` Prometheus `kafka_server_replicamanager_offlinereplicacount` Telegraf `kafka_replica_manager_offlinereplicacount_value` MegaEase Dashboard - `kafka-replica-manager-offlinereplicacount-value-max-metric`
Number of partitions on this broker	MegaEase`kafka-replica-manager-partitioncount-value-metric` Datadog`kafka.replication.partition_count` Prometheus `kafka_server_replicamanager_partitioncount` Telegraf `kafka_replica_manager_partitioncount_value` MegaEase Dashboard - `kafka-replica-manager-partitioncount-value-max-metric`
Number of under-replicated partitions	MegaEase`kafka-replica-manager-underreplicatedpartitions-value-metric` Datadog`kafka.replication.under_replicated_partitions` Prometheus `kafka_server_replicamanager_underreplicatedpartitions` Telegraf `kafka_replica_manager_underreplicatedpartitions_value` MegaEase Dashboard - `kafka-replica-manager-underreplicatedpartitions-value-max-metric`
Number of partitions whose in-sync replicas count is less than minIsr	MegaEase`kafka-replica-manager-underminisrpartitioncount-value-metric` Datadog`kafka.replication.under_min_isr_partition_count` Prometheus `kafka_server_replicamanager_underminisrpartitioncount` Telegraf `kafka_replica_manager_underminisrpartitioncount_value` MegaEase Dashboard - `kafka-replica-manager-underminisrpartitioncount-value-min-metric` - `kafka-replica-manager-underminisrpartitioncount-value-max-metric` - `kafka-replica-manager-underminisrpartitioncount-value-avg-metric`
Byte in rate from clients per topic	MegaEase`kafka-topic-bytesinpersec-oneminuterate-metric` Datadog`kafka.topic.net.bytes_in.rate` Prometheus `rate(kafka_server_brokertopicmetrics_bytesout_total[1m])` Telegraf `kafka_topic_bytesinpersec_oneminuterate` MegaEase Dashboard - `kafka-topic-bytesinpersec-oneminuterate-max-by-topic-metric`
Byte out rate to clients per topic	MegaEase`kafka-topic-bytesoutpersec-oneminuterate-metric` Datadog`kafka.topic.net.bytes_out.rate` Prometheus `rate(kafka_server_brokertopicmetrics_bytesout_total[1m])` Telegraf `kafka_topic_bytesoutpersec_oneminuterate` MegaEase Dashboard - `kafka-topic-bytesoutpersec-oneminuterate-max-by-topic-metric`
Aggregate incoming message rate per topic	MegaEase`kafka-topic-messagesinpersec-oneminuterate-metric` Datadog`kafka.topic.messages_in.rate` Prometheus `rate(kafka_server_brokertopicmetrics_messagesin_total[1m])` Telegraf `kafka_topic_messagesinpersec_oneminuterate` MegaEase Dashboard - `kafka-topic-messagesinpersec-oneminuterate-max-by-topic-metric`
Byte in rate from clients	MegaEase`kafka-topics-bytesinpersec-oneminuterate-metric` Datadog`kafka.net.bytes_in.rate` Prometheus `rate(kafka_server_brokertopicmetrics_bytesout_total[1m])` Telegraf `kafka_topics_bytesinpersec_oneminuterate` MegaEase Dashboard - `kafka-topics-bytesinpersec-oneminuterate-max-metric`
Byte out rate to clients	MegaEase`kafka-topics-bytesoutpersec-oneminuterate-metric` Datadog`kafka.net.bytes_out.rate` Prometheus `rate(kafka_server_brokertopicmetrics_bytesout_total[1m])` Telegraf `kafka_topics_bytesoutpersec_oneminuterate` MegaEase Dashboard - `kafka-topics-bytesoutpersec-oneminuterate-max-metric`
Aggregate incoming message rate	MegaEase`kafka-topics-messagesinpersec-oneminuterate-metric` Datadog`kafka.messages_in.rate` Prometheus `rate(kafka_server_brokertopicmetrics_messagesin_total[1m])` Telegraf `kafka_topics_messagesinpersec_oneminuterate` MegaEase Dashboard - `kafka-topics-messagesinpersec-oneminuterate-max-metric`
Byte-in rate from other brokers	MegaEase`kafka-topics-replicationbytesinpersec-count-metric` Datadog`NotSupport` Prometheus `rate(kafka_server_brokertopicmetrics_replicationbytesin_total[1m])` Telegraf `kafka_topics_replicationbytesinpersec_count` MegaEase Dashboard - `kafka-topics-replicationbytesinpersec-count-max-metric` - `kafka-topics-replicationbytesinpersec-count-avg-metric` - `kafka-topics-replicationbytesinpersec-count-min-metric`
Byte-out rate to other brokers	MegaEase`kafka-topics-replicationbytesoutpersec-count-metric` Datadog`NotSupport` Prometheus `rate(kafka_server_brokertopicmetrics_replicationbytesout_total[1m])` Telegraf `kafka_topics_replicationbytesoutpersec_count` MegaEase Dashboard - `kafka-topics-replicationbytesoutpersec-count-avg-metric` - `kafka-topics-replicationbytesoutpersec-count-min-metric` - `kafka-topics-replicationbytesoutpersec-count-max-metric`
the highest CurrentLag value across all partitions	MegaEase`kafka-group-topic-lag-metric` Datadog`kafka.consumer_lag` Prometheus `kafka_consumergroup_lag` Telegraf `kafka_group_topic_lag` MegaEase Dashboard - `kafka-group-topic-lag-max-by-group-metric`
total offset of all partitions for the group	MegaEase`kafka-group-topic-offset-sum-metric` Datadog`avg:kafka.consumer_offset{*}` Prometheus `kafka_consumergroup_current_offset_sum` Telegraf `kafka_group_topic_offset_sum` MegaEase Dashboard - `kafka-group-topic-offset-sum-avg-by-group-metric` - `kafka-group-topic-offset-sum-min-by-group-metric` - `kafka-group-topic-offset-sum-max-by-group-metric`
A count of the total number of partitions that the group has committed offsets for	MegaEase`kafka-group-topic-partition-count-metric` Datadog`kafka.replication.partition_count` Prometheus `kafka_topic_partitions` Telegraf `kafka_group_topic_partition_count` MegaEase Dashboard - `kafka-group-topic-partition-count-max-by-group-metric`
The sum of all partition CurrentLag values for the group	MegaEase`kafka-group-topic-total-lag-metric` Datadog`kafka.consumer_lag` Prometheus `kafka_consumergroup_lag` Telegraf `kafka_group_topic_total_lag` MegaEase Dashboard - `kafka-group-topic-total-lag-max-by-group-metric`
offset of single partition in single topic	MegaEase`kafka-topic-offset-offset-metric` Datadog`kafka.consumer_offset` Prometheus `kafka_topic_partition_current_offset` Telegraf `kafka_topic_offset_offset` MegaEase Dashboard - `kafka-topic-offset-offset-ratio-metric`
the CurrentLag value for single partition	MegaEase`kafka-topic-partition-lag-metric` Datadog`kafka.consumer_lag` Prometheus `kafka_consumergroup_lag` Telegraf `kafka_topic_partition_lag` MegaEase Dashboard - `kafka-topic-partition-lag-max-by-group-metric`
the number of failed produce requests per second in a Kafka topic	MegaEase`kafka-topics-failedproducerequestspersec-count-metric` Datadog`jmx.kafka.server.FailedProduceRequestsPerSec.count` Prometheus `kafka_topics_failedproducerequestspersec_count` Telegraf `kafka_topics_failedproducerequestspersec_count` MegaEase Dashboard - `kafka-topics-failedproducerequestspersec-count-ratio-metric`
the rate at which messages are converted by a Kafka producer before they are sent to a topic When a producer sends a message to a Kafka topic, it may need to convert the message to a different format, such as Avro or Protobuf, before it can be written to the topic This metric tracks the number of message conversions per second that occur during this process.	MegaEase`kafka-topics-producemessageconversionspersec-count-metric` Datadog`jmx.kafka.server.ProduceMessageConversionsPerSec.count` Prometheus `kafka_topics_producemessageconversionspersec_count` Telegraf `kafka_topics_producemessageconversionspersec_count` MegaEase Dashboard - `kafka-topics-producemessageconversionspersec-count-ratio-metric`
the total number of produce requests per second in a Kafka topic	MegaEase`kafka-topics-totalproducerequestspersec-count-metric` Datadog`jmx.kafka.server.TotalProduceRequestsPerSec.count` Prometheus `kafka_topics_totalproducerequestspersec_count` Telegraf `kafka_topics_totalproducerequestspersec_count` MegaEase Dashboard - `kafka-topics-totalproducerequestspersec-count-ratio-metric`
the total amount of CPU time consumed by a process	MegaEase`kafka-process-cpu-seconds-total-metric` Datadog`jmx.java.lang.process_cpu_time` Prometheus `rate(process_cpu_seconds_total[1m])` Telegraf `kafka_process_cpu_seconds_total` MegaEase Dashboard - `kafka-process-cpu-seconds-total-ratio-metric`
the amount of memory used by a Java Virtual Machine (JVM)	MegaEase`NotSupport` Datadog`jmx.java.lang.memory_usage.used` Prometheus `jvm_memory_bytes_used`
the approximate accumulated collection elapsed time in seconds	MegaEase`NotSupport` Datadog`NotSupport` Prometheus `jvm_gc_collection_seconds_sum`
Rate of replicas joining the ISR pool	MegaEase`kafka-replica-manager-isrexpandspersec-count-metric` Datadog`max:kafka.replication.isr_expands.rate` Prometheus `NotSupport` Telegraf `kafka_replica_manager_isrexpandspersec_count` MegaEase Dashboard - `kafka-replica-manager-isrexpandspersec-count-min-metric` - `kafka-replica-manager-isrexpandspersec-count-avg-metric` - `kafka-replica-manager-isrexpandspersec-count-max-metric`
Rate of replicas leaving the ISR pool	MegaEase`kafka-replica-manager-isrshrinkspersec-count-metric` Datadog`max:kafka.replication.isr_shrinks.rate` Prometheus `NotSupport` Telegraf `kafka_replica_manager_isrshrinkspersec_count` MegaEase Dashboard - `kafka-replica-manager-isrshrinkspersec-count-min-metric` - `kafka-replica-manager-isrshrinkspersec-count-avg-metric` - `kafka-replica-manager-isrshrinkspersec-count-max-metric`
Total time in ms to serve the specified request	MegaEase`NotSupport` Datadog`avg:kafka.request.fetch_follower.time.avg` Prometheus `NotSupport`
Average time for a produce request	MegaEase`NotSupport` Datadog`avg:kafka.request.produce.time.avg` Prometheus `NotSupport`
Total time in ms to serve the specified request	MegaEase`NotSupport` Datadog`avg:kafka.request.fetch_consumer.time.avg` Prometheus `NotSupport`
Number of requests waiting in the producer purgatory	MegaEase`NotSupport` Datadog`sum:kafka.request.producer_request_purgatory.size` Prometheus `NotSupport`
Number of requests waiting in the producer purgatory	MegaEase`NotSupport` Datadog`sum:kafka.request.fetch_request_purgatory.size` Prometheus `NotSupport`
Average time for a produce request	MegaEase`NotSupport` Datadog`avg:kafka.request.produce.time.avg` Prometheus `NotSupport`
Total time in ms to serve the specified request	MegaEase`NotSupport` Datadog`avg:kafka.request.fetch_consumer.time.avg` Prometheus `NotSupport`
Total time in ms to serve the specified request	MegaEase`NotSupport` Datadog`avg:kafka.request.fetch_follower.time.avg` Prometheus `NotSupport`
Producer bytes out rate	MegaEase`NotSupport` Datadog`sum:kafka.producer.bytes_out` Prometheus `NotSupport`
Number of producer requests per second	MegaEase`NotSupport` Datadog`sum:kafka.producer.request_rate` Prometheus `NotSupport`
Number of producer responses per second	MegaEase`NotSupport` Datadog`sum:kafka.producer.response_rate` Prometheus `NotSupport`
Producer average request latency	MegaEase`NotSupport` Datadog`sum:kafka.producer.request_latency_avg` Prometheus `NotSupport`
Producer I/O wait time	MegaEase`NotSupport` Datadog`sum:kafka.producer.io_wait` Prometheus `NotSupport`
Consumer bytes in rate	MegaEase`NotSupport` Datadog`sum:kafka.consumer.bytes_in` Prometheus `NotSupport`
Rate of consumer message consumption	MegaEase`NotSupport` Datadog`sum:kafka.consumer.messages_in` Prometheus `NotSupport`
The minimum rate at which the consumer sends fetch requests to a broker	MegaEase`NotSupport` Datadog`sum:kafka.consumer.fetch_rate` Prometheus `NotSupport`

Nginx Metrics Zookeeper Metrics