Available in VPC
This section provides information on the performance and operational metrics of the Cloud Hadoop service. Performance/operational metrics can be managed in an integrated manner through NAVER Cloud Platform's Cloud Insight service and can be used for various monitoring purposes.
Note
See the Cloud Insight API Guide for information on how to use performance/operational metrics for monitoring.
Basic information
The following is the basic information of the Cloud Hadoop service for using performance/operational metrics.
- Product name (
prodName)- System/Cloud Hadoop(VPC)
- Product key (
cw_key)- 564076991498817536
- Metric type
- cpu: CPU performance data
- memory: Memory performance data
- disk: Disk performance data
- fs: File system performance data
- ntwk: Network performance data
- process: Processes information
- svr: Server statistical data
- gpu: GPU performance data
- plugin_process: Plugin process data
- plugin_file: Plugin file data
- plugin_port: Plugin port data
- agent: Agent performance data
- hadoop: hadoop statistical data
- Queue
Performance/operation metrics
The performance/operational metrics of the Cloud Hadoop service are as follows.
CPU
CPU-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation user_rto FLOAT % Basic user ratio (%) Min1,Min5,Min30,Hour2,Day1 idle_rto FLOAT % Basic idle ratio (%) Min1,Min5,Min30,Hour2,Day1 sys_rto FLOAT % Basic system ratio (%) Min1,Min5,Min30,Hour2,Day1 used_rto FLOAT % Basic used ratio (%) Min1,Min5,Min30,Hour2,Day1 nice_rto (Linux) FLOAT % Basic nice ratio (%) Min1,Min5,Min30,Hour2,Day1 irq_rto (Linux) FLOAT % Basic interrupt ratio (%) Min1,Min5,Min30,Hour2,Day1 softirq_rto (Linux) FLOAT % Basic soft interrupt ratio (%) Min1,Min5,Min30,Hour2,Day1 io_wait_rto (Linux) FLOAT % Basic IO wait ratio (%) Min1,Min5,Min30,Hour2,Day1 prv_mde_exec_tm_rto (Windows) FLOAT % Basic CPU used ratio in the privileged mode (%) Min1,Min5,Min30,Hour2,Day1 dly_pcd_call_tm_rto (Windows) FLOAT % Basic CPU used ratio for deferred procedure calls (DPCs) (%) Min1,Min5,Min30,Hour2,Day1 interrupt_tm_rto (Windows) FLOAT % Basic CPU used ratio for interrupts (%) Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo cpu_idx
Memory
Memory-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation mem_mb FLOAT MB Basic total memory (MB) Min1,Min5,Min30,Hour2,Day1 used_mem_mb FLOAT MB Basic used memory (MB), used_mem_mb = total_mem_mb - free_mem_mb - bffr_mb - cache_mb Min1,Min5,Min30,Hour2,Day1 free_mem_mb FLOAT MB Basic free memory (MB) Min1,Min5,Min30,Hour2,Day1 mem_usert FLOAT % Basic, Default Memory Utilization(%), mem_usert = used_mem_mb / total_mem_mb * 100 Min1,Min5,Min30,Hour2,Day1 shared_mem_mb (Linux) FLOAT MB Basic shared memory (MB) Min1,Min5,Min30,Hour2,Day1 bffr_mb (Linux) FLOAT MB Basic buffers (MB) Min1,Min5,Min30,Hour2,Day1 cache_mb (Linux) FLOAT MB Basic cache (MB) Min1,Min5,Min30,Hour2,Day1 pgin_mb FLOAT MB Basic page in (MB) Min1,Min5,Min30,Hour2,Day1 pgout_mb FLOAT MB Basic page out (MB) Min1,Min5,Min30,Hour2,Day1 pgfault_tcnt (Linux) LONG num/sec Basic page fault count per second Min1,Min5,Min30,Hour2,Day1 pgfault_rto (Windows) FLOAT % Basic page fault ratio (%) Min1,Min5,Min30,Hour2,Day1 swap_mb FLOAT MB Basic swap (MB) Min1,Min5,Min30,Hour2,Day1 swap_used_mb FLOAT MB Basic swap used (MB) Min1,Min5,Min30,Hour2,Day1 swap_free_mb FLOAT MB Basic swap free (MB) Min1,Min5,Min30,Hour2,Day1 swap_usert FLOAT % Basic swap used ratio (%) Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo
Disk
Disk-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation read_byt_cnt FLOAT bytes/sec Basic read bytes Min1,Min5,Min30,Hour2,Day1 write_byt_cnt FLOAT bytes/sec Basic write bytes Min1,Min5,Min30,Hour2,Day1 read_cnt FLOAT num/sec Basic disk read count per second Min1,Min5,Min30,Hour2,Day1 write_cnt FLOAT num/sec Basic disk write count per second Min1,Min5,Min30,Hour2,Day1 avgrq_sz FLOAT num Basic The average size (in sectors) of the requests that were issued to the device Min1,Min5,Min30,Hour2,Day1 avgqu_sz FLOAT num/sec Basic The average queue length of the requests that were issued to the device Min1,Min5,Min30,Hour2,Day1 await_tm FLOAT millisecond Basic The average time (in milliseconds) for I/O requests issued to the device to be served including the time spent by the requests in queue and the time spent servicing them Min1,Min5,Min30,Hour2,Day1 svc_tm FLOAT millisecond Basic The average service time (in milliseconds) for I/O requests that were issued to the device Min1,Min5,Min30,Hour2,Day1 used_rto FLOAT % Basic Percentage of elapsed time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%. Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo disk_idx
File System
File system-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation all_byt_cnt FLOAT MB Basic fs bytes Min1,Min5,Min30,Hour2,Day1 used_byt_cnt FLOAT MB Basic used bytes Min1,Min5,Min30,Hour2,Day1 free_byt_cnt FLOAT MB Basic free bytes Min1,Min5,Min30,Hour2,Day1 fs_usert FLOAT % Basic used ratio Min1,Min5,Min30,Hour2,Day1 ind_usert FLOAT % Basic inodes usage(Linux) Min1,Min5,Min30,Hour2,Day1 dev_nm STRING - Basic device name Min1 mnt_stat_cd INTEGER - Basic mount state(1: mounted, 0: unmounted) Min1 fs_type_nm STRING - Basic file system type Min1 - Dimension
Dimension name Data type type instanceNo mnt_nm
Network
Network-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation snd_bps FLOAT bits/sec Basic send bits per second Min1,Min5,Min30,Hour2,Day1 rcv_bps FLOAT bits/sec Basic receive bits per second Min1,Min5,Min30,Hour2,Day1 snd_pps FLOAT packets/sec Basic send packets per second Min1,Min5,Min30,Hour2,Day1 rcv_pps FLOAT packets/sec Basic receive packets per second Min1,Min5,Min30,Hour2,Day1 snd_fail_packt_cnt INTEGER num Basic send fail packets Min1,Min5,Min30,Hour2,Day1 rcv_fail_packt_cnt INTEGER num Basic receive fail packets Min1,Min5,Min30,Hour2,Day1 clsn_packt_cnt (Linux) INTEGER num Basic collisions packets Min1,Min5,Min30,Hour2,Day1 nic_ip STRING - Basic IP Address Min1 macaddr STRING - Basic MAC Address Min1 - Dimension
Dimension name Data type type instanceNo nic_desc
Process
Process-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation cpu_usert FLOAT % Basic per process CPU used ratio (%) Min1,Min5,Min30,Hour2,Day1 p_proc_mem_usert FLOAT % Basic per process memory used ratio (%) Min1,Min5,Min30,Hour2,Day1 vir_mem_byt_cnt FLOAT KB Basic virtual memory size Min1,Min5,Min30,Hour2,Day1 cpu_tm_ss FLOAT - Basic cpu time Min1,Min5,Min30,Hour2,Day1 prit_rnk INTEGER - Basic priority Min1,Min5,Min30,Hour2,Day1 nice_val (Linux) INTEGER - Basic nice Min1,Min5,Min30,Hour2,Day1 thd_cnt INTEGER num Basic threads count Min1,Min5,Min30,Hour2,Day1 proc_t_cpu_usert (Linux) FLOAT % Basic cpu usage (%) Min1,Min5,Min30,Hour2,Day1 pf (Windows) LONG num/sec Basic page fault count Min1,Min5,Min30,Hour2,Day1 proc_id INTEGER - Basic process ID Min1 parent_pid INTEGER - Basic parent process ID Min1 proc_nm STRING - Basic process name Min1 proc_cpu_cnt INTEGER num Basic cpu core count being used Min1,Min5,Min30,Hour2,Day1 proc_stat_cd STRING - Basic process state Min1 strt_ymdt (Linux) STRING - Basic process start time Min1 - Dimension
Dimension name Data type type instanceNo
Server
Server-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation boot_time LONG sec Basic boot time (sec) Min1 time_deviation LONG millisecond Basic time deviation with NTP server Min1 user_cnt INTEGER num Basic user count Min1 is_on INTEGER flag Basic (deprecated) 1 if server is alive, 0 otherwise Min1 is_rebooted INTEGER flag Basic 1 if reboot is detected, 0 otherwise Min1 avg_cpu_used_rto FLOAT % Basic, Default CPU Utilization Average Min1,Min5,Min30,Hour2,Day1 max_cpu_used_rto FLOAT % Basic CPU used ratio maximum (%) Min1,Min5,Min30,Hour2,Day1 avg_cpu_idle_rto FLOAT % Basic CPU idle ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_cpu_user_rto FLOAT % Basic CPU user ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_cpu_sys_rto (Linux) FLOAT % Basic CPU system ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_nice_rto (Linux) FLOAT % Basic CPU nice ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_irq_rto (Linux) FLOAT % Basic CPU interrupt ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_softirq_rto (Linux) FLOAT % Basic CPU soft interrupt ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_io_wait_rto (Linux) FLOAT % Basic CPU IO wait ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_prv_mde_exec_tm_rto (Windows) FLOAT % Basic CPU used ratio in the privileged mode Min1,Min5,Min30,Hour2,Day1 avg_dly_pcd_call_tm_rto (Windows) FLOAT % Basic CPU used ratio for deferred procedure calls (DPCs) (%) Min1,Min5,Min30,Hour2,Day1 avg_interrup_tm_rto (Windows) FLOAT % Basic CPU used ratio for interrupts (%) Min1,Min5,Min30,Hour2,Day1 avg_read_byt_cnt FLOAT bytes/sec Basic disk read bytes average Min1,Min5,Min30,Hour2,Day1 avg_write_byt_cnt FLOAT bytes/sec Basic disk write bytes average Min1,Min5,Min30,Hour2,Day1 avg_read_cnt FLOAT num/sec Basic disk read count per second average Min1,Min5,Min30,Hour2,Day1 avg_write_cnt FLOAT num/sec Basic disk write count per second average Min1,Min5,Min30,Hour2,Day1 max_read_byt_cnt INTEGER bytes/sec Basic disk read bytes maximum Min1,Min5,Min30,Hour2,Day1 max_write_byt_cnt INTEGER bytes/sec Basic disk write bytes maximum Min1,Min5,Min30,Hour2,Day1 max_read_cnt INTEGER num/sec Basic disk read count per second maximum Min1,Min5,Min30,Hour2,Day1 max_write_cnt INTEGER num/sec Basic disk write count per second maximum Min1,Min5,Min30,Hour2,Day1 fs_all_mb FLOAT MB Basic all block file systems size (MB) Min1,Min5,Min30,Hour2,Day1 fs_used_mb FLOAT MB Basic all block file systems used (MB) Min1,Min5,Min30,Hour2,Day1 fs_free_mb FLOAT MB Basic all block file systems used ratio (%) Min1,Min5,Min30,Hour2,Day1 avg_fs_usert FLOAT % Basic, Default File System Utilization Average Min1,Min5,Min30,Hour2,Day1 max_fs_usert FLOAT % Basic file system used ratio maximum (%) Min1,Min5,Min30,Hour2,Day1 avg_ind_usert (Linux) FLOAT % Basic file system inodes used ratio average (%) Min1,Min5,Min30,Hour2,Day1 avg_snd_bps FLOAT bits/sec Basic, Default Network Out Average Min1,Min5,Min30,Hour2,Day1 max_snd_bps FLOAT bits/sec Basic network send bits/sec maximum Min1,Min5,Min30,Hour2,Day1 avg_rcv_bps FLOAT bits/sec Basic, Default Network In Average Min1,Min5,Min30,Hour2,Day1 max_rcv_bps FLOAT bits/sec Basic network receive bits/sec maximum Min1,Min5,Min30,Hour2,Day1 avg_snd_pps FLOAT packets/sec Basic network send packets/sec average Min1,Min5,Min30,Hour2,Day1 max_snd_pps FLOAT packets/sec Basic network send packets/sec maximum Min1,Min5,Min30,Hour2,Day1 avg_rcv_pps FLOAT packets/sec Basic network receive packets/sec average Min1,Min5,Min30,Hour2,Day1 max_rcv_pps FLOAT packets/sec Basic network receive packets/sec maximum Min1,Min5,Min30,Hour2,Day1 avg_snd_fail_packt_cnt FLOAT num Basic network send fail packets average Min1,Min5,Min30,Hour2,Day1 avg_rcv_fail_packt_cnt FLOAT num Basic network receive fail packets average Min1,Min5,Min30,Hour2,Day1 avg_clsn_packt_cnt (Linux) FLOAT num Basic network collisions packets average Min1,Min5,Min30,Hour2,Day1 proc_cnt INTEGER num Basic processes count Min1,Min5,Min30,Hour2,Day1 run_que_len INTEGER num Basic running processes Min1,Min5,Min30,Hour2,Day1 proc_mem_usert FLOAT % Basic all processes memory used ratio (%) Min1,Min5,Min30,Hour2,Day1 load_average_1m FLOAT % Basic CPU load 1 minute (%) Min1,Min5,Min30,Hour2,Day1 load_average_5m FLOAT % Basic CPU load 5 minute average (%) Min1,Min5,Min30,Hour2,Day1 load_average_15m FLOAT % Basic CPU load 15 minute average (%) Min1,Min5,Min30,Hour2,Day1 gpu_count INTEGER count Basic Total GPU Count Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo
GPU
GPU-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation vmem_usage INTEGER % Basic Current GPU MEM Usage Min1,Min5,Min30,Hour2,Day1 vmem_usage_mib INTEGER MiB Basic Current GPU MEM Usage Min1,Min5,Min30,Hour2,Day1 usage INTEGER % Basic Current GPU Usage Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo gpu_idx
Plugin Process
Plugin process-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation is_process_up INTEGER flag Basic 1 if process is up, 0 otherwise Min1,Min5,Min30,Hour2,Day1 process_count INTEGER num Basic process count Min1,Min5,Min30,Hour2,Day1 avg_cpu_usert FLOAT % Basic process used Average Min1,Min5,Min30,Hour2,Day1 avg_mem_usert FLOAT % Basic average memory usage % Min1,Min5,Min30,Hour2,Day1 avg_mem_byt_cnt FLOAT KB Basic memory usage Min1,Min5,Min30,Hour2,Day1 avg_thd_cnt FLOAT num Basic thread thread count Min1,Min5,Min30,Hour2,Day1 tot_cpu_usert FLOAT % Basic total cpu usage % Min1,Min5,Min30,Hour2,Day1 tot_mem_usert FLOAT % Basic total memory usage % Min1,Min5,Min30,Hour2,Day1 tot_mem_byt_cnt FLOAT KB Basic total memory usage Min1,Min5,Min30,Hour2,Day1 tot_thd_cnt INTEGER num Basic total thread count Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo proc_name
Plugin File
Plugin file-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation file_modified INTEGER flag Basic 1: modified, 0: not modified Min1,Min5,Min30,Hour2,Day1 file_exist INTEGER flag Basic 1: exist, 0: not exist Min1,Min5,Min30,Hour2,Day1 file_size LONG bytes Basic file size(byte) Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo path
Plugin Port
Plugin port-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation is_port_up INTEGER flag Basic 1: port is up, 0: is down Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type type instanceNo port
Agent
Agent-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation agent_version INTEGER - Basic - Min1 - Dimension
Dimension name Data type type instanceNo
Hadoop
Hadoop-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation active_nodes INTEGER num Basic, Default active_nodes Min1,Min5,Min30,Hour2,Day1 allocated_container INTEGER num Basic allocated_container Min1,Min5,Min30,Hour2,Day1 allocated_mb INTEGER MB Basic, Default allocated_mb Min1,Min5,Min30,Hour2,Day1 allocated_v_cores INTEGER num Basic allocated_v_cores Min1,Min5,Min30,Hour2,Day1 apps_completed INTEGER num Basic, Default apps_completed Min1,Min5,Min30,Hour2,Day1 apps_failed INTEGER num Basic, Default apps_failed Min1,Min5,Min30,Hour2,Day1 apps_killed INTEGER num Basic, Default apps_killed Min1,Min5,Min30,Hour2,Day1 apps_pending INTEGER num Basic, Default apps_pending Min1,Min5,Min30,Hour2,Day1 apps_running INTEGER num Basic, Default apps_running Min1,Min5,Min30,Hour2,Day1 apps_submitted INTEGER num Basic, Default apps_submitted Min1,Min5,Min30,Hour2,Day1 available_mb INTEGER MB Basic, Default available_mb Min1,Min5,Min30,Hour2,Day1 capacity_remaining_gb INTEGER GB Basic, Default capacity_remaining_gb Min1,Min5,Min30,Hour2,Day1 corrupt_blocks INTEGER num Basic corrupt_blocks Min1,Min5,Min30,Hour2,Day1 decommissioned_nodes INTEGER num Basic decommissioned_nodes Min1,Min5,Min30,Hour2,Day1 hdfs_bytes_read INTEGER bytes Basic hdfs_bytes_read Min1,Min5,Min30,Hour2,Day1 hdfs_bytes_written INTEGER bytes Basic hdfs_bytes_written Min1,Min5,Min30,Hour2,Day1 hdfs_utilization FLOAT % Basic, Default hdfs_utilization Min1,Min5,Min30,Hour2,Day1 lost_nodes INTEGER num Basic, Default lost_nodes Min1,Min5,Min30,Hour2,Day1 missing_blocks INTEGER num Basic, Default missing_blocks Min1,Min5,Min30,Hour2,Day1 num_live_data_nodes INTEGER num Basic, Default num_live_data_nodes Min1,Min5,Min30,Hour2,Day1 pending_containers INTEGER num Basic pending_containers Min1,Min5,Min30,Hour2,Day1 pending_deletion_blocks INTEGER num Basic pending_deletion_blocks Min1,Min5,Min30,Hour2,Day1 pending_replication_blocks INTEGER num Basic pending_replication_blocks Min1,Min5,Min30,Hour2,Day1 pending_v_cores INTEGER num Basic pending_v_cores Min1,Min5,Min30,Hour2,Day1 rebooted_nodes INTEGER num Basic rebooted_nodes Min1,Min5,Min30,Hour2,Day1 reserved_containers INTEGER num Basic reserved_containers Min1,Min5,Min30,Hour2,Day1 reserved_mb INTEGER MB Basic reserved_mb Min1,Min5,Min30,Hour2,Day1 total_load INTEGER num Basic total_load Min1,Min5,Min30,Hour2,Day1 total_mb INTEGER MB Basic, Default total_mb Min1,Min5,Min30,Hour2,Day1 total_nodes INTEGER num Basic, Default total_nodes Min1,Min5,Min30,Hour2,Day1 under_replicated_blocks INTEGER num Basic under_replicated_blocks Min1,Min5,Min30,Hour2,Day1 unhealthy_nodes INTEGER num Basic, Default unhealthy_nodes Min1,Min5,Min30,Hour2,Day1 yarn_memory_available_percentage FLOAT % Basic, Default yarn_memory_available_percentage Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type idDimension type STRING instanceNo STRING clusterName STRING
Queue
Queue-related performance/operational metrics are as follows.
- Metric
Metric name Data type Unit Metric type Description Interval Aggregation queue_used_memory FLOAT GB Basic, Default queue_used_memory Min1,Min5,Min30,Hour2,Day1 queue_used_memory_percent FLOAT % Basic, Default queue_used_memory_percent Min1,Min5,Min30,Hour2,Day1 queue_used_v_cores INTEGER num Basic, Default queue_used_v_cores Min1,Min5,Min30,Hour2,Day1 queue_used_v_cores_percent FLOAT % Basic, Default queue_used_v_cores_percent Min1,Min5,Min30,Hour2,Day1 - Dimension
Dimension name Data type idDimension type STRING mbrNo STRING instanceNo STRING clusterName STRING queueName STRING