Cloud Hadoop (VPC) metrics

Prev Next

Available in VPC

This section provides information on the performance and operational metrics of the Cloud Hadoop service. Performance/operational metrics can be managed in an integrated manner through NAVER Cloud Platform's Cloud Insight service and can be used for various monitoring purposes.

Note

See the Cloud Insight API Guide for information on how to use performance/operational metrics for monitoring.

Basic information

The following is the basic information of the Cloud Hadoop service for using performance/operational metrics.

  • Product name (prodName)
    • System/Cloud Hadoop(VPC)
  • Product key (cw_key)
    • 564076991498817536
  • Metric type

Performance/operation metrics

The performance/operational metrics of the Cloud Hadoop service are as follows.

CPU

CPU-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    user_rto FLOAT % Basic user ratio (%) Min1,Min5,Min30,Hour2,Day1
    idle_rto FLOAT % Basic idle ratio (%) Min1,Min5,Min30,Hour2,Day1
    sys_rto FLOAT % Basic system ratio (%) Min1,Min5,Min30,Hour2,Day1
    used_rto FLOAT % Basic used ratio (%) Min1,Min5,Min30,Hour2,Day1
    nice_rto (Linux) FLOAT % Basic nice ratio (%) Min1,Min5,Min30,Hour2,Day1
    irq_rto (Linux) FLOAT % Basic interrupt ratio (%) Min1,Min5,Min30,Hour2,Day1
    softirq_rto (Linux) FLOAT % Basic soft interrupt ratio (%) Min1,Min5,Min30,Hour2,Day1
    io_wait_rto (Linux) FLOAT % Basic IO wait ratio (%) Min1,Min5,Min30,Hour2,Day1
    prv_mde_exec_tm_rto (Windows) FLOAT % Basic CPU used ratio in the privileged mode (%) Min1,Min5,Min30,Hour2,Day1
    dly_pcd_call_tm_rto (Windows) FLOAT % Basic CPU used ratio for deferred procedure calls (DPCs) (%) Min1,Min5,Min30,Hour2,Day1
    interrupt_tm_rto (Windows) FLOAT % Basic CPU used ratio for interrupts (%) Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    cpu_idx

Memory

Memory-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    mem_mb FLOAT MB Basic total memory (MB) Min1,Min5,Min30,Hour2,Day1
    used_mem_mb FLOAT MB Basic used memory (MB), used_mem_mb = total_mem_mb - free_mem_mb - bffr_mb - cache_mb Min1,Min5,Min30,Hour2,Day1
    free_mem_mb FLOAT MB Basic free memory (MB) Min1,Min5,Min30,Hour2,Day1
    mem_usert FLOAT % Basic, Default Memory Utilization(%), mem_usert = used_mem_mb / total_mem_mb * 100 Min1,Min5,Min30,Hour2,Day1
    shared_mem_mb (Linux) FLOAT MB Basic shared memory (MB) Min1,Min5,Min30,Hour2,Day1
    bffr_mb (Linux) FLOAT MB Basic buffers (MB) Min1,Min5,Min30,Hour2,Day1
    cache_mb (Linux) FLOAT MB Basic cache (MB) Min1,Min5,Min30,Hour2,Day1
    pgin_mb FLOAT MB Basic page in (MB) Min1,Min5,Min30,Hour2,Day1
    pgout_mb FLOAT MB Basic page out (MB) Min1,Min5,Min30,Hour2,Day1
    pgfault_tcnt (Linux) LONG num/sec Basic page fault count per second Min1,Min5,Min30,Hour2,Day1
    pgfault_rto (Windows) FLOAT % Basic page fault ratio (%) Min1,Min5,Min30,Hour2,Day1
    swap_mb FLOAT MB Basic swap (MB) Min1,Min5,Min30,Hour2,Day1
    swap_used_mb FLOAT MB Basic swap used (MB) Min1,Min5,Min30,Hour2,Day1
    swap_free_mb FLOAT MB Basic swap free (MB) Min1,Min5,Min30,Hour2,Day1
    swap_usert FLOAT % Basic swap used ratio (%) Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo

Disk

Disk-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    read_byt_cnt FLOAT bytes/sec Basic read bytes Min1,Min5,Min30,Hour2,Day1
    write_byt_cnt FLOAT bytes/sec Basic write bytes Min1,Min5,Min30,Hour2,Day1
    read_cnt FLOAT num/sec Basic disk read count per second Min1,Min5,Min30,Hour2,Day1
    write_cnt FLOAT num/sec Basic disk write count per second Min1,Min5,Min30,Hour2,Day1
    avgrq_sz FLOAT num Basic The average size (in sectors) of the requests that were issued to the device Min1,Min5,Min30,Hour2,Day1
    avgqu_sz FLOAT num/sec Basic The average queue length of the requests that were issued to the device Min1,Min5,Min30,Hour2,Day1
    await_tm FLOAT millisecond Basic The average time (in milliseconds) for I/O requests issued to the device to be served including the time spent by the requests in queue and the time spent servicing them Min1,Min5,Min30,Hour2,Day1
    svc_tm FLOAT millisecond Basic The average service time (in milliseconds) for I/O requests that were issued to the device Min1,Min5,Min30,Hour2,Day1
    used_rto FLOAT % Basic Percentage of elapsed time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%. Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    disk_idx

File System

File system-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    all_byt_cnt FLOAT MB Basic fs bytes Min1,Min5,Min30,Hour2,Day1
    used_byt_cnt FLOAT MB Basic used bytes Min1,Min5,Min30,Hour2,Day1
    free_byt_cnt FLOAT MB Basic free bytes Min1,Min5,Min30,Hour2,Day1
    fs_usert FLOAT % Basic used ratio Min1,Min5,Min30,Hour2,Day1
    ind_usert FLOAT % Basic inodes usage(Linux) Min1,Min5,Min30,Hour2,Day1
    dev_nm STRING - Basic device name Min1
    mnt_stat_cd INTEGER - Basic mount state(1: mounted, 0: unmounted) Min1
    fs_type_nm STRING - Basic file system type Min1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    mnt_nm

Network

Network-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    snd_bps FLOAT bits/sec Basic send bits per second Min1,Min5,Min30,Hour2,Day1
    rcv_bps FLOAT bits/sec Basic receive bits per second Min1,Min5,Min30,Hour2,Day1
    snd_pps FLOAT packets/sec Basic send packets per second Min1,Min5,Min30,Hour2,Day1
    rcv_pps FLOAT packets/sec Basic receive packets per second Min1,Min5,Min30,Hour2,Day1
    snd_fail_packt_cnt INTEGER num Basic send fail packets Min1,Min5,Min30,Hour2,Day1
    rcv_fail_packt_cnt INTEGER num Basic receive fail packets Min1,Min5,Min30,Hour2,Day1
    clsn_packt_cnt (Linux) INTEGER num Basic collisions packets Min1,Min5,Min30,Hour2,Day1
    nic_ip STRING - Basic IP Address Min1
    macaddr STRING - Basic MAC Address Min1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    nic_desc

Process

Process-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    cpu_usert FLOAT % Basic per process CPU used ratio (%) Min1,Min5,Min30,Hour2,Day1
    p_proc_mem_usert FLOAT % Basic per process memory used ratio (%) Min1,Min5,Min30,Hour2,Day1
    vir_mem_byt_cnt FLOAT KB Basic virtual memory size Min1,Min5,Min30,Hour2,Day1
    cpu_tm_ss FLOAT - Basic cpu time Min1,Min5,Min30,Hour2,Day1
    prit_rnk INTEGER - Basic priority Min1,Min5,Min30,Hour2,Day1
    nice_val (Linux) INTEGER - Basic nice Min1,Min5,Min30,Hour2,Day1
    thd_cnt INTEGER num Basic threads count Min1,Min5,Min30,Hour2,Day1
    proc_t_cpu_usert (Linux) FLOAT % Basic cpu usage (%) Min1,Min5,Min30,Hour2,Day1
    pf (Windows) LONG num/sec Basic page fault count Min1,Min5,Min30,Hour2,Day1
    proc_id INTEGER - Basic process ID Min1
    parent_pid INTEGER - Basic parent process ID Min1
    proc_nm STRING - Basic process name Min1
    proc_cpu_cnt INTEGER num Basic cpu core count being used Min1,Min5,Min30,Hour2,Day1
    proc_stat_cd STRING - Basic process state Min1
    strt_ymdt (Linux) STRING - Basic process start time Min1
  • Dimension
    Dimension name Data type
    type
    instanceNo

Server

Server-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    boot_time LONG sec Basic boot time (sec) Min1
    time_deviation LONG millisecond Basic time deviation with NTP server Min1
    user_cnt INTEGER num Basic user count Min1
    is_on INTEGER flag Basic (deprecated) 1 if server is alive, 0 otherwise Min1
    is_rebooted INTEGER flag Basic 1 if reboot is detected, 0 otherwise Min1
    avg_cpu_used_rto FLOAT % Basic, Default CPU Utilization Average Min1,Min5,Min30,Hour2,Day1
    max_cpu_used_rto FLOAT % Basic CPU used ratio maximum (%) Min1,Min5,Min30,Hour2,Day1
    avg_cpu_idle_rto FLOAT % Basic CPU idle ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_cpu_user_rto FLOAT % Basic CPU user ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_cpu_sys_rto (Linux) FLOAT % Basic CPU system ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_nice_rto (Linux) FLOAT % Basic CPU nice ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_irq_rto (Linux) FLOAT % Basic CPU interrupt ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_softirq_rto (Linux) FLOAT % Basic CPU soft interrupt ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_io_wait_rto (Linux) FLOAT % Basic CPU IO wait ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_prv_mde_exec_tm_rto (Windows) FLOAT % Basic CPU used ratio in the privileged mode Min1,Min5,Min30,Hour2,Day1
    avg_dly_pcd_call_tm_rto (Windows) FLOAT % Basic CPU used ratio for deferred procedure calls (DPCs) (%) Min1,Min5,Min30,Hour2,Day1
    avg_interrup_tm_rto (Windows) FLOAT % Basic CPU used ratio for interrupts (%) Min1,Min5,Min30,Hour2,Day1
    avg_read_byt_cnt FLOAT bytes/sec Basic disk read bytes average Min1,Min5,Min30,Hour2,Day1
    avg_write_byt_cnt FLOAT bytes/sec Basic disk write bytes average Min1,Min5,Min30,Hour2,Day1
    avg_read_cnt FLOAT num/sec Basic disk read count per second average Min1,Min5,Min30,Hour2,Day1
    avg_write_cnt FLOAT num/sec Basic disk write count per second average Min1,Min5,Min30,Hour2,Day1
    max_read_byt_cnt INTEGER bytes/sec Basic disk read bytes maximum Min1,Min5,Min30,Hour2,Day1
    max_write_byt_cnt INTEGER bytes/sec Basic disk write bytes maximum Min1,Min5,Min30,Hour2,Day1
    max_read_cnt INTEGER num/sec Basic disk read count per second maximum Min1,Min5,Min30,Hour2,Day1
    max_write_cnt INTEGER num/sec Basic disk write count per second maximum Min1,Min5,Min30,Hour2,Day1
    fs_all_mb FLOAT MB Basic all block file systems size (MB) Min1,Min5,Min30,Hour2,Day1
    fs_used_mb FLOAT MB Basic all block file systems used (MB) Min1,Min5,Min30,Hour2,Day1
    fs_free_mb FLOAT MB Basic all block file systems used ratio (%) Min1,Min5,Min30,Hour2,Day1
    avg_fs_usert FLOAT % Basic, Default File System Utilization Average Min1,Min5,Min30,Hour2,Day1
    max_fs_usert FLOAT % Basic file system used ratio maximum (%) Min1,Min5,Min30,Hour2,Day1
    avg_ind_usert (Linux) FLOAT % Basic file system inodes used ratio average (%) Min1,Min5,Min30,Hour2,Day1
    avg_snd_bps FLOAT bits/sec Basic, Default Network Out Average Min1,Min5,Min30,Hour2,Day1
    max_snd_bps FLOAT bits/sec Basic network send bits/sec maximum Min1,Min5,Min30,Hour2,Day1
    avg_rcv_bps FLOAT bits/sec Basic, Default Network In Average Min1,Min5,Min30,Hour2,Day1
    max_rcv_bps FLOAT bits/sec Basic network receive bits/sec maximum Min1,Min5,Min30,Hour2,Day1
    avg_snd_pps FLOAT packets/sec Basic network send packets/sec average Min1,Min5,Min30,Hour2,Day1
    max_snd_pps FLOAT packets/sec Basic network send packets/sec maximum Min1,Min5,Min30,Hour2,Day1
    avg_rcv_pps FLOAT packets/sec Basic network receive packets/sec average Min1,Min5,Min30,Hour2,Day1
    max_rcv_pps FLOAT packets/sec Basic network receive packets/sec maximum Min1,Min5,Min30,Hour2,Day1
    avg_snd_fail_packt_cnt FLOAT num Basic network send fail packets average Min1,Min5,Min30,Hour2,Day1
    avg_rcv_fail_packt_cnt FLOAT num Basic network receive fail packets average Min1,Min5,Min30,Hour2,Day1
    avg_clsn_packt_cnt (Linux) FLOAT num Basic network collisions packets average Min1,Min5,Min30,Hour2,Day1
    proc_cnt INTEGER num Basic processes count Min1,Min5,Min30,Hour2,Day1
    run_que_len INTEGER num Basic running processes Min1,Min5,Min30,Hour2,Day1
    proc_mem_usert FLOAT % Basic all processes memory used ratio (%) Min1,Min5,Min30,Hour2,Day1
    load_average_1m FLOAT % Basic CPU load 1 minute (%) Min1,Min5,Min30,Hour2,Day1
    load_average_5m FLOAT % Basic CPU load 5 minute average (%) Min1,Min5,Min30,Hour2,Day1
    load_average_15m FLOAT % Basic CPU load 15 minute average (%) Min1,Min5,Min30,Hour2,Day1
    gpu_count INTEGER count Basic Total GPU Count Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo

GPU

GPU-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    vmem_usage INTEGER % Basic Current GPU MEM Usage Min1,Min5,Min30,Hour2,Day1
    vmem_usage_mib INTEGER MiB Basic Current GPU MEM Usage Min1,Min5,Min30,Hour2,Day1
    usage INTEGER % Basic Current GPU Usage Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    gpu_idx

Plugin Process

Plugin process-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    is_process_up INTEGER flag Basic 1 if process is up, 0 otherwise Min1,Min5,Min30,Hour2,Day1
    process_count INTEGER num Basic process count Min1,Min5,Min30,Hour2,Day1
    avg_cpu_usert FLOAT % Basic process used Average Min1,Min5,Min30,Hour2,Day1
    avg_mem_usert FLOAT % Basic average memory usage % Min1,Min5,Min30,Hour2,Day1
    avg_mem_byt_cnt FLOAT KB Basic memory usage Min1,Min5,Min30,Hour2,Day1
    avg_thd_cnt FLOAT num Basic thread thread count Min1,Min5,Min30,Hour2,Day1
    tot_cpu_usert FLOAT % Basic total cpu usage % Min1,Min5,Min30,Hour2,Day1
    tot_mem_usert FLOAT % Basic total memory usage % Min1,Min5,Min30,Hour2,Day1
    tot_mem_byt_cnt FLOAT KB Basic total memory usage Min1,Min5,Min30,Hour2,Day1
    tot_thd_cnt INTEGER num Basic total thread count Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    proc_name

Plugin File

Plugin file-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    file_modified INTEGER flag Basic 1: modified, 0: not modified Min1,Min5,Min30,Hour2,Day1
    file_exist INTEGER flag Basic 1: exist, 0: not exist Min1,Min5,Min30,Hour2,Day1
    file_size LONG bytes Basic file size(byte) Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    path

Plugin Port

Plugin port-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    is_port_up INTEGER flag Basic 1: port is up, 0: is down Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type
    type
    instanceNo
    port

Agent

Agent-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    agent_version INTEGER - Basic - Min1
  • Dimension
    Dimension name Data type
    type
    instanceNo

Hadoop

Hadoop-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    active_nodes INTEGER num Basic, Default active_nodes Min1,Min5,Min30,Hour2,Day1
    allocated_container INTEGER num Basic allocated_container Min1,Min5,Min30,Hour2,Day1
    allocated_mb INTEGER MB Basic, Default allocated_mb Min1,Min5,Min30,Hour2,Day1
    allocated_v_cores INTEGER num Basic allocated_v_cores Min1,Min5,Min30,Hour2,Day1
    apps_completed INTEGER num Basic, Default apps_completed Min1,Min5,Min30,Hour2,Day1
    apps_failed INTEGER num Basic, Default apps_failed Min1,Min5,Min30,Hour2,Day1
    apps_killed INTEGER num Basic, Default apps_killed Min1,Min5,Min30,Hour2,Day1
    apps_pending INTEGER num Basic, Default apps_pending Min1,Min5,Min30,Hour2,Day1
    apps_running INTEGER num Basic, Default apps_running Min1,Min5,Min30,Hour2,Day1
    apps_submitted INTEGER num Basic, Default apps_submitted Min1,Min5,Min30,Hour2,Day1
    available_mb INTEGER MB Basic, Default available_mb Min1,Min5,Min30,Hour2,Day1
    capacity_remaining_gb INTEGER GB Basic, Default capacity_remaining_gb Min1,Min5,Min30,Hour2,Day1
    corrupt_blocks INTEGER num Basic corrupt_blocks Min1,Min5,Min30,Hour2,Day1
    decommissioned_nodes INTEGER num Basic decommissioned_nodes Min1,Min5,Min30,Hour2,Day1
    hdfs_bytes_read INTEGER bytes Basic hdfs_bytes_read Min1,Min5,Min30,Hour2,Day1
    hdfs_bytes_written INTEGER bytes Basic hdfs_bytes_written Min1,Min5,Min30,Hour2,Day1
    hdfs_utilization FLOAT % Basic, Default hdfs_utilization Min1,Min5,Min30,Hour2,Day1
    lost_nodes INTEGER num Basic, Default lost_nodes Min1,Min5,Min30,Hour2,Day1
    missing_blocks INTEGER num Basic, Default missing_blocks Min1,Min5,Min30,Hour2,Day1
    num_live_data_nodes INTEGER num Basic, Default num_live_data_nodes Min1,Min5,Min30,Hour2,Day1
    pending_containers INTEGER num Basic pending_containers Min1,Min5,Min30,Hour2,Day1
    pending_deletion_blocks INTEGER num Basic pending_deletion_blocks Min1,Min5,Min30,Hour2,Day1
    pending_replication_blocks INTEGER num Basic pending_replication_blocks Min1,Min5,Min30,Hour2,Day1
    pending_v_cores INTEGER num Basic pending_v_cores Min1,Min5,Min30,Hour2,Day1
    rebooted_nodes INTEGER num Basic rebooted_nodes Min1,Min5,Min30,Hour2,Day1
    reserved_containers INTEGER num Basic reserved_containers Min1,Min5,Min30,Hour2,Day1
    reserved_mb INTEGER MB Basic reserved_mb Min1,Min5,Min30,Hour2,Day1
    total_load INTEGER num Basic total_load Min1,Min5,Min30,Hour2,Day1
    total_mb INTEGER MB Basic, Default total_mb Min1,Min5,Min30,Hour2,Day1
    total_nodes INTEGER num Basic, Default total_nodes Min1,Min5,Min30,Hour2,Day1
    under_replicated_blocks INTEGER num Basic under_replicated_blocks Min1,Min5,Min30,Hour2,Day1
    unhealthy_nodes INTEGER num Basic, Default unhealthy_nodes Min1,Min5,Min30,Hour2,Day1
    yarn_memory_available_percentage FLOAT % Basic, Default yarn_memory_available_percentage Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type idDimension
    type STRING
    instanceNo STRING
    clusterName STRING

Queue

Queue-related performance/operational metrics are as follows.

  • Metric
    Metric name Data type Unit Metric type Description Interval Aggregation
    queue_used_memory FLOAT GB Basic, Default queue_used_memory Min1,Min5,Min30,Hour2,Day1
    queue_used_memory_percent FLOAT % Basic, Default queue_used_memory_percent Min1,Min5,Min30,Hour2,Day1
    queue_used_v_cores INTEGER num Basic, Default queue_used_v_cores Min1,Min5,Min30,Hour2,Day1
    queue_used_v_cores_percent FLOAT % Basic, Default queue_used_v_cores_percent Min1,Min5,Min30,Hour2,Day1
  • Dimension
    Dimension name Data type idDimension
    type STRING
    mbrNo STRING
    instanceNo STRING
    clusterName STRING
    queueName STRING