AmirHossein Sojoodi

AmirHossein Sojoodi

PhD student in Computer Engineering at Queen's University

NVIDIA Management Library (NVML)

Published:
4 minute read

In this post, I will provide two examples on how to use the NVIDIA Management Library (NVML) to query the GPU information. The first example is a simple C++ program that prints the GPU power and temperature, and the second one is about getting PCIe info.

NVML

The NVIDIA Management Library (NVML) is a C-based API for monitoring and managing various states of NVIDIA GPU devices. The library is used to query the GPU information, such as power, temperature, and utilization. The library is also used to manage the GPU devices, such as setting the power limit and the clock speed. The complete documentation of NVML can be found here. Some famous tools that use NVML are nvidia-smi and nvtop.

Example 1: Power and Temperature

The following C++ program prints the GPU power and temperature.

#include <iostream>
#include <nvml.h>

#define CHECK_NVML(result, message) \
    if (result != NVML_SUCCESS) { \
        std::cerr << message << ": " << nvmlErrorString(result) << std::endl; \
        return 1; \
    }

int main() {
    
    nvmlDevice_t device;
    unsigned int power;
    unsigned int temperature;

    CHECK_NVML(nvmlInit(), "Failed to initialize NVML");
    CHECK_NVML(nvmlDeviceGetHandleByIndex(0, &device), "Failed to get device handle");
    CHECK_NVML(nvmlDeviceGetPowerUsage(device, &power), "Failed to get power usage");
    CHECK_NVML(nvmlDeviceGetTemperature(device, NVML_TEMPERATURE_GPU, &temperature), "Failed to get temperature");
    
    std::cout << "Power: " << power << " mW" << std::endl;
    std::cout << "Temperature: " << temperature << " C" << std::endl;

    nvmlShutdown();
    return 0;
}

The program uses the NVML library, therefore, it should be compiled with the -lnvidia-ml flag.

g++ -o nvml_power_temp nvml_power_temp.cpp -lnvidia-ml
./nvml_power_temp
# If you want a specific GPU, you can set the CUDA_VISIBLE_DEVICES environment variable:
# CUDA_VISIBLE_DEVICES=0 ./nvml_power_temp

Example 2: Some PCIe Information

The following C++ program prints some PCIe information.

#include <iostream>
#include <nvml.h>

#define CHECK_NVML(result, message) \
    if (result != NVML_SUCCESS) { \
        std::cerr << message << ": " << nvmlErrorString(result) << std::endl; \
        return 1; \
    }

int main() {
    
    unsigned int device_count;
    
    CHECK_NVML(nvmlInit(), "Failed to initialize NVML");
    CHECK_NVML(nvmlDeviceGetCount(&device_count), "Failed to get device count");
    
    for (unsigned int i = 0; i < device_count; i++) {
        nvmlDevice_t device;
        nvmlPciInfo_t pci_info;
        unsigned int speed;
        unsigned int max_speed;
        unsigned int max_width;
        unsigned int curr_width;
        unsigned int tx_throughput;
        unsigned int rx_throughput;

        CHECK_NVML(nvmlDeviceGetHandleByIndex(i, &device), 
                    "Failed to get device handle");
        CHECK_NVML(nvmlDeviceGetPciInfo(device, &pci_info), 
                    "Failed to get PCI info");
        CHECK_NVML(nvmlDeviceGetMaxPcieLinkWidth(device, &max_width), 
                    "Failed to get PCIe link max width");
        CHECK_NVML(nvmlDeviceGetCurrPcieLinkWidth(device, &curr_width), 
                    "Failed to get PCIe link current width");
        // For whatever reasons, the following metric doesn't make sense to me.
        CHECK_NVML(nvmlDeviceGetPcieLinkMaxSpeed(device, &max_speed), 
                    "Failed to get PCIe link max speed");
        CHECK_NVML(nvmlDeviceGetPcieSpeed(device, &speed), 
                    "Failed to get PCIe link speed");
        CHECK_NVML(nvmlDeviceGetPcieThroughput(device, NVML_PCIE_UTIL_TX_BYTES, &tx_throughput), 
                    "Failed to get PCIe throughput");
        CHECK_NVML(nvmlDeviceGetPcieThroughput(device, NVML_PCIE_UTIL_RX_BYTES, &rx_throughput), 
                    "Failed to get PCIe throughput");

        std::cout << "Device " << i << ":" << std::endl;
        std::cout << "  Bus ID: " << pci_info.busId << std::endl;
        std::cout << "  PCIe Max Link Width: " << max_width << " lanes" << std::endl;
        std::cout << "  PCIe Current Link Width: " << curr_width << " lanes" << std::endl;
        std::cout << "  PCIe Max Link Speed: " << max_speed << " MB/s" << std::endl;
        std::cout << "  PCIe Link Speed: " << speed << " MB/s" << std::endl;
        std::cout << "  PCIe TX Throughput: " << tx_throughput << " KB/s" << std::endl;
        std::cout << "  PCIe RX Throughput: " << rx_throughput << " KB/s" << std::endl;
    }

    nvmlShutdown();
    return 0;
}

Again, the program uses the NVML library, therefore, it should be compiled with the -lnvidia-ml flag.

You May Also Enjoy

Setup LAMMPS

4 minute read

Published:

Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) is a classical molecular dynamics code that can be used to model atoms or, more generally, as a parallel particle simulator at various scales. The complete documentation of LAMMPS can be found here. In this post, I will provide a guide on how to setup LAMMPS on a Linux machine. My setup is on a cluster with NVIDIA GPUs, UCX, and OpenMPI. Also, we have a built-in module system to load the necessary modules.

MPS on Multi-Instance GPU

9 minute read

Published:

In previous posts, (MPS and MIG), I have explained how to enable MPS and MIG on NVIDIA GPUs. In this post, I will explain how to use both technologies at the same time. In more detail, I would like to enable MPS on all of the MIG instances. For more information, you can refer to the NVIDIA document.

Wasm + WebGPU example on DCP

12 minute read

Published:

This example is a follow-up to my previous post on how to write a cross-platform WebGPU example. In this one, I’ll demonstrate how to deploy a matmult example written in C/C++ and WebGPU in a DCP worker using WASM. Note that for verification purposes, I provide dawn-based native test, too, but this example doesn’t need to build/install dawn in order to work.

Cross Platform WebGPU Example - Compute API

5 minute read

Published:

As a part of my work at Distributive, I developed an example providing a simple matrix multiplication using WebGPU Compute API in C++. This example is written in C++ and can be compiled to WebAssembly using Emscripten, native executable file using Dawn. The example is based on here, and its complete source can be found there.