Monitoring Gpu Usage: A Comprehensive Guide To Tracking Performance

Monitoring GPU usage is essential for ensuring that your system runs smoothly and efficiently, especially when performing computationally intensive tasks. GPU, or Graphics Processing Unit, is a dedicated hardware component designed to accelerate graphics and video rendering. By keeping track of GPU usage, you can identify performance bottlenecks and optimise your system's resources. This is particularly important in the fields of data science and machine learning, where GPUs are widely used for their ability to perform parallel computations.

Using the built-in Task Manager on Windows

To monitor your GPU usage on Windows, you can use the built-in Task Manager. This feature was added in the Fall Creators Update for Windows 10, so if you're using an older version of Windows, you won't be able to see these tools in your Task Manager.

To check if you have the right version of Windows, press Windows+R, type "dxdiag" into the box, and then press Enter to open the DirectX Diagnostic tool. Go to the "Display" tab and look for "WDDM 2.x" under "Driver Model" in the "Drivers" section. If you see "WDDM 1.x", your GPU isn't compatible.

Once you've confirmed compatibility, here's how you can use the Task Manager to monitor GPU usage:

• Open the Task Manager by right-clicking on any empty space on your taskbar and selecting "Task Manager", or by pressing Ctrl+Shift+Esc on your keyboard.
• If you see the standard, simple view, click on "More details" at the bottom of the Task Manager window.
• In the full view of the Task Manager, go to the "Processes" tab. Right-click on any column header and enable the "GPU" option. This adds a GPU column that shows the percentage of GPU resources used by each application.
• You can also enable the "GPU Engine" option to see which GPU engine an application is using.
• The total GPU usage of all applications is displayed at the top of the GPU column. You can click on this column to sort the list and see which applications are using the most GPU resources.
• The GPU Engine column displays the physical GPU and the engine being used by each application.
• If you want to see how much video memory an application is using, switch to the "Details" tab. Right-click on any column header, click "Select Columns", and then enable the "Dedicated GPU Memory" and "Shared GPU Memory" columns.
• The "Dedicated GPU Memory" column shows how much memory an application is using on your GPU. If you have a discrete graphics card, this is the amount of VRAM being used. If you have integrated graphics, it shows how much of your system RAM is reserved for your graphics hardware.
• The "Shared GPU Memory" column shows how much of the system's normal DRAM memory is being used by an application for video features.
• To monitor overall GPU resource usage, go to the "Performance" tab and select "GPU" from the sidebar. This will show you real-time GPU usage and graphs of dedicated and shared GPU memory usage.

While this process allows you to monitor GPU usage, it's important to note that the GPU section in the Task Manager disappears every time you close it. However, the columns you enabled under the "Details" section will remain.

Using third-party tools like MSI Afterburner

If you're looking for a more advanced way to monitor your GPU usage, you can turn to third-party tools like MSI Afterburner. This tool is primarily an overclocking utility designed to help your graphics card achieve higher performance. However, when paired with the RivaTuner Statistics Server (RTSS) from Guru3D.com, it becomes a powerful in-game performance monitoring solution.

To get started with MSI Afterburner, you'll need to download and install both it and the RTSS on your Windows PC. Once you have the applications installed, you'll be greeted by the Afterburner interface, which displays two dials showing the current status of your graphics cards, including GPU and memory clock frequencies, voltage, and temperature.

To access the performance monitoring features, click the Settings cog in Afterburner, then navigate to the "On-Screen Display" section. Here, you can set your global hotkeys for accessing the on-screen display. Next, click the "Monitoring" tab to choose which hardware stats you want to see while gaming.

In the "Active Hardware Monitoring Graphs" section, you can select the specific metrics you're interested in, such as GPU usage, temperature, memory usage, and more. It's worth noting that by default, none of these options are displayed on-screen, so you'll need to enable them manually. Additionally, you can adjust the display format, choosing between text or graphs for each metric.

One particularly useful metric to monitor is the framerate, which can help you ensure your machine is achieving a smooth 60 frames per second. To enable this, simply tick the checkbox next to "Framerate" and "Show in On-Screen Display." Similarly, if you have a multi-core processor, you can monitor CPU usage per thread to keep an eye on how work is distributed.

MSI Afterburner also allows you to monitor GPU temperature, which is crucial for ensuring your GPU fans are functioning optimally to prevent overheating. You can also monitor other system parameters like CPU temperature, RAM usage, and power draw.

While this feature is particularly useful for gaming, it can also be helpful when trying out a new game or after updating your drivers or game to see how your system handles the changes.

Using the NVIDIA System Management Interface (nvidia-smi)

The NVIDIA System Management Interface, or nvidia-smi, is a command-line utility that provides a comprehensive view of GPU usage and performance. It is a powerful tool for monitoring and troubleshooting GPU-related issues in NVIDIA-based systems. Here are some detailed instructions on using nvidia-smi to monitor your GPU usage:

To get an overview of your GPU usage, you can use the following query:

$ nvidia-smi --query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,utilization.memory,memory.total,memory.free,memory.used --format=csv -l 5

This query provides a range of information, including the GPU name, PCI bus ID, driver version, power state, PCIe link generation, core GPU temperature, GPU and memory utilisation, and memory statistics. The `--format=csv` option formats the output as a CSV file, making it easier to read. The `-l 5` option sets the query to run every 5 seconds, providing real-time updates.

You can also query specific aspects of your GPU's performance. For example, to check the VBIOS version of each device, use the following query:

$ nvidia-smi --query-gpu=gpu_name,gpu_bus_id,vbios_version --format=csv

This will return the official product name of the GPU, the PCI bus ID, and the VBIOS version.

If you want to monitor GPU behaviour over time and output the data to a CSV file, you can use the following command:

$ nvidia-smi --query-gpu=index,timestamp,power.draw,clocks.sm,clocks.mem,clocks.gr --format=csv -l 1

This command provides continuous monitoring of detailed statistics such as power draw, SM clocks, memory clocks, and graphics clocks, with timestamps included.

Additionally, you can enable persistence mode to ensure that your clock and power settings persist across program runs or driver invocations:

$ nvidia-smi -pm 1

This command makes your settings persistent, so you don't have to reset them each time.

By utilising these nvidia-smi queries, you can effectively monitor and manage your GPU usage, ensuring optimal performance and troubleshooting any potential issues.

Using nvtop, a tool similar to htop but for NVIDIA GPUs

NVTOP stands for Neat Videocard TOP, a (h)top-like task monitor for GPUs and accelerators. It can handle multiple GPUs and print information about them in a familiar way. NVTOP supports a variety of vendors, including NVIDIA (Linux proprietary drivers).

To install NVTOP, you can use the following commands:

For Ubuntu/Debian

Sudo apt install nvtop

For Red Hat/CentOS

Sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E %{rhel}).noarch.rpm

Sudo dnf install nvtop

Once installed, you can launch NVTOP by simply typing `nvtop` in your terminal. This will bring up a terminal-based interface with updating graphs and a summary of the processes running on your GPUs.

NVTOP allows you to customize the display, including the information shown in graphs, how processes are sorted, and how frequently updates should occur. These settings can be configured interactively and saved to disk, so your preferences persist between sessions.

Additionally, NVTOP includes a feature similar to htop, allowing you to interactively kill processes from the GUI. This can be useful if you need to terminate any buggy or resource-intensive processes.

Using the NVIDIA Control Panel

To monitor your GPU usage using the NVIDIA Control Panel, you will first need to launch the Control Panel as an administrator. To do this, log in to Windows as an administrator, then launch the NVIDIA Control Panel. Alternatively, use the Windows file manager to navigate to the Control Panel Client and run it as an administrator.

Once you have accessed the NVIDIA Control Panel, you can view GPU utilisation by clicking on the 'Select a Task' pane under 'Workstation', then clicking 'Manage GPU Utilisation'. This will bring up a page that displays information such as the high-end Quadro and Tesla GPUs installed in your system, the utilisation of each GPU, and whether ECC is enabled for each GPU. You can also view a graph of GPU utilisation over time by clicking on 'GPU Utilisation Graphs'.

If you wish to change the usage mode of a GPU, you can do so by selecting the 'Select a Task' pane and clicking 'Manage GPU Utilisation' under 'Workstation'. Under 'Usage Mode', select the appropriate option for each Quadro card. You can choose whether the Quadro card or SLI group handles professional graphics and compute (CUDA) needs, or professional graphics needs only. Please note that Tesla cards are automatically dedicated to compute needs and cannot be changed. After making your changes, click 'Apply'.

The Usage Mode setting will apply to all applications and programs by default. However, you can set a specific usage mode for a particular program by clicking on the 'Manage 3D Settings' link at the bottom of the page and adjusting the CUDA-GPUs setting accordingly.

Frequently asked questions