mirror of
https://github.com/DualCoder/vgpu_unlock.git
synced 2024-05-19 19:53:04 +12:00
feat: update readme with patching instructions
This commit is contained in:
Igor Moura
parent
886c1cfe61
commit
d4561a43e0
55
README.md
55
README.md
|
|
@ -1,13 +1,11 @@
|
|||
# vgpu\_unlock
|
||||
# vgpu_unlock
|
||||
|
||||
Unlock vGPU functionality for consumer grade GPUs.
|
||||
|
||||
|
||||
## Important!
|
||||
|
||||
This tool is very untested, use at your own risk.
|
||||
|
||||
|
||||
## Description
|
||||
|
||||
This tool enables the use of Geforce and Quadro GPUs with the NVIDIA vGPU
|
||||
|
|
@ -15,23 +13,33 @@ software. NVIDIA vGPU normally only supports a few Tesla GPUs but since some
|
|||
Geforce and Quadro GPUs share the same physical chip as the Tesla this is only
|
||||
a software limitation for those GPUs. This tool aims to remove this limitation.
|
||||
|
||||
|
||||
## Dependencies:
|
||||
|
||||
* This tool requires Python3, the latest version is recommended.
|
||||
* The python package "frida" is required. `pip3 install frida`.
|
||||
* The tool requires the NVIDIA GRID vGPU driver.
|
||||
* "dkms" is required as it simplifies the process of rebuilding the
|
||||
driver alot. Install DKMS with the package manager in your OS.
|
||||
|
||||
- This tool requires Python3, the latest version is recommended.
|
||||
- The python package "frida" is required. `pip3 install frida`.
|
||||
- The tool requires the NVIDIA GRID vGPU driver.
|
||||
- "dkms" is required as it simplifies the process of rebuilding the
|
||||
driver alot. Install DKMS with the package manager in your OS.
|
||||
|
||||
## Installation:
|
||||
|
||||
### Easy patch:
|
||||
|
||||
In the following instructions, it's assumed that you have the `vgpu_unlock` folder under `/opt/`. As in, the file `vgpu_unlock_hooks.c` should be located at `/opt/vgpu_unlock/vgpu_unlock_hooks.c`. If that's not the case, then proceed with the manual installation bellow.
|
||||
|
||||
- Make sure that both the NVIDIA vGPU-KVM driver installer (the `.run` file) and the `vgpu_unlock.patch` are in the same folder.
|
||||
- Apply the patch with `sh NVIDIA-INSTALLER.run --apply-patch vgpu_unlock.patch`. This should create a new file ending with `-custom.run`.
|
||||
- Install the driver using the new custom installer: `sh NVIDIA-INSTALLER-custom.run --dkms`.
|
||||
- Reboot.
|
||||
|
||||
### Manually:
|
||||
|
||||
In the following instructions `<path_to_vgpu_unlock>` need to be replaced with
|
||||
the path to this repository on the target system and `<version>` need to be
|
||||
replaced with the version of the NVIDIA GRID vGPU driver.
|
||||
|
||||
Install the NVIDIA GRID vGPU driver, make sure to install it as a dkms module.
|
||||
|
||||
```
|
||||
./nvidia-installer --dkms
|
||||
```
|
||||
|
|
@ -39,11 +47,13 @@ Install the NVIDIA GRID vGPU driver, make sure to install it as a dkms module.
|
|||
Modify the line begining with `ExecStart=` in `/lib/systemd/system/nvidia-vgpud.service`
|
||||
and `/lib/systemd/system/nvidia-vgpu-mgr.service` to use `vgpu_unlock` as
|
||||
executable and pass the original executable as the first argument. Ex:
|
||||
|
||||
```
|
||||
ExecStart=<path_to_vgpu_unlock>/vgpu_unlock /usr/bin/nvidia-vgpud
|
||||
```
|
||||
|
||||
Reload the systemd daemons:
|
||||
|
||||
```
|
||||
systemctl daemon-reload
|
||||
```
|
||||
|
|
@ -51,22 +61,26 @@ systemctl daemon-reload
|
|||
Modify the file `/usr/src/nvidia-<version>/nvidia/os-interface.c` and add the
|
||||
following line after the lines begining with `#include` at the start of the
|
||||
file.
|
||||
|
||||
```
|
||||
#include "<path_to_vgpu_unlock>/vgpu_unlock_hooks.c"
|
||||
```
|
||||
|
||||
Modify the file `/usr/src/nvidia-<version>/nvidia/nvidia.Kbuild` and add the
|
||||
following line at the bottom of the file.
|
||||
|
||||
```
|
||||
ldflags-y += -T <path_to_vgpu_unlock>/kern.ld
|
||||
```
|
||||
|
||||
Remove the nvidia kernel module using dkms:
|
||||
|
||||
```
|
||||
dkms remove -m nvidia -v <version> --all
|
||||
```
|
||||
|
||||
Rebuild and reinstall the nvidia kernel module using dkms:
|
||||
|
||||
```
|
||||
dkms install -m nvidia -v <version>
|
||||
```
|
||||
|
|
@ -74,12 +88,12 @@ dkms install -m nvidia -v <version>
|
|||
Reboot.
|
||||
|
||||
---
|
||||
|
||||
**NOTE**
|
||||
|
||||
This script will only work if there exists a vGPU compatible Tesla GPU that
|
||||
uses the same physical chip as the actual GPU being used.
|
||||
|
||||
|
||||
---
|
||||
|
||||
## How it works
|
||||
|
|
@ -92,18 +106,18 @@ ID is unique for each type of PCI device. In order to enable vGPU support we
|
|||
need to tell the driver that the PCI device ID of the installed GPU is one of
|
||||
the device IDs used by a vGPU capable GPU.
|
||||
|
||||
### Userspace script: vgpu\_unlock
|
||||
### Userspace script: vgpu_unlock
|
||||
|
||||
The userspace services nvidia-vgpud and nvidia-vgpu-mgr uses the ioctl syscall
|
||||
to communicate with the kernel module. Specifically they read the PCI device ID
|
||||
and determines if the installed GPU is vGPU capable.
|
||||
|
||||
The python script vgpu\_unlock intercepts all ioctl syscalls between the
|
||||
The python script vgpu_unlock intercepts all ioctl syscalls between the
|
||||
executable specified as the first argument and the kernel. The script then
|
||||
modifies the kernel responses to indicate a PCI device ID with vGPU support
|
||||
and a vGPU capable GPU.
|
||||
|
||||
### Kernel module hooks: vgpu\_unlock\_hooks.c
|
||||
### Kernel module hooks: vgpu_unlock_hooks.c
|
||||
|
||||
In order to exchange data with the GPU the kernel module maps the physical
|
||||
address space of the PCI bus into its own virtual address space. This is done
|
||||
|
|
@ -111,7 +125,7 @@ using the ioremap\* kernel functions. The kernel module then reads and writes
|
|||
data into that mapped address space. This is done using the memcpy kernel
|
||||
function.
|
||||
|
||||
By including the vgpu\_unlock\_hooks.c file into the os-interface.c file we can
|
||||
By including the vgpu_unlock_hooks.c file into the os-interface.c file we can
|
||||
use C preprocessor macros to replace and intercept calls to the iormeap and
|
||||
memcpy functions. Doing this allows us to maintain a view of what is mapped
|
||||
where and what data that is being accessed.
|
||||
|
|
@ -127,9 +141,9 @@ to let us know where that section begins and ends.
|
|||
### How it all comes together
|
||||
|
||||
After boot the nvidia-vgpud service queries the kernel for all installed GPUs
|
||||
and checks for vGPU capability. This call is intercepted by the vgpu\_unlock
|
||||
and checks for vGPU capability. This call is intercepted by the vgpu_unlock
|
||||
python script and the GPU is made vGPU capable. If a vGPU capable GPU is found
|
||||
then nvidia-vgpu creates an MDEV device and the /sys/class/mdev\_bus directory
|
||||
then nvidia-vgpu creates an MDEV device and the /sys/class/mdev_bus directory
|
||||
is created by the system.
|
||||
|
||||
vGPU devices can now be created by echoing UUIDs into the `create` files in the
|
||||
|
|
@ -137,7 +151,7 @@ mdev bus representation. This will create additional structures representing
|
|||
the new vGPU device on the MDEV bus. These devices can then be assigned to VMs,
|
||||
and when the VM starts it will open the MDEV device. This causes nvidia-vgpu-mgr
|
||||
to start communicating with the kernel using ioctl. Again these calls are
|
||||
intercepted by the vgpu\_unlock python script and when nvidia-vgpu-mgr asks if
|
||||
intercepted by the vgpu_unlock python script and when nvidia-vgpu-mgr asks if
|
||||
the GPU is vGPU capable the answer is changed to yes. After that check it
|
||||
attempts to initialize the vGPU device instance.
|
||||
|
||||
|
|
@ -148,7 +162,7 @@ The kernel module maps the physical PCI address range 0xf0000000-0xf1000000 into
|
|||
its virtual address space, it then performs some magical operations which we
|
||||
don't really know what they do. What we do know is that after these operations
|
||||
it accesses a 128 bit value at physical address 0xf0029624, which we call the
|
||||
magic value. The kernel module also accessses a 128 bit value at physical
|
||||
magic value. The kernel module also accessses a 128 bit value at physical
|
||||
address 0xf0029634, which we call the key value.
|
||||
|
||||
The kernel module then has a couple of lookup tables for the magic value, one
|
||||
|
|
@ -170,7 +184,7 @@ to generate a valid HMAC-SHA256 signature and the AES-128 decrypted data blocks
|
|||
has to contain a vGPU capable PCI device ID. If any of these checks fail, then
|
||||
the error code 0x56 "Call not supported" is returned.
|
||||
|
||||
In order to make these checks pass the hooks in vgpu\_unlock\_hooks.c will look
|
||||
In order to make these checks pass the hooks in vgpu_unlock_hooks.c will look
|
||||
for a ioremap call that maps the physical address range that contain the magic
|
||||
and key values, recalculate the addresses of those values into the virtual
|
||||
address space of the kernel module, monitor memcpy operations reading at those
|
||||
|
|
@ -180,4 +194,3 @@ the signature and data bocks, validate the signature, decrypt the blocks, edit
|
|||
the PCI device ID in the decrypted data, reencrypt the blocks, regenerate the
|
||||
signature and insert the magic, blocks and signature into the table of vGPU
|
||||
capable magic values. And that's what they do.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue