FYI: Resizable BAR (Base Address Register) is a PCI Express feature that removes the traditional 256MB memory limit, allowing your CPU to access your graphics card's entire VRAM at once for a potential 5% to 15% boost in gaming performance
Only on specific devices, but generally every GPU released in last few generations, and every dedicated GPU from intel[1], supports "large BAR" this way to map entire GPU memory into MMIO space. This has to be supported on both target device and AFAIK the PCI Root Port[2], and of course needs to be properly handled in firmware - which this tool fixes.
[1] Intel GPUs actually can't work without it, because they do not have legacy "windowing" mode where portions of 256MB BAR window are mapped to different areas in GPU memory
[2] This tool implies that it's possible to handle root port side entirely in firmware, but it's new to me at least, so I can't vouch for that :)
These kind of BIOS modding shenannigans is why I got interesed in Coreboot, so that enhancing a Firmware capabilities becomes possible without touching propietary tools/code.
Other similar mods included adding NVMe Boot capabilities to older Firmwares that didn't supported them so that you could boot from modern NVMe drives if you used some kind of adapter (As none of those Motherboards have M.2 Slots, but you can use them with a PCIe-to-M.2 adapter card). Also adding required stuff to get certain CPUs booting in Motherboard that were never officially supported (Depending on generation, could be as simple as adding new CPU Microcode).
These days you might be able to just point an AI at the original BIOS and tell it what you need done, instead of attempting to install a complete rewrite like Coreboot (which comes with its own limitations, the biggest one being that only a rather small subset of mobos are actually supported.)
Coreboot in desktops is perhaps in the best shape it has been in more than a decade because you have the MSI Z690-A/Z790-P series for LGA 1700 (Which were current gen when released, and still viable if you want Intel with DDR4, albeit the boards aren't available in retail anymore) and the recently released port for the MSI B850-P for AM5 Phoenix. It was a major leap from previously, where it only supported ancient platforms or unobtanium reference boards. I bashed Coreboot a lot for that, but now you actually have usable options. Same with Servers, with the Gigabyte MZ33-AR1 for AMD Turin. When was the last time you saw Coreboot running on current gen boards available in retail, if ever?
The last time I looked at Coreboot it didn't have any configuration UI at all ("you have to set the options and recompile/reflash" was the answer I got), which basically makes it next to useless for a regular PC... and apparently it still doesn't. They've already done the hard work of RE'ing and all the hardware-specific stuff; how hard can it be to at least have a simple setup menu? Hopefully someone vibecodes (vibeports?) one in the next little while.
(Incidentally this is also one of my gripes with the Intel Macs, wanting to put Linux on one but having zero configurability pre-OS is a huge pain. Meanwhile a standard PC BIOS with the equivalent hardware has plenty of options. Maybe I'll get around to (vibe)porting a regular BIOS to mine...)
The menu part is the task of the payload.
If you compile Coreboot with a payload you can get a menu no problem. EDK II, Seabios or grub payloads all support this.
If you want to you may want to point your vibes for vibecoding towards making a libpayload universal menu for Coreboot , that way you at least conform with the architecture.
I was an early user of the Z690-A from 2022 and while it was dull in features, the basics (Boot Menu) were already present. The configuration UI was TianoCore/EDK2:
Regarding Coreboot, it is far worse than that. They literally do not care anymore; focus is not on corebooting all the boards, but on serving some sort of commercial "customer".
e.g. they do seldom if ever take patches for adding support or fixing issues in older boards. They just sit there ignored.
A fork or entire new project with different focus is sorely needed.
There is also the Libreboot which may be of your interest. What I see is that there is a lot of development on Coreboot but it is indeed mostly driven by companies working on their latest platforms. There are not many contributors working on older platforms. Though in my experience the response to patches is relatively quick.
I had the exact opposite impression, there are a lot of user contributions working on old platforms, and even specific platform communities. For example, you have 15h.org which is a community dedicated to AMD Bulldozer / Piledriver generation, and they managed to fix a few things that were left broken in Coreboot. I also saw an influx of users working in Sandy Bridge / Ivy Bridge / Haswell era Intel platforms with an occasional Skylake. The problem is, precisely, that most of the user contributions are for old platforms that nobody that does NOT currently owns those will want to touch if you're purchasing new. I would have been very enthusiastic about that work if it was done about 10 years ago or so, nowadays they seem to be hobby projects that don't advance anything forward.
There do exists individual users that work on newer stuff, after the MSI Z690-A / Z790-P there were 2 people that did ports for H610 since at that point the platform was supported and they only had to port the Motherboard. However, they are rather rare, since the kind of user that Coreboot seems to attract are from the privacy and security anti-ME/PSP crowd that don't even want to use newer platforms, which is the exact opposite of my view.
A lot of the work companies does may be just for their internal systems or reference platform and these may not get to the hands of end users, ever. I still recall Phoronix articles covering Coreboot releases always taking a jab at the new supported Motherboards until the MSI Z690-A DDR4 port from 2022.
Since about 5 or 6 years ago, PCIe Video Cards advertise a capability known as Resizeable BAR. PCI Devices like GPUs require some memory to use for PCI MMIO, which is directly visible on the CPU Address Space. As a side note, this Address Space is shared with RAM, and anyone that was around when having 4 GiB RAM with a 32 Bit OS was common (Earlier than 2010 or so) knows that you only saw about 3.25 GiB RAM or so because of sharing the Address Space with PCI MMIO, and those fortunate enough to have used SLI usually saw even less than that, like 2.87 GiB RAM.
GPUs has been using a 256 MiB PCI MMIO window regardless of how much VRAM they actually have since... nearly forever? At least since PCIe is a thing, since I recall than AGP Aperture Size was seteable in era accurate BIOSes. PCIe 3.0 specification introduced a feature known as Resizeable BAR, where the PCI Device can tell a compatible Firmware how much MMIO it actually wants. GPUs uses that to tell a ReBAR capable UEFI Firmware that it wants more MMIO (Usually as big as the GPU VRAM), or uses legacy 256 MiB otherwise.
GPUs have been using a 256MB window since they started coming with 256MB of VRAM, as it would otherwise be a pointless waste of address space to have a window larger than the actual amount of VRAM. Previously, they would have a window exactly equal to the size of VRAM (likely rounded up to the next power of 2.)
There would also be a block of memory mapped hardware registers also needing address space that you have to poke to make the GPU actually do GPU things, instead of just being an expensive way to add extra memory to a system, no?
And if my experience from embedded development is in any way transferable, they're probably fairly spread out and probably takes a fairly big chunk of address space too.
Yes, typically the memory mapped memory is in one (64-bit) bar, memory mapped registers in another (64-bit) bar, plus for compatability with vga, probably a 32-bit memory bar and an i/o bar. 64-bit bars take up two bar slots, so that fills all six slots in the PCI config.
Just to make sure I understand: Is the resizable BAR/MMIO a RAM buffer for PCie packets? Or does it have some deeper integration with DMA or something? I assume it's not like memory mapped peripherals on an AXI bus which is why you need the buffer?
No idea on the AXI Bus you're talking about, so can't make comparisons.
MMIO (Memory Mapped I/O) is essentially memory (Whenever RAM or ROM) from OTHER devices that is directly visible on the CPU Address Space. My understanding is that from the CPU side, MMIO is mostly transparent (Except for the massive increase in latency) because it gets used like if it was interacting with its own workspace with regular instructions like MOV.
What PCIe ReBAR changes is that before, you could only see a 256 MiB window onto the GPU VRAM, so there was an added overhead since the GPU may need to relocate things from inside that window somewhere else on its total VRAM (So yes, it may be interpreted as if what you see from the CPU side is just some kind of exchange buffer).
I believe the best way to describe how it operates is comparing it to EMS (Expanded Memory) from the DOS days since it also worked with a similar, if not the same idea. You could only see a portion of the total memory from what was installed on the EMS card (A 128 KiB window located on the upper part of the 1 MiB address space from the 8086 CPU), so you had to switch which Page (Region) of the memory was visible, adding a lot of overhead and most likely requiring an additional buffer in main RAM to move data from one Page to another. However, since I have no knowledge if the GPUs really work like that I can't confirm. I never knew whenever the 256 MiB is "fixed" (You always see the same Region) or if you can decide which section of the VRAM to make visible.
For historic reasons (e.g. 32 bit address spaces, plus the need to reserve the space for multiple pci peripherals) it has been a narrow, movable aperture.
Resizable BAR lets the size of the aperture be chosen (which is usually chosen to allow all of VRAM to fit in and be directly accessible).
There is no buffer at all. BAR is "base address register" (hint, there is no "the BAR"). Each PCIe device / function can have up to 6 32-bit BARs or 3 64 bit BARs.
When you insert a GPU into a PCIe slot, the memory mapped regions in memory can't be put in a hard wired location because any arbitrary device can be inserted and it can provide an arbitrary amount of memory (yes the GPU provides its own memory to the CPU). A BAR reserves a memory mapped region in the CPU space that is backed by the PCIe device.
When the BAR is smaller than the memory of the inserted device, the CPU cannot communicate with all of the memory on the inserted device directly anmore. This means if you want to perform a write to a region in the GPU outside a BAR region you have to go through the BAR region anyway. It's not a RAM buffer for PCIe packets.
>I assume it's not like memory mapped peripherals on an AXI bus which is why you need the buffer?
It's hard to explain without going into a few low-level details of PCI Express, but let me try.
Most PCI devices expose some memory and/or I/O ports to the CPU. That memory (or I/O ports) is mapped to somewhere in the address space visible to the CPU. Besides the memory and I/O ports, all PCI devices also expose a separate set of configuration registers; among these registers, there are the Base Address Registers (BARs), which configure where the memory or I/O ports is mapped.
Here's an example output from "lspci -vv" for a GPU:
Region 0: Memory at 7c00000000 (64-bit, prefetchable) [size=8G]
Region 2: Memory at 7e00000000 (64-bit, prefetchable) [size=256M]
Region 4: I/O ports at f000 [size=256]
Region 5: Memory at fca00000 (32-bit, non-prefetchable) [size=1M]
Expansion ROM at fcb00000 [disabled] [size=128K]
Note that regions 0 and 2 are above the 4GB addressable by old 32-bit CPUs. To be compatible with these old CPUs, this card and many others like it allow the firmware (and/or the operating system) to choose not only where the memory is mapped, but also its size. We can see this in the same "lspci -vv" output for this GPU:
Capabilities: [200 v1] Physical Resizable BAR
BAR 0: current size: 8GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB
BAR 2: current size: 256MB, supported: 2MB 4MB 8MB 16MB 32MB 64MB 128MB 256MB
Older systems which do not understand this extended capability will still treat these regions as fixed size, probably with the first size in this list (256MB for region 0, 2MB for region 2). Newer systems can tell the device to "resize" the BAR to a bigger size, which obviously needs the first region to be placed above the 4GB barrier since it's too big.
Why is this useful? This particular GPU has 8GB of VRAM; it's quite obvious that region 0 is a direct view into that VRAM. When using the maximum BAR size, the CPU can directly read and write anywhere into the VRAM; when using a smaller BAR, the CPU can only see a small window into the VRAM, and has to use less direct methods to access it.
(As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)
> (As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)
IIRC Linux doesn't need resizable bar enabled in the BIOS since the kernel will resize the bar if supported by the GPU, Windows however relies on the UEFI doing it which is where it being enabled in the BIOS is needed.
ReBar's commercial name is AMD Smart Access Memory, it allows a PCIe device such as a GPU to map more VRAM to the system at once, which improves performance by reducing access overhead. This generated much fanfare in the early 2020s after AMD officially supported it in the newly released AMD Zen 3 CPUs with RX6800 series GPUs. It was marketed as a new technology to boost GPU/gaming performance. What AMD did was just rebranding an obscure feature in the PCIe specification [1]. As shown by this project, it was actually supported by the PCIe controller since Sandy Bridge, just disabled in the firmware. For a decade nobody bothered to use it. Presumably, AMD saw an opportunity and enabled it, presumably after validating the hardware and fixing any driver compatibility problems.
[1] This is nothing new in the tech industry. Intel rebrands DVFS as SpeedStep, IOMMU as VT-d, AMD rebrands the NX bit as Enhanced Virus Protection, etc.
I feel part of the "Smart Access Memory" name is actually the driver features and paths to actually make use of it.
It's not like ReBar is a single toggle "Make Things Faster", but a different option in how it can map gpu memory to the cpu. The driver still needs to use it - and decide where it's use vs the "staging buffer" approach would actually be be benefitial.
Resizable Base Address Register. In short, rather than the 256MB of mapped memory available to the CPU for any PCI device, ReBAR capable devices can map larger memory to the CPU’s addressable memory space. Without ReBAR or sufficient tricks (that Intel GPUs lack) you have to shuffle 256MB chunks around between GPU and CPU.
Using an Intel discrete GPU on a system that lacks ReBAR support results in somewhat lower performance, not a complete inability to use more than 256MB of VRAM.
Source? I don't know what happened for the Linux drivers, but the Windows drivers definitely booted and ran benchmarks without ReBAR when the Arc A380 launched, because there were lots of published comparisons of the performance impact on workloads that definitely didn't fit in 256MB.
Unfortunately I can't edit my earlier post, because I can't find it - might have been distorted rumour that reached me, then :/
Wasn't helped by my experience reacting very strongly to story of early Arc drivers being problematic because the devs were too used to GPU being just a ring-bus away ;-)
It should come with a warning label. If you're in a scenario where VRAM is being maxed out and thrashing main memory then ReBAR will worsen the frequency and magnitude of the stutters. I ran into this with a 3070 (8 GB) recently.
It's entirely possible a game manages paging VRAM badly. But allowing the game more flexibility isn't a problem with a larger BAR, it's that the game is stupid and gets dumber the more VRAM you give it.
If a program runs slower when you give it more RAM, the problem isn't giving more RAM.
I think you’re experiencing paging to main memory rather than anything ReBAR related. ReBAR itself reduces memory copy operations. It’s of course possible you’re experiencing some sort of BIOS bug.
BARs are part of the PCI spec, they're the way for firmware and OS to determine the size of the memory or i/o window the card controls as well as the way for the system firmware (usually) to pick what address range the card should use.
In the old days, they were a fixed size. If your GPU has 16 GB of ram and you want to access all of it via memory addresses, you'd need a 16 GB BAR ... but lots of (older) systems wouldn't be able to map in a large BAR because of a lack of address lines (or wahtever), so GPUs had stayed with 256MB for VRAM access because it was compatible. With a smaller BAR than the VRAM, you have to use some sort of windowing / paging setup. Resizable BAR lets the BAR start small so older systems will work, but grow larger with capable systems.
Much better than having a jumper to set the BAR to big or small, and you can skip VRAM window management.
ReBAR stands for "Resizable BAR (Base Address Register)" - it makes the CPU's access window into GPU memory resizable; instead of accessing VRAM in small chunks, the CPU can access much more at once, which can improve performance on some modern GPUs.
i literally don't understand how people move through life - do you expect every single link on this site (or any site) to be ELI5 for you specifically? do you not understand that some things require effort/homework on your part?
What is the purpose of your comments? You failed to answer his question, then attack him for asking the question. Consider the two possibilities: a) doing that homework is easy, b) it's hard. If it's easy, you could have just posted the relevant link and be done. If it's hard, then his question is warranted. Either way, your comments add nothing. One can only conclude that you don't actually know how to answer the question. The real question, though, is: why even bother replying?
Also, ReBAR can get complicated as you can see from other responses above. The articles on the web are also not really good as they tend to replicate marketing material and make vague statements about performance, but fail to explain how specifically performance is improved.
They wanted a basic explanation in response to that.
Or to put it differently, I would take the sentence "He asked for a purpose and a use-case not an explanation." and replace the word "not" with a comma.
this fixes a real annoyance on older workstation boards where above 4g decoding is a requirement but the option isn't even in the bios menu. saw a build recently on an x79 board where it had to be exposed with a hand patched bios before anything like this or a modern gpu would work at all.
thanks for the throwback, I remember having to hand patch the bios on my x79 system to use a gpu with 12GB vram. Fun times. My motherboard also mysteriously stopped working shortly after…
I patched my old Haswell system to add an NVMe DXE so I could boot from a modern SSD. I currently have a Lets Note laptop that definitely has UEFI, but pretty much pretends to be BIOS - apparently it's possible to modify to expose UEFI features but I have yet to try...
Since I don't want to tinker with my HP Firmware, a brief investigation led me to OpenCore's ability to enable Resizable BAR in the bootloader before Windows boots. Not sure how reliable it is.
https://github.com/xCuri0/ReBarUEFI/discussions/118
"Do I need to flash a new UEFI image on the motherboard, to enable ReBAR on the GPU ? Yes, this is how it works for Turing GPUs (GTX 1600 / RTX 2000)."
Sounds scary. Like there is a good chance to brick the motherboard permanently.
I think i'm going to skip this for my ye olde pc with crappy GTX 1660 Ti card. Can't afford a new system.
just did it, worked perfectly from the guides on my MSI motherboard. Only scary moment was the actual bios flashing, because it took a small while and you shouldn't touch anything around you that risks blowing a fuse.
Can anyone verify if this works on a Lenovo Thinkpad? I checked the List of Working Motherboards [1]. They often have Thunderbolt but no way to enable ReBAR.
Tempted to see if this will work on the work dell laptop, which hardware supports rebar, but Dell just can't be assed to enable it - use an external ARC GPU to properly support 3 screens (again, Dells awful drivers / firmware - the same hardware supports 3 screens just fine).
This made me finally try to migrate an old system of mine. Migrate what you ask? Well, ReBAR driver stuff requires GPT-based disks, and on that machine it's still running on MBR...
Real question, why go through the hassle to install chrome on a linux machine?
I moved away from chrome a while ago, but last time I installed it on linux it was a pain, and firefox and its derivatives are essentially at feature and speed parity or better than chrome across the board.
You most likely have this enabled and unless you run some ancient board.
This UEFI mod is for motherboards from circa 2018.
For workstation machines that would be 1st and 2nd generation Threadripper. So, if your CPU's model name starts with 3 or higher (and you have an AMD workstation), you likely have this already enabled by default.
FYI: Resizable BAR (Base Address Register) is a PCI Express feature that removes the traditional 256MB memory limit, allowing your CPU to access your graphics card's entire VRAM at once for a potential 5% to 15% boost in gaming performance
Thanks for the quick explanation. I'd been reading through the README to figure out what it was for, but wasn't able to.
Thanks, that context was definitely missing. Is that a feature that exists on any GPU brand/model?
Only on specific devices, but generally every GPU released in last few generations, and every dedicated GPU from intel[1], supports "large BAR" this way to map entire GPU memory into MMIO space. This has to be supported on both target device and AFAIK the PCI Root Port[2], and of course needs to be properly handled in firmware - which this tool fixes.
[1] Intel GPUs actually can't work without it, because they do not have legacy "windowing" mode where portions of 256MB BAR window are mapped to different areas in GPU memory
[2] This tool implies that it's possible to handle root port side entirely in firmware, but it's new to me at least, so I can't vouch for that :)
These kind of BIOS modding shenannigans is why I got interesed in Coreboot, so that enhancing a Firmware capabilities becomes possible without touching propietary tools/code.
Other similar mods included adding NVMe Boot capabilities to older Firmwares that didn't supported them so that you could boot from modern NVMe drives if you used some kind of adapter (As none of those Motherboards have M.2 Slots, but you can use them with a PCIe-to-M.2 adapter card). Also adding required stuff to get certain CPUs booting in Motherboard that were never officially supported (Depending on generation, could be as simple as adding new CPU Microcode).
These days you might be able to just point an AI at the original BIOS and tell it what you need done, instead of attempting to install a complete rewrite like Coreboot (which comes with its own limitations, the biggest one being that only a rather small subset of mobos are actually supported.)
Yes, I know that there are examples in the wild of that kind of AI modding:
https://www.downtowndougbrown.com/2026/08/fixing-an-emachine...
https://www.tomshardware.com/pc-components/cpus/modder-uses-...
Coreboot in desktops is perhaps in the best shape it has been in more than a decade because you have the MSI Z690-A/Z790-P series for LGA 1700 (Which were current gen when released, and still viable if you want Intel with DDR4, albeit the boards aren't available in retail anymore) and the recently released port for the MSI B850-P for AM5 Phoenix. It was a major leap from previously, where it only supported ancient platforms or unobtanium reference boards. I bashed Coreboot a lot for that, but now you actually have usable options. Same with Servers, with the Gigabyte MZ33-AR1 for AMD Turin. When was the last time you saw Coreboot running on current gen boards available in retail, if ever?
The last time I looked at Coreboot it didn't have any configuration UI at all ("you have to set the options and recompile/reflash" was the answer I got), which basically makes it next to useless for a regular PC... and apparently it still doesn't. They've already done the hard work of RE'ing and all the hardware-specific stuff; how hard can it be to at least have a simple setup menu? Hopefully someone vibecodes (vibeports?) one in the next little while.
(Incidentally this is also one of my gripes with the Intel Macs, wanting to put Linux on one but having zero configurability pre-OS is a huge pain. Meanwhile a standard PC BIOS with the equivalent hardware has plenty of options. Maybe I'll get around to (vibe)porting a regular BIOS to mine...)
The menu part is the task of the payload. If you compile Coreboot with a payload you can get a menu no problem. EDK II, Seabios or grub payloads all support this.
If you want to you may want to point your vibes for vibecoding towards making a libpayload universal menu for Coreboot , that way you at least conform with the architecture.
I was an early user of the Z690-A from 2022 and while it was dull in features, the basics (Boot Menu) were already present. The configuration UI was TianoCore/EDK2:
https://youtu.be/eSz2VPH64SE?t=343
https://docs.dasharo.com/dasharo-menu-docs/dasharo-system-fe...
I do recall hearing the same thing than you but for those that used SeaBIOS and needed to recompile and reflash just to change boot order...
Regarding Coreboot, it is far worse than that. They literally do not care anymore; focus is not on corebooting all the boards, but on serving some sort of commercial "customer".
e.g. they do seldom if ever take patches for adding support or fixing issues in older boards. They just sit there ignored.
A fork or entire new project with different focus is sorely needed.
There is also the Libreboot which may be of your interest. What I see is that there is a lot of development on Coreboot but it is indeed mostly driven by companies working on their latest platforms. There are not many contributors working on older platforms. Though in my experience the response to patches is relatively quick.
I had the exact opposite impression, there are a lot of user contributions working on old platforms, and even specific platform communities. For example, you have 15h.org which is a community dedicated to AMD Bulldozer / Piledriver generation, and they managed to fix a few things that were left broken in Coreboot. I also saw an influx of users working in Sandy Bridge / Ivy Bridge / Haswell era Intel platforms with an occasional Skylake. The problem is, precisely, that most of the user contributions are for old platforms that nobody that does NOT currently owns those will want to touch if you're purchasing new. I would have been very enthusiastic about that work if it was done about 10 years ago or so, nowadays they seem to be hobby projects that don't advance anything forward.
There do exists individual users that work on newer stuff, after the MSI Z690-A / Z790-P there were 2 people that did ports for H610 since at that point the platform was supported and they only had to port the Motherboard. However, they are rather rare, since the kind of user that Coreboot seems to attract are from the privacy and security anti-ME/PSP crowd that don't even want to use newer platforms, which is the exact opposite of my view.
A lot of the work companies does may be just for their internal systems or reference platform and these may not get to the hands of end users, ever. I still recall Phoronix articles covering Coreboot releases always taking a jab at the new supported Motherboards until the MSI Z690-A DDR4 port from 2022.
[dead]
What does this do? I perused the README but I'm still clueless on what the purpose and use-case is for ReBAR.
(Posting this under the assumption others will also appreciate a bit of quick context.)
Since about 5 or 6 years ago, PCIe Video Cards advertise a capability known as Resizeable BAR. PCI Devices like GPUs require some memory to use for PCI MMIO, which is directly visible on the CPU Address Space. As a side note, this Address Space is shared with RAM, and anyone that was around when having 4 GiB RAM with a 32 Bit OS was common (Earlier than 2010 or so) knows that you only saw about 3.25 GiB RAM or so because of sharing the Address Space with PCI MMIO, and those fortunate enough to have used SLI usually saw even less than that, like 2.87 GiB RAM.
GPUs has been using a 256 MiB PCI MMIO window regardless of how much VRAM they actually have since... nearly forever? At least since PCIe is a thing, since I recall than AGP Aperture Size was seteable in era accurate BIOSes. PCIe 3.0 specification introduced a feature known as Resizeable BAR, where the PCI Device can tell a compatible Firmware how much MMIO it actually wants. GPUs uses that to tell a ReBAR capable UEFI Firmware that it wants more MMIO (Usually as big as the GPU VRAM), or uses legacy 256 MiB otherwise.
GPUs have been using a 256MB window since they started coming with 256MB of VRAM, as it would otherwise be a pointless waste of address space to have a window larger than the actual amount of VRAM. Previously, they would have a window exactly equal to the size of VRAM (likely rounded up to the next power of 2.)
There would also be a block of memory mapped hardware registers also needing address space that you have to poke to make the GPU actually do GPU things, instead of just being an expensive way to add extra memory to a system, no?
And if my experience from embedded development is in any way transferable, they're probably fairly spread out and probably takes a fairly big chunk of address space too.
Yes, typically the memory mapped memory is in one (64-bit) bar, memory mapped registers in another (64-bit) bar, plus for compatability with vga, probably a 32-bit memory bar and an i/o bar. 64-bit bars take up two bar slots, so that fills all six slots in the PCI config.
Just to make sure I understand: Is the resizable BAR/MMIO a RAM buffer for PCie packets? Or does it have some deeper integration with DMA or something? I assume it's not like memory mapped peripherals on an AXI bus which is why you need the buffer?
No idea on the AXI Bus you're talking about, so can't make comparisons.
MMIO (Memory Mapped I/O) is essentially memory (Whenever RAM or ROM) from OTHER devices that is directly visible on the CPU Address Space. My understanding is that from the CPU side, MMIO is mostly transparent (Except for the massive increase in latency) because it gets used like if it was interacting with its own workspace with regular instructions like MOV.
What PCIe ReBAR changes is that before, you could only see a 256 MiB window onto the GPU VRAM, so there was an added overhead since the GPU may need to relocate things from inside that window somewhere else on its total VRAM (So yes, it may be interpreted as if what you see from the CPU side is just some kind of exchange buffer). I believe the best way to describe how it operates is comparing it to EMS (Expanded Memory) from the DOS days since it also worked with a similar, if not the same idea. You could only see a portion of the total memory from what was installed on the EMS card (A 128 KiB window located on the upper part of the 1 MiB address space from the 8086 CPU), so you had to switch which Page (Region) of the memory was visible, adding a lot of overhead and most likely requiring an additional buffer in main RAM to move data from one Page to another. However, since I have no knowledge if the GPUs really work like that I can't confirm. I never knew whenever the 256 MiB is "fixed" (You always see the same Region) or if you can decide which section of the VRAM to make visible.
Just to close the loop, AXI bus[1] refers to a very common implementation, in the gateware/verilog, of a MIMO.
[1] https://en.wikipedia.org/wiki/Advanced_eXtensible_Interface
It’s also similar to how mmap() on 32-bit systems maps files larger than 2GB.
It's just memory mapping the peripheral.
For historic reasons (e.g. 32 bit address spaces, plus the need to reserve the space for multiple pci peripherals) it has been a narrow, movable aperture.
Resizable BAR lets the size of the aperture be chosen (which is usually chosen to allow all of VRAM to fit in and be directly accessible).
There is no buffer at all. BAR is "base address register" (hint, there is no "the BAR"). Each PCIe device / function can have up to 6 32-bit BARs or 3 64 bit BARs.
When you insert a GPU into a PCIe slot, the memory mapped regions in memory can't be put in a hard wired location because any arbitrary device can be inserted and it can provide an arbitrary amount of memory (yes the GPU provides its own memory to the CPU). A BAR reserves a memory mapped region in the CPU space that is backed by the PCIe device.
When the BAR is smaller than the memory of the inserted device, the CPU cannot communicate with all of the memory on the inserted device directly anmore. This means if you want to perform a write to a region in the GPU outside a BAR region you have to go through the BAR region anyway. It's not a RAM buffer for PCIe packets.
>I assume it's not like memory mapped peripherals on an AXI bus which is why you need the buffer?
The PCIe controller is an AXI peripheral...
It's hard to explain without going into a few low-level details of PCI Express, but let me try.
Most PCI devices expose some memory and/or I/O ports to the CPU. That memory (or I/O ports) is mapped to somewhere in the address space visible to the CPU. Besides the memory and I/O ports, all PCI devices also expose a separate set of configuration registers; among these registers, there are the Base Address Registers (BARs), which configure where the memory or I/O ports is mapped.
Here's an example output from "lspci -vv" for a GPU:
Note that regions 0 and 2 are above the 4GB addressable by old 32-bit CPUs. To be compatible with these old CPUs, this card and many others like it allow the firmware (and/or the operating system) to choose not only where the memory is mapped, but also its size. We can see this in the same "lspci -vv" output for this GPU: Older systems which do not understand this extended capability will still treat these regions as fixed size, probably with the first size in this list (256MB for region 0, 2MB for region 2). Newer systems can tell the device to "resize" the BAR to a bigger size, which obviously needs the first region to be placed above the 4GB barrier since it's too big.Why is this useful? This particular GPU has 8GB of VRAM; it's quite obvious that region 0 is a direct view into that VRAM. When using the maximum BAR size, the CPU can directly read and write anywhere into the VRAM; when using a smaller BAR, the CPU can only see a small window into the VRAM, and has to use less direct methods to access it.
(As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)
> (As an aside: go right now and do a "sudo lspci -vv" on your computer, if you see a Resizable BAR capability which isn't using the maximum size, you can probably gain a bit more speed for free by going into the BIOS and enabling "Resizable BAR" and/or "Above 4G decoding". If you can't find these options, well, AFAIU that's what this project is all about..)
IIRC Linux doesn't need resizable bar enabled in the BIOS since the kernel will resize the bar if supported by the GPU, Windows however relies on the UEFI doing it which is where it being enabled in the BIOS is needed.
ReBar's commercial name is AMD Smart Access Memory, it allows a PCIe device such as a GPU to map more VRAM to the system at once, which improves performance by reducing access overhead. This generated much fanfare in the early 2020s after AMD officially supported it in the newly released AMD Zen 3 CPUs with RX6800 series GPUs. It was marketed as a new technology to boost GPU/gaming performance. What AMD did was just rebranding an obscure feature in the PCIe specification [1]. As shown by this project, it was actually supported by the PCIe controller since Sandy Bridge, just disabled in the firmware. For a decade nobody bothered to use it. Presumably, AMD saw an opportunity and enabled it, presumably after validating the hardware and fixing any driver compatibility problems.
[1] This is nothing new in the tech industry. Intel rebrands DVFS as SpeedStep, IOMMU as VT-d, AMD rebrands the NX bit as Enhanced Virus Protection, etc.
I feel part of the "Smart Access Memory" name is actually the driver features and paths to actually make use of it.
It's not like ReBar is a single toggle "Make Things Faster", but a different option in how it can map gpu memory to the cpu. The driver still needs to use it - and decide where it's use vs the "staging buffer" approach would actually be be benefitial.
IOMMU branding at least makes a little bit of sense. AMDs and Intels virtualisation ISA are completely different, including their IOMMU.
SAM wasn't just about the resizable BAR capability but also about driver adjustments to make efficient use of the larger BAR.
Resizable Base Address Register. In short, rather than the 256MB of mapped memory available to the CPU for any PCI device, ReBAR capable devices can map larger memory to the CPU’s addressable memory space. Without ReBAR or sufficient tricks (that Intel GPUs lack) you have to shuffle 256MB chunks around between GPU and CPU.
> sufficient tricks (that Intel GPUs lack)
Isn't it just being able to shift the window of GPU memory visible to the CPU?
Yes, intel lacks a remapping mechanism to support BAR smaller than the total VRAM size.
Using an Intel discrete GPU on a system that lacks ReBAR support results in somewhat lower performance, not a complete inability to use more than 256MB of VRAM.
Then it changed, because early on I recall people reporting the drivers wouldn't work at all
Source? I don't know what happened for the Linux drivers, but the Windows drivers definitely booted and ran benchmarks without ReBAR when the Arc A380 launched, because there were lots of published comparisons of the performance impact on workloads that definitely didn't fit in 256MB.
Unfortunately I can't edit my earlier post, because I can't find it - might have been distorted rumour that reached me, then :/
Wasn't helped by my experience reacting very strongly to story of early Arc drivers being problematic because the devs were too used to GPU being just a ring-bus away ;-)
It allows the CPU to access all GPU VRAM. This improves gaming performance, especially for AMD GPUs.
It should come with a warning label. If you're in a scenario where VRAM is being maxed out and thrashing main memory then ReBAR will worsen the frequency and magnitude of the stutters. I ran into this with a 3070 (8 GB) recently.
It's entirely possible a game manages paging VRAM badly. But allowing the game more flexibility isn't a problem with a larger BAR, it's that the game is stupid and gets dumber the more VRAM you give it.
If a program runs slower when you give it more RAM, the problem isn't giving more RAM.
I think you’re experiencing paging to main memory rather than anything ReBAR related. ReBAR itself reduces memory copy operations. It’s of course possible you’re experiencing some sort of BIOS bug.
> what the purpose and use-case is for ReBAR
Literally the second sentence in the repo:
> This provides performance benefits and is even required for Intel Arc GPUs to function optimally.
Your quote doesn’t help me understand this project at all.
What is a BAR let alone a resizable one? Readme just jumps in, which is fine, but I’m not sure why this is on HN or why I should care.
BARs are part of the PCI spec, they're the way for firmware and OS to determine the size of the memory or i/o window the card controls as well as the way for the system firmware (usually) to pick what address range the card should use.
In the old days, they were a fixed size. If your GPU has 16 GB of ram and you want to access all of it via memory addresses, you'd need a 16 GB BAR ... but lots of (older) systems wouldn't be able to map in a large BAR because of a lack of address lines (or wahtever), so GPUs had stayed with 256MB for VRAM access because it was compatible. With a smaller BAR than the VRAM, you have to use some sort of windowing / paging setup. Resizable BAR lets the BAR start small so older systems will work, but grow larger with capable systems.
Much better than having a jumper to set the BAR to big or small, and you can skip VRAM window management.
ReBAR stands for "Resizable BAR (Base Address Register)" - it makes the CPU's access window into GPU memory resizable; instead of accessing VRAM in small chunks, the CPU can access much more at once, which can improve performance on some modern GPUs.
Its also neccessary in many cases using any PCIe device with BAR-addressable memory on-board above the threshold, not just vram.
i literally don't understand how people move through life - do you expect every single link on this site (or any site) to be ELI5 for you specifically? do you not understand that some things require effort/homework on your part?
What is the purpose of your comments? You failed to answer his question, then attack him for asking the question. Consider the two possibilities: a) doing that homework is easy, b) it's hard. If it's easy, you could have just posted the relevant link and be done. If it's hard, then his question is warranted. Either way, your comments add nothing. One can only conclude that you don't actually know how to answer the question. The real question, though, is: why even bother replying?
Also, ReBAR can get complicated as you can see from other responses above. The articles on the web are also not really good as they tend to replicate marketing material and make vague statements about performance, but fail to explain how specifically performance is improved.
> What is the purpose of your comments?
Same as his: to voice my frustration about something on the internet.
Your original comment sure didn't do that. It just gave an incorrect answer.
What exactly was "incorrect" about it? He asked for a purpose and a use-case not an explanation.
> not an explanation
Wrong.
.... It's literally right there:
> I'm still clueless on what the purpose and use-case is for ReBAR
Are you seriously debating this?
They wanted a basic explanation in response to that.
Or to put it differently, I would take the sentence "He asked for a purpose and a use-case not an explanation." and replace the word "not" with a comma.
I quoted the direct response. I have no idea what you're debating.
I'm not sure how to put it simpler.
I agree with "He asked for a purpose and a use-case". (This is what your quote supports.)
I disagree with "not an explanation".
When asking for a purpose and use case, they were asking for an explanation.
this fixes a real annoyance on older workstation boards where above 4g decoding is a requirement but the option isn't even in the bios menu. saw a build recently on an x79 board where it had to be exposed with a hand patched bios before anything like this or a modern gpu would work at all.
Does this help for more than just gaming ?
thanks for the throwback, I remember having to hand patch the bios on my x79 system to use a gpu with 12GB vram. Fun times. My motherboard also mysteriously stopped working shortly after…
I patched my old Haswell system to add an NVMe DXE so I could boot from a modern SSD. I currently have a Lets Note laptop that definitely has UEFI, but pretty much pretends to be BIOS - apparently it's possible to modify to expose UEFI features but I have yet to try...
Since I don't want to tinker with my HP Firmware, a brief investigation led me to OpenCore's ability to enable Resizable BAR in the bootloader before Windows boots. Not sure how reliable it is. https://github.com/xCuri0/ReBarUEFI/discussions/118
"Do I need to flash a new UEFI image on the motherboard, to enable ReBAR on the GPU ? Yes, this is how it works for Turing GPUs (GTX 1600 / RTX 2000)."
Sounds scary. Like there is a good chance to brick the motherboard permanently. I think i'm going to skip this for my ye olde pc with crappy GTX 1660 Ti card. Can't afford a new system.
just did it, worked perfectly from the guides on my MSI motherboard. Only scary moment was the actual bios flashing, because it took a small while and you shouldn't touch anything around you that risks blowing a fuse.
>GTX 1660 Ti
reports on github show 0% speedup for 1xxx series
Can anyone verify if this works on a Lenovo Thinkpad? I checked the List of Working Motherboards [1]. They often have Thunderbolt but no way to enable ReBAR.
[1] https://github.com/xCuri0/ReBarUEFI/issues/11
Finally, a way to squeeze more life out of my old gaming rig. My 3070 will appreciate this.
Tempted to see if this will work on the work dell laptop, which hardware supports rebar, but Dell just can't be assed to enable it - use an external ARC GPU to properly support 3 screens (again, Dells awful drivers / firmware - the same hardware supports 3 screens just fine).
This made me finally try to migrate an old system of mine. Migrate what you ask? Well, ReBAR driver stuff requires GPT-based disks, and on that machine it's still running on MBR...
Do you inject a DXE into OEM firmware for that map, or wait for the board vendor and live with the small window?
On my linux machine chrome was crashing almost daily before activating rebar by patching bios.
Real question, why go through the hassle to install chrome on a linux machine?
I moved away from chrome a while ago, but last time I installed it on linux it was a pain, and firefox and its derivatives are essentially at feature and speed parity or better than chrome across the board.
Sweet! Been wishing for ReBAR on my X99 rig forever. This could finally make that old setup sing with a modern GPU.
Should I be enabling this on a workstation setup doing ML ? Mostly GPU protein folding
You most likely have this enabled and unless you run some ancient board. This UEFI mod is for motherboards from circa 2018. For workstation machines that would be 1st and 2nd generation Threadripper. So, if your CPU's model name starts with 3 or higher (and you have an AMD workstation), you likely have this already enabled by default.
I am in need of this from the opposite direction - an RTX 3080 20G BIOS which adds ReBAR to the card.
Claude tells me the BAR is outside the encrypted part of the card firmware so theoretically should be possible?
They link https://github.com/terminatorul/NvStrapsReBar in the repo which does what you want for 2000 series cards. Start there, burn some credits.