Molly Rocket

[sylvan]

It would be cool if the GPU could basically do this whole thing for us. I.e. it just spits out a list of page faults each frame (and automatically tags that page as "invalid" in the TLB to avoid too many duplicates and then moves down the mipchain), then the CPU reads this and periodically uploads one page at a time to the GPU. These pages wouldn't have any notion of texels or anything, it would just be a chunk of memory (4K, 8K?)

This would be very nice because we wouldn't have to worry about filtering and all that noise, the GPU would just say "oi, I need 4K of data starting at virtual adress A", and the CPU would look this adress up (in some range set structure) to see which texture is responsible for that virtual address, then go off and load that bit of data from disk (or a compressed intermediate cache), grab a page from a pool somewhere to stick it in (using "second chance" or something similar to find an available page) and then post it to the GPU which would then enter it into its page table.

Even more flexible would be to obviously just get an interrupt on the CPU and handle it however you want to, but that may be slow...

Obviously if we get more flexible gpus (larrabee?) we could basically write this ourselves.

[Won]

My guess is that TomF and company have already implemented much of this on Larrabee.

How does this system chose which pages to replace? Does the GPU or CPU decide? Aside from that, it sounds much like the asynchronous query system. I'm not sure why you would rather traffic in bytes than pixels.

It would be nice you didn't have to do anything special to deal with filtering, which means the hardware would have to do some funky stuff at tile boundaries, where some of the filter samples could come from tiles that are from radically different mip levels. I imagine this would be similar to what you would need to support texture borders or filtering across cubemap faces.

Speaking of which, borders and cubemaps might be an interesting thing to generalize; like allow users to stitch 2-D textures borders in various patterns, although I'm not sure how you would do a page map for this.

It would also be nice if the indirection provided by the page map could be used to control rendering to physical textures as well (including masking chunks), to make it easier/faster to do something like rendering to adaptive shadow maps. You'd have to have different edge increments or whatever for rendering at different resolutions simultaneously.

[sylvan]

I'd like the CPU to decide which pages to replace. Basically you'd just keep a pool of physical pages, and then send up information about what page goes where to the GPU which will enter it into its page table. The various bits for each page has to be accessible by the CPU for this to work though (e.g. the "referenced" bit would be needed for the page replacement strategy). So it may be better to actually keep the page table on the CPU and do all of this logic yourself. A TLB miss would then trigger an interrupt on the CPU, and your job then is to quickly return the physical address of the virtual address in question (if it's there). If it's not there we still need a way for the page fault to "fail" which would cause the GPU to try the next MIP level instead. And I guess the GPU can keep an "invalid" bit in the TLB for this address so it won't trigger an interrupt for that same page again (unless that TLB entry happens to be overwritten). Then when the CPU has loaded a page it would just flush the TLB entry for the virtual address which caused the page fault, which will cause that address to trigger another interrupt, which means you can just return the newly allocated physical address for that page.

More flexible, but if we have to go to the CPU every time the TLB misses that could be costly (the goal would be that the vast majority of TLB misses would be actual page faults, rather than just running out of TLB entries). Though with lots of threads to hide latency (and assuming this only happens a few times per frame) it may be totally acceptable. It may be that a large TLB is cheaper than implementing a bunch of page logic on the GPU, though... Any hardware people care to chime in?

I'd prefer to deal directly with bytes rather than texels because it simplifies things greatly. You don't have to worry about different formats, different compression schemes etc., just chunks of bytes.

[Won]

Well, things like texture format/layout is exactly what you want the GPU to worry about, not the application. At least this is the OpenGL style.

I don't think CPU interrupts for GPU virtual texture faults would be good for interactive performance, since those things have enough trouble masking the latency from GPU memory, let alone some arbitrary CPU callback. If the interrupt doesn't actually block the GPU, then why make it an interrupt rather than an asynchronous query? Something like a GPU callback might make sense here; conditional rendering can/should be triggered on faults as well as occlusion queries. This could be how you implement lazy, sparse rendering of things like reflection maps without much CPU intervention. But you'd still likely have a frame of delay.

Here's an idea: allow conditional renders to be specified to be delayed a frame, so that in an AFR system a separate GPU performs the conditional rendering overlapped with the current frame.

Also, I think discussions about TLBs confuse the issue somewhat, since the application developer should really only have to worry about page maps and not TLBs, and the API should transparently invalidate the TLB when necessary. Not to say that a TLB isn't important (could double fillrate or better), but it is an implementation detail. But if you insist, you might as well start the debate about whether texture caches are virtually or physically addressed, and how this might interact with render-to-sparse-texture or explicit page map control.

[sylvan]

Yes, precisely, the GPU should be the one that worries about texture format and layout (not the CPU), which is why the pages should be bytes, not texels. Not sure if you were agreeing with me there or if I'm just misunderstanding your argument. It seems reasonably simple to stick a mostly standard virtual memory system (with the notable addition of "failure") on the GPU, that just requests regular "chunks of bytes" in the standard way, and leave everything else the same (except, as you point out, extra logic to do filtering properly when the initial adressing of a sample returns "failure" and you have to go down the mip chain).

The GPU does need to block on the page fault if the page table is on the CPU side of things (because it needs to know the physical address before it can continue, if the page is indeed available, else it needs to be told that the page is not available), which is why it needs to be an interrupt. If the page fault is handled on the GPU entirely then it could (should) be asynchronous (like I said in my original post - you just read it at the end of the frame). You'd really just need a fixed buffer of page fault entries, as it's actually completely okay to ignore a page fault when the buffer is full (they'll just get handled next frame).

The problem is that if we want the CPU to be able to decide which pages to replace (and I guess we do? But maybe not, it could just implement a reasonable strategy on the GPU and I'd be happy with that) then the CPU needs to gather some form of usage statistics (typically a "referenced" bit), so if the GPU needs to pass data over to the CPU about page usage anyway, it may make sense to just keep the page table on the CPU for extra flexibility... But yeah, latency would be an issue. Though I'm not convinced it would be prohibitive, if you have enough threads to hide the latency on the GPU and a TLB large enough so that it doesn't hit the CPU interrupt needlessly (i.e. when the page is available, and was used fairly recently)... I'm just brainstorming though, I may be completely off...

[TomF]

[sylvan]