[FrontPage] [TitleIndex] [WordIndex] 

NOTE: This page is outdated and is only retained for historical purposes. Suspend with KMS has been implemented and working for all cards exept for TNT2.

Latest news:
10.10.2009 posted by benkai
Ben Skeggs has given a great update on the status of Nouveau including an update on Suspend/Resume:

Suspend/Resume

Yes, that's right, Nouveau finally supports suspend/resume. In theory the support is there for all chipsets. In the very least, every chipset I own (at least one from each family, and 5 G80-class cards) can suspend/resume reliably except for a TNT2 that has other issues.. This support is only available when kernel modesetting is switched on, so that's even more reason to use it!

The following status only applies to TNT2:

1. Intro / FAQ

Q: Is suspend working with nouveau?

2. Outlook

Ben Skeggs has been hired by RedHat and so he can work on the features which are important for RedHat and distributions in general, like KMS and suspend/resume which seems to be fully done now. In any case, you can come to the nouveau IRC channel, report any issueas and ask if you could help.

3. Possibilities for suspending and resuming the card

3.1. s2ram

Working suspend2ram requires ability to cold-boot (also called POSTing - "Power-On Self-Test") the card. This is very card-specifc, and may work on some hardware with some method, or not. Currently, there are the following ways how the card can be posted. This does much more than just turning on the backlight (eg. like xset dpms). Fore example it needs to init the nvidia ASIC, upload the firmware into it and boot it.

3.1.1. Cold-booting the card using x86 BIOS routines

The VESA BIOS Extensions (VBE) provide a way to POST the card even after boot with cards whose VESA BIOS is working after boot. For many (nmost?) nvidia cards, it's not, but the BIOS of G8x cards (nv50) cards seems to work.

3.1.2. Cold-booting the card by interpreting the init scripts in the Video BIOS

This depends on nouveau's BIOS parser to correctly interpret the reverse-engineered init script of your Nvidia BIOS.

These BIOS parsers do not set or restore a text mode after cold-booting the card, and they only work for nv4x cards and older. (For nv5x cards the VBE POST can be used.)

3.2. s2disk

For s2disk, the machine is ultimatively turned off completely and all power can be removed. The resume process is preceeded by a normal system boot thru the BIOS, so the card is initialized by the VGA BIOS in this case, so we do not need to have cold-booting the card working.

4. Support for suspend in other drivers

5. About Option "ShadowFB" "on"

"ShadowFB" disables Xv, EXA and hardware-accelerated RENDER as well as hardware 3D. If you only need 2D graphics and your system is fast, ShadowFB is likely also fast, but you have to switch to the text console before suspending.

6. Possible tools for reverse-engineering

Hopefully, MmioTrace works during suspending and resume. mmiotrace does not provide much symboic information on nv5x cards yet.

7. Low-level intro

Note: Below, you may Nvidia-specific terms which you may not know yet. Read HonzaHavlicek to understand what they mean.

Nvidia cards are composed of several separate units: PMC, PTIMER, PFB, PGRAPH and PFIFO. Nouveaus's GPU initialisation code in nouveau.ko is called during X server startup thru an ioctl which is issued during DRI initialisation. nouveau.ko skips this step if nouveau.ko's internal status indicates that the GPU is already initialized.

Even the oldest cards supported by nouveau (NV04 = TNT2) have a GPU unit which has support for multi-tasking like a full-blown CPU and they even appear to have other CPU-like features like memory protection so that one "GPU process" cannot mess up the state of other "GPU processes", and Nouveau does a context switch between two such GPU processes even as part of the X startup. For 2D and 3D acceleration in parallel, these context switches have to be working.

All the GPU processes which were set up before suspending need to be saved and have to be restored on resume - after the various parts of the GPU are initialized. Contexts of these "GPU processes" live in differnt parts of the GPU, one part of these GPU contexts lives in the actual drawing engine (PGRAPH) while one part lives in the part which feeds the drawing engine with the commands for drawing (the PFIFO).

7.1. Low-Level Card initialisation in nouveau.ko

The initialisation routines for the nvidia chip components differ from chip generation to chip generation, so the 1st step which nouveau.ko does is checking which chip generation the card uses. Depending on the generation, the pointers to the card-specific functions are set to the card-specific functions for the detected chip generation. When the GPU initialisation is finally called, the card-specific initialisation functions for the various initialsation stages are executed. First, the initialisaiton functions for rather low-level units are called. The first one is the PMC. It seems to be someting like a power switch for the other units, at least it seems to be used by the initialisation functions of the other units to power-cycle the respective unit before it is initialized. At least on NV04, init of PMC, PTIMER, PFB and PGRAPH are trivial. PFIFO init is where things get more involved. PFIFO needs to know the locations of several parts of the cards "instance memory"(a special memory region in VRAM, reserved for the internal GPU status and commands, size is 512 KiB or 1MiB, depending on the card): RAMHT (hash tables), RAMFC (FIFO context) and RAMRO (FIFO runout table). This information is configured and written to the FIFO in fifo_instance_mem_init() and that part is still trivial.

All drawing commands are sent to the GPU inside a "channel" (someting like a process for a CPU). A channel is composed of it's FIFO context, it's PGRAPH context, as well as allocated GPU objects and DMA objects.

DMA and Interrupt setup are the final stages of the GPU initialisation routine in nouveau.ko. Current status is that I can run the init routines of PMC, PTIMER and PFB during EnterVT when switching from console text mode back to graphics mode on NV04 and have EXA still working, to add make GPRAPH init I have to call the graph_save_context and graph_load_context routines around graph_init(), but for PFIFO init, it gets more tricky, see below, and for that, we may need some mmiotraces to know what to do to get the combination of PGRAPH and PFIFO init work.

Off-Topic: Regarding saving power): Depending on wether the PMC really can really power-down all the other units of the GPU, the PMC and the suspend code in general could used to save power (= gain of battery time) using nouveau (in addition to reducing the clock speed with nvclock) when no drawing is done thru the GPU or when all drawing is done directly by the CPU like when ShadowFB is enabled. When nothing is drawn for some time, we could already (with current code) test wether we can turn off PTIMER, PFB and PGRAPH and turn them on again quickly triggered by trapping writes the the user-space command submission buffer. When suspend is working, we could turn off most chip compenents to save power and restore their status when drawing starts again.

7.1.1. Examples of what we need to do on Nvidia chip generations

malc0's suspend patch does this by calling mc.init(), timer.init(), fb.init(), graph.init() and fifo.init(). Afterwards, he restores the RAMIN area.

8. Channel data objects

What should we do with all the gpu data objects in the channels?

When a channel is allocated and used, lots of gpu data objects are allocated in VRAM. Data structures are complex: Each channel has a fifo context which has a pushbuf attached which has a number of gpu objechts (look at nouveau_fifo_alloc). FIFO alloc is started from user space, goes into the kernel and back to do remaining things in user space again.

malc0's patch saves and restore the whole RAMIN area instead. The space currently allocated to RAMIN by nouveau is 1MiB for >=NV40 and 512kib for all older cards. It would take less memory to save and restore RAMFC, RAMRO and then not the complete RAMIN heap but only the allocated GPU objects in it. For that, we'd have to traverse thru the list of allocated gpu_objects within the memory management of nouveau/drm and back up those.

In theory, it would be nice to allocate the RAMIN backup space in swappable memory (may have to be allocated by the user process?) and use copy_to/from_user for the transfer in order to not require a large continous chunk of memory (which may not be available if memory gets fragmented), but swapping would need to be running during suspend/resume for this to work.

10. Obsolete progress reports from Spring 2008

2013-03-24 13:16