The irregular Nouveau-Development companion
Issue for June, 28th
1. Intro
This is number 22 of the carefully crafted irregular Nouveau Development Companion. It contains all the accumulated wisdom of the Nouveau project (including mishaps and progresses) and - probably as always - I will get correction requests from Darktama even before I officially announce its release (come on admit it, you are running a script to check for new releases 
).
It is good to be cautious. There is a reason why we call this The irregular Nouveau Development Companion. As these weeks didn't see much activity, this issue was delayed.
Marcheu was MIA (claimed something about "renovating home") and Darktama was caught beta testing Wine with steam and Tomb Raider Anniversary.
2. Current status
But still, we managed to get some work done.
Well, rules-ng and the MMioTrace tool chain seems to be a good tool for reverse engineering. Shortly after our last issue went online, Philip Ezolt from the radeon driver project dropped into our channel and offered additions for the rules-ng sub project. He asked whether we would oppose to add radeon specific data to the rules-ng xml files. Goal is to make MMio-parse give out useful text for radeon dumps too. Of course we accepted and he is busy working on his additions (http://nouveau.cvs.sourceforge.net/nouveau/rules-ng/databases/).
By the way this seems a good moment to finally plug the TiRDC blog of the radeon community (http://tirdc.livejournal.com/). After 3 issues like ours they switched over to a blog kind of publishing. They claim that they got their inspiration from our companion. We are honered but still, this is no real innovation from us either. We in turn were inspired by the Wine Weekly News and the LWN.net kernel summaries.
It seems our idea of the irregular newsletter is catching on (besides the radeon one we have seen at least one for the Dtrace tool and for XML issues).
When pq started comparing the real life names of registers and constant values in the driver sources to those which would be generated through rules-ng and its tools, he came up with some problems. After discussing the issues with Marcheu it became clear that we want to change names only if no other options exists. That is to avoid reference problems when some sources call a register FOO while other sources call it FOOBAR. This problem is apparent especially when browsing the Haiku NVidia driver sources. It is simply confusing.
pq changed some restrictions on his naming scheme accordingly and thus life was fine again.
Jwstolk got somewhat sidetracked regarding his macro finder project. As he noticed that we have different kind of MMio dumps (32 and 64 bit, old version without timestamps / markers and a new version with plus tracing may be either raw or text (when run through mmio-parse), he started some work to convert between the different versions. You can find his current effort at this address (http://jwstolk.xs4all.nl/nouveau/code/mmio-convert.c). In his own words:
- Mmio-convert does the opposite of mmio-parse. as well as convert between 32-bit and 64-bit binary traces, and the (pending) new binary format. (and a simple text format)
Pmdata is doing a face lifting on renouveau. He wants to get rid of the huge internal data structure nv_object[] and load it from a XML-database. Furthermore, the tool will be probably be split into a dumper and a parser part. This will allow us to send smaller dumps and reparse it once we found out newer functionality. Today we would need to ask the users to dump one more time.
Darktama came back into action by adding fixes lingering on the mailing list for init of NV49 and NV4b cards. Patches came from jb17some and stephan_2303 and were fixed up by Darktama. Of course he added some fixes into the mix as well.
Furthermore he added all changes from his tree to the main tree. As the next changes would break the API, Darktama asks for tests. Please do test on your hardware and report back to him whether it works or not (please name hardware and add some details). Currently NV4x is working, Pmdata got his hard lockups back (but only when trying to run glxgears a second time). Still he confirmed breakage for NV1x.
As already mentioned, the next step includes breaking the API between DRM and DDX once again. So it would be wise to make sure that the latest updates didn't introduce new bugs.
No progress on the G8x side of the project as Darktama was cleaning up his pile of patches of generic parts of the driver. KoalaBR contacted Matthew Garrett (who wrote libx86 / emux86) for the vbetool and was asked to build a new XOrg server with this lib enabled (Xorg has its own version included), even if a version of Xorg on x86 normally wouldn't need it. That is to compare behaviour of Xorg with the behaviour we see. Btw, the "display is going into suspend mode" problem is the same on Darktama's G84 card too, whatever tool is used. KoalaBR hasn't mustered enough courage yet to tackle that task.
Well ahuillet is currently setting up his Xv project. First thing he did, was writing an executive summary of things he plans to do within the scope of his project. See http://nouveau.freedesktop.org/wiki/ArthurHuillet. He added it at the bottom of the page beneath his already existing docs.
After getting an okay from Darktama he setup his git repository on fd.o but got some problems understanding the concepts of git. Especially setting up using our DDX main repository as the upstream source, while feeding his changes from his local copy led to some confusion.
After resolving those, he started with a simple patch that changed the structure of XvPutImage so that later on DMA transfers can be easily added. Next step was writing an AGP patch to support DMA copy from the aperture to the video ram.
DMA can be done easily with AGP, because AGP was designed exactly for this : it provides a GART (basically a relocation table) that enables the card to have access to a large amount of system memory, that appears to be contiguous to the card. This feature makes it quite easy to do DMA : Basically you copy data to the aperture and then DMA from it by pointing the card at it and setting the size of the data to copy. PCI-e should be similar but needs more testing. Pmdata and cmn volunteered to test the patch, results from CMN on a NV35 GPU indicate that the FIFO times out after about 1 minute. While nv works, vanilla nouveau does time out for him too.
Darktama chimed in and asked him to comment out the call to NVWaitVSync(). CMN reported success and Darktama said it was obviously his fault. The code now setups a "blank" PGRAPH context which clears some bits in the NV10_PGRAPH_SURFACE register. The values are "magic" which means that we got them from our reverse engineering tools and are not fully (or even worse not at all) understood. A solution is pending.
When detailing his patch to marcheu a discussion about memory object and DMA copy ensued, from which I understood that:
- NVidia cards offer a PCI memory object and a AGP memory object. Plus it has its own memory (frame buffer). PCI object and AGP object differ in one major point: the PCI object points to a list of allocated memory pages, while the AGP points to an address and has a size. Both are objects needed for DMA transfers, but please do read on.
- the DRM allocates the AGP, PCI (not yet, see below) or FB memory and returns a physical adress
- the card expects an offset inside your current memory object where each of those objects refers to a piece of memory, either AGP, PCI or FB in our setup, we have one AGP (called NvDmaAGP) and one FB (called NvDmaFB) object. Both cover the whole range of memory available to them (FB the memory on card and AGP the memory of the aperture.
- Now if you want to start a memory to memory copy you take the NVidia Memory to Memory object and plug the two memory objects as source and destination into it.
That's the theory of it, but unfortunately we can't create PCI Memory objects just yet. On the other hand, rivatv has already done exactly that. So we only need to look at how they do it and rewrite it to work within our driver environment.
Marcheu fixed the double call of the projection matrix setup but claimed that he still has problems with swtcl (software transform, clipping and lighting). Due to real life constraints, he restrained himself to answering questions.
Still the viewport issue mentioned in the last issue still prevails.
Finally, kmeyer got his fd.o account and created a new dump directory. He and JussiP are now in the process to setup the status page (http://users.tkk.fi/~jpakkane/ren/) once again.
3. Help needed
Please do check out current DRM and DDX and report back your success or failure. We need to know that everything is working the same as before, as the next step will break the internal API which would make tracking bugs introduced much harder.
If you don't mind, please do test ahuillet's patches at git://people.freedesktop.org/~ahuillet/xf86-video-nouveau and give him feedback. However, be prepared for problems, misfeatures and crashes as this is definately a work in progress!