[FrontPage] [TitleIndex] [WordIndex] 

I am currently trying to update the page, make it more readable and then transfer to NouveauBits (I guess no one updates others personal page).

TODO: are channels compleatly independent? Each channel has its own objects and so on.

Intro

Nvidia overview

Gr objects

FIFO

DMA objects

subchannels and channels

nv_demo

MMIO registers

RAMIN, RAMFC, RAMHT, RAMRO

RAMIN

RAMFC

RAMHT

RAMRO

DMA object

  address - here it means 32 byte offset of memory area DMA object references. Example: DMA notifier is a small memory area (32-256 bytes) somewhere in AGP or VRAM. Offset in this case is offset from AGP/VRAM starting offset.
  '''Entry[0]'''
  11:0  class (seems like I can always use 0 here)
  12    page table present?
  13    page entry linear?
  15:14 access: 0 rw, 1 ro, 2 wo
  17:16 target: 0 NV memory, 1 NV memory tiled, 2 PCI, 3 AGP
  31:20 dma adjust (bits 0-11 of the address)
  '''entry[1]'''
  dma limit
  '''entry[2]'''
  1     0 readonly, 1 readwrite
  31:12 dma frame address (bits 12-31 of the address)
  Non linear page tables seem to need a list of frame addresses afterwards,
  the rivatv project has some info on this.

DMA objects are stored in RAMIN heap. Each object is identified by handle , handle is 4bytes integer. Creation of object consists from few steps:

Context object (Graphic object)

NouveauBits

* Null object (in source NvNull) is object passed as argument to some object functions where you don't know what you will fill in later. It is more or less default object. Its handle is 0x00000000 and it has to be created to be used(so you pass handle 0x0000 to functions of other objects). Class of null object is 0x30.

* There are no "implicit" objects (= objects that don't have to be created and can be used). All objects have to be created. During creation of object a user defined handle is assigned to the object and the handle is inserted into RAMHT along with some other info(like where is the object stored). Some functions require a handle of some object as parameter (NOT a address, if handle is not found in RAMHT, PFIFO error will occur).

* Nv20 either doesn't implement scissors or we dont know how to init it

* The INSTANCE_WR writes in *_graph_context_init are setting up the default graphics context.

* The intialized objects and MMIO setup survive the X restart. It will survive a lot, object setup will survive 20 seconds long shutdown. Beware when trying initialize drm using nouveau own code instead of nvidia.

* 19:09 < marcheu> but PGRAPH can be either put there through the graphics context, or through the object (or through direct writes, but nvidia doesn't do much of those)

* NV_PFB are registers that control frame buffer ram timings, fb layout, tiling...

* Offsets of the depth and color buffer must be multiple of 64 (otherwise the card will choke). If depth is disabled, you can set depth offset to same value as color buffer.

* In EXA A8+A8 is very important common operation. EXA uses it to compute font/icons masks.

* When DRM log says "invalid channel" whatever, you must reboot.

* PGRAPH context shouldnt be modified by user/driver. PGRAPH context is modified by card itself as you write to FIFO (Nvidia is writing to it in nvsdk, but there is used completly different approach).

21:48 < matc> pq: AFAIK PIPE_DATA stuff is a mirror to 3D state. And on nv10 we need to init them with certain value, otherwise 3D command produce errors IRRC. 21:49 < matc> yes may be if the 3D pipe is in a random state

Software methods (GPU system call handled in DPC)

* How are interrupts handled?

Ftom now on is Old Content(TM): update, spellcheck, move.

Opened questions :

  1. Is the FIFO only way to modify objects? Nope it is not. But it is a convenient, fast and safe way. You can also change objects by directly writing to memory area, where objects are stored. When you fire FIFO, it will take all commands you wrote and modify object. I dont remember address where objects are stored, but if you want to see how PIO structure looks, it is in nvsdk, file nvhw32.h structures like Nv04Dx5TexturedTriangle or Nv03GdiRectangleText.

  2. If there is a sequence of commands on a FIFO object where offset of actions differs by 4, is it possible to send more than one command in one packet? I have seen similar things in some renouveau dumps. 

Will all these FIFO actions produce same result?
1.
   BEGIN_RING_SIZE(7, 0x00000304, 3)
   OUT_RING(0x11111111)
   OUT_RING(0x22222222)
   OUT_RING(0x33333333)
2.
   BEGIN_RING_SIZE(7, 0x00000304, 1)
   OUT_RING(0x11111111)
   BEGIN_RING_SIZE(7, 0x00000308, 1)
   OUT_RING(0x22222222)
   BEGIN_RING_SIZE(7, 0x0000030c, 1)
   OUT_RING(0x33333333)
3.
   BEGIN_RING_SIZE(7, 0x00000304, 2)
   OUT_RING(0x11111111)
   OUT_RING(0x22222222)
   BEGIN_RING_SIZE(7, 0x0000030c, 1)
   OUT_RING(0x33333333)
4.
   BEGIN_RING_SIZE(7, 0x00000304, 1)
   OUT_RING(0x11111111)
   BEGIN_RING_SIZE(7, 0x00000308, 2)
   OUT_RING(0x22222222)
   OUT_RING(0x33333333)

Channels and subchannels

NVidia HW can easily support multiple opengl processes at once thanks to channels. Each OpenGl process has its own channel and is using it for all operations. Each channel can be thought of as individual processing unit (context switching is necessary for this illusion).

Each channel has 8 subchannels. Subchannel is something like a pointer to "object" which can be things like NV04_CONTEXT_SURFACES_3D or NV04_DX5_TEXTURED_TRIANGLE or NV04_SCALED_IMAGE_FROM_MEMORY... and each "object" can do a given kind of action, as their names imply. So subchannel with NV04_SCALED_IMAGE_FROM_MEMORY cant draw an triangle on screen, but can put image on screen. You can see objects in nouveau_regs.h together with possible commands for fifo (lets call them rings).

How does it work together

Object

Object type

info/subchannel

NvDmaFB

DMA object

VRAM of card

NvDmaAGP

DMA object

AGP bus

Nv3D

class_3d

NvSub3D, default card primitive for drawing 3D at the card

NvCtxSurf2D

NV04_CONTEXT_SURFACES_2D

NvSubCtxSurf2D

NvCtxSurf3D

NV04_CONTEXT_SURFACES_3D

NvSubCtxSurf3D, only on NV04, used for setting up buffer context

NvImageBlit

NV_IMAGE_BLIT

NvSubImageBlit

NvMemFormat

NV_MEMORY_TO_MEMORY_FORMAT

NvSubMemFormat, used for copy of buffers 

static void nv20ClipPlane(GLcontext *ctx, GLenum plane, const GLfloat *equation)
{
        nouveauContextPtr nmesa = NOUVEAU_CONTEXT(ctx);
        BEGIN_RING_CACHE(NvSub3D, NV20_TCL_PRIMITIVE_3D_CLIP_PLANE_A(plane), 4); // We are using previously created subchannel NvSub3D for definition of clip plane
        OUT_RING_CACHEf(equation[0]);
        OUT_RING_CACHEf(equation[1]);
        OUT_RING_CACHEf(equation[2]);
        OUT_RING_CACHEf(equation[3]);
}

How to start to do some work with FIFO

During your work with fifo, you put actions into FIFO. Each action is a packet like structure, that starts with size, command and subchannel. 

  BEGIN_RING(NvSub3D_0, NV30_TCL_PRIMITIVE_3D_CLEAR_VALUE_ARGB, 1);
  OUT_RING  (0xffffffff);

Before you start putting actions to FIFO, you have to assign context object to subchannel (context object has to be of course created even before this). Well, driver should take care of this for you, but during development you need to do it yourself. 

        // how to assign context object to subchannel
        BEGIN_RING(number_of_subchannel, 0, 1);  // assign object with handle_of_object
        OUT_RING  (handle_of_object);            // to subchannel number_of_subchannel

Now there is assigned object in subchannel number_of_subchannel (NoS). You can assign another context objects to another subchannels anytime. As soon as there is at least one subchannel with assigned object, you can put actions to FIFO (other actions than assigning object to subchannel).

Readl actions that *do* *something* are specified in packets (in most cases one action in one packet) os actions. Packet/action starts with writing info about action to FIFO through BEGIN_RING(subchannel_with_assigned_object_OBJ, command, size). You can use only commands that are valid on object OBJ, list can be found nouveau_reg.h, under class of OBJ. 

Object NV30_CLEAR_BUFFER (this is class of OBJ)
#define                 NV30_CLEAR_BUFFER                        0x00000066  // this is class
#       define          NV30_CLEAR_BUFFER_SET_DMA_NOTIFY         0x00000180  // this command for class NV30_CLEAR_BUFFER
#       define          NV30_CLEAR_BUFFER_SET_IMAGE_PATTERN      0x00000188  // this command for class NV30_CLEAR_BUFFER
#       define          NV30_CLEAR_BUFFER_SET_RASTER_OP          0x0000018c  // this command for class NV30_CLEAR_BUFFER
#       define          NV30_CLEAR_BUFFER_SET_CONTEXT_SURFACE_2D 0x00000198  // this command for class NV30_CLEAR_BUFFER
#       define          NV30_CLEAR_BUFFER_UNK002fc               0x000002fc  // this command for class NV30_CLEAR_BUFFER
!!! you cant BEGIN_RING(subchannel_with_assigned_object_NV30_CLEAR_BUFFER, NV30_TCL_PRIMITIVE_3D_BEGIN_END, 0x1)

BEGIN_RING marks start of packet with a command. It also has size, size is always at least 0x1 and it means number of arguments of command. Size is command specific and it is necessary to use renouveau dumps or other sources for precise syntax. Each argument is 4 bytes. Header of packet is also 4 bytes long (see macro). 

  // example of action, it wont do anything useful or maybe crash.
  NvSub3D = 7;
  Nv3D_handle = 0xbeef1234;
  // create a context object
  _context_obj_create(Nv3D_handle, NV30_TCL_PRIMITIVE_3D, init.channel)
  // set Nv3D_handle context object to subchannel 7
  BEGIN_RING(NvSub3D, 0, 1);
  OUT_RING  (Nv3D_handle);
  // set clear color
  BEGIN_RING(NvSub3D, NV30_TCL_PRIMITIVE_3D_CLEAR_VALUE_ARGB, 1);
  OUT_RING  (0xff0000ff);
  // clear color buffer
  BEGIN_RING(NvSub3D, NV30_TCL_PRIMITIVE_3D_CLEAR_WHICH_BUFFERS, 1);
  OUT_RING  (0x000000f0);

This may not be the only way specifying actions. See questions for packed actions

DMA notify - example

In this part I am attempting to describe some stuff about DMA, like commands ending with DMA_NOTIFY. Nearly every object has command ...DMA_NOTIFY, parameter of the command is DMA notifier.

A DMA notifier is a DMA object that references a small (32 byte it seems, we use 256 for safety) memory area that will be used by the HW to give feedback about a DMA operation. 

// This is not exactly what you write to FIFO, but it gives an idea.
set subchannel 0 to context object NV_IMAGE_FROM_CPU (beef6101)
  // now at each line is one action. First is COMMAND - parameters (here mostly objects)
  NV_IMAGE_FROM_CPU_DMA_NOTIFY       - DMA obj NV_DMA_TO_MEMORY (beef0301)
  NV_IMAGE_FROM_CPU_CLIP_RECTANGLE   - context obj NV01_CONTEXT_CLIP_RECTANGLE) | patch = SRCCOPY_AND (beef1901)
  NV_IMAGE_FROM_CPU_PATTERN          - context obj NV04_IMAGE_PATTERN (***beef4401)
  NV_IMAGE_FROM_CPU_ROP              - context obj NV03_PRIMITIVE_RASTER_OP (beef4301)
  NV_IMAGE_FROM_CPU_SURFACE          - context obj NV04_CONTEXT_SURFACES_2D (beef4201)
  NV_IMAGE_FROM_CPU_OPERATION        - SRCCOPY // It tells card copy image.

Well, what exactly is NV_IMAGE_FROM_CPU doing? It probably copy image from CPU to screen. First you map subchannel and do some commands at subchannel/object (0/beef6101). The last action (NV_IMAGE_FROM_CPU_OPERATION - SRCCOPY) tells GPU to copy image from source to destination. It can take a while, and during that time, GPU is processing another FIFO actions (if it is possible, GPU will be able to tell if next action interfere with DMA transfer and if it does, then GPU will wait for end of transfer). So FIFO process few (4-5) actions after SRCCOPY and now action (NV_IMAGE_FROM_CPU_OPERATION - SRCCOPY) ends.

Since it specified NV_IMAGE_FROM_CPU_DMA_NOTIFY, there is a DMA notifier and we can be notified about end of transfer. There are two ways to check:

It is unknown, how the 'nvdriver' reacts if it gets notify events that are not registered.

Writing NV_NOTIFY_WRITE_LE_AWAKEN into the 'Notify' field of an object in a channel really causes an interrupt in the PGRAPH engine. Thus we can determine whether a DMA transfer has finished in the interrupt handler.

We can't use interrupts in user land, so we do the simple polling approach.

For further info look into nv_dma.c.

What is action - command

  BEGIN_RING(NvSub3D_0, NV30_TCL_PRIMITIVE_3D_CLEAR_VALUE_ARGB, 1);
  OUT_RING  (0xffffffff);

NV04 initialization

The purpose of this document is to store for another developers as well as for myself some informations and principles of NVidia cards I discovered. But the original goal of my journey is to make glxgears work on NV04 class of cards.

From test_startup dump of REnouveau it seems that I should follow this : 

 1. Create DMA objects for FB and AGP.
 1. Create DMA memory area for surface ('''NvDma3Dsurf'''), where will be drawn triangle. In startup it looks like this:
          Searching for object beef0201
          Context is 82001495
          Software object
          ENGINE_SW[0] = 0000303d = class = 003d (NV_DMA_IN_MEMORY) | page table | page entry linear | dma_access = rw | dma_target = NV mem | dma adjust = 000
          ENGINE_SW[1] = 00fd00ff = dma limit = 00fd00ff
          ENGINE_SW[2] = 00000003 = dma page address = 00000 | r/w = TRUE | UNKNOWN = 00000001
          ENGINE_SW[3] = 00000003 = dma page address = 00000 | r/w = TRUE | UNKNOWN = 00000001
 1. Create context object NV04_CONTEXT_SURFACES_3D (NvCtxSurf3D)
   1. NV04_CONTEXT_SURFACES_3D_DMA_COLOR = NvDma3Dsurf
   1. NV04_CONTEXT_SURFACES_3D_DMA_ZETA  = NvDma3Dsurf
   1. NV04_CONTEXT_SURFACES_3D_FORMAT = color = !A8R8G8B8 | type = pitch | width = 0 | height = 0
   1. NV04_CONTEXT_SURFACES_3D_CLIP_HORIZONTAL = x = 0 | width = 512
   1. NV04_CONTEXT_SURFACES_3D_CLIP_VERTICAL = y = 0 | height = 512
   1. NV04_CONTEXT_SURFACES_3D_PITCH = color = 5120 | zeta = 2112
   1. NV04_CONTEXT_SURFACES_3D_OFFSET_COLOR = 0x0001f400
   1. NV04_CONTEXT_SURFACES_3D_OFFSET_ZETA = 0x00ec8000
 1. Create context object NV04_DX5_TEXTURED_TRIANGLE.
   1. NV04_DX5_TEXTURED_TRIANGLE_DMA_1 = NvDma3Dsurf
   1. NV04_DX5_TEXTURED_TRIANGLE_DMA_2 = DMA object at AGP aparature, with dma page address = 48000
   1. NV04_DX5_TEXTURED_TRIANGLE_SURFACE = NvCtxSurf3D
 1. Draw triangle... somehow.Copy code from default test.

The trouble is, it doesn't work. I made a mistake somewhere.

2013-03-24 13:16