Graphics linkspam: A maze of twisty passages

One of the things I’ve been frustrated with lately is an overview document of Intel graphics architecture. There’s the massive Intel graphics Programmer Reference Manuals (which are so big they need to be separate PDFs for each chapter), but if you’re skimming each chapter intro for an overview, you’re likely to get lost. Fortunately, I stumbled across a 22-page Intel graphics architecture overview document that looks much more promising.

Another cool find this week was two sites that document which OpenGL extensions hardware manufacturers implement on Linux. If you’re a game designer looking to improve performance by using the latest wiz-bang extension, it’s important to know if that extension is actually implemented in all the hardware you care about. There’s the Mesa Matrix site that says whether a particular hardware vendor implements an extension, and another site that further breaks it down per-platform.

How to approach a new system: Linux graphics and Mesa

By now, most of the Linux world knows I’ve stopped working on the Linux kernel and I’m starting work on the Linux graphics stack, specifically on the userspace graphics project called Mesa.

What is Mesa?

Like the Linux kernel, Mesa falls into the “operating system plumbing” category. You don’t notice it until it breaks. Or clogs, slowing down the rest of the system. Or you need plumbing for your shiny new hot tub, only you’re missing some new essential feature, like hot water or tessellation shaders.

The most challenging and exciting part of working on systems “plumbing” projects is optimization, which requires a deep understanding of both the hardware limitations below you in the stack, and the needs of the software layers above you. So where does Mesa fit into the Linux graphics stack?

Mesa is the most basic part of the userspace graphics stack, sitting above the Linux kernel graphics driver that handles command submission to the hardware, and below 3D libraries like qt, kde, or game engines like Unity or Unreal. A game engine creates a specialized 3D program, called a shader, that conforms to a particular rev of a graphics spec (such as EGL 3.0 or GLES 3.1). Mesa takes that shader program and compiles it into graphics hardware commands specific to the system Mesa is running on.

What’s cool about Mesa?

The exciting thing for me is the potential for optimizing graphics around system hardware limitations. For example, you can optimize the compiler to generate less graphics commands. In theory, less commands means less for the hardware to execute, and thus better graphics performance. However, if each of those commands are really expensive to run on your graphics hardware, that optimization can actually end up hurting your performance.

This kind of work is fun for me, because I get to touch and learn about hardware, without actually writing Verilog or VHDL. I get to make hardware do interesting things, like add a new features that makes the latest Steam game actually render or add an optimization that improves the performance of a new open source Indie game.

Understand how much you don’t know

Without knowledge of both the hardware and software layers above, it’s impossible to evaluate what optimizations are useful to pursue. For example, the first time I heard modern Intel GPUs have a completely separate cache from the CPU, I asked two questions: “What uses that cache?” and “What happens when the cache is full?” The hardware Execution Units (EUs) that execute graphics commands use the cache to store metadata called URBs. My next question, on discovering that URB size could vary, and that newer hardware had an ever-increasing maximum URB size, was to ask, “How does the graphics stack pick which URB size to use?” Obviously, picking a smaller URB size means that more URBs can fit in the cache, which means it’s less likely that an EU will have to stall until there’s room in the cache for the URB it needs for the set of graphics commands its executing. Picking an URB size that was too small would mean programs that use a lot of metadata wouldn’t be able to run.

Mesa is used by a lot of different programs with different needs, and each may require different URB sizes. The hardware has to be programmed with the maximum URB size, and changing that requires stopping any executing commands in the graphics pipeline. So you can’t go changing the URB size on the fly every time a new 3D graphics program starts running, or you would risk stalling the whole graphics stack. Imagine your entire desktop system freezing every time you launched a new Steam game.

Asking questions helps you ramp faster

It’s my experience with other operating system components that lead me to ask questions of my more experienced graphics developer co-workers. I really appreciate working in a community where I know I can ask these kinds of basic questions without fear of backlash at a newcomer. I love working on a team that chats about tech (and non tech!) over lunch. I love the easy access of the Intel graphics and DRI irc channels, where people are willing to answer simple questions like “How do I build the documentation?” (Which turns out to be complex when you run into a bug in doxygen that causes an infinite loop and increasing memory consumption until a less-than-capable build box starts to freeze under memory pressure. Imagine what would have happened if the experienced devs assumed I was a n00b and ignored me?)

My experience with other complex systems makes me understand that the deep, interesting problems can’t be approached without a long ramp up period and lots of smaller patches to gain understanding of the system. I do chafe a bit as I write those patches, knowing the interesting problems are out there, but I know I have to walk before I start running. In the meantime, I find joy in making blog posts about what I’m learning about the graphics pipeline, and I hope we can walk together.

Progress on Graphics

I’m back from my 8-week sabbatical!  My roadtrip was a lot of fun, but now it’s time to get back to work.

In the month I’ve been back, I’ve been making good progress on learning about ChromeOS and graphics.   I have some really awesome, patient mentors (Josh Triplett and Chad Versace) who have been helping me learn about the ChromeOS build system and graphics.  At this point, I can build and install my own ChromeOS images and individual packages.  The build system is both complex and kinda crufty, like some sort of horribly rusty swiss army knife.  (Really, you have two separate build systems for subversion and git projects?)  However, I’m really impressed with ChromeOS test suite and the ability to kick off remote tests.  If only the Google developers would stop breaking the build…

On the graphics side, four of my patches got merged into mesa and piglit to add support and tests for a ChromeOS-specific EGL extension.  Chad has been creating graphics tutorials to introduce both Josh and I to userspace graphics concepts.  There’s a lot of new concepts to learn, everything from vertices and model transforms to relearning my Linear Alegebra.

Getting to work in userspace again has been so much fun.  Being able to use tools like strace and gdb to step through the code simply isn’t possible when you’re in a kernel interrupt service routine.  Some days I do miss hardware, poking at bits to make lights blink, but it’s really gratifying to see the tangible graphics work on my screen.  I’ve completed the GL version of “hello world” and completed Chad’s tutorial to make a triangle appear on the screen.  It was really satisfying to figure out how to make it spin around the Z axis.

Onwards to my next adventure!

I’m joining the Intel OTC ChromeOS differentiation team, and transitioning maintainership of the xHCI driver over to Mathias Nyman.

I’ve only been officially on the team for a couple of weeks, but I’m already playing with webGL tutorials and learning about vertices, shaders, EGL extensions, piglet tests, and loads more about graphics.  It’s been really great working with Josh Triplett and Chad Versace. I managed to join the team in time to attend their group quarterly event at Ground Kontrol.  We played pinball, 80’s games, and DDR for hours. 🙂