Festival of updates #3: Snow Leopard and "huge pages"!

Nerds only. I mentioned yesterday that the elegant 23-page Ars Technica review of the new Mac Snow Leopard OS should give as much tech detail as "anyone" would want. Au contraire! (Someday I will learn to avoid saying "anyone," "everyone," "no one," etc.) After the jump, a remaining question apparently left unanswered even by Ars Technica -- namely, whether the latest Mac OS supports "huge pages." An explanation of what this is and why it matters, via reader and software guy Ken Broomfield, follows. This goes into the "there are always more details" category, and is offered as a public service.

Ken Broomfield writes:

"ArsTechnica deserves a lot of credit for doing in-depth stuff like this that's becoming hard to find anywhere except in dry, poorly-written journal articles (though Ars has done less of this lately). But they lost me with this part, about the desirability of a 64-bit OS X kernel:

>"Tracking 96GB of RAM requires 1.5GB of kernel address space. Using more than a third of the kernel's address space just to track memory is a pretty uncomfortable situation."<

".... Intel x86 processors can manage memory either in 4K byte units (called "pages") or in 2MB or 4MB pages ("huge pages"). Huge pages would vastly reduce the accounting overhead for managing memory.... I haven't been able to find definitive references, but apparently OS X doesn't support huge pages; Windows Vista, Windows 7, recent Windows Server and Linux apparently do. Huge pages would also significantly improve performance. I saw one reference saying that general CPU performance increases by 15% because address translation for memory accesses is more efficient (the CPU's Translation Lookaside Buffer (TLB) sees fewer misses because there are far fewer pages). Virtual memory swapping to and from disk -- when your computer is running low on real memory, etc. -- is also faster because it's happening in larger chunks. The 4K page size was something IBM came up with in the pleistocene era of mainframes (the 1960s and 70s) as an optimal trade-off, and it's no longer optimal in an age when people routinely open a dozen 20-megapixel images in Photoshop.

"The main thing is that where the Ars article talks about a 64-bit kernel solving the problem of memory management overhead taking up a lot of room, it's really missing the point. If Snow Leopard supported huge pages, the problem would be solved regardless; if it doesn't, it should, for many reasons.

"Searching far and wide, I see no reference to OS X supporting huge pages, but I can confirm that Linux, Windows Vista and later, and Windows Server 2003 and later support huge pages, even under a 32-bit kernel...

"[The Ars article] is derived from this Apple document (danger -- wonky) and if this doesn't mention huge pages, they probably don't support it yet.

"For regular users out there, you may want to ask why Photoshop is 64-bit on Windows and not on the Mac. Apple's been touting 64-bit computing for a few years now, somewhat dishonestly, but has dropped the ball in a couple of areas. Some of this goes back to the origins of the Mac, 25 years ago."

I know there is someone for whom this is not enough detail -- but it's enough for me, for now.