Exadata Roadmap – More Speculation

Oracle Sun Flash Accelerator F40 card

It’s silly season. In the run up to Oracle Open World there are always rumours and whispers about what products will be announced – and this year is no different. I know this because I’m one of the people partaking in the spread of baseless and unfounded speculation.

Clearly the thing that most people are talking about is the almost certain release of a new Exadata generation called the X3. There appear to be both the X3-2 and X3-8 generations coming, as well as an interesting “Exadata X3-2 Eighth Rack” (that’s eighth as in 1/8 not as in 8th I presume). You don’t need me to tell you any of this, because Andy Colvin from those excellent guys at Enkitec has written a great article all about it right here.

And as if that wasn’t enough, Kevin Closson, the ex-Performance Architect of Exadata, has added his own speculative article in which he walks the fine line of legal requirement placed upon anyone who used to work in the Oracle Development organisation (because Oracle’s expensively assembled legal team often finds time to stretch its muscles about these things: to quote Kevin, “I’m only commenting about the rumors I’ve read and I will neither confirm nor deny even whether I *can* confirm or deny.” But did you notice how he didn’t confirm whether he could confirm that he could confirm it?)

Anyway, with Andy and Kevin on the case, there is little point in me trying to add anything there. So let’s look at some of the other rumours.

Sun F40 Accelerator Flash Cards

It appears that the X3 will finally ditch the unloved Sun F20 flash cards that have been present since the introduction of flash when the V2 model came out in 2009. Flash technology has advanced rapidly over recent years – and the F20 cards were hardly at the forefront of the technology even in 2009.

The F20 cards contained four flash modules known as DOMs, each with 24GB of SLC flash and 64MB of DRAM. In order to ensure that writes made it to flash in the event of power loss, they also had a dirty great big super-capacitors strapped to the back. I’m no fan of supercaps in general, they tend to have reliability issues and go bang in the night. I’m not saying that Oracle’s cards had this issue though (because I also have to consider that expensively-assembled legal team). However it’s interesting to note this quote in the F20 user guide:  “Because high temperatures can have negative impact on life expectancy, it is best to locate the Sun Flash Accelerator F20 PCIe Card in PCIe slots that offer maximum airflow“.

The new F40 cards have now switched to using MLC flash and again contain four DOMs. This time they are 100GB in size, giving a total of 400GB usable (512GB raw). There is no mention of DRAM, but of course it must be there. The manual also offers no insight into whether there are any supercaps (unlike the F20 manual which had a lovely section on “Super Capacitors versus Batteries”) but I can see some fat little nodules on the picture up above which tell me that capacitance is still essential. The result of these changes (probably mainly the switch to MLC) is that the published mean time between failure has dropped by 50% from 2m hours to 1m hours. That’s taken 114 years off of the lifetime of the cards!

The power draw appears to have risen, because the F20 used around 16.5W during normal operation, whereas the F40 is described as using 25W max and 11.5W even when idle. On the other hand maybe they just picked a value in the middle and called that “normal”.

What will be interesting is to see how Oracle handles the flash write cliff. Flash media is very fast for reads; in the case of the F40 the latency is 251 microseconds (not impressive against the 90 microseconds on a Violin system, but still better than disk). Flash is even faster for writes, with the F40 having a 95 microsecond latency (25 microseconds on Violin 🙂 ). The area to watch out for though is erasing. On flash you can only write to an empty block, so once the block is used it has to be erased again before you issue another write to it. Violin has all sorts of patented technology to ensure this doesn’t affect performance (but as I’ve already plugged Violin twice I’ll shut up about it). Oracle doesn’t – at least, nothing that any of the flash vendors would be worried about.

[Disclosure: In the comments section below, Alex asked a question about the block size which made me realise that the F40 datasheet numbers are showing latency figures for 8k, whereas I am quoting Violin latency figures for 4k blocks. Even so, it’s still obvious that there are some big differences there.]

That’s never really been a problem for Oracle before, because the Exadata flash was used as a write-through cache, where the write performance of the flash cards was not an issue. This time, with the new “flash for all writes” capabilities of the flash cache, write performance is going to matter – particularly for sustained writes, such as ETL jobs, batch loads, data imports etc. Unless Oracle has some way to avoid it, once the capacity of the cards is used and all of the flash cells have been written to, there will be a big drop-off in performance whilst the garbage collection takes place in the background to try and erase free cells. It will be interesting to see how the X3 behaves during this type of load.

Database Virtualisation

This is the other hot topic for me, since I am an avid believer that we are seeing a major trend in the industry towards the virtualisation of production Oracle databases. Oracle, it has to be said, has not had a massive amount of success with its Oracle VM product. I actually quite like it, but I appear to be in a minority. It’s not got anything like the market penetration of Hyper-V, whilst VMware is in a different league altogether.

History tells us that when Oracle has a product with which it wants to drive (or rather, enforce) more adoption, it uses “interesting” strategies. The addition of OVM to Exadata is, for me, almost certain. In this way, Oracle gets to push its own virtualisation product as something that a) is “engineered to work with Exadata”, b) is a “one throat to choke” support solution, and c) is the *only* choice you can have.

Expect to see lots of announcements around this, with particular hype over the features such as online migration and integration with OEM, as well as lots of talk about how the Infiniband network makes it all a million times faster than some unspecified alternative.

Update 10 September 2012

It’s come to my attention that the Sun F40 cards look incredibly similar to the LSI Nytro WarpDrive WLP4-200 flash cards. Just take a look at the pictures. I don’t know this for a fact, but the similarity is plain to see. Surely Oracle must be OEMing these?

A note for Oracle’s legal team: please note that this is all wild speculation and that I am in no way using any knowledge gained whilst an employee of Oracle. In fact the main thing I learned whilst an employee was that people on the outside who aren’t supposed to know get to have a lot more fun speculating than the people on the inside who are supposed to know but don’t.