Smart Flash Cache

Disclaimer: This post was written in 2012 and covers the Exadata X2 model. I have no plans to update it to cover the later models, so please be aware that the information below is no longer be up to date. In particular, the information about the Smart Flash Cache being a write-through cache is no longer correct for later versions of the Exadata software.

As part of the default installation process the flash storage is configured to contain “Exadata Smart Flash Cache”, an intelligent and automatically-managed caching mechanism which determines whether to keep an object in the Smart Flash Cache once it has been read; random reads against indexes and tables are likely to be cached whereas sequentially accessed data (e.g. scans) are not. Equally, activities such as backups will not populate the scan as they are expected not to be repeated in the near future.

The Exadata Smart Flash Cache is a write-through cache; write operations are written through to disk and then the caching software decides whether to stage them back to the cache. For this reason the Exadata machine does not benefit from flash storage speeds when making write IO requests.

Instead of using the Smart Flash Cache feature, customers are able to locate ASM diskgroups on their flash storage if they desire – so far Oracle has tended to recommend against this. The problem with this approach is that customers have to manually control which objects reside in these flash-based diskgroups and continually monitor them to ensure that they are the best-suited candidates. As load profiles change and these objects become less suitable, they must be manually rebuilt back into the hard drive-based disk groups.

Smart Flash Logging

An additional feature implemented in Exadata Storage software 11.2.2.4.2 (and database version 11.2.0.2 + BP11) is called Exadata Smart Flash Logging. With this feature 512MB of flash storage is reserved for redo writes and the database’s log writer (LGRW) process adopts a different pattern of behaviour. In a system which does not use this feature LGWR writes in parallel to multiplexed copies of the redo logs and then waits for all writes to complete. The result of this is that the time taken to perform these writes (indicated by the Oracle wait interface statistics log file parallel write) is the time taken for the slowest disk to complete the write. With Exadata Smart Flash Logging the redo log files remain on disk but the additional reserved 512MB of space is created on flash storage. When issuing a write call, LGWR write to the redo logs on disk as usual but will additionally make another parallel write to the flash area. LGWR then waits for whichever of these writes completes first to post it, after which it continues without waiting for the other; in this method the redo write time is now that of the fastest media – flash or disk. Oracle claims that this feature offers the ability to avoid spikes in redo write time, which is an important benefit for busy OLTP systems.

Greg Rahn has a good description of this feature on his blog, whilst Jason Arneil has an interesting article on just how much of an impact Smart Flash Logging actually has.

Next: OLTP Performance