Strange ASM Behaviour with 4k Devices

question-mark-dice

This is only a short post to document something I’ve seen and reproduced but still don’t understand. Storage devices generally have a physical sector size of 512 bytes or, more recently, 4k. This is a subject which causes much confusion (partly because some vendors seek to portray whichever sector size they use as “better”). You can read more about the subject here.

On my Oracle Linux 6.3 server running the Unbreakable Enterprise Kernel I have two block devices presented from Violin. These devices have physical block sizes of 4k but logical block sizes of 512 bytes:

[root@server1 ~]# uname -r
2.6.39-200.24.1.el6uek.x86_64

[root@server1 ~]# fdisk -l /dev/mapper/violin_data1 | grep "Sector size"
Sector size (logical/physical): 512 bytes / 4096 bytes

[root@server1 ~]# fdisk -l /dev/mapper/violin_data2 | grep "Sector size"
Sector size (logical/physical): 512 bytes / 4096 bytes

Using Oracle Grid Infrastructure 11.2.0.3, I’m now going to create an ASM diskgroup on just one of the devices, using the new clause SECTOR_SIZE=4096:

SQL> CREATE DISKGROUP DATA EXTERNAL REDUNDANCY
  2  DISK '/dev/mapper/violin_data1'
  3  ATTRIBUTE
  4       'sector_size'='4096',
  5       'compatible.asm' = '11.2',
  6       'compatible.rdbms' = '11.2';

Diskgroup created.

SQL> select GROUP_NUMBER, DISK_NUMBER, STATE, NAME, PATH, SECTOR_SIZE from v$asm_disk;

GROUP_NUMBER DISK_NUMBER STATE    NAME            PATH                      SECTOR_SIZE
------------ ----------- -------- --------------- ------------------------- -----------
           0           1 NORMAL                   /dev/mapper/violin_data2          512
           1           0 NORMAL   DATA_0000       /dev/mapper/violin_data1          512

As you can see, the disk which joined the diskgroup shows a sector size of 512 (which is correct, because ASM is reading the logical block size of the device). The other disk is also showing a 512 byte sector size. So now let’s add it to the diskgroup:

SQL> alter diskgroup data add disk '/dev/mapper/violin_data2';

Diskgroup altered.

SQL> select GROUP_NUMBER, DISK_NUMBER, STATE, NAME, PATH, SECTOR_SIZE from v$asm_disk;

GROUP_NUMBER DISK_NUMBER STATE    NAME            PATH                      SECTOR_SIZE
------------ ----------- -------- --------------- ------------------------- -----------
           1           1 NORMAL   DATA_0001       /dev/mapper/violin_data2         4096
           1           0 NORMAL   DATA_0000       /dev/mapper/violin_data1          512

Huh? The newly added disk has suddenly become a 4k sector device. Why? If I add both devices during the initial CREATE DISKGROUP statement this does not happen, it only seems to happen when I ADD the disk to an existing diskgroup.

Why?

About these ads

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 757 other followers