[Oracle Linux 6.8 ] ASMCA時發現無法順利create ASM DISKGROUP

今天在Oracle Linux 6.8 安裝完畢後, 於ASMCA時發現無法順利create ASM DISKGROUP, 且發生ORA 7445 , ORA 600等相關的錯誤

於Alert_+ASM.log 發現

Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0xFFFFFFFFFFFFFFF0] [PC:0xBBB6283, kghalf()+259] [flags: 0x0, count: 1]

Errors in file /u01/app/oracle/diag/asm/+asm/+ASM/trace/+ASM_ora_16372.trc  (incident=169):

ORA-07445: exception encountered: core dump [kghalf()+259] [SIGSEGV] [ADDR:0xFFFFFFFFFFFFFFF0] [PC:0xBBB6283] [Address not mapped to object] []

Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM/incident/incdir_169/+ASM_ora_16372_i169.trc

Errors in file /u01/app/oracle/diag/asm/+asm/+ASM/trace/+ASM_ora_16774.trc  (incident=2089):

ORA-00600: internal error code, arguments: [kfdds::badaddr], [0x7F3894182108], [], [], [], [], [], [], [], [], [], []

Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM/incident/incdir_2089/+ASM_ora_16774_i2089.trc

Use ADRCI or Support Workbench to package the incident.

解決方法

參考以下兩篇, 就可以正常create asm disks了

Oracle Linux 6 - ASM Instances Fail with 4K Sector Size LUN (Doc ID 2211975.1)

New Block Size Feature for OracleASM and oracleasm-support (Doc ID 1530578.1)

看起來是新的ASM RPM (oracleasm-support-2.1.8-1.el6.x86_64) 需要調整一個參數 ORACLEASM_USE_LOGICAL_BLOCK_SIZE 讓ASM(512 byte block) 配合Storage 使用的block size(4K block) (常發生於Storage Block Size : 4K 或是All SSD的設備)

Step 1.

# oracleasm configure -p

Writing Oracle ASM library driver configuration: done

[root@tpechdrdb01 ~]# oracleasm configure | grep ORACLEASM_USE_LOGICAL_BLOCK_SIZE

ORACLEASM_USE_LOGICAL_BLOCK_SIZE="true"

Step 2.

vi /etc/sysconfig/oracleasm

set ORACLEASM_USE_LOGICAL_BLOCK_SIZE="true"

Step 3.

Reboot

--

其他參考link

 http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

  The Oracle ASMLib kernel driver is now included in the Unbreakable Enterprise Kernel.  No driver package needs to be installed when using this kernel.

--

http://blog.csdn.net/msdnchina/article/details/52645174

来源于:  CRS Not Able to Discover Voting Disk after Kernel and Module kmod-oracleasm Upgrade (文档 ID 1994371.1)

适用于:
Linux 6.0以及更新版本
Linux x86-64bit

症状:

使用如下的参数来配置的asmlib:

[html] view plain copy

1. ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true   

当linux kernel和kmod-oracleasm从 2.0.6.rh1-2.el6.x86_64 升级到 2.0.8-4.el6_6.x86_64之后,CRS不能发现voting disk

[html] view plain copy

1. 2015-03-24 03:59:47.421: [ CSSD][2286941952]clssnmvDiskVerify: discovered a potential voting file  
2. 2015-03-24 03:59:47.421: [ SKGFD][2286941952]Handle 0x7feb7813ddf0 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:OCRVOTE11:  
3.   
4. 2015-03-24 03:59:47.422: [ CSSD][2286941952]clssnmvDiskVerify: TOC format mismatch expected(0x634c7373 0x546f636b), found(0x0000 0x0000)  
5. 2015-03-24 03:59:47.422: [ SKGFD][2286941952]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x7feb7813ddf0 for disk :ORCL:OCRVOTE11:  

变化:
Linux kernel升级之后,kmod-oracleasm从 2.0.6.rh1-2.el6.x86_64 升级到 2.0.8-4.el6_6.x86_64.

原因:
对于disk来说,有两种block size:

[html] view plain copy

1. •logical block size  
2. •physical block size  

对于传统的disk,在linux上,这两种block size均为512Bytes

现在越来越多的disk 的 physical block size等于4096,但是logical block size 依然是512

比如:

[html] view plain copy

1. # grep "" /sys/block/sdag/queue/*block_size  
2. /sys/block/sdag/queue/logical_block_size:512  
3. /sys/block/sdag/queue/physical_block_size:4096  

在老的配置中, kmod-oracleasm-2.0.6.rh1-2.el6.x86_64 不支持在 ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true 这个选项(该选项在/etc/sysconfig/oracleasm文件中).
因此,即使 ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true ,oracleasm的sector size是4096,而不是512

[html] view plain copy

1. $ modinfo ./kmod-oracleasm-2.0.6.rh1-2.el6.x86_64/lib/modules/2.6.32-358.el6.x86_64/extra/oracleasm/oracleasm.ko  
2. filename:       ./kmod-oracleasm-2.0.6.rh1-2.el6.x86_64/lib/modules/2.6.32-358.el6.x86_64/extra/oracleasm/oracleasm.ko  
3. description:    Kernel driver backing the Generic Linux ASM Library.  
4. author:         Joel Becker <joel.becker@oracle.com>  
5. version:        2.0.6  
6. license:        GPL  
7. srcversion:     B6C06FFEF593E8060B7BFEE  
8. depends:          
9. vermagic:       2.6.32-358.el6.x86_64 SMP mod_unload modversions  

但是,新的驱动kmod-oracleasm-2.0.8-4.el6_6.x86_64确实支持该参数

[html] view plain copy

1. $ modinfo ./kmod-oracleasm-2.0.8-4.el6_6.x86_64/lib/modules/2.6.32-504.1.3.el6.x86_64/extra/oracleasm/oracleasm.ko  
2. filename:       ./kmod-oracleasm-2.0.8-4.el6_6.x86_64/lib/modules/2.6.32-504.1.3.el6.x86_64/extra/oracleasm/oracleasm.ko  
3. description:    Kernel driver backing the Generic Linux ASM Library.  
4. author:         Joel Becker <joel.becker@oracle.com>  
5. version:        2.0.8  
6. license:        GPL  
7. srcversion:     223CD1891C9BE8AFC0A0E76  
8. depends:          
9. vermagic:       2.6.32-504.1.3.el6.x86_64 SMP mod_unload modversions  
10. parm:           use_logical_block_size:Prefer logical block size over physical (Y=logical, N=physical [default]) (bool)        <==== now it support  

当升级完kernel之后,并且ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,这样的话,使用的sector size 是512.
因此,会由于sector size的不匹配导致 CRS 不能发现voting disk.

[html] view plain copy

1. There are similar issues which are caused by opposite reason, please refer to:  
2. Linux: After NetApp ONTAP Upgrade: CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds (Doc ID 1641344.1)  
3. Alert: After SAN Firmware Upgrade, ASM Diskgroups ( Using ASMLIB) Cannot Be Mounted Due To ORA-15085: ASM disk "" has inconsistent sector size. (Doc ID 1500460.1)  

解决方案:
第一步:重新配置oracleasm以便使用physical block size 
# oracleasm configure -p

[html] view plain copy

1. It's not recommended to edit the file /etc/sysconfig/oracleasm directly, as you might hit another issue described in:  
2. Updates to ASMLib Configuration File "/etc/sysconfig/oracleasm" Not Effective (Doc ID 868352.1)  

第二步:reboot 操作系统.

参考文档:

[html] view plain copy

1. NOTE:1500460.1 - Alert: After SAN Firmware Upgrade, ASM Diskgroups ( Using ASMLIB) Cannot Be Mounted Due To ORA-15085: ASM disk "" has inconsistent sector size.  
2. NOTE:1641344.1 - Linux: After NetApp ONTAP Upgrade: CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds  
3. NOTE:868352.1 - Updates to ASMLib Configuration File "/etc/sysconfig/oracleasm" Not Effective  
4. NOTE:1530578.1 - New Block Size Feature for OracleASM and oracleasm-support  

译者总结:
其实这个文章说的比较绕,我个人总结如下:

前提: 

1.这个asm diskgroup用了asmlib来实现的
2.升级前:/etc/sysconfig/oracleasm文件中的ORACLEASM_USE_LOGICAL_BLOCK_SIZE等于true (意味着asm使用底层disk的logical block size)
3.asm disk 的physical block size等于4096,logical block size等于512

升级前:
kmod-oracleasm-2.0.6.rh1-2.el6.x86_64,但是该版本的kmod-oracleasm不支持ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,
也就是说Oracleasm其实还是用的4096的physical sector size,512的logical block size.其实这是4k sector size的emulition mode.


升级后:
kmod-oracleasm-2.0.8-4.el6_6.x86_64,该版本的kmod-oracleasm支持ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,
也就是说Oracleasm其实还是用的4096的physical sector size,4096的logical sector size.


问题被解决之后的状态:
/etc/sysconfig/oracleasm文件中的ORACLEASM_USE_LOGICAL_BLOCK_SIZE等于false(意味着asm使用底层disk的physical block size)