星期三, 5月 31, 2017

[Oracle Linux 6.8 ] ASMCA時發現無法順利create ASM DISKGROUP

今天在Oracle Linux 6.8 安裝完畢後, 於ASMCA時發現無法順利create ASM DISKGROUP, 且發生ORA 7445 , ORA 600等相關的錯誤

於Alert_+ASM.log 發現
Exception [type: SIGSEGV, Address not mapped to object] [ADDR:0xFFFFFFFFFFFFFFF0] [PC:0xBBB6283, kghalf()+259] [flags: 0x0, count: 1]
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM/trace/+ASM_ora_16372.trc  (incident=169):
ORA-07445: exception encountered: core dump [kghalf()+259] [SIGSEGV] [ADDR:0xFFFFFFFFFFFFFFF0] [PC:0xBBB6283] [Address not mapped to object] []
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM/incident/incdir_169/+ASM_ora_16372_i169.trc

Errors in file /u01/app/oracle/diag/asm/+asm/+ASM/trace/+ASM_ora_16774.trc  (incident=2089):
ORA-00600: internal error code, arguments: [kfdds::badaddr], [0x7F3894182108], [], [], [], [], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM/incident/incdir_2089/+ASM_ora_16774_i2089.trc
Use ADRCI or Support Workbench to package the incident.

解決方法
參考以下兩篇, 就可以正常create asm disks了
Oracle Linux 6 - ASM Instances Fail with 4K Sector Size LUN (Doc ID 2211975.1)
New Block Size Feature for OracleASM and oracleasm-support (Doc ID 1530578.1)
看起來是新的ASM RPM (oracleasm-support-2.1.8-1.el6.x86_64) 需要調整一個參數 ORACLEASM_USE_LOGICAL_BLOCK_SIZE 讓ASM(512 byte block) 配合Storage 使用的block size(4K block) (常發生於Storage Block Size : 4K 或是All SSD的設備)

Step 1.
# oracleasm configure -p
Writing Oracle ASM library driver configuration: done
[root@tpechdrdb01 ~]# oracleasm configure | grep ORACLEASM_USE_LOGICAL_BLOCK_SIZE
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="true"

Step 2.
vi /etc/sysconfig/oracleasm
set ORACLEASM_USE_LOGICAL_BLOCK_SIZE="true"

Step 3.
Reboot

--
其他參考link
 http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

  The Oracle ASMLib kernel driver is now included in the Unbreakable Enterprise Kernel.  No driver package needs to be installed when using this kernel.

--
http://blog.csdn.net/msdnchina/article/details/52645174


来源于:  CRS Not Able to Discover Voting Disk after Kernel and Module kmod-oracleasm Upgrade (文档 ID 1994371.1)

适用于:
Linux 6.0以及更新版本
Linux x86-64bit

症状:

使用如下的参数来配置的asmlib:
[html] view plain copy
  1. ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true   

当linux kernel和kmod-oracleasm从 2.0.6.rh1-2.el6.x86_64 升级到 2.0.8-4.el6_6.x86_64之后,CRS不能发现voting disk
[html] view plain copy
  1. 2015-03-24 03:59:47.421: [ CSSD][2286941952]clssnmvDiskVerify: discovered a potential voting file  
  2. 2015-03-24 03:59:47.421: [ SKGFD][2286941952]Handle 0x7feb7813ddf0 from lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: for disk :ORCL:OCRVOTE11:  
  3.   
  4. 2015-03-24 03:59:47.422: [ CSSD][2286941952]clssnmvDiskVerify: TOC format mismatch expected(0x634c7373 0x546f636b), found(0x0000 0x0000)  
  5. 2015-03-24 03:59:47.422: [ SKGFD][2286941952]Lib :ASM:/opt/oracle/extapi/64/asm/orcl/1/libasm.so: closing handle 0x7feb7813ddf0 for disk :ORCL:OCRVOTE11:  
变化:
Linux kernel升级之后,kmod-oracleasm从 2.0.6.rh1-2.el6.x86_64 升级到 2.0.8-4.el6_6.x86_64.

原因:
对于disk来说,有两种block size:
[html] view plain copy
  1. •logical block size  
  2. •physical block size  

对于传统的disk,在linux上,这两种block size均为512Bytes

现在越来越多的disk 的 physical block size等于4096,但是logical block size 依然是512

比如:
[html] view plain copy
  1. # grep "" /sys/block/sdag/queue/*block_size  
  2. /sys/block/sdag/queue/logical_block_size:512  
  3. /sys/block/sdag/queue/physical_block_size:4096  

在老的配置中, kmod-oracleasm-2.0.6.rh1-2.el6.x86_64 不支持在 ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true 这个选项(该选项在/etc/sysconfig/oracleasm文件中).
因此,即使 ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true ,oracleasm的sector size是4096,而不是512
[html] view plain copy
  1. $ modinfo ./kmod-oracleasm-2.0.6.rh1-2.el6.x86_64/lib/modules/2.6.32-358.el6.x86_64/extra/oracleasm/oracleasm.ko  
  2. filename:       ./kmod-oracleasm-2.0.6.rh1-2.el6.x86_64/lib/modules/2.6.32-358.el6.x86_64/extra/oracleasm/oracleasm.ko  
  3. description:    Kernel driver backing the Generic Linux ASM Library.  
  4. author:         Joel Becker <joel.becker@oracle.com>  
  5. version:        2.0.6  
  6. license:        GPL  
  7. srcversion:     B6C06FFEF593E8060B7BFEE  
  8. depends:          
  9. vermagic:       2.6.32-358.el6.x86_64 SMP mod_unload modversions  


但是,新的驱动kmod-oracleasm-2.0.8-4.el6_6.x86_64确实支持该参数
[html] view plain copy
  1. $ modinfo ./kmod-oracleasm-2.0.8-4.el6_6.x86_64/lib/modules/2.6.32-504.1.3.el6.x86_64/extra/oracleasm/oracleasm.ko  
  2. filename:       ./kmod-oracleasm-2.0.8-4.el6_6.x86_64/lib/modules/2.6.32-504.1.3.el6.x86_64/extra/oracleasm/oracleasm.ko  
  3. description:    Kernel driver backing the Generic Linux ASM Library.  
  4. author:         Joel Becker <joel.becker@oracle.com>  
  5. version:        2.0.8  
  6. license:        GPL  
  7. srcversion:     223CD1891C9BE8AFC0A0E76  
  8. depends:          
  9. vermagic:       2.6.32-504.1.3.el6.x86_64 SMP mod_unload modversions  
  10. parm:           use_logical_block_size:Prefer logical block size over physical (Y=logical, N=physical [default]) (bool)        <==== now it support  


当升级完kernel之后,并且ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,这样的话,使用的sector size 是512.
因此,会由于sector size的不匹配导致 CRS 不能发现voting disk.
[html] view plain copy
  1. There are similar issues which are caused by opposite reason, please refer to:  
  2. Linux: After NetApp ONTAP Upgrade: CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds (Doc ID 1641344.1)  
  3. Alert: After SAN Firmware Upgrade, ASM Diskgroups ( Using ASMLIB) Cannot Be Mounted Due To ORA-15085: ASM disk "" has inconsistent sector size. (Doc ID 1500460.1)  

解决方案:
第一步:重新配置oracleasm以便使用physical block size 
# oracleasm configure -p
[html] view plain copy
  1. It's not recommended to edit the file /etc/sysconfig/oracleasm directly, as you might hit another issue described in:  
  2. Updates to ASMLib Configuration File "/etc/sysconfig/oracleasm" Not Effective (Doc ID 868352.1)  
第二步:reboot 操作系统.

参考文档:
[html] view plain copy
  1. NOTE:1500460.1 - Alert: After SAN Firmware Upgrade, ASM Diskgroups ( Using ASMLIB) Cannot Be Mounted Due To ORA-15085: ASM disk "" has inconsistent sector size.  
  2. NOTE:1641344.1 - Linux: After NetApp ONTAP Upgrade: CRS-1714:Unable to discover any voting files, retrying discovery in 15 seconds  
  3. NOTE:868352.1 - Updates to ASMLib Configuration File "/etc/sysconfig/oracleasm" Not Effective  
  4. NOTE:1530578.1 - New Block Size Feature for OracleASM and oracleasm-support  

译者总结:
其实这个文章说的比较绕,我个人总结如下:

前提: 

1.这个asm diskgroup用了asmlib来实现的
2.升级前:/etc/sysconfig/oracleasm文件中的ORACLEASM_USE_LOGICAL_BLOCK_SIZE等于true (意味着asm使用底层disk的logical block size)
3.asm disk 的physical block size等于4096,logical block size等于512

升级前:
kmod-oracleasm-2.0.6.rh1-2.el6.x86_64,但是该版本的kmod-oracleasm不支持ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,
也就是说Oracleasm其实还是用的4096的physical sector size,512的logical block size.其实这是4k sector size的emulition mode.


升级后:
kmod-oracleasm-2.0.8-4.el6_6.x86_64,该版本的kmod-oracleasm支持ORACLEASM_USE_LOGICAL_BLOCK_SIZE=true,
也就是说Oracleasm其实还是用的4096的physical sector size,4096的logical sector size.


问题被解决之后的状态:
/etc/sysconfig/oracleasm文件中的ORACLEASM_USE_LOGICAL_BLOCK_SIZE等于false(意味着asm使用底层disk的physical block size)
張貼者： 於 2:50 下午

沒有留言:

張貼留言

訂閱： 張貼留言 (Atom)

LinkWithin-相關文件

Related Posts Plugin for WordPress, Blogger...