IT 研究室 ( 前IT DBA's 資訊站): 安裝 Oracle 10g Database R2 RAC for RHEL5 x86

安裝 Oracle 10g Database R2 RAC for RHEL5 x86_64 步驟

Oracle 10g RAC install steps at RHEL5 (ocfs cluster filesystem)

I. 安裝Oracle 10g Clusterware , patch to 10.2.0.4 約需 1500MB,

Oracle 10g Database R2 , patch to 10.2.0.4 約需 1900 MB, 全程費時約 4 小時.

1.前置作業

1.1 檢查OS版本

#cat /etc/redhat-release

Red Hat Enterprise Linux Server release 5.5 (Tikanga)

1.2 rpm檢查 (檢查以下rpm是否有安裝)

binutils-2.17.50 (x86_64)
compat-db-4.2.52 (x86_64)
compat-libstdc++-296(i386)
compat-libstdc++-33-3.2.3 (x86_64)
compat-libstdc++-33-3.2.3(i386)
control-center-2.16.0 (x86_64)
gcc-4.1.1 (x86_64)
gcc-c++-4.1.1 (x86_64)
glibc-2.5-12 (x86_64)
glibc-2.5-12 (i686)
glibc-common-2.5-12 (x86_64)
glibc-devel-2.5-12 (x86_64)
glibc-devel-2.5-12(i386)
glibc-headers-2.5-12 (x86_64)
ksh-20060214-1.4 (x86_64)
libaio-0.3.96 (x86_64)
libgcc-4.1.1(i386)
libgcc-4.1.1(x86_64)
libgnome-2.16.0 (x86_64)
libgnomeui-2.16.0 (x86_64)
libgomp-4.1.1 (x86_64)
libstdc++-4.1.1 (x86_64)
libstdc++-devel-4.1.1 (x86_64)
libXp-1.0.0-8 (i386)
make-3.81 (x86_64)
sysstat-7.0.0 (x86_64)

檢查:

|sort

另外需注意安裝

libXp-1.0.0-8.i386.rpm (java plugin required)

sysstat-7.0.2-1.el5.i386.rpm (iostat)

1.3 修改kernel參數

vi /etc/sysctl.conf

kernel.shmmax = 4136632320 (physical RAM in bytes , 在這邊為4GB)

kernel.shmall =1009920 (kernel.shmmax / 4096)

net.core.rmem_default = 1048576

net.core.rmem_max = 1048576

net.core.wmem_default = 262144

net.core.wmem_max = 262144

net.ipv4.ip_local_port_range=1024 65000

kernel.shmmni = 4096

kernel.sem= 250 32000 32 1024

fs.file-max = 65536

修改完後以root執行 sysctl -p (使參數生效)

Note:

rmem_default bytes of the socket receive buffer

rmem_max max bytes of the socket receive buffer

wmem_default bytes of the socket send buffer

wmem_max max bytes of the socket send buffer

其中 /proc/sys/kernel/sem 的四個參數分別為

SEMMSL 250 10 + max(#(PROCESSES)) of any OracleDB on the system

SEMMNS 1000 SUM(#(PROCESSES) of OracleDBs on the system) + max(#(PROCESSES)) + (10 * #(DB on the system)

SEMOPM 100 max(#(operations per semop call))

SEMMNI 100 max(#(semaphore sets in the entire system))

1.4 設定session限制

Add the following lines to the /etc/security/limits.conf file

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

Add the following line in the /etc/pam.d/login file

session required pam_limits.so

Add the following lines to the /etc/profile.

if [ $USER = "oracle" ]; then

ulimit -u 16384

ulimit -n 65536

1.5 新增使用者及群組

groupadd -g 500 dba

useradd -g dba -u 500 -s /bin/bash -d /home/oracle -m oracle

mkdir /oracle /oracle/db102 /oracle/crs102

chown -R oracle:dba /oracle

1.6 修改oracle環境參數檔

su - oracle; vi ~/.bash_profile

ORACLE_BASE=/oracle; export ORACLE_BASE

ORACLE_HOME=$ORACLE_BASE/db102; export ORACLE_HOME

ORACLE_SID=orcl; export ORACLE_SID

ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data; export ORA_NLS33

NLS_LANG=AMERICAN_AMERICA.ZHT16MSWIN950; export NLS_LANG

NLS_DATE_FORMAT='YYYYMMDD HH24:MI:SS'; export NLS_DATE_FORMAT

PATH=$ORACLE_HOME/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin:.; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:/usr/local/lib; export LD_LIBRARY_PATH

TMP=/tmp; export TMP

TMPDIR=$TMP; export TMPDIR

1.7 修改hosts (at each db node)

vi /etc/hosts

172.16.0.16 dbsv03

172.16.0.17 dbsv04

172.16.0.18 dbsv03-vip

172.16.0.19 dbsv04-vip

192.168.0.3 heartbeat1

192.168.0.4 heartbeat2

1.8 設定ssh authication(for dbsv03 , dbsv04)

1.8.1 For root authication

----------------------------------------------------------------------------------

db1:

ssh-keygen -t rsa

cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

scp -r ~/.ssh root@dbsv04:

The authenticity of host 'dbsv2 (172.16.0.17)' can't be established.

RSA key fingerprint is fd:d4:7d:38:e8:45:fb:b4:80:a9:17:97:fa:e9:8b:9e.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'dbsv04,172.16.0.17' (RSA) to the list of known hosts.

root@dbsv04's password:root123

id_rsa.pub 100% 392 0.4KB/s 00:00

known_hosts 100% 399 0.4KB/s 00:00

id_rsa 100% 1675 1.6KB/s 00:00

authorized_keys 100% 392 0.4KB/s 00:00

db1:

scp ~/.ssh/id_rsa.pub root@dbsv04:tempid

db2:

cat ~/tempid >> ~/.ssh/authorized_keys

scp ~/.ssh/id_rsa.pub root@dbsv03:tempid

db1:

cat ~/tempid >> ~/.ssh/authorized_keys

----------------------------------------------------------------------------------

1.8.2 For oracle user authication

passwd oracle

: oracle

su - oracle

ssh-keygen -t rsa

cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys

scp -r ~/.ssh oracle@dbsv04:

The authenticity of host 'dbsv04 (172.16.0.17)' can't be established.

RSA key fingerprint is fd:d4:7d:38:e8:45:fb:b4:80:a9:17:97:fa:e9:8b:9e.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'dbsv04,172.16.0.17' (RSA) to the list of known hosts.

oracle@dbsv04's password:oracle

id_rsa.pub 100% 392 0.4KB/s 00:00

known_hosts 100% 399 0.4KB/s 00:00

id_rsa 100% 1675 1.6KB/s 00:00

authorized_keys 100% 392 0.4KB/s 00:00

db1:

scp ~/.ssh/id_rsa.pub oracle@dbsv04:tempid

db2:

cat ~/tempid >> ~/.ssh/authorized_keys

scp ~/.ssh/id_rsa.pub oracle@dbsv03:tempid

db1:

cat ~/tempid >> ~/.ssh/authorized_keys

verify: 以root 或oracle使用者執行以下指令，確認可免密碼連結

ssh dbsv03

ssh dbsv04

2.安裝資料庫軟體

2.1 安裝10.2.0.1 clusterware

gunzip 10201_clusterware_linux_x86_64.cpio.gz

mv 10201_clusterware_linux_x86_64.cpio ./crs10201

cpio -idmv < 10201_clusterware_linux_x86_64.cpio

as root:

/oracle/source/crs10201/clusterware/rootpre/rootpre.sh

$ ./runInstaller -ignoreSysPrereqs

********************************************************************************

Please run the script rootpre.sh as root on all machines/nodes. The script can be found at the toplevel of the CD or stage-area. Once you have run the script, please type Y to proceed

Answer 'y' if root has run 'rootpre.sh' so you can proceed with Oracle Clusterware installation.

Answer 'n' to abort installation and then ask root to run 'rootpre.sh'.

********************************************************************************

Has 'rootpre.sh' been run by root? [y/n] (n)

Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be redhat-3, SuSE-9, redhat-4, UnitedLinux-1.0, asianux-1 or asianux-2

Failed <<<<

>>> Ignoring required pre-requisite failures. Continuing...

點選User Verified

[root@dbsv03 oradata]# /oracle/crs102/root.sh

WARNING: directory '/oracle' is not owned by root

Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory

Setting up NS directories

Oracle Cluster Registry configuration upgraded successfully

WARNING: directory '/oracle' is not owned by root

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: dbsv03 heartbeat1 dbsv03

node 2: dbsv04 heartbeat2 dbsv04

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Now formatting voting device: /oradata/voting.0

Now formatting voting device: /oradata/voting.1

Now formatting voting device: /oradata/voting.2

Format of 3 voting devices complete.

Startup will be queued to init within 90 seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

dbsv03

CSS is inactive on these nodes.

dbsv04

Local node checking complete.

Run root.sh on remaining nodes to start CRS daemons.

[root@dbsv04 crs102]# ./root.sh

WARNING: directory '/oracle' is not owned by root

Checking to see if Oracle CRS stack is already configured

Setting the permissions on OCR backup directory

Setting up NS directories

Oracle Cluster Registry configuration upgraded successfully

WARNING: directory '/oracle' is not owned by root

clscfg: EXISTING configuration version 3 detected.

clscfg: version 3 is 10G Release 2.

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: dbsv03 heartbeat1 dbsv03

node 2: dbsv04 heartbeat2 dbsv04

clscfg: Arguments check out successfully.

NO KEYS WERE WRITTEN. Supply -force parameter to override.

-force is destructive and will destroy any previous cluster

configuration.

Oracle Cluster Registry for cluster has already been initialized

Startup will be queued to init within 90 seconds.

Adding daemons to inittab

Expecting the CRS daemons to be up within 600 seconds.

CSS is active on these nodes.

dbsv03

dbsv04

CSS is active on all nodes.

Waiting for the Oracle CRSD and EVMD to start

Oracle CRS stack installed and running under init(1M)

Running vipca(silent) for configuring nodeapps

Error 0(Native: listNetInterfaces:[3])

[Error 0(Native: listNetInterfaces:[3])]

Error 0(Native: listNetInterfaces:[3])

[Error 0(Native: listNetInterfaces:[3])] 為redhate 5 上安裝Oracle RAC會發生的訊息，

需要執行以下動作,設定vipca才會正常

# /oracle/crs102/bin/vipca
Error 0(Native: listNetInterfaces:[3])
[Error 0(Native: listNetInterfaces:[3])]

Remember to re-edit these files on all nodes:
<CRS_HOME>/bin/vipca
<CRS_HOME>/bin/srvctl
<RDBMS_HOME>/bin/srvctl
<ASM_HOME>/bin/srvctl

if [ "$arch" = "i686" -o "$arch" = "ia64" -o "$arch" = "x86_64" ]
then

  LD_ASSUME_KERNEL=2.4.19

  export LD_ASSUME_KERNEL

fi

unset LD_ASSUME_KERNEL <<<== Line to be added

then cluvfy will fail due to vipca not completed successfully

手動設定

<CRS_HOME>/bin # ./oifcfg setif -global bond0/172.16.0.0:public
<CRS_HOME>/bin # ./oifcfg setif -global bond1/192.168.0.0:cluster_interconnect
<CRS_HOME>/bin # ./oifcfg getif
bond0 172.16.0.0 global public

bond1 192.168.0.0 global cluster_interconnect

再執行 vipca

2.2 安裝cluster patches (10.2.0.4)

unzip p6810189_10204_Linux-x86-64.zip

./Disk1/runInstaller

[oracle@dbsv03 10204]$ ./Disk1/runInstaller

Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be redhat-3, SuSE-9, SuSE-10, redhat-4, redhat-5, UnitedLinux-1.0, asianux-1, asianux-2 or asianux-3

Passed

All installer requirements met.

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-01-06_02-08-45PM. Please wait ...[oracle@dbsv03 10204]$ Oracle Universal Installer, Version 10.2.0.4.0 Production

[root@dbsv03 bin]# /oracle/crs102/bin/crsctl stop crs

Stopping resources.

Successfully stopped CRS resources

Stopping CSSD.

Shutting down CSS daemon.

Shutdown request successfully issued.

[root@dbsv03 crs102]# /oracle/crs102/install/root102.sh

Creating pre-patch directory for saving pre-patch clusterware files

Completed patching clusterware files to /oracle/crs102

Relinking some shared libraries.

Relinking of patched files is complete.

WARNING: directory '/oracle' is not owned by root

Preparing to recopy patched init and RC scripts.

Recopying init and RC scripts.

Startup will be queued to init within 30 seconds.

Starting up the CRS daemons.

Waiting for the patched CRS daemons to start.

This may take a while on some systems.

10204 patch successfully applied.

clscfg: EXISTING configuration version 3 detected.

clscfg: version 3 is 10G Release 2.

Successfully accumulated necessary OCR keys.

Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.

node <nodenumber>: <nodename> <private interconnect name> <hostname>

node 1: dbsv03 heartbeat1 dbsv03

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

clscfg -upgrade completed successfully

於dbsv03 , dbsv04 需安裝glibc-devel-2.5-49.i386.rpm後，再執行vipca

2.3 建立RAC vip

[root@dbsv03 bin]# ./vipca

檢查resource是否已經加入:

以root執行/oracle/crs102/bin/crs_stat -t

2.4 安裝10.2.0.1 database

gunzip 10201_database_linux_x86_64.cpio.gz

cpio -idmv < 10201_database_linux_x86_64.cpio.gz

./runInstaller -ignoreSysPrereqs

2.5 安裝database patches (10.2.0.4)

$ ./runInstaller

Starting Oracle Universal Installer...

Checking installer requirements...

Checking operating system version: must be redhat-3, SuSE-9, SuSE-10, redhat-4, redhat-5, UnitedLinux-1.0, asianux-1, asianux-2 or asianux-3

Passed

All installer requirements met.

Preparing to launch Oracle Universal Installer from /tmp/OraInstall2011-01-06_04-09-03PM. Please wait ...[oracle@dbsv03 Disk1]$ Oracle Universal Installer, Version 10.2.0.4.0 Production