登录 注册

11.2.0.4 + redhat server 6.4 + openfiler 建立RAC

本贴最后更新于 3105 天前，其中的信息可能已经水流花落

RAC 的存储划分：

  1. 创建 1G 的 lun for vote disk/ocr disk （ocrdisk:）
  2. 创建 20G 的 lun for flash recover area
  3. 创建 500G for data

所需软件：
drwxr-xr-x. 7 oracle oinstall 4096 Aug 27 12:46 database
drwxr-xr-x. 7 oracle oinstall 4096 Aug 27 12:33 grid
-rw-r--r--. 1 oracle oinstall 1395582860 Nov 17 21:01 p13390677_112040_Linux-x86-64_1of7.zip
-rw-r--r--. 1 oracle oinstall 1151304589 Nov 17 21:01 p13390677_112040_Linux-x86-64_2of7.zip
-rw-r--r--. 1 oracle oinstall 1205251894 Nov 17 21:03 p13390677_112040_Linux-x86-64_3of7.zip --grid

每个 rac 主机 hosts
[root@rac2 ~]# cat /etc/hosts
127.0.0.1 localhost.localdomain localhost

Public

192.168.12.91 rac1.localdomain rac1
192.168.12.93 rac2.localdomain rac2

Private

10.10.0.91 rac1-priv.localdomain rac1-priv
10.10.0.92 rac2-priv.localdomain rac2-priv

Virtual

192.168.12.92 rac1-vip.localdomain rac1-vip
192.168.12.94 rac2-vip.localdomain rac2-vip

SCAN

192.168.12.95 scan.localdomain scan

nas: 192.168.12.99

  1. 安装存储
    使用 iscsi　openfiler 作为共享存储，
    下载 ISO 安装操作存储系统。　下载页：　http://www.openfiler.com/
    安装完毕后访问：　https://192.168.12.99:446/ openfiler/password
    创建存储盘步骤：
    1). service menu:
    iSCSI Target > enable
    2).system->Network Access Configuration-> scan 192.168.12.0 255.255.255.0 Share
    3).block devices: volumns - > Block Devices -> Create a partition in /dev/sdb > create
    4). Volume Groups -> rac.sharedisk -> add volumn group
    5). volumns -> Add Volume
    me Name (no spaces. Valid characters [a-z,A-Z,0-9]): orcdisk1
    Volume Description: orcdisk1
    Required Space (MB): 1024
    Filesystem / Volume type: iscis

6). 创建 iscic target
system->iSCSI Targets->
重复上述动作，增加 5 个逻辑卷 orcdisk1,orcdisk2，orcdisk3,dbfile,fra

  1. 各 RAC 节点安装存储盘
    1). 各个节点安装　yum install iscsi-initiator-utils.x86_64
    2). 启动　iscsi 服务，service iscsi restart
    #chkconfig iscsi on
    3). scan iscsi
    [root@rac1 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.12.99:3260
    iscsiadm: This command will remove the record [iface: default, target: iqn.2006-01.com.openfiler:tsn.e9ccef7f9888, portal: 192.168.12.99,3260], but a session is using it. Logout session then rerun command to remove record.
    192.168.12.99:3260,1 iqn.2006-01.com.openfiler:fra
    192.168.12.99:3260,1 iqn.2006-01.com.openfiler:data
    192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk

[root@rac2 ~]# iscsiadm -m discovery -t sendtargets -p 192.168.12.99:3260
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:fra
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:dbfile
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk3
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk2
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk1

mount node:
iscsiadm -m node -T iqn.2006-01.com.openfiler:fra -p 192.168.12.99:3260 -l
iscsiadm -m node -T iqn.2006-01.com.openfiler:dbfile -p 192.168.12.99:3260 -l
iscsiadm -m node -T iqn.2006-01.com.openfiler:ocrdisk3 -p 192.168.12.99:3260 -l
iscsiadm -m node -T iqn.2006-01.com.openfiler:ocrdisk2 -p 192.168.12.99:3260 -l
iscsiadm -m node -T iqn.2006-01.com.openfiler:ocrdisk1 -p 192.168.12.99:3260 -l

查看当前会话连接的 iscsi target
iscsiadm -m node session
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:fra
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk2
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:dbfile
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk3
192.168.12.99:3260,1 iqn.2006-01.com.openfiler:ocrdisk1

#fdisk 查看　分区

如果第 ２ 步安装成功，则接着安装 RAC.

  1. 安装各个节点
    参考：　http://www.oracle-base.com/articles/rac/articles-rac.php

1）.安装 RPM 包

yum install libcap* libcap.so.1 compat-libcap1-1.10-1.x86_64 compat-libstdc++ elfutils-libelf-devel glibc-devel gcc gcc-c++ libaio-devel libstdc++-devel unixODBC unixODBC-devel sysstat binutils elfutils-libelf glibc glibc-common libaio libgcc libstdc++ make libXp libXp-devel libXau-devel

#　下载 pdksh-5.2.14.rpm
Download 安装包从 http://rpm.pbone.net/index.php3/stat/4/idpl/2394414/dir/redhat_6.x/com/pdksh-5.2.14-1.i386.rpm.html

rpm -ivh pdksh-5.2.14-1.i386.rpm

２）update system

vi /etc/sysctl.conf

echo " kernel.shmmni = 4096 " >> /etc/sysctl.conf
echo " kernel.sem = 250 32000 100 128 " >> /etc/sysctl.conf
echo " fs.file-max = 6815744 " >> /etc/sysctl.conf
echo " fs.aio-max-nr = 1048576 " >> /etc/sysctl.conf
echo " net.ipv4.ip_local_port_range = 9000 65500 " >> /etc/sysctl.conf
echo " net.core.rmem_default=262144 " >> /etc/sysctl.conf
echo " net.core.rmem_max=4194304 " >> /etc/sysctl.conf
echo " net.core.wmem_default=262144 " >> /etc/sysctl.conf
echo " net.core.wmem_max=1048586 " >> /etc/sysctl.conf

sysctl -p

3）. /etc/security/limits.conf
echo "grid soft nproc 2047 " >> /etc/security/limits.conf
echo "grid hard nproc 16384 " >> /etc/security/limits.conf
echo "grid soft nofile 1024 " >> /etc/security/limits.conf
echo "grid hard nofile 65536 " >> /etc/security/limits.conf

echo "oracle soft nproc 2047 " >> /etc/security/limits.conf
echo "oracle hard nproc 16384 " >> /etc/security/limits.conf
echo "oracle soft nofile 1024 " >> /etc/security/limits.conf
echo "oracle hard nofile 65536 " >> /etc/security/limits.conf

４）/etc/pam.d/login
echo "session required /lib64/security/pam_limits.so" >> /etc/pam.d/login

５）　vi /etc/profile,　附加以下内容

if [ $USER = "oracle" ] || [ $USER = "grid" ] ; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

６) 创建用户 oracle/grid user

groupadd -g 1000 oinstall
groupadd -g 1100 asmadmin
groupadd -g 1200 dba
groupadd -g 1201 oper
groupadd -g 1300 asmdba
groupadd -g 1301 asmoper

useradd -u 1100 -g oinstall -G asmadmin,asmdba,asmoper grid
useradd -u 1101 -g oinstall -G dba,oper,asmdba oracle

#modify password
passwd oracle
passwd grid

７）安装 rac 检测包。

2 个节点上安装 cvuqdisk-1.0.9-1.rpm (解压的 RAC 安装盘的 grid 目录内)

cd /home/oracle/software/grid
rpm -Uvh cvuqdisk-1.0.9-1.rpm

８）关闭防火墙

service iptables stop

#disable firewall
sed -ie "s#SELINUX=enforcing#SELINUX=disabled#g" /etc/selinux/config || setenforce 0
setenforce 0

10）　shutdown ntp service
service ntpd stop
mv /etc/ntp.conf /etc/ntp.conf.orig
rm /var/run/ntpd.pid
chkconfig ntpd off

#如果是使用 ntp 服务，需要增加-x
If you want to use NTP, you must add the "-x" option into the following line in the "/etc/sysconfig/ntpd" file.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"
Then restart NTP.

service ntpd restart

11)　创建主目录
mkdir -p /u01/app/11.2.0.4/grid
mkdir -p /u01/app/oracle/product/11.2.0.4/db_1
chown -R oracle:oinstall /u01
chmod -R 775 /u01/

12)设置环境变量　rac1/rac2

#su - oracle
vi /home/oracle/grid_env

ORACLE_SID=+ASM1; export ORACLE_SID
ORACLE_HOME=$GRID_HOME; export ORACLE_HOME
PATH=ORACLE_HOME/bin:BASE_PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=ORACLE_HOME/JRE:ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

$ chmod u+x grid_env

su - oracle

vi /home/oracle/.bashrc

Oracle Settings

TMP=/tmp; export TMP
TMPDIR=$TMP; export TMPDIR

ORACLE_HOSTNAME=rac1; export ORACLE_HOSTNAME
ORACLE_UNQNAME=RAC; export ORACLE_UNQNAME
ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
GRID_HOME=/u01/app/11.2.0.4/grid; export GRID_HOME
DB_HOME=$ORACLE_BASE/product/11.2.0.4/db_1; export DB_HOME
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
ORACLE_SID=RAC1; export ORACLE_SID
ORACLE_TERM=xterm; export ORACLE_TERM
BASE_PATH=/usr/sbin:$PATH; export BASE_PATH
PATH=ORACLE_HOME/bin:BASE_PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=ORACLE_HOME/JRE:ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

if [ $USER = "oracle" ]; then
if [ $SHELL = "/bin/ksh" ]; then
ulimit -p 16384
ulimit -n 65536
else
ulimit -u 16384 -n 65536
fi
fi

alias grid_env='. /home/oracle/grid_env'
alias db_env='. /home/oracle/db_env'

$ vi /home/oracle/db_env
ORACLE_SID=RAC1; export ORACLE_SID
ORACLE_HOME=$DB_HOME; export ORACLE_HOME
PATH=ORACLE_HOME/bin:BASE_PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
CLASSPATH=ORACLE_HOME/JRE:ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

#chech network
ifconfig -a | grep eth

  1. 创建用户等效性
    ２ 个节点上执行如下
    $
    mkdir ~/.ssh
    chmod 755 ~/.ssh
    ssh-keygen -t dsa

#将 rac1/rac2 节点的公钥拷贝到 rac1 节点的　authorized_keys
rac1$
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh oracle@rac2 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

#将 rac1/rac2 节点的公钥拷贝到 rac2 节点的　authorized_keys
rac2$
cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
ssh oracle@rac1 cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

检查环境

rac1
cd /home/oracle/grid
./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -fixup -verbose

  1. udev 绑定磁盘
    先将磁盘分区。　再 udev
    http://www.oracle-base.com/articles/linux/udev-scsi-rules-configuration-in-oracle-linux-5-and-6.php

echo "options=-g" >> /etc/scsi_id.config

create udev file

[root@rac1 ~]# /sbin/scsi_id -g -u -d /dev/sdb
14f504e46494c455248384a34676d2d445a474f2d69676250
[root@rac1 ~]# /sbin/scsi_id -g -u -d /dev/sdc
14f504e46494c45526c48306b36492d673370332d6631474f
[root@rac1 ~]# /sbin/scsi_id -g -u -d /dev/sdd
14f504e46494c455256335463674e2d53524e632d73574d54
[root@rac1 ~]# /sbin/scsi_id -g -u -d /dev/sde
14f504e46494c45524f47546550702d6f5541572d31397862
[root@rac1 ~]# /sbin/scsi_id -g -u -d /dev/sdf
14f504e46494c4552456d56624b352d42327a762d624a314e

vi /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c455248384a34676d2d445a474f2d69676250", NAME="asm-votedisk-1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526c48306b36492d673370332d6631474f", NAME="asm-votedisk-2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45524f47546550702d6f5541572d31397862", NAME="asm-votedisk-3", OWNER="oracle", GROUP="dba", MODE="0660"

KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c455256335463674e2d53524e632d73574d54", NAME="asm-dbfile", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c4552456d56624b352d42327a762d624a314e", NAME="asm-fra", OWNER="oracle", GROUP="dba", MODE="0660"

/sbin/start_udev

#检查 ams 的影射
ll /dev/asm-votedisk-

  1. 安装集群件
  1. 安装 grid 安装 votedisk/ocr 盘
    http://www.oracle-base.com/articles/rac/articles-rac.php
    http://www.oracle-base.com/articles/11g/oracle-db-11gr2-rac-installation-on-linux-using-nfs.php
    ./runInstaller
    Select the "Install and Configure Grid Infrastructure for a Cluster" option, then click the "Next" button.

Select the "Advanced Installation" option, then click the "Next" button.
Grid - Select Product Languages
Enter cluster information and uncheck the "Configure GNS" option, then click the "Next" button.

asm 搜索 path: /dev/asm*

  1. 只安装 database 软件
  2. asmca 创建 FR,DATA2 个目录, dbca 创建数据库。
  • Oracle

    Oracle（甲骨文）公司，全称甲骨文股份有限公司（甲骨文软件系统有限公司），是全球最大的企业级软件公司，总部位于美国加利福尼亚州的红木滩。1989 年正式进入中国市场。2013 年，甲骨文已超越 IBM，成为继 Microsoft 后全球第二大软件公司。

    105 引用 • 127 回帖 • 370 关注
1 赞同
743 4 115 143 46 108 44 257

相关帖子

回帖
11.2.0.4 + redhat server 6.4 + openfiler 建立RAC

欢迎来到这里！

我们正在构建一个小众社区，大家在这里相互信任，以平等 • 自由 • 奔放的价值观进行分享交流。最终，希望大家能够找到与自己志同道合的伙伴，共同成长。

注册 关于
请输入回帖内容 ...
cactus0509
回帖
0
 帖子
21
 积分
2005