【博客文章2025】Oracle Database 23ai（23.5.0.24.07）RAC集群（网格基础架构部分）
1. 起始环境：
本博客使用一套Oracle Database 19c（19.0.0.0.0）作为起始环境，将其升级为23.5.0.24.07：
[grid@station11 ~]$ crsctl query crs softwareversion -all
Oracle Clusterware version on node [station11] is [19.0.0.0.0]
Oracle Clusterware version on node [station12] is [19.0.0.0.0]
|
查看集群详细情况：
[grid@station11 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.chad
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.net1.network
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.ons
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE station12 STABLE
3 ONLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE station12 STABLE
3 OFFLINE OFFLINE STABLE
ora.FRA.dg(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE station12 STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE station11 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE station12 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE station12 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE station12 169.254.1.204 172.31
.118.12 172.31.118.2
12,STABLE
ora.asm(ora.asmgroup)
1 ONLINE OFFLINE Instance Shutdown,ST
ABLE
2 ONLINE ONLINE station12 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE OFFLINE STABLE
2 ONLINE ONLINE station12 STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE station12 STABLE
ora.mgmtdb
1 ONLINE ONLINE station12 Open,STABLE
ora.qosmserver
1 ONLINE ONLINE station12 STABLE
ora.racorcl.db
1 ONLINE ONLINE station11 Open,HOME=/u01/app/o
racle/product/19.3.0
/dbhome_1,STABLE
2 ONLINE ONLINE station12 Open,HOME=/u01/app/o
racle/product/19.3.0
/dbhome_1,STABLE
ora.scan1.vip
1 ONLINE ONLINE station11 STABLE
ora.scan2.vip
1 ONLINE ONLINE station12 STABLE
ora.scan3.vip
1 ONLINE ONLINE station12 STABLE
ora.station11.vip
1 ONLINE ONLINE station11 STABLE
ora.station12.vip
1 ONLINE ONLINE station12 STABLE
--------------------------------------------------------------------------------
|
所有节点操作系统均为Oracle Enterprise Linux 8.3：
[grid@station11 ~]$ uname -a
Linux station11.lab.example.com 5.4.17-2011.7.4.el8uek.x86_64 #2 SMP Fri Oct 2 14:39:04 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
|
2. 操作系统类操作（本类操作，所有节点都要操作一遍!）：
2.1 操作系统---升级操作系统（至少4个包）： 要挂载Oracle Enterprise Linux 8.10的光盘，配置yum源，升级以下这些组件：
binutils:
[root@station11 ~]# yum upgrade binutils.x86_64
BaseOS 2.3 MB/s | 2.3 kB 00:00
BaseOS 167 MB/s | 3.5 MB 00:00
AppStream 2.7 MB/s | 2.8 kB 00:00
AppStream 180 MB/s | 8.7 MB 00:00
Last metadata expiration check: 0:00:01 ago on Wed 05 Mar 2025 12:11:41 PM CST.
Dependencies resolved.
===================================================================================================
Package Architecture Version Repository Size
===================================================================================================
Upgrading:
binutils x86_64 2.30-123.0.2.el8 BaseOS 5.9 M
Transaction Summary
===================================================================================================
Total size: 5.9 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Upgrading : binutils-2.30-123.0.2.el8.x86_64 1/2
Running scriptlet: binutils-2.30-123.0.2.el8.x86_64 1/2
Running scriptlet: binutils-2.30-79.0.1.el8.x86_64 2/2
Cleanup : binutils-2.30-79.0.1.el8.x86_64 2/2
Running scriptlet: binutils-2.30-79.0.1.el8.x86_64 2/2
Verifying : binutils-2.30-123.0.2.el8.x86_64 1/2
Verifying : binutils-2.30-79.0.1.el8.x86_64 2/2
Upgraded:
binutils-2.30-123.0.2.el8.x86_64
Complete!
|
libgcc:
[root@station11 ~]# yum upgrade libgcc.x86_64
Last metadata expiration check: 0:06:30 ago on Wed 05 Mar 2025 12:11:41 PM CST.
Dependencies resolved.
====================================================================================================
Package Architecture Version Repository Size
====================================================================================================
libgcc x86_64 8.5.0-21.0.1.el8 BaseOS 92 k
Transaction Summary
====================================================================================================
Upgrade 1 Package
Total size: 92 k
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Upgrading : libgcc-8.5.0-21.0.1.el8.x86_64 1/2
Running scriptlet: libgcc-8.5.0-21.0.1.el8.x86_64 1/2
Cleanup : libgcc-8.3.1-5.1.0.1.el8.x86_64 2/2
Running scriptlet: libgcc-8.3.1-5.1.0.1.el8.x86_64 2/2
Verifying : libgcc-8.5.0-21.0.1.el8.x86_64 1/2
Verifying : libgcc-8.3.1-5.1.0.1.el8.x86_64 2/2
Upgraded:
libgcc-8.5.0-21.0.1.el8.x86_64
Complete!
|
libstdc++*:
[root@station11 ~]# yum upgrade libstdc++*
Last metadata expiration check: 0:11:44 ago on Wed 05 Mar 2025 12:11:41 PM CST.
Dependencies resolved.
=====================================================================================================
Package Architecture Version Repository Size
=====================================================================================================
Upgrading:
cpp x86_64 8.5.0-21.0.1.el8 AppStream 10 M
gcc x86_64 8.5.0-21.0.1.el8 AppStream 23 M
gcc-c++ x86_64 8.5.0-21.0.1.el8 AppStream 12 M
gcc-gdb-plugin x86_64 8.5.0-21.0.1.el8 AppStream 130 k
libgomp x86_64 8.5.0-21.0.1.el8 BaseOS 218 k
libstdc++ x86_64 8.5.0-21.0.1.el8 BaseOS 465 k
libstdc++-devel x86_64 8.5.0-21.0.1.el8 AppStream 2.1 M
Transaction Summary
=====================================================================================================
Total size: 49 M
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Upgrading : libstdc++-8.5.0-21.0.1.el8.x86_64 1/14
Running scriptlet: libstdc++-8.5.0-21.0.1.el8.x86_64 1/14
Upgrading : libstdc++-devel-8.5.0-21.0.1.el8.x86_64 2/14
Upgrading : cpp-8.5.0-21.0.1.el8.x86_64 3/14
Running scriptlet: cpp-8.5.0-21.0.1.el8.x86_64 3/14
Upgrading : libgomp-8.5.0-21.0.1.el8.x86_64 4/14
Running scriptlet: libgomp-8.5.0-21.0.1.el8.x86_64 4/14
Upgrading : gcc-8.5.0-21.0.1.el8.x86_64 5/14
Running scriptlet: gcc-8.5.0-21.0.1.el8.x86_64 5/14
Upgrading : gcc-c++-8.5.0-21.0.1.el8.x86_64 6/14
Upgrading : gcc-gdb-plugin-8.5.0-21.0.1.el8.x86_64 7/14
Running scriptlet: gcc-gdb-plugin-8.5.0-21.0.1.el8.x86_64 7/14
Cleanup : gcc-gdb-plugin-8.3.1-5.1.0.1.el8.x86_64 8/14
Running scriptlet: gcc-gdb-plugin-8.3.1-5.1.0.1.el8.x86_64 8/14
Cleanup : gcc-c++-8.3.1-5.1.0.1.el8.x86_64 9/14
Cleanup : libstdc++-devel-8.3.1-5.1.0.1.el8.x86_64 10/14
Running scriptlet: gcc-8.3.1-5.1.0.1.el8.x86_64 11/14
Cleanup : gcc-8.3.1-5.1.0.1.el8.x86_64 11/14
Running scriptlet: cpp-8.3.1-5.1.0.1.el8.x86_64 12/14
Cleanup : cpp-8.3.1-5.1.0.1.el8.x86_64 12/14
Running scriptlet: libgomp-8.3.1-5.1.0.1.el8.x86_64 13/14
Cleanup : libgomp-8.3.1-5.1.0.1.el8.x86_64 13/14
Running scriptlet: libgomp-8.3.1-5.1.0.1.el8.x86_64 13/14
Cleanup : libstdc++-8.3.1-5.1.0.1.el8.x86_64 14/14
Running scriptlet: libstdc++-8.3.1-5.1.0.1.el8.x86_64 14/14
Verifying : libgomp-8.5.0-21.0.1.el8.x86_64 1/14
Verifying : libgomp-8.3.1-5.1.0.1.el8.x86_64 2/14
Verifying : libstdc++-8.5.0-21.0.1.el8.x86_64 3/14
Verifying : libstdc++-8.3.1-5.1.0.1.el8.x86_64 4/14
Verifying : cpp-8.5.0-21.0.1.el8.x86_64 5/14
Verifying : cpp-8.3.1-5.1.0.1.el8.x86_64 6/14
Verifying : gcc-8.5.0-21.0.1.el8.x86_64 7/14
Verifying : gcc-8.3.1-5.1.0.1.el8.x86_64 8/14
Verifying : gcc-c++-8.5.0-21.0.1.el8.x86_64 9/14
Verifying : gcc-c++-8.3.1-5.1.0.1.el8.x86_64 10/14
Verifying : gcc-gdb-plugin-8.5.0-21.0.1.el8.x86_64 11/14
Verifying : gcc-gdb-plugin-8.3.1-5.1.0.1.el8.x86_64 12/14
Verifying : libstdc++-devel-8.5.0-21.0.1.el8.x86_64 13/14
Verifying : libstdc++-devel-8.3.1-5.1.0.1.el8.x86_64 14/14
Upgraded:
cpp-8.5.0-21.0.1.el8.x86_64 gcc-8.5.0-21.0.1.el8.x86_64 gcc-c++-8.5.0-21.0.1.el8.x86_64 gcc-gdb-plugin-8.5.0-21.0.1.el8.x86_64 libgomp-8.5.0-21.0.1.el8.x86_64 libstdc++-8.5.0-21.0.1.el8.x86_64
libstdc++-devel-8.5.0-21.0.1.el8.x86_64
Complete!
|
nfs-utils:
[root@station11 ~]# yum upgrade nfs-utils.x86_64
Last metadata expiration check: 0:13:11 ago on Wed 05 Mar 2025 12:11:41 PM CST.
Dependencies resolved.
=======================================================================================================
Package Architecture Version Repository Size
=======================================================================================================
Upgrading:
nfs-utils x86_64 1:2.3.3-59.0.1.el8 BaseOS 514 k
Transaction Summary
=======================================================================================================
Upgrade 1 Package
Total size: 514 k
Is this ok [y/N]: y
Downloading Packages:
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: nfs-utils-1:2.3.3-59.0.1.el8.x86_64 1/1
Running scriptlet: nfs-utils-1:2.3.3-59.0.1.el8.x86_64 1/2
Upgrading : nfs-utils-1:2.3.3-59.0.1.el8.x86_64 1/2
Running scriptlet: nfs-utils-1:2.3.3-59.0.1.el8.x86_64 1/2
Running scriptlet: nfs-utils-1:2.3.3-35.el8.x86_64 2/2
Cleanup : nfs-utils-1:2.3.3-35.el8.x86_64 2/2
Running scriptlet: nfs-utils-1:2.3.3-35.el8.x86_64 2/2
Verifying : nfs-utils-1:2.3.3-59.0.1.el8.x86_64 1/2
Verifying : nfs-utils-1:2.3.3-35.el8.x86_64 2/2
Upgraded:
nfs-utils-1:2.3.3-59.0.1.el8.x86_64
Complete!
|
2.2 操作系统---恢复scp： 在RHEL8系列的操作系统上安装19c，必需做：
[root@station11 ~]# which scp
/usr/bin/scp
[root@station11 ~]# ls -l /usr/bin/scp*
-rwxr-xr-x. 1 root root 23 Jan 1 2024 /usr/bin/scp
-rwxr-xr-x. 1 root root 90 May 11 2019 /usr/bin/scp-dbus-service
-rwxr-xr-x. 1 root root 105440 Jul 31 2020 /usr/bin/scp.ori
[root@station11 ~]# cat /usr/bin/scp
/usr/bin/scp.ori -T $*
|
安装23ai前，必需恢复scp：
[root@station11 ~]# rm /usr/bin/scp
rm: remove regular file '/usr/bin/scp'? y
[root@station11 ~]# mv /usr/bin/scp.ori /usr/bin/scp
|
2.3 操作系统---内存大页面配置： 将下面这段代码，保存成shell脚本hugepage.sh：
#!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
if [ $MIN_PG -gt 0 ]; then
NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
fi
done
# Finish with results
case $KERN in
'2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
'2.6' | '3.8' | '3.10' | '5.4' ) echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
*) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End
|
并执行：
[root@station11 ~]# ./hugepage.sh
Recommended setting: vm.nr_hugepages = 2366
|
配置/etc/sysctl.conf：
[root@station11 ~]# echo vm.nr_hugepages = 2366 >> /etc/sysctl.conf
[root@station11 ~]# sysctl -p
fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmall = 3355443
kernel.shmmni = 4096
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 1048576
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 11000 65000
fs.aio-max-nr = 1048576
kernel.shmmax = 13743895347
net.ipv4.conf.ens10.rp_filter = 2
net.ipv4.conf.ens11.rp_filter = 2
vm.nr_hugepages = 2366
[root@station11 ~]# grep -i hugepage /proc/meminfo
AnonHugePages: 1583104 kB
ShmemHugePages: 0 kB
FileHugePages: 0 kB
HugePages_Total: 1421
HugePages_Free: 1421
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
|
2.4 操作系统---禁止透明内存大页面配置： 执行：
[root@station11 ~]# echo never >> /sys/kernel/mm/transparent_hugepage/enabled
[root@station11 ~]# echo never >> /sys/kernel/mm/transparent_hugepage/defrag
|
编辑/etc/default/grub文件，在GRUB_CMDLINE_LINUX中添加transparent_hugepage=never：
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto resume=UUID=923d0eee-0295-41f4-96b2-fc501f71f38e rhgb transparent_hugepage=never quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true
|
如果是BIOS机型，配置grub.cfg：
[root@station11 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
done
|
如果是UEFI机型，配置grub.cfg：
[root@station11 ~]# grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
Generating grub configuration file ...
done
|
2.5 操作系统---添加oracle用户进asmadmin组：
[root@station11 ~]# id oracle
uid=500(oracle)
gid=500(oinstall)
groups=500(oinstall),501(dba),502(oper),503(backupdba),504(dgdba),505(kmdba),506(racdba),2004(asmdba)
[root@station11 ~]# gpasswd -a oracle asmadmin
Adding user oracle to group asmadmin
|
2.6 操作系统---准备安装目录：
[root@station11 ~]# cd /u01/app/
[root@station11 app]# chmod g+w ./
[root@station11 app]# su - grid
Last login: Wed Mar 5 18:06:50 CST 2025
[grid@station11 ~]$ cd /u01/app/
[grid@station11 app]$ mkdir -p 23.5.0/grid
|
在第集群第1个节点的/u01/app/23.5.0/grid/下，以grid用户解压缩23ai INFRASTRUCTURE软件（本操作，不需在其他节点进行）： [grid@station11 grid]$ unzip /stage/INFRA/V1043786-01.zip
|
3. 集群件其他资源类操作（本类操作，只要在集群的第一个节点都要操作）：
3.1 集群件其他资源---删除mgmtdb（root用户执行）：
[root@station11 ~]# . oraenv
ORACLE_SID = [+ASM1] ?
The Oracle base remains unchanged with value /u01/app/grid
[root@station11 ~]# srvctl status mgmtdb
Database is enabled
Instance -MGMTDB is running on node station11
[root@station11 ~]# srvctl stop mgmtdb -f
[root@station11 ~]# srvctl remove mgmtdb -f
[root@station11 ~]# srvctl stop mgmtlsnr -f
[root@station11 ~]# srvctl remove mgmtlsnr -f
|
3.2 集群件其他资源---删除mgmtdb（ASM用户执行）：
[grid@station11 ~]$ asmcmd
ASMCMD> ls
DATA/
FRA/
ASMCMD> cd data
ASMCMD> ls
ASM/
RACORCL/
_MGMTDB/
cluster11/
orapwasm
orapwasm_backup
ASMCMD> rm -r _MGMTDB/
You may delete multiple files and/or directories.
Are you sure? (y/n) y
|
3.3 集群件其他资源--- 如果内网网卡有多个，23ai不允许它们是同一个网段：
[root@station11 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/grid
[root@station11 ~]# oifcfg getif
ens10 172.31.118.0 global cluster_interconnect,asm
ens11 172.31.118.0 global cluster_interconnect,asm
ens3 172.25.250.0 global public
[root@station11 ~]# oifcfg delif -global ens11/172.31.118.0
[root@station11 ~]# oifcfg setif -global ens11/172.19.3.0:cluster_interconnect,asm
[root@station11 ~]# oifcfg getif
ens10 172.31.118.0 global cluster_interconnect,asm
ens3 172.25.250.0 global public
ens11 172.19.3.0 global cluster_interconnect,asm
|
[root@station11 ~]# nmcli con mod ens11 ipv4.addresses 172.19.3.11/24
[root@station11 ~]# nmcli con up ens11
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
|
[root@station12 ~]# nmcli con mod ens11 ipv4.addresses 172.19.3.12/24
[root@station12 ~]# nmcli con up ens11
Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/6)
[root@station12 ~]# ping 172.19.3.11
PING 172.19.3.11 (172.19.3.11) 56(84) bytes of data.
64 bytes from 172.19.3.11: icmp_seq=1 ttl=64 time=0.332 ms
64 bytes from 172.19.3.11: icmp_seq=2 ttl=64 time=0.212 ms
64 bytes from 172.19.3.11: icmp_seq=3 ttl=64 time=0.324 ms
64 bytes from 172.19.3.11: icmp_seq=4 ttl=64 time=0.348 ms
64 bytes from 172.19.3.11: icmp_seq=5 ttl=64 time=0.225 ms
^C
--- 172.19.3.11 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 132ms
|
4. ASM类操作（本类操作，只要在集群的第一个节点都要操作）：
4.1 ASM---ASM实例增加"_exadata_feature_on"参数：
[grid@station11 ~]$ sqlplus /nolog
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 5 13:17:58 2025
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
SQL> conn / as sysasm
Connected.
SQL> alter system set "_exadata_feature_on"=true scope=spfile;
System altered.
|
[grid@station11 ~]$ sqlplus /nolog
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 5 14:02:29 2025
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
SQL> conn / as sysasm
Connected.
SQL> alter diskgroup data set attribute 'compatible.rdbms'='19.0.0.0.0';
Diskgroup altered.
SQL> alter diskgroup fra set attribute 'compatible.rdbms'='19.0.0.0.0';
Diskgroup altered.
|
5. DATABASE---DB实例增加"_exadata_feature_on"参数：
[oracle@station11 ~]$ sqlplus /nolog
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 5 13:20:02 2025
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
SQL> conn / as sysdba
Connected.
SQL> alter system set "_exadata_feature_on"=true scope=spfile;
System altered.
|
6. 重新启动集群：
关闭集群：
[root@station11 ~]# crsctl stop cluster -all
CRS-2673: Attempting to stop 'ora.crsd' on 'station11'
CRS-2673: Attempting to stop 'ora.crsd' on 'station12'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'station11'
CRS-2673: Attempting to stop 'ora.racorcl.db' on 'station11'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'station12'
CRS-2673: Attempting to stop 'ora.racorcl.db' on 'station12'
CRS-2673: Attempting to stop 'ora.chad' on 'station12'
CRS-2673: Attempting to stop 'ora.chad' on 'station11'
CRS-2673: Attempting to stop 'ora.qosmserver' on 'station12'
CRS-2677: Stop of 'ora.racorcl.db' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'station11'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'station11'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'station11'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'station11' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.station11.vip' on 'station11'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'station11'
CRS-2677: Stop of 'ora.station11.vip' on 'station11' succeeded
CRS-2677: Stop of 'ora.scan1.vip' on 'station11' succeeded
CRS-2677: Stop of 'ora.qosmserver' on 'station12' succeeded
CRS-2677: Stop of 'ora.racorcl.db' on 'station12' succeeded
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'station12'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'station12'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'station12'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'station12'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'station12'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'station12'
CRS-2673: Attempting to stop 'ora.cvu' on 'station12'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'station12' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'station12' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'station12' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'station12' succeeded
CRS-2677: Stop of 'ora.FRA.dg' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'station12'
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'station12'
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'station12'
CRS-2673: Attempting to stop 'ora.station12.vip' on 'station12'
CRS-2677: Stop of 'ora.scan2.vip' on 'station12' succeeded
CRS-2677: Stop of 'ora.station12.vip' on 'station12' succeeded
CRS-2677: Stop of 'ora.asm' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'station12'
CRS-2677: Stop of 'ora.scan3.vip' on 'station12' succeeded
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'station12'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'station12' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server 'station12' succeeded.
CRS-4549: Unexpected disconnect while executing shutdown request.
CRS-2677: Stop of 'ora.cvu' on 'station12' succeeded
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'station11'
CRS-2673: Attempting to stop 'ora.chad' on 'station11'
CRS-2675: Stop of 'ora.chad' on 'station11' failed
CRS-2679: Attempting to clean 'ora.chad' on 'station11'
CRS-2681: Clean of 'ora.chad' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'station11'
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'station11'
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'station11'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'station11'
CRS-2673: Attempting to stop 'ora.FRA.dg' on 'station11'
CRS-2677: Stop of 'ora.DATA.dg' on 'station11' succeeded
CRS-2677: Stop of 'ora.FRA.dg' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'station11'
CRS-2677: Stop of 'ora.scan1.vip' on 'station11' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.station11.vip' on 'station11'
CRS-2677: Stop of 'ora.station11.vip' on 'station11' succeeded
CRS-2677: Stop of 'ora.asm' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'station11'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'station11'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'station11' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server 'station11' succeeded.
CRS-2673: Attempting to stop 'ora.ons' on 'station11'
CRS-2677: Stop of 'ora.ons' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'station11'
CRS-2677: Stop of 'ora.net1.network' on 'station11' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'station11' has completed
CRS-2677: Stop of 'ora.crsd' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.evmd' on 'station11'
CRS-2673: Attempting to stop 'ora.storage' on 'station11'
CRS-2677: Stop of 'ora.storage' on 'station11' succeeded
CRS-2677: Stop of 'ora.evmd' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'station11'
CRS-2673: Attempting to stop 'ora.asm' on 'station11'
CRS-2677: Stop of 'ora.ctssd' on 'station11' succeeded
CRS-2675: Stop of 'ora.chad' on 'station12' failed
CRS-2679: Attempting to clean 'ora.chad' on 'station12'
CRS-2681: Clean of 'ora.chad' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'station12'
CRS-2677: Stop of 'ora.ons' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'station12'
CRS-2677: Stop of 'ora.net1.network' on 'station12' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'station12' has completed
CRS-2677: Stop of 'ora.crsd' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'station12'
CRS-2673: Attempting to stop 'ora.evmd' on 'station12'
CRS-2673: Attempting to stop 'ora.storage' on 'station12'
CRS-2677: Stop of 'ora.storage' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'station12'
CRS-2677: Stop of 'ora.ctssd' on 'station12' succeeded
CRS-2677: Stop of 'ora.evmd' on 'station12' succeeded
CRS-2677: Stop of 'ora.asm' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'station11'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'station11' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'station11'
CRS-2677: Stop of 'ora.cssd' on 'station11' succeeded
CRS-2677: Stop of 'ora.asm' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'station12'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'station12' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'station12'
CRS-2677: Stop of 'ora.cssd' on 'station12' succeeded
|
打开集群：
[root@station11 ~]# crsctl start cluster -all
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'station11'
CRS-2672: Attempting to start 'ora.evmd' on 'station11'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'station12'
CRS-2672: Attempting to start 'ora.evmd' on 'station12'
CRS-2676: Start of 'ora.cssdmonitor' on 'station12' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'station12'
CRS-2672: Attempting to start 'ora.diskmon' on 'station12'
CRS-2676: Start of 'ora.diskmon' on 'station12' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'station11'
CRS-2672: Attempting to start 'ora.diskmon' on 'station11'
CRS-2676: Start of 'ora.diskmon' on 'station11' succeeded
CRS-2676: Start of 'ora.evmd' on 'station12' succeeded
CRS-2676: Start of 'ora.evmd' on 'station11' succeeded
CRS-2676: Start of 'ora.cssd' on 'station12' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'station12'
CRS-2672: Attempting to start 'ora.ctssd' on 'station12'
CRS-2676: Start of 'ora.cssd' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'station11'
CRS-2676: Start of 'ora.ctssd' on 'station12' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'station12' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'station12'
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'station11'
CRS-2676: Start of 'ora.ctssd' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'station11'
CRS-2676: Start of 'ora.asm' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'station11'
CRS-2676: Start of 'ora.asm' on 'station12' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'station12'
CRS-2676: Start of 'ora.storage' on 'station12' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'station12'
CRS-2676: Start of 'ora.crsd' on 'station12' succeeded
CRS-2676: Start of 'ora.storage' on 'station11' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'station11'
CRS-2676: Start of 'ora.crsd' on 'station11' succeeded
|
7. 开始升级INFRASTRUCTURE：
8. /etc/oratab：
[root@station11 ~]# cat /etc/oratab
#Backup file is /u01/app/grid/crsdata/station11/output/oratab.bak.station11.grid line added by Agent
#
# This file is used by ORACLE utilities. It is created by root.sh
# and updated by either Database Configuration Assistant while creating
# a database or ASM Configuration Assistant while creating ASM instance.
# A colon, ':', is used as the field terminator. A new line terminates
# the entry. Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
# $ORACLE_SID:$ORACLE_HOME::
#
# The first and second fields are the system identifier and home
# directory of the database respectively. The third field indicates
# to the dbstart utility that the database should , "Y", or should not,
# "N", be brought up at system boot time.
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
+ASM1:/u01/app/23.5.0/grid:N
racorcl1:/u01/app/oracle/product/19.3.0/dbhome_1:N |
暂时锁住/etc/oratab：
[root@station11 ~]# chattr +i /etc/oratab
|
[root@station12 ~]# cat /etc/oratab
#Backup file is /u01/app/grid/crsdata/station11/output/oratab.bak.station11.grid line added by Agent
#
# This file is used by ORACLE utilities. It is created by root.sh
# and updated by either Database Configuration Assistant while creating
# a database or ASM Configuration Assistant while creating ASM instance.
# A colon, ':', is used as the field terminator. A new line terminates
# the entry. Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
# $ORACLE_SID:$ORACLE_HOME::
#
# The first and second fields are the system identifier and home
# directory of the database respectively. The third field indicates
# to the dbstart utility that the database should , "Y", or should not,
# "N", be brought up at system boot time.
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
+ASM2:/u01/app/23.5.0/grid:N
racorcl2:/u01/app/oracle/product/19.3.0/dbhome_1:N |
暂时锁住/etc/oratab：
[root@station12 ~]# chattr +i /etc/oratab
|
9. 结束环境：
升级为23.5.0.24.07：
[grid@station11 ~]$ crsctl query crs softwareversion -all
Oracle Clusterware version on node [station11] is [23.0.0.0.0]
Oracle Clusterware version on node [station12] is [23.0.0.0.0] |
查看集群详细情况：
[grid@station11 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.chad
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.helper
OFFLINE OFFLINE station11 STABLE
OFFLINE OFFLINE station12 STABLE
ora.net1.network
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
ora.ons
ONLINE ONLINE station11 STABLE
ONLINE ONLINE station12 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE station11 STABLE
2 ONLINE ONLINE station12 STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE station11 STABLE
2 ONLINE ONLINE station12 STABLE
ora.FRA.dg(ora.asmgroup)
1 ONLINE ONLINE station11 STABLE
2 ONLINE ONLINE station12 STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE station11 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE station12 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE station12 STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE station11 Started,STABLE
2 ONLINE ONLINE station12 Started,STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE station11 STABLE
2 ONLINE ONLINE station12 STABLE
ora.cdp1.cdp
1 ONLINE ONLINE station11 STABLE
ora.cdp2.cdp
1 ONLINE ONLINE station12 STABLE
ora.cdp3.cdp
1 ONLINE ONLINE station12 STABLE
ora.cvu
1 ONLINE ONLINE station11 STABLE
ora.racorcl.db
1 ONLINE ONLINE station11 Open,HOME=/u01/app/o
racle/product/19.3.0
/dbhome_1,STABLE
2 ONLINE ONLINE station12 Open,HOME=/u01/app/o
racle/product/19.3.0
/dbhome_1,STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE station11 STABLE
ora.scan2.vip
1 ONLINE ONLINE station12 STABLE
ora.scan3.vip
1 ONLINE ONLINE station12 STABLE
ora.station11.vip
1 ONLINE ONLINE station11 STABLE
ora.station12.vip
1 ONLINE ONLINE station12 STABLE
-------------------------------------------------------------------------------- |
|
