Tales From A Lazy Fat DBA

Hello Everyone,
Today i would like to start series/chapters describing Oracle Automatic Storage Management (Oracle ASM) concepts and provides an overview of Oracle ASM features. Followed posts will covers subjects like Installation, Configuration, Administration/Management, Monitoring. Troubleshooting and Optimization etc.

In this maiden post (Part 1) i would like to discuss and elaborate about ASM installation and related areas.

Prerequisites:
Considering that you already have the OS ready with all packages per-installed before we begin our ASM installation on the top. I will start with right from the scratch.

Step 1:
Preparing Disks or Partitions which will be used while creating the ASM diskgroups.
I’ve created 3 Persistent Disks each of 4GB in size from the VM Disk (I will perform all steps in VM environment).

This is how the VM Setting will look like once you are done with the Disk creation.
*Forgot about the Fifth Hard Disk of 10GB for now. Will explain the usage later on the series.

Once you have the disks created, Next you’ll have to format the newly created disks to make them usable: Using fdisk command. Command displays the status of available newly created partitions/disks as:
/dev/sdb, /dev/sdc, /dev/sdd – Each of 4GB (4294 MBs) in size.

[root@localhost ~]# fdisk -l

Disk /dev/sda: 91.3 GB, 91268055040 bytes, 178257920 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000aab6c

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 178257919 88615936 8e Linux LVM

Disk /dev/sdb: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdd: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdc: 4294 MB, 4294967296 bytes, 8388608 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/ol-root: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/ol-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/mapper/ol-home: 34.8 GB, 34837889024 bytes, 68042752 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Next you have to make each of the newly created raw partition as valid and usable primary partition.
Below step performs the said for one of the disk /dev/sdc. You will have to perform the same step for rest of the two disks as well and save the change using ‘w’ argument of fdisk.

[root@localhost ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x2be914a8.

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-8388607, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-8388607, default 8388607):
Using default value 8388607
Partition 1 of type Linux and of size 4 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Step 2:

Next is to install the optional ASMLIB packages. ASMLib is an optional utility that can be used on Linux systems to manage Oracle ASM devices. You can download the ASM packages from Oracle’s website.
– All ASMLib installations require that the ‘kmod-oracleasm’,’oracleasmlib’ and ‘oracleasm-support’ packages be
installed on the system.
– Packages required (RHEL 64 Bit, 11gR2) are:
oracleasm-support-2.1.8-1.el6.x86_64.rpm, oracleasmlib-2.0.4-1.el6.x86_64.rpm

I prefer using Packages using YUM – Otherwise you can use the direct RPM’s.
Example:

[root@localhost 11.2.0]# yum install kmod-oracleasm
Loaded plugins: langpacks
Resolving Dependencies
……..
………
Complete!

[root@localhost 11.2.0]# yum install oracleasm*
Loaded plugins: langpacks
Resolving Dependencies
……..
……….
Complete!

Now that the ASMLib software is installed, a few steps have to be taken by the system administrator to make the ASM driver available. The ASM driver needs to be loaded, and the driver filesystem needs to be mounted.

Run the oracleasm script with the ‘configure’ option. It will ask for the user and group that default to owning the ASM driver access point. If the database was running as the ‘oracle’ user and the ‘dba’ group, the output would look like this:

[root@localhost 11.2.0]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets (‘[]’). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done

Next step is to initialize the oracle oracleasm modules. Use following command to do this. Output will look like below.

[root@localhost 11.2.0]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module “oracleasm”: oracleasm
Configuring “oracleasm” to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Next you will have to make disks available to ASMLib. Every disk that ASMLib is going to be accessing needs to be made available. This is accomplished by creating an ASM disk.

[root@localhost 11.2.0]# oracleasm createdisk ASM1 /dev/sdb1
Writing disk header: done
Instantiating disk: done
[root@localhost 11.2.0]# oracleasm createdisk ASM2 /dev/sdd1
Writing disk header: done
Instantiating disk: done
[root@localhost 11.2.0]# oracleasm createdisk ASM3 /dev/sdc1
Writing disk header: done
Instantiating disk: done

Existing disks can be listed and queried using below command.
With this you are done with ASMLib installation and configuration.

[root@localhost 11.2.0]# oracleasm listdisks
ASM1
ASM2
ASM3
[root@localhost 11.2.0]# ls -ltrh /dev/oracleasm/disks
total 0
brw-rw—-. 1 oracle dba 8, 17 Dec 30 11:36 ASM1
brw-rw—-. 1 oracle dba 8, 49 Dec 30 11:36 ASM2
brw-rw—-. 1 oracle dba 8, 33 Dec 30 11:36 ASM3

Step 3:

Next when you have all the disks ready to use – Install Grid Infrastructure Binaries!!
Now we are ready to perform the binary installations. Lets first install the GI binaries. To do it mount or copy the Oracle software.
Patch Name: ‘13390677’
You will have to download three of the packages out of the total bundle available. First 2 are for database and third zip contains Oracle Grid Infrastructure, includes: Oracle ASM, Oracle Clusterware, Oracle Restart software.
Names:
p13390677_112040_platform_1of7.zip (Oracle Database, including Oracle RAC components)
p13390677_112040_platform_2of7.zip (Oracle Database, including Oracle RAC components)
p13390677_112040_platform_3of7.zip (Oracle Grid Infrastructure, includes: Oracle ASM, Oracle Clusterware, Oracle Restart)

I have only pasted relevant screenshots for the GI Installation and skipped few.

Now you will have to run the root.sh script as root user. root.sh script will ask you to run one of the two scripts depending on the standalone or RAC configuration. In our case we need to execute below script/command for standalone database.

[root@localhost grid]# /u01/app/11.2.0/grid/perl/bin/perl -I/u01/app/11.2.0/grid/perl/lib -I/u01/app/11.2.0/grid/crs/install /u01/app/11.2.0/grid/crs/install/roothas.pl
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
LOCAL ADD MODE
Creating OCR keys for user ‘oracle’, privgrp ‘oinstall’..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user ‘root’, privgrp ‘root’..
Operation successful.
CRS-4664: Node localhost successfully pinned.
Adding Clusterware entries to oracle-ohasd.service

localhost 2016/01/01 12:28:47 /u01/app/11.2.0/grid/cdata/localhost/backup_20160101_122847.olr
Successfully configured Oracle Grid Infrastructure for a Standalone Server

With this our Grid Infrastructure installation completed successfully. You can verify the ASM instance running using
[oracle@Fatdba ~]$ ps -ef|grep pmon
oracle 3624 1 0 09:12 ? 00:00:01 asm_pmon_+ASM
oracle 11176 11152 0 13:00 pts/5 00:00:00 grep pmon

Step 4:

Install Oracle Database Binaries
Set the ORACLE_BASE and ORACLE_HOME (different from Grid Home) and then run the database runInstaller.
I’ve pasted only required screenshots of steps performed during DB installation.

Now we have the Grid and DB software’s installed on our system. Start CSSD service

Now we have Grid Infrastructure and Oracle Database binaries installed.

Step 5:

The next step is to create ASM instance and then the database instance. Before starting the creation of ASM instance we need to ensure that CSSD service is up and running. This can be done by issuing ‘crs_start ora.cssd‘ command.

[oracle@localhost ~]$ export ORACLE_HOME=/u01/11.2.0.2/grid
[oracle@localhost ~]$ export PATH=$ORACLE_HOME/bin:$PATH
[oracle@localhost ~]$ crs_stat -t
Name Type Target State Host
————————————————————
ora.cssd ora.cssd.type OFFLINE OFFLINE
ora.diskmon ora….on.type OFFLINE OFFLINE
ora.evmd ora.evm.type ONLINE ONLINE localhost
ora.ons ora.ons.type OFFLINE OFFLINE
[oracle@localhost ~]$ crs_start ora.cssd
Attempting to start `ora.cssd` on member `localhost`
Attempting to start `ora.diskmon` on member `localhost`
Start of `ora.diskmon` on member `localhost` succeeded.
Start of `ora.cssd` on member `localhost` succeeded.
[oracle@localhost ~]$ crs_stat -t
Name Type Target State Host
————————————————————
ora.cssd ora.cssd.type ONLINE ONLINE localhost
ora.diskmon ora….on.type ONLINE ONLINE localhost
ora.evmd ora.evm.type ONLINE ONLINE localhost
ora.ons ora.ons.type OFFLINE OFFLINE

Now you are free to create the database on ASM storage using dbca or manually. Whatever you are comfortable with.
Below are few of the important steps that you need to follow while creating database using DBCA.

Do a status check of all services.

[oracle@Fatdba ~]$ crs_stat -t
Name Type Target State Host
————————————————————
ora.DATA.dg ora….up.type ONLINE ONLINE fatdba
ora.FRA.dg ora….up.type ONLINE ONLINE fatdba
ora….ER.lsnr ora….er.type ONLINE ONLINE fatdba
ora.asm ora.asm.type ONLINE ONLINE fatdba
ora.btm666.db ora….se.type ONLINE ONLINE fatdba
ora.cssd ora.cssd.type ONLINE ONLINE fatdba
ora.diskmon ora….on.type OFFLINE OFFLINE
ora.evmd ora.evm.type ONLINE ONLINE fatdba
ora.ons ora.ons.type OFFLINE OFFLINE
ora.test666.db ora….se.type ONLINE ONLINE fatdba

With this you are done with the DB creation as well. You can also create the database using manual method.
I will try to cover that in future posts.

Hope That Helps!
Prashant Dixit