My initial goal was simply to create a test ASM environment on my PC running Oracle Enterprise Linux 5 and Oracle Database 11.2.0.2. The first step was easy…download and install Grid Infrastructure 11.2.0.1. This went without a problem until I ran the ASM Configuration Assistant (asmca) and realized that ASM couldn’t see any available disks/partitions/raw devices because my PC only had one physical disk drive in it. In order to get ASM to work, I used the “dd” command to create files that ASM can treat as raw devices. This is strictly for a test environment and should never be done like this in Production.

The commands that I used were:

#– create four 10GB files to be used for +DATA and +FRA (run as oracle)

mkdir /asmdisks

dd if=/dev/zero of=/asmdisks/data_disk1 bs=1024k count=10000
dd if=/dev/zero of=/asmdisks/data_disk2 bs=1024k count=10000
dd if=/dev/zero of=/asmdisks/fra_disk1 bs=1024k count=10000
dd if=/dev/zero of=/asmdisks/fra_disk2 bs=1024k count=10000

#– create the loop and raw devices.(run as root) This allows ASM to discover them as 4 raw devices
#– NOTE: You will also need to add the following commands to your server startup
#– scripts so that ASM can still see them after you reboot the server

losetup /dev/loop1 /asmdisks/data_disk1
losetup /dev/loop2 /asmdisks/data_disk2
losetup /dev/loop3 /asmdisks/fra_disk1
losetup /dev/loop4 /asmdisks/fra_disk2

raw /dev/raw/raw1 /dev/loop1
raw /dev/raw/raw2 /dev/loop2
raw /dev/raw/raw3 /dev/loop3
raw /dev/raw/raw4 /dev/loop4

#– change the permission to oracle:dba

chown oracle:dba /dev/raw/raw1
chown oracle:dba /dev/raw/raw2
chown oracle:dba /dev/raw/raw3
chown oracle:dba /dev/raw/raw4

chmod 660 /dev/raw/raw1
chmod 660 /dev/raw/raw2
chmod 660 /dev/raw/raw3
chmod 660 /dev/raw/raw4

After doing this, I was able to run asmca and create a +DATA and a +FRA disk group, each 10 GB in size (the raw devices are each 10GB and ASM mirrors them, so you end up with 10GB per disk group). Everything was going fine so far and I was able to create a database using the ASM disk groups that I just created.

Just when everything was working so perfectly, I decided to change things around a bit and upgrade from 11.2.0.1 Grid Infrastructure to 11.2.0.2. This release introduced Oracle Restart - billed as ‘High Availability for a non-RAC database’ and I wanted to try it out. It was also at this point that I thought to myself “I should have installed 11.2.0.2 Grid Infrastructure to begin with, rather than upgrading from 11.2.0.1″. Oh well, too late. On the other hand, this does provide a chance to do an upgrade that quite a few of our clients may actually do.

I downloaded disk3 of Patch #10098816 and followed the typical Oracle installation steps. One new thing with this version is that Oracle no longer does “In-place” upgrades, meaning each new release will have its own $ORACLE_HOME. This could cause problems for folks who hard-code the $ORACLE_HOME in scripts. Everything seemed to go without any problem until I tried to start the database I had earlier created. Upon startup, Oracle complained that it couldn’t read the spfile. Using the ASM Command Line (amscmd), I created a copy of the spfile in /tmp and tried to start with that. The file was created and readable, but this time, Oracle couldn’t read the control files. The disk groups were mounted, everything seemed fine but ASM was in an inconsistent state, sort of “stuck” between 11.2.0.1 and 11.2.0.2. After working on this to no avail for a while, I decided to take the “easy” way out. I would uninstall 11.2.0.1 and 11.2.0.2, re-install 11.2.0.2 and completely rebuild my +ASM instance. This was a test environment after all… probably not the way I would approach it in a Production situation.

With previous versions of the Oracle Installer, you could easily uninstall an ORACLE_HOME from the OUI. Now, when you click Uninstall, it directs to you to run a deinstall script from $ORACLE_HOME/deinstall. This failed for both versions and eventually I decided to just delete the $ORACLE_HOMEs and edit the oraInventory/ContentsXML/inventory.xml to remove all references to both of them. This seemed to work until running root.sh at the end the 11.2.0.2 install when I got the error and helpful suggestion:

“Improper Clusterware configuration found on this host. Deconfigure the existing clusterware by running “$CRS_HOME/install/roothas.pl -deconfig “

Of course, the roothas.pl also failed but I was able to use the –force option and it then succeeded. I re-ran root.sh and finally 11.2.0.2 Grid Infrastructure was installed.

I then ran asmca and was able to create an +ASM instance and two disk groups (+DATA and +FRA) with the files I had created earlier. The final test was whether I could use ASM to store datafiles for another instance. So I fired up the Database Configuration Assistanct (dbca) and created a database with all of its files located in the +DATA diskgroup. Success!

Now, back to Oracle Restart for a moment…like I said, Oracle Restart is new with 11.2.0.2 and is advertised as High Availability for a Non-RAC instance. It is essentially a daemon that runs (ohasd) and will restart the database/Listener/ASM if they crash unexpectedly. A shutdown abort or shutdown normal will not cause it to restart, as Oracle assumes you intend for it to be down. This could turn out to be a great thing for situations where it may take the DBA a while to login to restart a database. No need for extra downtime if the database restart can be automated!

Oracle Restart also takes care of stopping and starting the database(s) when the server reboots. This is a big change for those of us who have come to know and the love the dbora start/stop scripts. Once you start using Oracle Restart, the /etc/oratab (or /var/opt/oracle/oratab) file is no longer used by Oracle to start the database after the system reboots. After I registered my database with Oracle Restart with:
srvctl add database -d DEVDB01 -o /opt/oracle/product/11.2.0/dbhome_1
I noticed that Oracle had modified my /etc/oratab, changing the Y to an N.

A quick list of useful Oracle Restart commands are below:

srvctl stop|start|status database –d <database name>
srvctl stop|start|status asm
srvctl stop|start|status listener

In then end, I got my test environment configured exactly the way I wanted. If I could do things differently, I would have installed Grid Infrastructure 11.2.0.2 to begin with, rather than upgrading from 11.2.0.1. I also would have spent more time trying to figure out what went wrong with my initial upgrade from 11.2.0.1 to 11.2.0.2. Ideally, the upgrade would have worked and I wouldn’t have had to go through all the extra steps. This does raise a concern about doing this upgrade for our clients. I think my next step will be to setup a new environment specifically to perform the upgrade again and figure out what went wrong.