shutdown the Exadata database nodes and storage cells in a rolling fashion

How to shutdown the Exadata database nodes and storage cells in a rolling fashion so certain hardware tasks can be performed

The goal of this document is to provide a way to shutdown the Exadata database nodes and storage cells in a rolling fashion so certain hardware tasks may be performed on each server. For example - replacing the raid controller battery requires downtime for each server. This document will provide a way to perform these tasks without taking down the database.

FIX

This note separates the tasks into two different sections.

1. Instructions on shutting down and restarting each database node in a rolling fashion.
2. Instructions on shutting down and restarting each storage cell in a rolling fashion.

 SECTION1: Shutting down and restarting each Exadata database node in a rolling fashion

As root perform the following on each database server - one at a time:

1)Check to see if AUTOSTART is enabled:

# $GRID_HOME/bin/crsctl config crs

 2) Disable the Grid Infrastructure for autostart on the database servers if the previous step indicated it is currently enabled for autostart.

# $GRID_HOME/bin/crsctl disable crs

•  Note: This is step is [Optional] and it can required during maintenance operation like "firmware patches" which requires to reboot the Compute Node several times.

3) Stop the Grid Infrastructure stack on the first database server locally:

# $GRID_HOME/bin/crsctl stop crs

 4) Verify that the Grid Infrastructure stack has shutdown successfully on the database server.

The following command should show no output if the GI stack has shutdown:

# ps -ef | grep diskmon

 5) Confirm that the clusterware resources are still up and running on the other nodes:

cd /opt/oracle.SupportTools/onecommand
# dcli -g dbs_group -l root $GRID_HOME/bin/crsctl check crs

7) Before shutting down the DB node, check /etc/fstab for any nfs mounts that should be unmounted.

8) Shut the db node down so you can peform maintenance.

# poweroff

 9) Perform the scheduled maintenance on the DB node.

You can proceed now with hardware replacement or maintenance.

10) After hardware has been replaced or maintenance performed and your ready to bring the server back up, power on the DB node by using the power button on the front panel of the Exadata Storage Servers.

11) Start the Grid Infrastructure stack on the database server once it comes up:

# $GRID_HOME/bin/crsctl start crs

 12) Wait until the Grid Infrastructure stack has successfully started. To check the status of the Grid Infrastructure stack, run the following command and verify that the "ora.asm" instance is started. Note that the command below will continue to report that it is unable to communicate with the Grid Infrastructure software for several minutes after issuing the "crsctl start crs" command above:

# /u01/app/11.2.0/grid/bin/crsctl status resource -t

 13) Reenable the Grid Infrastructure for autostart again since we disabled it earlier in Section 1 # 3:

# $GRID_HOME/bin/crsctl enable crs

 14) Confirm that the clusterware resources are up and running on all nodes:

cd /opt/oracle.SupportTools/onecommand

# dcli -g dbs_group -l root $GRID_HOME/bin/crsctl check crs

 After completing the steps above, repeat same steps again in Section 1 on the remaining nodes (one at a time) until all hardware has been replaced on each DB node.

SECTION 2: Shutting down and restarting the storage cells in a rolling fashion

1) Refer to the instructions listed in the note below:
Steps to shut down or reboot an Exadata storage cell without affecting ASM (Doc ID 1188080.1)

Reference Doc ID - . (Doc ID 1539451.1)

Steps To Shut Down / Start Up The Exadata

Steps To Shut Down / Start Up The Exadata

Complete shutdown of DATABASE AND ASM .

To change any hardware replacement ,flashdrive, Disks.

Follow the steps as defined.

Shut Down

·         Log in to the first database server as root.

·         Change to the One Command directory:

# cd /opt/oracle.SupportTools/onecommand

Note whether the Grid Infrastructure is currently enabled for autostart, so that this state can be restored later:

# dcli -g dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl config crs

[Optional] If the preceding step indicated that the Grid Infrastructure is currently enabled for autostart, disable it for autostart on the database servers:

# dcli -g dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl disable crs

Note: This step is optional. It may be required during maintenance operations like “firmware patches,” which require rebooting the compute node several times.

·         Stop the Grid Infrastructure stack on the database servers (compute nodes):

# dcli -g dbs_group -l root "/u01/app/12.1.0.2/grid/bin/crsctl stop crs"

Verify that the Grid Infrastructure stack has shut down successfully on the database servers. The following command should show no output if the Grid Infrastructure stack has shut down:

# dcli -g dbs_group -l root "ps -ef | grep diskmo[n]"

[Optional] Disable email/SNMP alerts if applicable. If alerts will be enabled, first note the notification method in use so that it can be restored later. The following command will show the current notification method:

# dcli -g cell_group -l root "su - celladmin -c \"cellcli -e list cell attributes name,notificationMethod\""

Now set the notification method to null to disable alerts:

# dcli -g cell_group -l root "su - celladmin -c \"cellcli -e alter cell notificationMethod=null\""

Shut down all cells:

# dcli -g cell_group -l root "su - celladmin -c \"cellcli -e alter cell shutdown services all \""

Power off all cells:

# dcli -g cell_group -l root poweroff

Copy the /opt/oracle.SupportTools/onecommand/dbs_group to /opt/oracle.SupportTools/onecommand/dbs_group2:

# cp /opt/oracle.SupportTools/onecommand/dbs_group /opt/oracle.SupportTools/onecommand/dbs_group2

·         Remove the “first compute node name” from the /opt/oracle.SupportTools/onecommand/dbs_group2 file (using any editor, e.g., vi).

·         Power off all the compute nodes (except compute node # 1):

dcli -g dbs_group2 -l root poweroff

Power off compute node #1:

# poweroff

Now, both cell servers and compute nodes are down for maintenance.

Start Up

·         Power on “compute node #1” by using the power button on the front panel of the Exadata Storage Servers.

·         Change to the OneCommand directory:

# cd /opt/oracle.SupportTools/onecommand

Power on the cells either by using the power button on the front panel of the Exadata Storage Servers or by executing the following script from the first database server (from compute node #1):

for host in `cat cell_group`; do

echo ${host}: `ipmitool -H ${host}-ilom -U root -P welcome1 chassis power on`

done

Wait until all the cell servers are up. Then, power on the compute nodes either by using the power button on the front panel of the Exadata Storage Servers or by executing the following script from the first database server:

for host in `cat dbs_group2`; do

echo ${host}: `ipmitool -H ${host}-ilom -U root -P welcome1 chassis power on`

done

Verify that all Exadata Storage Servers have booted successfully:

# dcli -g cell_group -l root hostname

Verify that all the cells are up (MS, RS, and CELLSRV services must be running):

# dcli -g cell_group -l root "su - celladmin -c \"cellcli -e list cell detail \""

Re-enable email/SNMP alerts if they were disabled during the pre-maintenance steps. The following example illustrates the command to use if both email and SMTP alerting were used. Adjust the command as necessary to restore the notification method noted during the pre-maintenance steps.

# dcli -g cell_group -l root "su - celladmin -c \"cellcli -e alter cell notificationMethod=\'mail,snmp\'\""

Start the Grid Infrastructure stack on the first database server:

# /u01/app/12.1.0.2/grid/bin/crsctl start crs

Wait until the Grid Infrastructure stack has started successfully on the first database server. To check the status of the Grid Infrastructure stack, run the following command and verify that the “ora.asm” instance is started. Note that the following command will continue to report that it is unable to communicate with the Grid Infrastructure software for several minutes after issuing the “crsctl start crs” command above:

# /u01/app/12.1.0.2/grid/bin/crsctl status resource -t

Start the Grid Infrastructure stack on all remaining database servers with a one-minute delay between servers. Note that the following command will issue the start command for all database servers, including the first database server where it is already started. The output for the first server will therefore indicate that the software is already started, but this message can be safely ignored:

for h in `cat dbs_group2`; do

echo "Starting CRS on host $h"

ssh $h /u01/app/12.1.0.2/grid/bin/crsctl start crs

echo "Waiting 60 seconds…"

sleep 60

done

Monitor the startup progress (this could take several minutes):

# /u01/app/12.1.0.2/grid/bin/crsctl status resource -t

Re-enable the Grid Infrastructure for autostart if it was found to be enabled for autostart at the beginning of the pre-maintenance steps.

# dcli -g dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl enable crs

Note: If dcli/ssh is not enabled due to security reasons, then you will need to connect and execute the preceding steps on each node at the time.

Reference - 

Metalink Note - (Doc ID 1093890.1)

ILOM (Integrated Lights Out Manager) on Oracle EXADATA

ILOM (Integrated Lights Out Manager)

(ORACLE EXADATA=Combination of SERVERS+STORAGE CELL)

Each of this has ILOM interface which is used for hardware management and maintenance activity

ILOM is system management firmware you can use to monitor, manage, and configure a variety of Sun server platforms.

Oracle ILOM’s dedicated hardware and software is pre-installed on a variety of Oracle Sun server platforms.

ILOM is to provide support for a server in a manner such that access to the data center is not required for day-to-day support functions.

An ILOM also provides access to a variety of diagnostic features and has integration with Oracle’s Automatic Service Requests to provide call-home functionality to report hardware failures to Oracle for fast support.

The Oracle ILOM service processors provide a wide array of features, and its functional aspects improve with every release of the ILOM.

The Oracle ILOM service processor (SP) runs its own embedded operating system and has a dedicated Ethernet port, which together provide out-of-band management capability

·                     Remotely control the power state of your server

·                     System monitoring

·                     Remote KVMS capabilities

·                     Fault display

·                     Integration with a variety of authentication systems, such as LDAP, SSL, Radius and Active Directory

·                     Remote syslog setup

·                     SNMP and SMTP-based alerting

·                     Updating the firmware

·                     Environmental reporting

·                     View the graphical and non-graphical consoles for the host

·                     Gather the hardware diagnostics

·                     Monitoring host states remotely

·                     Access to various part and serial numbers

·                     Determine the hardware configuration of your system

Oracle ILOM automatically initializes as soon as power is applied to the server. It provides a full-featured, browser-based web interface and has an equivalent command-line interface (CLI).

Note -

ILOM supports the Intelligent Platform Management Interface (IPMI), which enables you to monitor and control your server platform, as well as to retrieve information about your server platform.

ILOM supports alerts in the form of IPMI Platform Event Trap (PET) alerts. Alerts provide advance warning of possible system failures.

Oracle ILOM supports the Simple Network Management Protocol (SNMP), which is used to exchange data about network activity.(ILOM on database servers can use SNMP to send sensor alerts to an Enterprise Manager Agent.)

Exachk in Exadata

What is Exachk and use in Exadata?

Oracle Document ID to get Exacheck Utility : 1070954.1

Exachk is designed to audit configuration of Oracle Exadata Machine which includes 

• DB Server, 
• Cell Storage, 
• Infiniband switch, 
• Software,
• Database  
• Ethernet Network.

Exachk report should be fetched after the initial Oracle Exadata Database Machine deployment, as a part of the routine maintenance schedule ,before and after any system configuration change and patching activity.

Phases:

There are four phase in exachk execution.

1) Minimal Input 

2) Data Collection

3) Analysis

4) HTML Report Generation

1. Minimal Input

• Check user equivalence between all the DB servers for Oracle user
• Search database which are running on Exadata and registered in OCR
• Provide output, which database should part of best practice check
• Search Oracle home of selected database which are part of best practice check
• Check the status of Oracle software stack Clusterware, ASM, RDBMS
• Copy of plug-ins which required to execute Exacheck report
• It asks for Storage server password
• Provide output based on given options
• Provide storage server root credential
• Exachk will verify root password for storage servers
• Provide Database server credential
• Exachk will verify root password for compute nodes
• It will create collection and audit checks log file
• Checking of .bash_profile of Oracle user on compute nodes
• It will start exachk on one of the node

2. Data Collection

Collect the required data from compute node, Storage server and Infiniband switch.

Data Collection includes below details.

• Database --Initialize Parameters, RMAN, Listener, Patch, Data corruption prevention, Database failure prevention, MAA
• GI      -- Patch, Version, Status, node failure prevention, Cluster Failure prevention
• Infiniband  -- Network check, /etc/hosts file, NTP configuration, firmware, health status, switch type, ports
• Cell      -- Cell disk configuration, Temperature, flash cache configuration, flash log configuration
• ASM     -- Initialize parameters, Data corruption prevention, Disk/cell failure prevention,
• OS          -- CPU, parameter of sysctl.conf file, users, time zones, Kernel, OFED, System logs, Network, ILOM, NTP,LVM,Imageinfo,
• Others  -- Exadata critical issue, Consolidation check, Site failure prevention,software maintenance, version for all the software, Machine Model/details, Firmware, Alert configuration

3. Analysis

Check best practice on all the devices and installed software as well database based on collected data.

4. Generate HTML report 

Once best practice check completes, it generates HTML report which you can review for further system improvement.

Exacheck Report Structure:

• Cluster Summary
• Findings Needing Attention
• MAA Scorecard
• Findings Passed
• System wide Automatic Service Request (ASR) health check
• Killed Processes
• Skipped Checks
• Excluded Checks
• Top 10 Time Consuming Checks

It gives five types of indication based upon that you can act on particular findings.

  

FAIL     =>    If best practice doesn't meet or not set as per best practice which is critical

WARNING => If best practice doesn't meet or not set as per best practice which is non critical

PASS          =>    If something is configured as per best practice than it will indicate with PASS

INFO             =>    For information purpose

ERROR         =>   If there is any configuration mismatch

Two Important Oracle Exadata Initialization Parameters

By: Richard Niemiec

---

Oracle has two very important parameters to use to test major features of Exadata.

The first is CELL_OFFLOAD_PROCESSING,

 which relates to Smart Scans (cell scans). The default value is TRUE, which means that Smart Scans are turned ON for Exadata (if you are using it). 

You can set this value to FALSE to turn off Smart Scans and check other features or compare speeds with and without this feature.

The second parameter is undocumented and should only be used with the consent of Oracle Support and also only for testing purposes. The second is _KCFIS_STORAGEIDX_DISABLED, 
which is used to disable storage indexes. This tells Oracle not to use storage index optimization on storage cells if set to TRUE. The default is FALSE. 

Another undocumented parameter (again check with Oracle Support) is also related to how the Smart Scan works and uses Bloom Filters. 
The parameter _BLOOM_FILTER_ENABLED is set to TRUE by default.

 With Exadata, bloom filters are used for join filtering with Smart Scans. _BLOOM_PRUNING_ENABLED also has a default of TRUE. Set these parameters to FALSE to disable them.

Best Practices for Getting the Most from Oracle Exadata

By: Richard Niemiec

---

Many best practices will help you get the most out of Exadata. I’ll start with a list of must haves and don’t do’s and then list some best practices!

Note that these can change at any time (check Oracle’s documentation for the latest as these are subject to change).

• Must have Bundle Patch 5. (See Note: 888828.1 for latest patch.)
• Must have ASM to use Exadata; Automatic Storage Management (ASM) serves as the filesystem and volume manager for Exadata.
• Must have the correct data center cooling! This is very important (I suggest doing some research on this).
• Must have three floor tiles on a raised floor (must support 2219 lbs./964 kg) with holes (cooling) for a full rack (between 1560 CFM and 2200 CFM front to back). You don’t want to melt it! All of this is subject to change so please check the latest specifications.
• Must have the correct power needs.
• Must have Oracle Linux 5.3 (x86_64) and Oracle DB 11.2 (currently).
• Must have RMAN for backups.
• Consider StorageTek SL500 Tape backup (many positive reviews but pricey).
• Don’t add any foreign hardware or … no support!
• Don’t change BIOS/firmware or … no support!

Here are some other best practices:

• Use an ASM allocation unit (AU) size of 4M (currently).
• “CREATE ALL” celldisk and griddisks.
• Use DCLI to run on all storage servers at once (this is both helpful and saves time).
• Use IORM for resource management.
• Decide your Fast Recovery Area (FRA) and MAA needs before you install.
• Use Database 11.2.0.1+ (11.2.1.3.1) and ASM 11.2.0.1+ (minimums currently).
• Must be compatible with 11.2.0.1+ (current minimum).
• Has a logfile size of 32G (Whoa!).
• Use Locally Managed Tablespaces (LMT) with 4M uniform extents.
• Move data with Data Pump (usually, but you have many other options).
• Start with the default initialization parameters for tuning and then adjust as needed for your workload.

What Makes Oracle Exadata So Great

Article By: Richard Niemiec

---

With the acquisition of Sun, a paradigm shift occurred in the database market, the hardware world, and the IT industry. While this shift is just beginning, the early tremors are showing that the major shift is coming and will be big over the next decade. Here are a few points about what is coming. Exadata is hardware that combines the power of the database, while leveraging features Oracle added at the hardware level that other hardware providers will not be able to replicate easily (or at all). Some of the major Exadata benefits include:

• Flash Cache  The Flash Cache is comprised of solid-state disks (information stored on chips—over 5 terabytes in a full rack), which is around 20× to 50× faster than disks (depending on the disks). The Flash Cache caches hot data (data frequently used). It does this as the last step (so it returns data to the user first and then caches it for the next time based on the settings you give it).
  NOTE
  The Flash Cache memory (hardware/chips) is not the same as the Database Flash Cache. You can use Database Flash Cache with the Oracle Database 11g and Oracle RAC 11g. The Database Flash Cache is an optional area that you can add if your database is running on Solaris or Oracle Linux. It is a file-based extension of the SGA-resident buffer cache (similar to a swap area), providing a level-2 cache for database blocks. The file is used as the database is aged out of the SGA. To get more information, see the initialization parameter DB_FLASH_CACHE_SIZE.

• Storage index  A storage index utilizes minimum/maximum (min/max) values to help queries run faster. A performance gain of 10× is common with storage indexes. They primarily maintain summary information about the data (like metadata in a way). The memory structure resides at the cell level (disk/storage cell). For a cell (a unit of storage), the min/max values for up to eight queried table columns are stored in that Exadata storage cell and in the storage index. So a storage index is an index of how the data is stored across cells (like the min/max values for partitions). When a query is run against a table, the storage index is checked to see which cells need to be accessed to retrieve the data. Oracle searches data based on the min/max for various columns and eliminates I/Os where there is no match. The process is 100 percent transparent to the user. It is performed at the hardware level and typically has one index for every 1M of disk. No setup is required; Oracle does this internally and automatically. Storage indexes can also provide a huge benefit for highly skewed data. The storage index offers quick access to the highly selective values, whereas the normal Exadata offloading handles the nonselective values.
• Smart scans  Smart scans are performed internally at the hardware level by Oracle and 10× savings are common. Oracle filters things based on the WHERE clause (predicates) and filters on row/column/join condition. No setup is required; Oracle does this internally and automatically.
• Hybrid Columnar Compression (HCC)  Hybrid Columnar Compression (HCC), also known as Exadata Hybrid Columnar Compression (EHCC), is data that is organized by a hybrid of column and row formats and then compressed. A performance gain anywhere from 10× to 30× is common. The tables are organized in Compression Units (CU), which contain around 1000 rows (more or less depending on the amount of data in the rows). CUs span many blocks. HCC is good for data that is bulk loaded, but it’s not built for OLTP or single block-read operations. It is primarily built for Data Warehouses and Queried Data and not for frequently updated data. Compared to Gzip/Bzip2, HCC typically has 2× the compression and is 10× faster.
• Enterprise Manager Grid Control  Oracle Enterprise Manager Grid Control has a specific plug-in for Exadata, which allows you to manage and view many of the advantages that Oracle provides with Exadata easily. It is also a good way to ensure things are working properly and efficiently.
• Enterprise Manager Exadata Simulation  This allows you to check the benefits of Exadata before implementing it by running a simulation on your current system. This feature is a part of SQL Performance Analyzer, which is a part of Oracle Real Application Testing. Additional licensing is required.
• I/O Resource Manager  Oracle’s latest I/O Resource Management (IORM) tool can be used with Exadata to manage multiple workloads and set resources as you deem necessary for each database, user, or task. IORM gives you the capability to manage many systems easily with a single machine.
• Security  Secure encrypted backup with change tracking file (which is much faster), in which the storage nodes offer hardware decryption acceleration. Exadata takes advantage of CPU hardware decryption acceleration with AES-NI, so you can encrypt and decrypt even when using TDE tablespace or column encryption (encrypted data is now decrypted before Smart-Scan is applied).
• Utilities  Cellcli is a command-line interface for queries at the hardware level and for monitoring various health metrics for individual cells; DCLI is for querying multiple cells, and ADR Command Interpreter (ADRCI) is for diagnostics.

SRVCTL COMMANDS

Useful Srvctl Commands

The best and nicely written 

Web Reference - SRVCTL COMMANDS

SRVCTL is known as server control utility, which is used to add, remove,relocate and manage different crs services or components in RAC database.

1. STOP DATABASE :

SYNTAX – srvctl stop database -d db_name [-o stop_options] where stop_options is normal/immediate(default)/transactional/abort

e.g

1 2 3 4 5 6 7

srvctl stop database -d PRODB -o normal srvctl stop database -d PRODB -o immediate srvctl stop database -d PRODB -o transactional srvctl stop database -d PRODB -o abort

2. START DATABASE
SYNTAX – srvctl start database -d db_name [-o start_options] where start_option is nomount/mount/open(default)

e.g

1 2 3 4 5 6

srvctl start database -d PRODB -o nomount srvctl start database -d PRODB -o mount srvctl start database -d PRODB -o open

3. STOP AN INSTANCE

SYNTAX – srvctl stop instance -d db_unique_name [-i “instance_name_list”]} [-o stop_options] [-f]
e.g

1 2 3 4

srvctl stop instance -d PRODB -i PRODB1

4. START AN INSTANCE

SYNTAX – srvctl start instance -d db_unique_name [-i “instance_name_list”} [-o start_options]
e.g

1 2 3 4

srvctl start instance -d PRODB -i PRODB1

5. REMOVING DB FROM CRS:
SYNTAX – srvctl remove database -d db_unique_name [-f] [-y] [-v]
e.g

1 2 3 4

srvctl remove database -d PRODB -f -y

6. ADDING DB IN CRS :
SYNTAX – srvctl add database -d db_unique_name -o ORACLE_HOME [-p spfile]
e.g

1 2 3 4

srvctl add database -d PRODB -o /u01/app/oracle/product/12.1.0.2/dbhome_1 -p +DATA/PRODDB/parameterfile/spfilePRODB.ora

7. REMOVING AN INSTANCE FROM CRS:
SYNTAX – srvctl remove instance -d DB_UNIQUE_NAME -i INSTANCE_NAME
e.g

1 2 3 4

srvctl remove instance -d PRODB - I PRODB1

8.ADDING AN INSTANCE TO CRS:
SYNTAX – srvctl add instance –d db_unique_name –i inst_name -n node_name
e.g

1 2 3 4

srvctl add instance -d PRODB - i PRODB1 -n rachost1

9. Enable/disable auto restart of the instance

1 2 3 4 5

srvctl enable instance -d DB_UNIQUE_NAME-i INSTANCE_NAME srvctl disable instance -d DB_UNIQUE_NAME-i INSTANCE_NAME

10. Enable/disable auto restart of the database

1 2 3 4 5

srvctl enable database -d DB_UNIQUE_NAME srvctl disable database -d DB_UNIQUE_NAME

11. ADDING A SERVICE:

SYNTAX – srvctl add servicec -d {DB_NAME} -s {SERVICE_NAME} -r {“preferred_list”} -a {“available_list”} [-P {BASIC | NONE | PRECONNECT}]

e.g

1 2 3 4

srvctl add service -d PREDB -s PRDB_SRV -r "PREDB1,PREDB2" -a "PREDB2" -P BASIC

12.REMOVING A SERVICE:

SYNTAX – srvctl remove service -d {DB_NAME} -s {SERVICE_NAME}
e.g

1 2 3 4

srvctl remove service -d PREDB -s PRDB_SRV

13. START A SERVICE
SYNTAX– srvctl start servicec -d {DB_NAME} -s {SERVICE_NAME}
e.g

1 2 3 4

srvctl start service -d PREDB -s PRDB_SRV

14. START A SERVICE

SYNTAX– srvctl stop servicec -d {DB_NAME} -s {SERVICE_NAME}
e.g

1 2 3 4

srvctl stop service -d PREDB -s PRDB_SRV

15. RELOCATE A SERVICE

SYNTAX – srvctl relocate service -d {database_name} -s {service_name} -i {old_inst_name} -r {new_inst_name}

EXAMPLE: (Relocating service PRDB_SRV from PREDB2 to PREDB1)

1 2 3 4

srvctl relocate service -d PREDB -s PRDB_SVC -i PREDB2 -t PREDB1

16. Check the status of service
SYNTAX – srvctl status service -d {database_name} -s {service_name}

1 2 3 4

srvctl status service -d PREDB -s PRDB_SVC

17. Check the configuration of service
SYNTAX – srvctl config service -d {database_name} -s {service_name}

1 2 3 4

srvctl config service -d PREDB -s PRDB_SVC

18. Check scan listener configuration

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

srvctl config scan_listener   SCAN Listener LISTENER_SCAN1 exists. Port: TCP:1522 Registration invited nodes: Registration invited subnets: SCAN Listener is enabled. SCAN Listener is individually enabled on nodes: SCAN Listener is individually disabled on nodes: SCAN Listener LISTENER_SCAN2 exists. Port: TCP:1522 Registration invited nodes: Registration invited subnets: SCAN Listener is enabled. SCAN Listener is individually enabled on nodes: SCAN Listener is individually disabled on nodes: SCAN Listener LISTENER_SCAN3 exists. Port: TCP:1522 Registration invited nodes: Registration invited subnets: SCAN Listener is enabled. SCAN Listener is individually enabled on nodes: SCAN Listener is individually disabled on nodes:

19. Modify scan_listener port:

1 2 3 4 5 6 7 8 9 10 11 12

srvctl modify scan_listener -p {new-SCAN-port}   srvctl modify scan_listener -p 1523     $GRID_HOME/bin/srvctl stop scan_listener $GRID_HOME/bin/srvctl start scan_listener   Alter system set remote_listener='orcl-scan.stc.com.sa:1523' scope=both sid='*';

20. Manage MGMTDB in oracle 12c:

1 2 3 4 5 6 7 8 9 10 11 12 13

srvctl status mgmtdb Database is enabled Instance -MGMTDB is running on node node12-1       -- stop and start MGMT db.   srvctl stop mgmtdb srvctl start mgmtdb

21. Enable trace for srvctl commands:

1 2 3 4 5 6 7 8 9 10

-- set this to enable trace at os   SRVM_TRACE=true export SRVM_TRACE   -- run any srvctl command srvctl status database -d ORACL

22. Set environment variables through srvctl.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

-- setenv to set env variables.(ORCL is the db_unique_name)   srvctl setenv database -db ORCL -env "ORACLE_HOME=/oracle/app/oracle/product/12.1.0.2/dbhome_1" srvctl setenv database -db ORCL -env "TNS_ADMIN=/oracle/app/oracle/product/12.1.0.2/dbhome_1/network/admin"     --getenv to view the env setting:   srvctl getenv database -db ORCL   ORCL: ORACLE_HOME=/oracle/app/oracle/product/12.1.0.2/dbhome_1 TNS_ADMIN=/oracle/app/oracle/product/12.1.0.2/dbhome_1/network/admin

23. Check status and config of ASM instance:

1 2 3 4 5 6 7 8 9 10 11

srvctl config asm ASM home: Password file: +MGMT/orapwASM ASM listener: LISTENER     srvctl status asm ASM is running on ses11-4,ses11-5

24. Stop and start services running from ORACLE_HOME

1 2 3 4 5 6

srvctl stop home -oraclehome /oracle/product/12.1.0.2/dbhome_1 -statefile /home/oracle/state.txt  -node dbhost-1   srvctl start home -oraclehome /oracle/product/12.1.0.2/dbhome_1 -statefile /home/oracle/state.txt  -node dbhost-1

25. Create a TAF policy

1 2 3 4 5 6

srvctl add service -db ORCLDB -service TAF_ORCL -preferred ORCLDB1 -available ORCLDB2 -tafpolicy BASIC -failovertype SELECT   srvctl start service -db OMPREDB -service TAF_ORCL