Monday, November 11, 2013

ORA-01078,ORA-29701 errors while starting standalone ASM on 12c


Error:

[grid@cloud1 ohasd]$ sqlplus "/as sysasm"

SQL*Plus: Release 12.1.0.1.0 Production on Mon Nov 11 08:17:49 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL> exit


Solution 1:

 execute below command as root user

$GRID_HOME/localconfig reset


Solution 2:

start cssd daemon as root user

[root@cloud1 bin]# ./crsctl start resource ora.cssd
CRS-2672: Attempting to start 'ora.cssd' on 'cloud1'
CRS-2672: Attempting to start 'ora.diskmon' on 'cloud1'
CRS-2676: Start of 'ora.diskmon' on 'cloud1' succeeded
CRS-2676: Start of 'ora.cssd' on 'cloud1' succeeded
[root@cloud1 bin]#

then start asm instance

[grid@cloud1 ohasd]$ sqlplus "/as sysasm"

SQL*Plus: Release 12.1.0.1.0 Production on Mon Nov 11 08:20:11 2013

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ASM instance started

Total System Global Area 1135747072 bytes
Fixed Size                  2297344 bytes
Variable Size            1108283904 bytes
ASM Cache                  25165824 bytes
ASM diskgroups mounted

Sunday, October 27, 2013

Resolved : The slave I/O thread stops because master and slave have equal MySQL server UUIDs; these UUIDs must be different for replication to work.

Starting mysql 5.6 the server generates a true UUID in addition to the –server-id supplied by the user.
 So remove auto.cnf file under data directory then restart mysql server after that new auto.cnf file will be created .

Error:


  Last_IO_Error: Fatal error: The slave I/O thread stops because master and slave have equal MySQL server UUIDs; these UUIDs must be different for replication to work.

 Workaroud:

*Remove auto.cnf file under data dir
*Restart mysql server



Thursday, October 24, 2013

Starting MySQL...The server quit without updating PID file [FAILED]/mysqld.pid)

Error 1:


Starting MySQL...The server quit without updating PID file [FAILED]/mysqld.pid)

Check Mysql logfile


131014 14:12:53 [ERROR] Plugin 'audit_log' init function returned error.

131014 14:12:53 [ERROR] Plugin 'audit_log' registration as a AUDIT failed.

131014 14:12:53 [ERROR] Failed to initialize plugins.

131014 14:12:53 [ERROR] Aborting


Workaround:

rename or remove audit.log files and restart mysql services


Ref:
http://dev.mysql.com/doc/refman/5.6/en/audit-log-plugin-logging-control.html



Error 2:


InnoDB: Error: log file ./ib_logfile0 is of different size 0 50331648 bytes
InnoDB: than specified in the .cnf file 0 5242880 bytes!
131029 18:33:09 [ERROR] Plugin 'InnoDB' init function returned error.
131029 18:33:09 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
131029 18:33:09 [ERROR] Unknown/unsupported storage engine: InnoDB
131029 18:33:09 [ERROR] Aborting

Workaround :

remove ib_logfile1,ib_logfile0 files and start mysql services

 

ERROR 1123 (HY000): Can't initialize function 'audit_log'; Plugin initialization function failed

Error:

Master >install plugin audit_log soname 'audit_log.so';
ERROR 1123 (HY000): Can't initialize function 'audit_log'; Plugin initialization function failed.

Workaround:

Add below lines on my.cnf file and restart mysql services

plugin-load=audit_log.so
audit-log=FORCE_PLUS_PERMANENT



Saturday, September 21, 2013

ORA-12528: TNS:listener: all appropriate instances are blocking new connections

Error:

ORA-12528: TNS:listener: all appropriate instances are blocking new connections

Workaround:

Add (UR = A) entry in tns entry like below


TESTS =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.0.34)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = tests)
     (UR = A)
    )
  )

Wednesday, August 21, 2013

ORA-27504: IPC error creating OSD context

Error:

Errors in file /u01/app/oracle/admin/DEMO/udump/demo1_ora_29106.trc:
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:skgxnqtsz failed with status: 0
ORA-27301: OS failure message: Error 0
ORA-27302: failure occurred at: SKGXN not av
clsssinit ret = 21
interconnect information is not available from OCR
  WARNING: No cluster interconnect has been specified. Depending on
           the communication driver configured Oracle cluster traffic
           may be directed to the public interface of this machine.
           Oracle recommends that RAC clustered databases be configured
           with a private interconnect for enhanced security and
           performance.

Workaround:

Add this parameter in init file cluster_interconnects=192.168.1.201

ORA-15183: ASMLIB initialization error

Below errors received while creating 10g database on 11gR2 grid infrastructure 


Error:

Errors in file /u01/app/oracle/admin/DEMO/bdump/demo1_rbal_30217.trc:
ORA-15183: ASMLIB initialization error [driver/agent not installed]
Thu Jul 25 21:31:19 IST 2013
Errors in file /u01/app/oracle/admin/DEMO/bdump/demo1_rbal_30217.trc:
ORA-15183: ASMLIB initialization error [/opt/oracle/extapi/32/asm/orcl/1/libasm.so]
ORA-15183: ASMLIB initialization error [driver/agent not installed]
Thu Jul 25 21:31:19 IST 2013
Errors in file /u01/app/oracle/admin/DEMO/bdump/demo1_psp0_30162.trc:
ORA-00600: internal error code, arguments: [kfkLoadByNum03], [0x20006728], [], [], [], [], [], []
Thu Jul 25 21:31:20 IST 2013
Errors in file /u01/app/oracle/admin/DEMO/bdump/demo1_psp0_30162.trc:
ORA-00600: internal error code, arguments: [kfkLoadByNum03], [0x20006728], [], [], [], [], [], []
Thu Jul 25 21:31:20 IST 2013
PSP0: terminating instance due to error 490
Thu Jul 25 21:31:20 IST 2013
Trace dumping is performing id=[cdmp_20130725213120]
Thu Jul 25 21:31:22 IST 2013
Dump system state for local instance only
System State dumped to trace file /u01/app/oracle/admin/DEMO/bdump/demo1_diag_30160.trc
Thu Jul 25 21:31:22 IST 2013
Trace dumping is performing id=[cdmp_20130725213122]

Workaround:
          
as root user:
# cd $ORACLE_HOME/bin
# chgrp asmadmin oracle
# chmod 6751 oracle
# ls -l oracle


          
$ su - grid
[grid@vm11 ]$ /usr/sbin/oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
[grid@vm11 ]$ cd /u01/app/11.2.0.3/bin/
[grid@vm11 bin]$ ./setasmgidwrap o=/u01/app/oracle/product/10.2.0/bin/oracle
[grid@vm11 bin]$
 

How to add shared disks in Virtual Box





VBoxManage createhd --filename G:\VBOX\shared\asm1.vdi --size 1024 --format VDI --variant Fixed
VBoxManage createhd --filename G:\VBOX\shared\asm2.vdi --size 1024 --format VDI --variant Fixed
VBoxManage createhd --filename G:\VBOX\shared\asm3.vdi --size 4096 --format VDI --variant Fixed



VBoxManage.exe storageattach vm31 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "G:\VBOX\shared\asm1.vdi" --mtype shareable
VBoxManage.exe storageattach vm31 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "G:\VBOX\shared\asm2.vdi" --mtype shareable
VBoxManage.exe storageattach vm31 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "G:\VBOX\shared\asm3.vdi" --mtype shareable


VBoxManage.exe storageattach vm32 --storagectl "SATA" --port 1 --device 0 --type hdd --medium "G:\VBOX\shared\asm1.vdi" --mtype shareable
VBoxManage.exe storageattach vm32 --storagectl "SATA" --port 2 --device 0 --type hdd --medium "G:\VBOX\shared\asm2.vdi" --mtype shareable
VBoxManage.exe storageattach vm32 --storagectl "SATA" --port 3 --device 0 --type hdd --medium "G:\VBOX\shared\asm3.vdi" --mtype shareable

PRCT-1011 : Failed to run "kfod" error while upgrading GI from 11.2.0.3 to 12.1.0.1



ERROR:
PRCT-1011 : Failed to run "kfod". Detailed error: WARNING: Using brute force method to determine the size of /dev/raw/rawctl.| There will be performance issues. Please check configuration to determine the cause for the failure of ioctl


Workaround:

Please remove  /dev/raw/rawctl file then retry upgrade

Saturday, July 6, 2013

Oracle 12c Database Installation on Oracle Linux x86-64


 Install 64bit OS:

Download and Install Oracle Linux 64bit on Virtual Box.
Download OTN: Oracle Database 12c Release 1 (12.1.0.1) Software (64-bit).



Require RPMs

rpm -Uvh zlib-devel-1.2.3-25.el6.x86_64.rpm
rpm -Uvh binutils*
rpm -Uvh compat-libcap1-1.10-1.x86_64.rpm
rpm -Uvh compat-libstdc++-33-3.2.3-69.el6.x86_64.rpm
rpm -Uvh kernel-headers-2.6.32-71.el6.x86_64.rpm
rpm -Uvh glibc-headers-2.12-1.7.el6.x86_64.rpm
rpm -Uvh glibc-devel-2.12-1.7.el6.x86_64.rpm
rpm -Uvh unixODBC-2.2.14-11.el6.x86_64.rpm
rpm -Uvh unixODBC-devel-2.2.14-11.el6.x86_64.rpm
rpm -Uvh libaio-devel-0.3.107-10.el6.x86_64.rpm
rpm -Uvh libaio-0.3.107-10.el6.i686.rpm
rpm -Uvh libaio-devel-0.3.107-10.el6.i686.rpm
rpm -Uvh libaio-0.3.107-10.el6.x86_64.rpm
rpm -Uvh ksh-20100621-2.el6.x86_64.rpm
rpm -Uvh ppl-0.10.2-11.el6.x86_64.rpm
rpm -Uvh cloog-ppl-0.15.7-1.2.el6.x86_64.rpm
rpm -Uvh mpfr-2.4.1-6.el6.x86_64.rpm
rpm -Uvh cpp-4.4.4-13.el6.x86_64.rpm
rpm -Uvh gcc-4.4.4-13.el6.x86_64.rpm
rpm -Uvh gcc-c++-4.4.4-13.el6.x86_64.rpm


 
Kernel Parameters :

Add below parameters on /etc/sysctl.conf

fs.file-max = 6815744
kernel.sem = 250 32000 100 128
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
fs.aio-max-nr = 1048576
net.ipv4.ip_local_port_range = 9000 65500


Add below entries on /etc/security/limits.conf

oracle   soft   nofile    1024
oracle   hard   nofile    65536
oracle   soft   nproc    2047
oracle   hard   nproc    16384
oracle   soft   stack    10240
oracle   hard   stack    32768


Create group and user:

groupadd -g 54321 oinstall
groupadd -g 54322 dba
groupadd -g 54323 oper
useradd -g oinstall -G dba,oper oracle


Invoke runinstaller

./runInstaller

 





SQL> select BANNER from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
PL/SQL Release 12.1.0.1.0 - Production
CORE    12.1.0.1.0      Production
TNS for Linux: Version 12.1.0.1.0 - Production
NLSRTL Version 12.1.0.1.0 - Production
 

Friday, June 7, 2013

Manual Switchover in Databguard

Primary :
SQL>select switchover_status from v$database;
SQL>alter database commit to switchover to physical standby with session shutdown;
SQL>shutdown immediate
SQL>startup nomount
SQL>alter database mount standby database;
Standby :
SQL>select switchover_status from v$database;
SQL>alter database commit to switchover to primary;
SQL>shutdown immediate
SQL>startup
New standby:SQL>alter database recover managed standby database disconnect from session;

Wednesday, June 5, 2013

pre-upgrade check for GI home

Pre-upgrade check tool

Syntax : runcluvfy.sh stage -pre crsinst -upgrade [-n ] [-rolling] -src_crshome -dest_crshome -dest_version [-fixup [-fixupdir]] [-verbose]

For example, to upgrade a 3-node Oracle Clusterware in /u01/app/11.1 to 11.2.0.3 in /u01/app/grid in
rolling fashion, execute the following:
$ runcluvfy.sh stage -pre crsinst -upgrade -n ,, - rolling -src_crshome /u01/app/11.1 -dest_crshome /u01/app/grid -dest_version 11.2.0.3.0 -fixup -fixupdir /tmp -verbose

Monday, March 18, 2013

Register listener manually on OCR

crs_stat -p ora.vm2.LISTENER_VM2.lsnr > /tmp/list

Modify relevent parameters on /tmp/list file after that register that profile on OCR

mv /tmp/list /tmp/ora.vm2.LISTENER_VM2_T.lsnr.cap
crs_register ora.vm2.LISTENER_VM2_T.lsnr -dir /tmp/
srvctl start listener -l LISTENER_VM2_T -n vm2

Saturday, March 16, 2013

Mysql Replication (Master-Slave)

Mysql Replication (Master - Slave):

Master Server    : 192.168.0.109
Slave Server    : 192.168.0.201

Master Node:
1. Backup the cnf file and add below entries on cnf file

key_buffer = 128m
myisam_sort_buffer_size = 32m
query_cache_size= 32M
sort_buffer_size = 128M
server-id = 1 
log-slow-queries = /applog/mysqld-slow.log
long_query_time = 2
log-bin = /applog/mysql-bin


2. Create Replication user on Master Node

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO  repl@"192.168.0.201" IDENTIFIED BY 'repl';

Verify user and grants

select host,user from user;
MASTER >show grants for repl@"192.168.0.201";
+-------------------------------------------------------------------------------------------------------------------------------------------------+
| Grants for repl@192.168.0.201                                                                                                                   |
+-------------------------------------------------------------------------------------------------------------------------------------------------+
| GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'repl'@'192.168.0.201' IDENTIFIED BY PASSWORD '*A424E797037BF97C19A2E88CF7891C5C2038C039' |
+-------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

MASTER >


3.Restart mysql

sudo /etc/init.d/mysql restart


4.Backup Master site directory.

block the tables for updates, record bin log position as root user

mysql –u root –p mysql

MASTER>FLUSH TABLES WITH READ LOCK;

record bin log position
MASTER >show master status;
+-------------------+----------+--------------+------------------+
| File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+-------------------+----------+--------------+------------------+
| master-bin.000021 |      107 |              |                  |
+-------------------+----------+--------------+------------------+
1 row in set (0.00 sec)


backup mysql directory

tar -cvf /backup/mysql_bkp.tar /u04/

unlock the tables

MASTER >UNLOCK TABLES;


transfer mysql_bkp.tar backup from master to slave node

scp mysql_bkp.tar mysql@192.168.0.201:/tmp/.


SLAVE Node:

1. Stop mysql

sudo /etc/init.d/mysql stop

2. delete old mysql files
$rm –rf mysql
$tar xvf mysql_bkp.tar

3.Remove  bin log files and bin log index files
$cd /u04/mysql/data
$rm db1-bin.*

4.Add below entry on my.cnf file

server-id = 2

5.start mysql

sudo /etc/init.d/mysql start

6.Issue below statement to add master

CHANGE MASTER TO  MASTER_HOST='192.168.0.109',  MASTER_USER='repl',  MASTER_PASSWORD='repl', MASTER_LOG_FILE='master-bin.000021', MASTER_LOG_POS=107;


7. Start slave

mysql> start slave;


*Check Master staus

mysql>SHOW MASTER STATUS \G
mysql>SHOW MASTER LOGS;

*Check Slave Status

mysql> SHOW SLAVE STATUS \G
If we will see below status means slave is running fine

Slave_IO_Running: Yes
Slave_SQL_Running: Yes,


Test:

create table emp (name char(10),age integer(2));
insert into emp(name,age) values('Tommy',20);


MASTER >use test;
Database changed
MASTER >create table emp (name char(10),age integer(2));
Query OK, 0 rows affected (0.27 sec)

MASTER >insert into emp(name,age) values('Tommy',20);
Query OK, 1 row affected (0.01 sec)

MASTER >

Verify on Slave site

SLAVE >use test;
Database changed
SLAVE >show tables;
+----------------+
| Tables_in_test |
+----------------+
| customers      |
| emp            |
+----------------+
2 rows in set (0.00 sec)

SLAVE >select * from emp;
+-------+------+
| name  | age  |
+-------+------+
| Tommy |   20 |
+-------+------+
1 row in set (0.00 sec)



Notes:

Using MySQL 5.0 and back
SHOW VARIABLES LIKE 'Slave_running';

Using MySQL 5.1+/5.5
select COUNT(1) SlaveThreads
from information_schema.processlist
where user = 'system user';


show slave status;

How To Repair MySQL Replication

SHOW SLAVE STATUS \G

STOP SLAVE;

 simply skip the invalid SQL query:

mysql> SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;

 If you'd like to skip two queries, you'd use SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 2; instead and so on.



mysql> START SLAVE;

Check Slave status

mysql> SHOW SLAVE STATUS \G

mysql> SHOW SLAVE STATUS \G

Tuesday, March 12, 2013

How to enable SSL on MYSQL?

Verify SSL status:

MASTER >show variables like "%ssl%";
+---------------+-----------------------------------+
| Variable_name | Value                             |
+---------------+-----------------------------------+
| have_openssl  | DISABLED                          |
| have_ssl      | DISABLED                          |
| ssl_ca        |                                   |
| ssl_capath    |                                   |
| ssl_cert      |                                   |
| ssl_cipher    |                                   |
| ssl_key       |                                   |
+---------------+-----------------------------------+

Disabled means --> mysql has ssl support but it's not enabled.

NO means --> don't have ssl support.



Create SSL Certificate:

cd /home/mysql/certs
openssl genrsa -out ca-key.pem 2048;
openssl req -new -x509 -nodes -days 3 -key ca-key.pem -out ca-cert.pem;
openssl req -newkey rsa:2048 -days 3 -nodes -keyout server-key.pem -out server-req.pem;
openssl x509 -req -in server-req.pem -days 3 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out server-cert.pem;
openssl req -newkey rsa:2048 -days 3 -nodes -keyout client-key.pem -out client-req.pem;
openssl x509 -req -in client-req.pem -days 3 -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out client-cert.pem;


Add configuration:

Add below configuration in /etc/my.cnf


[mysqld]

ssl
ssl-cipher=DHE-RSA-AES256-SHA
ssl-ca=/home/mysql/certs/ca-cert.pem
ssl-cert=/home/mysql/certs/server-cert.pem
ssl-key=/home/mysql/certs/server-key.pem


[client]
ssl-ca=/home/mysql/certs/ca-cert.pem
ssl-cert=/home/mysql/certs/client-cert.pem
ssl-key=/home/mysql/certs/client-key.pem



Restart mysql:

sudo /etc/init.d/mysql restart

Test SSL :

MASTER >show variables like "%ssl%";
+---------------+-----------------------------------+
| Variable_name | Value                             |
+---------------+-----------------------------------+
| have_openssl  | YES                               |
| have_ssl      | YES                               |
| ssl_ca        | /home/mysql/certs/ca-cert.pem     |
| ssl_capath    |                                   |
| ssl_cert      | /home/mysql/certs/server-cert.pem |
| ssl_cipher    | DHE-RSA-AES256-SHA                |
| ssl_key       | /home/mysql/certs/server-key.pem  |
+---------------+-----------------------------------+



How to check ssl certificate validation?

$ openssl x509 -noout -in server-cert.pem -dates
notBefore=Mar 12 15:36:27 2013 GMT
notAfter=Mar 15 15:36:27 2013 GMT
 

$ openssl x509 -noout -in client-cert.pem  -dates
notBefore=Mar 12 15:36:52 2013 GMT
notAfter=Mar 15 15:36:52 2013 GMT




ERROR 2026 (HY000): SSL connection error: ASN: after date in the past

ssl certificate might be expired




Enable Auditing Mysql:

MASTER >show variables like '%audit%';
Empty set (0.00 sec)

MASTER >install plugin audit_log soname 'audit_log.so';
Query OK, 0 rows affected (0.23 sec)

MASTER >show variables like '%audit%';
+--------------------------+--------------+
| Variable_name            | Value        |
+--------------------------+--------------+
| audit_log_buffer_size    | 1048576      |
| audit_log_file           | audit.log    |
| audit_log_flush          | OFF          |
| audit_log_policy         | ALL          |
| audit_log_rotate_on_size | 0            |
| audit_log_strategy       | ASYNCHRONOUS |
+--------------------------+--------------+
6 rows in set (0.00 sec)



Mysql Monitor Agent Starting Issue:

Verify Agent Logfile.

2013-07-10 12:55:14: (critical) [unix:/var/lib/mysql/mysql.sock] the hostid from `mysql`.inventory doesn't match our agent's host-id (ssh:{9d:d3:97:9f:1f:3e:cc:c4:cf:a6:63:00:9f:08:2a:04} != ssh:{80:c0:99:c4:fc:6d:d4:eb:ed:70:80:07:cb:df:3b:ec}). Shutting down
2013-07-10 12:55:14: (critical) agent_mysqld.c:1349: agent_mysqld.c:1127: operation canceled as we are shutting down
2013-07-10 12:55:14: (critical) last message repeated 37 times
2013-07-10 12:55:14: (critical) job_collect_mysql.c:428: [unix:/var/lib/mysql/mysql.sock] executing 'SHOW /*!50000 ENGINE */ INNODB STATUS' failed:  (0)
2013-07-10 12:55:14: (critical) agent_mysqld.c:1349: agent_mysqld.c:1127: operation canceled as we are shutting down


Workaround :

Truncate truncate mysql.inventory table then start agent
 

Thursday, February 28, 2013

UTL_FILE example

The UTL_FILE package lets your PL/SQL programs read and write operating system (OS) text files.

Example:

declare
l_output utl_file.file_type;
begin
l_output := utl_file.fopen('DUMP_DIR','output.txt','w');
utl_file.put(l_output,'utl_file example');
utl_file.fclose(l_output);
end;

 

Sunday, February 3, 2013

11gR2 RAC Tips

Configure User Equivalence :

Run below script from Grid Software location to configure user equivalance.

./sshUserSetup.sh -user grid -hosts "vm3 vm4" -advanced -noPromptPassphrase

Role Separation in 11gR2 RAC:

1.Creating groups and  Users for GI software.

$groupadd -g 1000 oinstall
$groupadd -g 1200 asmadmin
$groupadd -g 1201 asmdba
$groupadd -g 1202 asmoper

$useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid -S /bin/bash -c "GI Owner"  grid

Select below options while installing GI software.

ASM Database Administrator(OSDBA) group --> asmdba
ASM Instance Administration Operator (OSOPER) group --> asmoper
ASM Instance Administrator (OSASM) group --> asmadmin

2.Creating groups and  Users for Database software.
$groupadd -g 1300 dba
$groupadd -g 1301 oper

useradd -m -u 1101 -g oinstall -G dba,oper,asmdba -d /home/oracle -S /bin/bash -c "Database Owner"  oracle


select below options while installing oracle software

Database administrator (OSDBA)group --> dba
Database operator (OSOPER) group --> oper


Friday, January 18, 2013

Errors & Workaround



ORA-01591: lock held by in-doubt distributed transaction

solution:

SQL> select local_tran_id from dba_2pc_pending;

LOCAL_TRAN_ID
------------------------------------------------------------------
6.33.124689

SQL> rollback force '6.33.124689';

Rollback complete.

SQL> exec dbms_transaction.purge_lost_db_entry('6.33.124689');

PL/SQL procedure successfully completed.

SQL> select local_tran_id from dba_2pc_pending;

******************************************************************************

ERROR:
ORA-12534: TNS:operation not supported

Work around

set WALLET_LOCATION parameter in sqlnet.ora instead of ENCRYPTION_WALLET_LOCATION

*****************************************************************************

ORA-01017: invalid username/password; logon denied


[oracle@vm6 admin]$ sqlplus /@hat  ==>even if correct password

SQL*Plus: Release 10.2.0.5.0 - Production on Fri Jan 18 19:17:24 2013

Copyright (c) 1982, 2010, Oracle.  All Rights Reserved.

ERROR:
ORA-01017: invalid username/password; logon denied


Enter user-name:



Solution : Remove encryption from sqlnet.ora

Before (sqlnet.ora)

ENCRYPTION_WALLET_LOCATION =
 (SOURCE=
   (METHOD=file)
     (METHOD_DATA=
       (DIRECTORY=/u01/app/oracle/admin/hat/wallet)))

After:

[oracle@vm6 admin]$ cat sqlnet.ora
WALLET_LOCATION =
 (SOURCE=
   (METHOD=file)
     (METHOD_DATA=
       (DIRECTORY=/u01/app/oracle/admin/hat/wallet)))

SQLNET.WALLET_OVERRIDE=TRUE
[oracle@vm6 admin]$ sqlplus /@hat

SQL*Plus: Release 10.2.0.5.0 - Production on Fri Jan 18 19:20:14 2013

Copyright (c) 1982, 2010, Oracle.  All Rights Reserved.


Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 - Production
With the Partitioning, Oracle Label Security, OLAP, Data Mining,
Oracle Database Vault and Real Application Testing options

SQL>
**********************************************************************

Error :

RMAN> register database;

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of register command at 01/24/2013 21:34:10
RMAN-03014: implicit resync of recovery catalog failed
RMAN-03009: failure of full resync command on default channel at 01/24/2013 21:34:10
ORA-06502: PL/SQL: numeric or value error
                              
Solution :

Change the compatible parameter to 11.2.0.2 and try again
*************************************************************************************
Error:

RMAN> register database;

database registered in recovery catalog
starting full resync of recovery catalog
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03008: error while performing automatic resync of recovery catalog
ORA-02291: integrity constraint (RMAN.RLH_F1) violated - parent key not found


Solution:


connect to catalog database and run

select 'EXEC DBMS_RCVCAT.UNREGISTERDATABASE('||DB_KEY||','||DBID||');'
 command, resetlogs_time
from rc_database
where name = '&db_name';


COMMAND                                                                                                                 RESETLOGS
------------------------------------------------------------------------------------------------------------ ---------
EXEC DBMS_RCVCAT.UNREGISTERDATABASE(110446,2179054324);                                                                 11-MAY-11


SQL> EXEC DBMS_RCVCAT.UNREGISTERDATABASE(110446,2179054324);

PL/SQL procedure successfully completed.

************************************************************************************************
Error:

P: [INS-40420] The current installation user is not the same as the installation owner of the existing Oracle Grid Infrastructure software.

Workaround:

While upgrading 10g RAC to 11gR2 RAC, GI(Grid Infrastructure) should be installed as existing 10g crs owner not grid user.


Friday, January 11, 2013

Relink New Feature on 11gR2

Normal relink method is $ORACLE_HOME/bin/relink all


We can use runInstaller to relink oracle binaries like below

$ORACLE_HOME/oui/bin/runInstaller -relink -waitForCompletion -maketargetsxml  $ORACLE_HOME/inventory/make/makeorder.xml  -logLocation $ORACLE_HOME/install ORACLE_HOME=$ORACLE_HOME > $ORACLE_HOME/install/relink.log 2>&1

Rename ASM disk group on 11gR2

1.First unmount the diskgroup
 

[oracle@vm6 ~]$ asmcmd umount DATA1

2.Verify whether diskgroup has been mounted or not

[oracle@vm6 ~]$ asmcmd lsdg
 

State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  1048576      5114     1525                0            1525              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576     10236    10186                0           10186              0             N  DATA2/

3.Rename diskgroup

[oracle@vm6 ~]$ renamedg phase=both dgname=DATA1 newdgname=DATA3 verbose=true

Parsing parameters..

Parameters in effect:

         Old DG name       : DATA1
         New DG name          : DATA3
         Phases               :
                 Phase 1
                 Phase 2
         Discovery str        : (null)
         Clean              : TRUE
         Raw only           : TRUE
renamedg operation: phase=both dgname=DATA1 newdgname=DATA3 verbose=true
Executing phase 1
Discovering the group
Performing discovery with string:
Identified disk ASM:/opt/oracle/extapi/32/asm/orcl/1/libasm.so:ORCL:VOL2 with disk number:0 and timestamp (32970697 -482714624)
Checking for hearbeat...
Re-discovering the group
Performing discovery with string:
Identified disk ASM:/opt/oracle/extapi/32/asm/orcl/1/libasm.so:ORCL:VOL2 with disk number:0 and timestamp (32970697 -482714624)
Checking if the diskgroup is mounted or used by CSS
Checking disk number:0
Generating configuration file..
Completed phase 1
Executing phase 2
Looking for ORCL:VOL2
Modifying the header
Completed phase 2
Terminating kgfd context 0xb7e70050

above command fails use asm_diskstring command

 renamedg phase=both dgname=DATA1 newdgname=DATA3 verbose=true asm_diskstring='ORCL:VOL2'

4.Mount the new diskgroup

[oracle@vm6 ~]$ asmcmd mount DATA3

5.Verify the diskgroup
 

[oracle@vm6 ~]$ asmcmd lsdg
State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  1048576      5114     1525                0            1525              0             N  DATA/
MOUNTED  EXTERN  N         512   4096  1048576     10236    10186                0           10186              0             N  DATA2/
MOUNTED  EXTERN  N         512   4096  1048576      1019      967                0             967              0             N  DATA3/
[oracle@vm6 ~]$

Reconfigure HAS and CSS on non RAC ASM 11gR2

run below script as root user

/perl/bin/perl -I/perl/lib -I/crs/install /crs/install/roothas.pl -deconfig

/perl/bin/perl -I/perl/lib -I/crs/install /crs/install/roothas.pl 

above commands give error .use force option

/perl/bin/perl -I/perl/lib -I/crs/install /crs/install/roothas.pl -deconfig -force
/perl/bin/perl -I/perl/lib -I/crs/install /crs/install/roothas.pl



disable/enable auto start

# cd /bin
# ./crsctl stop has
# ./crsctl start has
# ./crsctl disable has
# ./crsctl enable has



catrelod script gives error while downgrading database from 11.2.0.3 to 11.2.0.2

Issue:

catrelod scripts gives error

ERROR at line 1:
ORA-20000: Upgrade from version 11.2.0.2.0 cannot be downgraded to version 11.2.0.1
ORA-06512: at line 7


Workaroud:

Refer Metalink ID : 1367333.1.this is due to unpublished bug

Tuesday, January 8, 2013

Configure TAF


Configure  the server side failover parameters to the service:


srvctl add service -d car -s car-srv -r "car1,car2" -P BASIC --->10g

srvctl add service -d car -s car-srv -l PRIMARY -e SELECT -m BASIC -q TRUE -r car1,car2 -w 5 -z 180 -P BASIC --->11gR2

srvctl start service -d car -s car-srv

srvctl config service -d car

Configure Local_Listener and Remote_Listener parameters on db side.

Show parameter listener

Below statement only for 10g 

execute dbms_service.modify_service (service_name => 'car-srv' -
, aq_ha_notifications => true -
, failover_method => dbms_service.failover_method_basic -
, failover_type => dbms_service.failover_type_select -
, failover_retries => 180 -
, failover_delay => 5 -
, clb_goal => dbms_service.clb_goal_long);

Verify status:

select name, failover_method, failover_type, failover_retries,goal, clb_goal,aq_ha_notifications
from dba_services where service_id = 3

NAME METHOD TYPE RETRIES GOAL CLB_GOAL AQNOT
------ ----------- ---------- -------- ---------- -------- -----
car-srv BASIC SELECT 180 NONE LONG YES

Check Listener services:

lsnrctl services


Client side connect time failover :

This is done by tnsentry.Don't need update on db level.

CAR=
 (DESCRIPTION =
    (ADDRESS_LIST =
      (ADDRESS = (PROTOCOL = TCP)(HOST = vm1-vip.domain.com)(PORT = 1521))
      (ADDRESS = (PROTOCOL = TCP)(HOST = vm2.vip.domain.com)(PORT = 1521))
     )
    (LOAD_BALANCE=YES)
    (CONNECT_DATA =
      (SERVICE_NAME = car-srv)
      (FAILOVER_MODE =
       (TYPE=SELECT)
       (METHOD=BASIC)
      )
    )
  )

Virtual IP:

VIP address is public ip because it's attached with public ip interface. VIP is managd by oracle clusterware.
We should use vip instead of public ip .Because vip is doing Fast Failover.Private IP  won't visible from outside of server and it should not be used.

For ex: If we will public ip address and one of the rac node will be crased then the client will make a request to the node on the public IP and will wait for a TCP timeout (usually 60 seconds) before moving to the next host in the connection list.

If we will user vip ,the client does not have to wait for a TCP timeout to move to the next node in the connection list



Failover Method:

BASIC               :session will not reconnect to a surviving node until the failover occurs.
PRECONNECT :shadow process will be  created on a backup instance to reduce failover time.

Failover Type:

SELECT : session will be authenticated on a surviving node but as well as SELECT statements will be re-executed
Session    :  In a SESSION failover, only the users session will be  re-authenticated.

11gR2 TAF Example:

TAF is a client side features that allows clients to reconnect on surviving node in the event of failure on instance.Reconnect happens automatically from with in OCI library.
*Any uncommited transactions will be rolled back.
*Server side program variables and session properties will be lost
*The select statements automatically re-executed on the new connection with the cursor positioned on the row on which it was positioned prior to the failover.

srvctl add service -d ten -s tenpp -q TRUE -P BASIC -j LONG -e SELECT -m BASIC -z 5 -w 5 -r ten1,ten2

Options:
-d  db_unique name
-s  Service name
-r  preferred instances
-a available instances.specify available instances if preferred instances are not specified
-g Serverpool name
-c uniform,singleton
-p failover method (None,Basic,Preconnect)
-e fileover type(Session,Select,None)
-y specify service startup (Manaul,Automatic)
-j connection load balancing goal(Short,Long)
-B enable or disable load balancing advisory(none,service_time,throughput)
-m Failover method optin(none,basic)
-z specify number of times to attempt to connect after a failover (default 5 attempts)
-w Specify the amount of time in seconds to wait between connect attempts (default 1 sec)



[oracle@vm12 ~]$ srvctl config service -d ten -s tenpp
Service name: tenpp
Service is enabled
Server pool: ten_tenpp
Cardinality: 2
Disconnect: false
Service role: PRIMARY
Management policy: AUTOMATIC
DTP transaction: false
AQ HA notifications: true
Failover type: SELECT
Failover method: BASIC
TAF failover retries: 5
TAF failover delay: 5
Connection Load Balancing Goal: LONG
Runtime Load Balancing Goal: NONE
TAF policy specification: BASIC
Edition:
Preferred instances: ten1,ten2
Available instances: