Archive

Author Archive

WARNING: Disk 0 (DATADG2_0000) in group 2 will be dropped in: (1101045) secs on ASM inst 1

April 24, 2018 Leave a comment

Symptoms

  1. One of disk in the diskgroup is offline
  2. WARNING: Disk 0 (DATADG2_0000) in group 2 will be dropped in: (1101045) secs on ASM inst

Fixed.

  1. Make sure the disk is not used
  2. Force to drop it from the diskgroup as below

SQL> alter diskgroup DATADG2 drop disk DATADG2_0000 force;

Diskgroup altered.

Advertisements
Categories: ASM, Uncategorized

cron job can’t run

April 19, 2018 Leave a comment

Findings

1. Cron Job can’t run

2. Cron log show

# cd /var/cron

# tail log
! c queue max run limit reached Thu Apr 19 10:31:00 2018
! rescheduling a cron job Thu Apr 19 10:31:00 2018
! c queue max run limit reached Thu Apr 19 10:31:00 2018
! rescheduling a cron job Thu Apr 19 10:31:00 2018
! c queue max run limit reached Thu Apr 19 10:31:00 20183.

3. Check Running cron jobs

# svcs -p system/cron
STATE STIME FMRI
online 2016 svc:/system/cron:default
2016 381 cron
# ptree 381
381 /usr/sbin/cron
17147 sh -c /export/home/oracle/dba/stby/apply_tsmsstby.sh >> /export/home/oracle/dba
17150 /bin/sh /export/home/oracle/dba/stby/apply_tsmsstby.sh
17530 /bin/sh /export/home/oracle/dba/stby/apply_tsmsstby.sh
17573 scp /export/home/oracle/dba/stby/tsmsstby/thread_1_seq_190481.1264.973347007 10
17574 /usr/bin/ssh -x -oForwardAgent no -oClearAllForwardings yes 10.252.16.14 scp -t
26963 sh -c /export/home/oracle/dba/stby/apply_empcstby.sh >> /export/home/oracle/dba
26968 /bin/sh /export/home/oracle/dba/stby/apply_empcstby.sh
26996 ssh 10.252.16.14 /export/home/oracle/dba/stby/get_archive_seq.sh empcstby 1
2972 sh -c /export/home/oracle/dba/stby/apply_tsmsstby.sh >> /export/home/oracle/dba
2975 /bin/sh /export/home/oracle/dba/stby/apply_tsmsstby.sh
2987 ssh 10.252.16.14 /export/home/oracle/dba/stby/get_archive_seq.sh tsmsstby 1
7505 sh -c /export/home/oracle/dba/stby/chksync_tsmsstby.sh >> /export/home/oracle/d
7509 /bin/bash /export/home/oracle/dba/stby/chksync_tsmsstby.sh
7518 /bin/bash /export/home/oracle/dba/stby/chksync_tsmsstby.sh
7524 ssh 10.252.16.14 /export/home/oracle/dba/stby/get_archive_time.sh tsmsstby

Since there were exceeded cron jobs running, it caused the errors in cron log

Fix: Kill the cron jobs by kill -9 381 (cron pid)

Categories: Solaris 11

cannot enable executable stack as shared object requires: Permission denied

March 20, 2017 Leave a comment

Symptoms

  1. Can’t automatically mount the filesystem

ora.acfsdg.vol1.acfs

OFFLINE ONLINE       testprdvm01            mounted on /utility_

files,STABLE

OFFLINE OFFLINE      testprdvm02            volume /utility_file

s offline,STABLE

2. Manually start by command srvctl

[root@testprdvm02 ~]# srvctl start filesystem -d /dev/asm/vol1-116

PRCR-1079 : Failed to start resource ora.acfsdg.vol1.acfs

CRS-5016: Process “/bin/mount” spawned by agent “ORAROOTAGENT” for action “start” failed: details at “(:CLSN00010:)” in “/u01/app/grid/diag/crs/testprdvm02/crs/trace/crsd_orarootagent_root.trc”

Error mounting file system ‘/dev/asm/vol1-116’ on ‘/utility_files’: /sbin/mount.acfs.bin: error while loading shared libraries: libclntsh.so.12.1: cannot enable executable stack as shared object requires: Permission denied

CRS-2674: Start of ‘ora.acfsdg.vol1.acfs’ on ‘testprdvm02’ failed

Fix:

Solution :
1. Open file /etc/selinux/config
2. change “SELINUX=enforcing” to “SELINUX=disabled”
3. Reboot

 

Categories: ORA-XXX, Uncategorized

ORA-00603: ORACLE server session terminated by fatal error

March 13, 2017 Leave a comment

Symptoms

  1. JDBC Client reported connection failure happened
  2. DB alert showed below errors at time the connection failure happened

mtype: 61 process 14042 failed because of a resource problem in the OS. The OS has most likely run out of buffers (rval: 4)
Errors in file /u02/oradiag/diag/rdbms/roamprd/roamprd1/trace/roamprd1_ora_14042.trc  (incident=29338):
ORA-00603: ORACLE server session terminated by fatal error
ORA-27504: IPC error creating OSD context
ORA-27300: OS system dependent operation:sendmsg failed with status: 105
ORA-27301: OS failure message: No buffer space available
ORA-27302: failure occurred at: sskgxpsnd2
opiodr aborting process unknown ospid (14042) as a result of ORA-603
2017-03-12 05:34:31.770000 +08:00
Sweep [inc][29338]: completed

Cause

High value of MTU for loop back adapter  on UEK3 causes the issue.

Fix

  1. Change loop back adapter MTU size

ifconfig lo mtu 16436

  1. Add MTU=16436 to /etc/sysconfig/network-scripts/ifcfg-lo
  2. Set vm.min_free_kbytes=32768 (32M)

 

Categories: ORA-XXX, Uncategorized

DBCA, running on oracle zone, pops up “Out of Memory”

April 20, 2016 Leave a comment

Runing on oracle zone, dbca showed “out of memory”. It may caused by  setting sga size higher than the capped-memory: [ locked: 2G] in zone configuration.

capped-memory:
physical: 4G
[swap: 4G]
[locked: 2G]

Fix: Incraese the capped-memory :[locked: <value higer than SGA>]

 

ORA-00392: log 1 of thread 1 is being cleared, operation not allowed

January 26, 2016 Leave a comment

Sympton

1. After restoring the backup of database from ASM to filesystem, it showed messages follows when opening the database with restlogs

SQL> alter database open resetlogs;
alter database open resetlogs
*
ERROR at line 1:
ORA-00392: log 1 of thread 1 is being cleared, operation not allowed
ORA-00312: online log 1 thread 1: ‘/u02/oradata/neprd6/redo01.log’

2. Check the database log status, it showed status of log files

SQL> select group#, status from

GROUP# STATUS
———- —————-
1 CLEARING_CURRENT
2 CLEARING
3 CLEARING
4 CLEARING
5 CLEARING
6 CLEARING_CURRENT
7 CLEARING
8 CLEARING
9 CLEARING

9 rows selected.

Fix:

  1. Clear the log manually SQL> alter database clear logfile group 1;

    Database altered.

    SQL> alter database clear logfile group 6;

    Database altered.

    SQL> select group#, status from v$log;

    GROUP# STATUS
    ———- —————-
    1 CURRENT
    2 CLEARING
    3 CLEARING
    4 CLEARING
    5 CLEARING
    6 CURRENT
    7 CLEARING
    8 CLEARING
    9 CLEARING

    9 rows selected.

  2. Start the database with resetlogs
    SQL> alter database open resetlogs;

    Database altered.

 

vnet: [ID 214931 kern.info] vnet0: vnet_addmac: programming macaddr(2:8:20:ad:ea:21) is not supported

November 10, 2015 Leave a comment

I created VNIC withing LDom Guest (devdbvm) by dladm create-vnic -l net0 vnic3, it showed failures and

Nov 10 11:21:58 devdbvm vnet: [ID 214931 kern.info] vnet0: vnet_addmac: programming macaddr(2:8:20:ad:ea:21) is not supported

Solution:

  1. Stop the guest

root@jecdb02:~# ldm stop -f devdbvm
LDom devdbvm stopped

2. Set mac-address of vnet with 8 X alt-mac-addres with command
root@jecdb02:~#  ldm set-vnet alt-mac-addrs=auto,auto,auto,auto,auto,auto,auto,auto vnet0 devdbvm

3. Show the binding

ldm list-bindings devdbvm

vnet0            primary-vsw0@primary        0    network@0  00:14:4f:fa:f8:68        1                         1500
00:14:4f:fa:6a:04
00:14:4f:f8:76:eb
00:14:4f:fa:de:4d
00:14:4f:fb:aa:d2
00:14:4f:fa:da:e2
00:14:4f:fb:9a:40
00:14:4f:fb:d3:6b
00:14:4f:fb:63:f5

4. Start the VM

Categories: LDom