Adop is not to able to detect any valid application tier nodes

Adop is not to able to detect any valid application tier nodes

Issue:

Running ADOP on 12.2 is failing at initial Validation with error “adop is not able to detect any valid application tier nodes”
Enter the APPS password: 
Enter the SYSTEM password: 
Enter the WLSADMIN password: 

Validating credentials... 

Initializing... 
Run Edition context : /u01/oracle/VIS/fs1/inst/apps/VIS_ora01/appl/admin/VIS_ora01.xml 
Patch edition context: / u01/oracle/VIS/fs2/inst/apps/VIS_ora01/appl/admin/VIS_ora01.xml 
*******FATAL ERROR******* 
PROGRAM : (/u01/oracle/VIS /fs1/EBSapps/appl/ad/12.0.0/bin/adzdoptl.pl) 
TIME : Wed Apr 22 04:40:38 2015 
FUNCTION: ADOP::GlobalVars::_GetMandatoryArgs [ Level 1 ] 
ERRORMSG: adop is not able to detect any valid application tier nodes in 
ADOP_VALID_NODES table. Ensure autoconfig is run on all nodes. 


[STATEMENT] Please run adopscanlog utility, using the command 

"adopscanlog -latest=yes" 

to get the list of the log files along with snippet of the error message corresponding to each log file. 


adop exiting with status = 255 (Fail) 

Cause

Upon investigation we found that Database listener is not listening ebs_patch service which is mandatory for 12.2 to work on any adop activities.

[oracle@ora01 ~]$ lsnrctl status $ORACLE_SID

LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 23-APR-2015 12:20:02

Copyright (c) 1991, 2013, Oracle.  All rights reserved.

Alias                     VIS

Listening Endpoints Summary...

.

.

Services Summary...

Service "VIS" has 2 instance(s).

  Instance "VIS", status UNKNOWN, has 1 handler(s) for this service...

  Instance "VIS", status READY, has 1 handler(s) for this service...

The command completed successfully

Solution

1)     Make sure that Database init parameter has service name ebs_patch defined

2)     Connect to Database as sysdba

sqlplus / as sysdba

SQL> show parameter service_name

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

service_names                        string      VIS, ebs_patch

                If this is there, set the parameter as below

                SQL> alter system set service_names=’ebs_patch’,’VIS’ scope=spfile;

3)     Verify init parameter LOCAL_LISTENER and  set it to <DB_NAME>_LOCAL

                                 SQL> alter system set local_listener=’VIS_LOCAL’ scope=spfile;

4)     Bounce the application & database services

Site Name profile in 12.2

We generally define “Site Name” profile at site level to identify forms session distinctly when we have multiple EBS Sessions. In prior releases upto 12.1.3 this can be directly modified and the change remains permanent until you make changes manually.

From 12.2, Oracle changed the way how it is handled. The value of Site Name defaults to parameter value s_sitename defined in context file. So, whenever we execute autoconfig it overwrites “Site Name” with s_sitename  parameter

The script that  updates the value when autoconfig is ran: $INST_TOP/admin/install/txkappsprf.sql

 In order to customize the change according to our requirement we can perform steps as below:

1)      Identify the template file that needs to be customized.

$AD_TOP/bin/adtmplreport.sh contextfile=$CONTEXT_FILE target=$INST_TOP/admin/install/txkappsprf.sql

This script provides log file which gives template file details:

TEMPLATE FILE   : /u01/tech_st/fs1/EBSapps/appl/fnd/12.0.0/admin/template/txkappsprf.sql

TARGET FILE     : /u01/tech_st/fs1/inst/apps/endecebs_endecaapps/admin/install/txkappsprf.sql

2)      Navigate to TEMPLATE File location $FNS_TOP/template and create directory “custom”

$ mkdir custom

3)      Switch to “custom” directory and copy the template file to this location.

4)      Edit the content of this newly copied file txkappsprf.sql under $FND_TOP/admin/template/custom as below

DEFINE SITE_NAME = "%s_sitename%"

To

DEFINE SITE_NAME = "<Your customized message>"

After these changes are made whenever you run autoconfig, the Site Name will be updated based on your custom template file. This can be added to your post clone steps based on your requirements.

Failed to start EBS 12.2.4 Managed services

Failed to start EBS 12.2.4 Managed services

Completed the upgrade of E-Business Suite to 12.2.4 and tried to start all the application services using adstrtal.sh script as below.

$ adstrtal.sh apps/<appspwd>

The script exited with status ‘1’ and examining the log file $INST_TOP/logs/appl/admin/log/adstrtal.log showed failure of all the managed servers

  Web Application Services                                                                            Enabled

  Web Application Services   oacore_server1                                admanagedsrvctl.sh         Failed

  Web Application Services   forms_server1                                 admanagedsrvctl.sh         Failed

  Web Application Services   oafm_server1                                  admanagedsrvctl.sh         Failed

  Web Application Services   forms-c4ws_server1                            admanagedsrvctl.sh         Failed

Reviewed one of the managed server startup logfile to identify the root cause of the failure.

Oacore_server1 logfile under $FMW_HOME/ user_projects/domains/EBS_domain_orcl/servers/oacore_server1/logs/   showed below error details

<Authentication denied: Boot identity not valid; The user name and/or password from the boot identity file (boot.properties) is not valid. The boot identity may have been changed since the boot identity file was created. Please edit and update the boot identity file with the proper values of username and password. The first time the updated boot identity file is used to start the server, these new values are encrypted.>

This error indicated that there is corruption happened with the boot.properties and this needs to be fixed.

Solution:

As initial fix, backed up existing boot.properties file under $FMW_HOME/ user_projects/domains/EBS_domain_orcl/servers/oacore_server1/security and created a new boot.properies with below details

$cat boot.properties

username=weblogic

password=weblogic

After creation of this to encrypt this start oacore_server1 managed server as below

$FMW_HOME/ user_projects/domains/EBS_domain_orcl/bin/startManagedWebLogic.sh oacore_server1 <admin_url>

Performed these steps to the remaining failed managed servers.

Now stopped all the managed services from console and then remaining services using adstpall.sh script.

Then restarted all the services using adstrtal.sh script and all the services came up successfully. But when later tried to stop services using adstpall.sh script couldn’t bring down the managed services.

Upon further troubleshooting found that the startup/shutdown arguments for managed servers are incorrect resulting in failures. This is found in config.xml file under $FMW_HOME/user_projects/domains/<domain_name>/config.

By reviewing them process argument details it showed that there is an invalid value for the property -Dweblogic.management.server for all the managed servers. Performed the below action plan to fix the issue.

1.       Log in in to the Administration Console.

2.       In the Change Center, click Lock & Edit.

3.       In the left pane of the Console, expand Environment and select Servers.

4.       In the Servers table, click the name of a Managed Server(oacore_server1).

5.       Select Configuration > Server Start

6.       In 'Arguments', correct the property value -Dweblogic.management.server to the correct value as below.

-XX:PermSize=128m -XX:MaxPermSize=384m -Xms512m -Xmx512m -….

-Dweblogic.Name=oacore_server1 -Dweblogic.management.server=http://orc11.localdomain:7006 -Djava.library.path=$MW_HOME/webtier/jdk/jre/lib/amd64:$MW_HOME/webtier/jdk/jre/lib/amd64/server:$MW_HOME/webtier/jdk/jre/lib/amd64/native_threads…

To

-XX:PermSize=128m -XX:MaxPermSize=384m -Xms512m -Xmx512m -….

-Dweblogic.Name=oacore_server1 -Dweblogic.management.server=http://orcl.localdomain:7006 -Djava.library.path=$MW_HOME/webtier/jdk/jre/lib/amd64:$MW_HOME/webtier/jdk/jre/lib/amd64/server:$MW_HOME/webtier/jdk/jre/lib/amd64/native_threads…

7.       Save the changes.

8.       Start all managed servers using using the command:

$ADMIN_SCRIPTS_HOME admanagedsrvctl.sh start <server name>

Performed these changes to all the other managed servers.