adop phase=prepare failed with ERROR: Script failed, exit code 255 while creating WLS domain

Issue: 
We are trying to run adop prepare phase on our EBS 12.2 Patch file system and it came out with below errors in FSCloneApplyAppsTier_08200431.log file.

START: Creating new WLS domain.
Running /u01/app/EBSDB/fs2/FMW_Home/oracle_common/bin/pasteConfig.sh -javaHome /u01/app/EBSDB/fs2/EBSapps/comn/util/jdk64 -al /u01/app/EBSDB/fs1/EBSapps/comn/adopclone_ebsdb01/FMW/WLS/EBSdomain.jar -tdl /u01/app/EBSDB/fs2/FMW_Home/user_projects/domains/EBS_domain_EBSDB -tmw /u01/app/EBSDB/fs2/FMW_Home -mpl /u01/app/EBSDB/fs1/EBSapps/comn/adopclone_ebsdb01/FMW/WLS/plan/moveplan.xml -ldl /u01/app/EBSDB/fs1/inst/apps/EBSDB_ebsdb01/admin/log/clone/wlsT2PApply -silent true -debug true -domainAdminPassword /u01/app/EBSDB/fs1/EBSapps/comn/adopclone_ebsdb01/FMW/tempinfo.txtScript Executed in 1602 milliseconds, returning status 255
ERROR: Script failed, exit code 255


The above information didn't give sufficient information and had to review CLONE2015-08-20_04-31-47AM.log &  CLONE2015-08-20_04-31-47AM.error logfiles under $ADOP_LOG_HOME/22/<prepare..>/<CONTEXT_NAME>/ directory.  These have provided below additional information which helped in resolving the issue.

CLONE2015-08-20_04-31-47AM.error:

SEVERE : Aug 20, 2015 4:31:49 AM - ERROR - CLONE-20372   Server port validation failed.
SEVERE : Aug 20, 2015 4:31:49 AM - CAUSE - CLONE-20372   Ports of following servers - AdminServer(7004) - are not available.  <-- This port is of Run filesystem instead it should look for 7005
SEVERE : Aug 20, 2015 4:31:49 AM - ACTION - CLONE-20372   Provide valid free ports.
oracle.as.t2p.exceptions.FMWT2PPasteConfigException: PasteConfig failed. Make sure that the move plan and the values specified in moveplan are correct.

CLONE2015-08-20_04-31-47AM.log:FINE : Aug 20, 2015 4:31:49 AM - [J2EEGenericValidationUtil:validateAndGetServerPort] Listen Port(7805) of server forms-c4ws_server1 is valid and free.FINE : Aug 20, 2015 4:31:49 AM - [J2EEGenericValidationUtil:validateServerConfig] Mapping of ports vs servers from moveplan = {AdminServer=[7004], forms-c4ws_server1=[7805], forms_server1=[7405], oacore_server1=[7205], oafm_server1=[7605]} FINE : Aug 20, 2015 4:31:49 AM - [J2EEGenericValidationUtil:validateServerConfig] Mapping of temporary ports assigned vs servers = {}
FINE : Aug 20, 2015 4:31:49 AM - [J2EEGenericValidationUtil:validateServerConfig] map_serverName_vs_actualPort_vs_TempPort = {}

Above highlighted area confirms that except AdminServer port all the other ports are properly picked up.

Additional info noticed in FSCloneApplyAppsTier_08200431.log
m_LstnAddrPrtsDBMap size = 6
Total no. of configGroup = 6
 configGroup Type name = SERVER_CONFIG
configProperty id = Server1
Count for NodeIterator nextNode = 3

Input for AdminServer Listen Address OR Port is NULL
Moveplan won't be updated for AdminServer Listen Address
Moveplan won't be updated for AdminServer Listen Port+++++++++++++++++++++++++
configProperty id = Server2
Count for NodeIterator nextNode = 3
Input for oacore_server1 Listen Address OR Port is NULL
 Updating oacore_server1 Listen Address to ushtc0app02.jacobs.com
 Updating oacore_server1 Listen Port to 7205

These highlighted details above confirmed that adop is unable to update AdminPort to correct value as part of its preparations.

Fix:
Upon further investigation, Listen Address value for AdminServer on Run Filesystem is removed manually.

1) Login to weblogic console of Run  Filesystem. Perform Lock & Edit. Then Navigate to Domain Structure -> Servers -> Select Adminserver.
2) Under Configuration screen -> Update Listen Address value to application server hostname(FQDN).
3) Bounce all the application services.
4) Run adop phase=prepare.

Workaround:
If you want the Listen address value to be Null. Run fs_clone for the EBS Apps. Then proceed with adop prepare phase.

FRM-92101 - There was a failure in the Forms Server during startup. This could happen due to invalid configuration.

FRM-92101 - Unable to Launch EBS & Discoverer in Same Browsers

Issue :

Getting error frm-92101: There was a failure in the Forms Server during startup. This could happen due to invalid configuration.

Unable to launch EBS and Discoverer plus applications together from Internet Explorer. This issue is occurring only if both Discoverer and EBS services are running on same server.

Issue occurs when we launch them together as below

1. Launch EBS URL and launch any forms
2. Then now open a new tab and launch discoverer URL
3. Then EBS Forms fail with below error message.

And we can see below error messages in java console

oracle.forms.net.ConnectionException: 500

    at oracle.forms.net.ConnectionException.createConnectionException(Unknown Source)

    acache:  Read manifest for https://ebs.domain.com:4443/OA_JAVA/oracle/apps/fnd/jar/fndewt.jar: read=45 full=87491

java.io.EOFException

    at java.io.DataInputStream.readUnsignedByte(Unknown Source)

   

4. Then later when we close and retry to launch forms it still fails with frm-92101 message as below

5. Below is the error recorded in application.log file

Unable to switch to Working Directory: /u01/oravis /apps/tech_st/10.1.3/forms

14/08/28 18:37:22.839 formsweb: Forms session <5> failed during startup: no response from runtime process

Cause

This issue is occurring because of improper handling path variable when a discoverer is launched. When EBS application is launched  it sets JSESSIONID cookie with a restriction of path= /OA_HTML so that this is restricted to EBS Applications. But when the discoverer application is launched it is setting JSESSIONID without any restriction.

This makes that the DISCOVERER JSESSIONID cooke is also passed with request send to EBS server and since DISCOVERER lacks the "path=/discoverer" when setting the cookie the HTTP requests for Forms Servlet receive an 'unexpected' JSESSIONID cookie value and fail with the error.

EBS http:

Set-Cookie: JSESSIONID=3c3046608609362f0bfe2e0ffc36096ef65ada33c71207aac7f9a4cc881ea06a.e3qPc3mMa34Ne34TahyLa3iKaN50; path=/OA_HTML; secure

Discoverer http:

Set-Cookie: JSESSIONID=hHq6TnsdMJhJlQvTk4nRTjSh2x00BKphgT6ktw376Xx1V30L1DfX!1397667650; domain=.domain.com; path=/; secure; HttpOnly

Solution:

There are two workarounds to prevent this issue

Workaround 1:

This workaround fixes issue of launching both forms and discoverer together but this restricts users launching more than one forms for their activity

1. In the $CONTEXT_FILE change
   <forms_tracking_cookies oa_var="s_forms_tracking_cookies">disabled</forms_tracking_cookies>
   to
   <forms_tracking_cookies oa_var="s_forms_tracking_cookies">enabled</forms_tracking_cookies>
2. Run autoconfig to reflect changes and bounce application services.

(or)

1. If autoconfig execution cannot be performed rightway, update orion-web.xml file under $INST_TOP/ora/10.1.3/j2ee/forms/application-deployments/forms/formsweb

<session-tracking cookies="disabled" />

to

<session-tracking cookies="enabled" />

2. Bounce opmn services using adopmnctl.sh

Workaround 2:

This workaround is most suggestable than prior and this is performed on Discoverer side and this changes make the discoverer to restrict usage of its JSessionid cookie.

1. Need to add following entries to weblogic.xml file of discoverer.war file.

<session-descriptor>
<cookie-name>JSESSIONID</cookie-name>
<cookie-path>/discoverer</cookie-path>
</session-descriptor>

2. The below are the files that needs to be modified with above changes. The paths may not be exact but should be similar as below

$DOMAIN_HOME/servers/<DomainName>/tmp/_WL_user/oracle.discoverer/yn4f2j/WEB-INF/weblogic.xml
$DOMAIN_HOME /servers/<DomainName>/tmp/_WL_user/discoverer_11.1.1.2.0/51oeh7/war/WEB-INF/weblogic.xml

3. Once the changes are done reploy discoverer.ear using fusion middleware console
4. Login FMW Console, click on “Lock & Edit” and go to Deployments -> Select Discoverer (11.1.1.X.0) -> Click on Update -> Select “Redeploy this application using the following deployment files” -> Next -> Finish -> “Activate Changes”
5. Bounce opmnctl services

Now you should be able to launch both EBS and discoverer without any restrictions

 

SSL for Weblogic Server

Hi,

Below are the high level steps of configuring Weblogic admin server and managed servers with SSL.

1) Create a keystore
2) Generate Certificate Request file using keytool
3) Export keystore file and upload to CA authority
4) Upload the Digital certificates obtained by Trusted CA to Key store
5) Login Weblogic console and go to configuration page of admin server
6) Enable SSL port under General, Modify Keystore details with Cutsom keystores and and keystore password
7) Update the same keystore under SSL Tab.
8) Save the changes and activate.
9) Shutdown admin server and restart it

After restart you should be able to launch Console with https (SSL Enabled).

The detailed primer of this activity will be published soon.. Wait for the updates