adop fs_clone/prepare fails with [UNEXPECTED]Invalid worker Count: 0

We recently added a node to our current EBS application. As part of the node addition steps, tried to executed adop phase=fs_clone but it failed immediately with errors.


Enter the APPS password:
Enter the SYSTEM password:
Enter the WLSADMIN password:
Validating credentials.
Initializing.
    Run Edition context  : /app/EBD//fs2/inst/apps/EBD_oraapp01/appl/admin/EBD_oraapp01.xml
    Patch edition context: /app/EBD//fs1/inst/apps/EBD_oraapp01/appl/admin/EBD_oraapp01.xml
    Patch file system free space: 41.85 GB
Validating system setup.
    [UNEXPECTED]Invalid worker Count: 0
    [UNEXPECTED]Error validating worker count

Later tried to assign worker count of value 8 manually as below but it also failed
  $ adop phase=fs_clone workers=8
 
Validating credentials.
Initializing.
    Run Edition context  : /app/EBD//fs2/inst/apps/EBD_oraapp01/appl/admin/EBD_oraapp01.xml
    Patch edition context: /app/EBD//fs1/inst/apps/EBD_oraapp01/appl/admin/EBD_oraapp01.xml
    Patch file system free space: 41.85 GB
    [UNEXPECTED]Specified value 8 is not supported by the database.
    [UNEXPECTED]A maximum of 1 workers can be invoked.
    [UNEXPECTED]Error validating worker count

Observed the same issue with prepare phase as well.

Solution:
The default worker count information(recomm & max)  is stored in a file adpawc.xml that can be found under $APPL_TOP/admin/$TWO_TASK/log

Somehow this file has been modified with recommended value as 0 and max value as 1 during node addition.

FileContent:
<?xml version="1.0"?>
<WORKER_COUNT>
        <RECOMMENDED>0</RECOMMENDED>
        <MAX>1</MAX></WORKER_COUNT>

Update recommended &  max value based on cpu count.

Here I updated recommended value as 8 and max value as 64 and saved the file.

Once the changes are made, fs_clone went smoothly.

adop fs_clone failed after cutover port issue

Issue:

FS_CLONE failed during vallidation and below are errors in Validation logfiles
Started execution   : ADOPValidations.java
Node Name                       File Edition                    Port Name                       Port Value
----------------------------------------------------------------------------------------------------------------------------
ebs001                      run                     s_java_object_cache_port                        12366
ebs002                      run                     s_java_object_cache_port                        12367
ebs001                      patch                   s_java_object_cache_port                        12366
ebs002                      patch                   s_java_object_cache_port                        12367

ERROR:
The following ports are out of sync on RUN Edition-> [s_java_object_cache_port]
Corrective Action: update the context files so that all the erroneous ports have same value across all nodes
and run AutoConfig to sync with the value in the database.
The following ports are out of sync on PATCH Edition-> [s_java_object_cache_port]
Corrective Action: update the context files so that all the erroneous ports have same value across all nodes
and  run AutoConfig with the -syncctx option to sync with the value in the database.

Fix:
There is a port mismatch for variable s_java_object_cache_port. Based on port pool, values should be as below:
Node Name                       File Edition                    Port Name                       Port Value
----------------------------------------------------------------------------------------------------------------------------
ebs001                      run                     s_java_object_cache_port                        12367
ebs002                      run                     s_java_object_cache_port                      12367
ebs001                      patch                   s_java_object_cache_port                       12366
ebs002                      patch                   s_java_object_cache_port                    12366

1) On ebs001 update run context file with correct port value 12367 & run autoconfig.sh
2) On ebs002 update patch contextfile and and adconfig with syncctc option as below
$ sh $AD_TOP/bin/adconfig.sh contextfile=$CONTEXT_FILE -syncctx