ADOP phase=apply Failed — How to Diagnose and Recover Without Aborting | punitoracledba

Oracle EBS DBA Series · May 2026 · punitoracledba.blogspot.com

ADOP phase=apply Failed — How to Diagnose and Recover Without Aborting

Oracle EBS R12.2 | ADOP Troubleshooting | Workers | Recovery | 10 min read

A failed adop phase=apply is one of the most stressful moments in EBS patching. Your maintenance window is ticking, users are waiting, and the terminal is showing errors. The instinct of many DBAs is to immediately run adop phase=abort and start again — but this is almost always the wrong move.

In most cases a failed apply can be diagnosed quickly, fixed in place, and restarted using restart=yes — without losing your prepare phase, without aborting the session, and without starting the entire cycle over from scratch.

This post walks through the exact steps to diagnose a failed apply, find the root cause in the logs, fix it, and restart cleanly.

Rule of thumb: Never run adop phase=abort as your first response to a failure. Always try restart=yes first. Abort is a last resort — it leaves the system in an inconsistent state and forces a full new cycle.

Why adop phase=apply Fails

The apply phase uses multiple parallel workers to process patch jobs. If any single worker encounters an error it cannot handle, that worker is marked as Failed and the apply session stops. The most common causes are:

Cause	What It Looks Like
Database object errors	ORA- errors in worker log — invalid object, constraint violation, compile error
Insufficient disk space	Worker stops mid-task — check APPL_TOP, PATCH_TOP, /tmp
File permission issues	Permission denied errors in the worker log
Missing prerequisite patch	Object or file expected by the patch does not exist
Network or DB connectivity drop	Worker loses connection mid-task — ORA-03113, ORA-03114
Locked or busy database objects	ORA-00054 resource busy — object locked by another session

Step 1 — Check the ADOP Session Status

The first thing to do after a failure is check what ADOP knows about the current session. This tells you which phase failed, on which node, and what the overall state is.

Check ADOP session status

# Source the run environment first
source /u01/oracle/R122/EBSapps.env run

# Check overall session status
adop phase=status

# Or query the database directly for more detail
sqlplus apps/<apps_password> <<EOF
SELECT adop_session_id,
       status,
       start_date,
       end_date,
       abandon_flag
FROM   ad_adop_sessions
ORDER  BY adop_session_id DESC
FETCH  FIRST 5 ROWS ONLY;
EOF

Find failed workers specifically sqlplus apps/<apps_password> <<EOF SELECT worker_id, status, start_date, end_date FROM ad_workers WHERE status = 'Failed' ORDER BY worker_id; EOF

Note the worker_id of every failed worker. You will use these numbers to find the correct log files in the next step.

Step 2 — Find and Read the Worker Log Files

ADOP stores all log files on the non-editioned filesystem under $APPL_TOP_NE. The log structure follows a predictable pattern based on the session ID and execution timestamp.

Locate the ADOP log directory # Base log location echo $APPL_TOP_NE # Logs are at: $APPL_TOP_NE/admin/log/adop/<session_id>/ # List sessions — find your current session ID ls -lt $APPL_TOP_NE/admin/log/adop/ # Navigate to the apply phase logs for your session cd $APPL_TOP_NE/admin/log/adop/<session_id>/<timestamp>/ # List contents to find worker log files ls -la

Read the failed worker log — replace N with the worker number

# View the last 200 lines of the failed worker log
tail -200 $APPL_TOP_NE/admin/log/adop/<session_id>/<timestamp>/adwork<N>.log

# Search specifically for ERROR lines
grep -i "error\|ORA-\|failed\|fatal" \
  $APPL_TOP_NE/admin/log/adop/<session_id>/<timestamp>/adwork<N>.log

# Also check the main adop log for context
tail -100 $APPL_TOP_NE/admin/log/adop/<session_id>/<timestamp>/adop.log

Always read the log from the bottom up. The actual error is usually the last meaningful line before the worker stopped. Lines above it are the normal processing that succeeded before the failure.

Step 3 — Diagnose the Root Cause

Once you have read the worker log, match the error to one of the common scenarios below and apply the fix before attempting to restart.

Scenario A — ORA- database errors

The most common type. Look for the ORA- error code in the worker log and resolve it before restarting.

Common ORA- errors and what to check ORA-00054: resource busy and acquire with NOWAIT --> A database object is locked. Find and kill the blocking session: SELECT sid, serial#, username, status, machine FROM v$session WHERE status = 'ACTIVE' AND username IS NOT NULL ORDER BY last_call_et DESC; -- Kill the blocking session: ALTER SYSTEM KILL SESSION '<sid>,<serial#>' IMMEDIATE; ORA-04031: unable to allocate memory in shared pool --> Increase shared_pool_size or flush the shared pool: ALTER SYSTEM FLUSH SHARED_POOL; ORA-01536: space quota exceeded --> Increase quota for APPS user or add space to the tablespace

Scenario B — Disk space exhausted

Check disk space on all critical mounts

# Check disk space — need minimum 20GB free on each
df -h $APPL_TOP $PATCH_TOP $APPL_TOP_NE /tmp

# Find large files consuming space in PATCH_TOP
du -sh $PATCH_TOP/*
find $PATCH_TOP -name "*.log" -size +500M

# Clean up old adop logs from previous sessions if needed
# Only remove logs from sessions that are fully completed
ls $APPL_TOP_NE/admin/log/adop/

Scenario C — File permission error

Check and fix file permissions # Look for permission denied in the worker log grep -i "permission denied" \ $APPL_TOP_NE/admin/log/adop/<session_id>/<timestamp>/adwork<N>.log # Fix permissions on APPL_TOP if needed # Run as the applmgr OS user chmod -R 755 $APPL_TOP/<product_short_name>/

Scenario D — Missing prerequisite patch

The worker log will show something like object does not exist or file not found for a file or DB object that the patch expects to already be present. This means a prerequisite patch was not applied.

Check the patch README.txt for listed prerequisites
Confirm those prerequisite patches are registered in AD_BUGS
If missing, abort the current session, apply the prerequisite, then start a fresh cycle

Verify a prerequisite patch is applied sqlplus apps/<apps_password> <<EOF SELECT bug_number, last_update_date FROM ad_bugs WHERE bug_number = '<prerequisite_patch_number>'; EOF

Step 4 — Restart the Failed Apply Session

Once the root cause is fixed, restart the apply session using restart=yes. ADOP will pick up from where it failed — it does not re-run jobs that already completed successfully.

Restart the apply session

# Source the run environment
source /u01/oracle/R122/EBSapps.env run

# Restart with the same patch number — ADOP resumes from failure point
adop phase=apply patches=<patch_number> restart=yes

# If you need fewer workers on restart due to resource constraints
adop phase=apply patches=<patch_number> restart=yes workers=4

# Monitor the restarted session
adop phase=status

Important: When restarting, specify the same patch number as the original apply command. Do not change the patches parameter — ADOP uses it to validate the session matches what was started.

Watch the apply session after restart. If the same worker fails again on the same task, the root cause was not fully resolved. Read the log again and look more carefully for the underlying error.

Step 5 — Verify Apply Completed Successfully

After a successful restart, confirm that all workers finished cleanly before moving to the finalize phase.

Confirm all workers completed sqlplus apps/<apps_password> <<EOF -- Should return no rows if all workers completed SELECT worker_id, status FROM ad_workers WHERE status NOT IN ('Completed', 'Assigned') ORDER BY worker_id; -- Check overall session is in apply completed state SELECT adop_session_id, status FROM ad_adop_sessions ORDER BY adop_session_id DESC FETCH FIRST 3 ROWS ONLY; EOF

Once all workers show Completed and the apply phase status is clean, continue to the finalize phase as normal.

Continue the patching cycle adop phase=finalize adop phase=cutover adop phase=cleanup

When Abort Is Actually the Right Choice

There are situations where adop phase=abort is genuinely necessary. Use it only when:

A prerequisite patch is missing and cannot be applied within the current cycle
The patch itself is incorrect or was downloaded for the wrong platform
The database edition is corrupted and cannot be recovered
Oracle Support has reviewed the logs and explicitly recommends abort
The apply failure is due to a known bug in the patch itself

After abort — what happens: An aborted session leaves the patch edition in an unusable state. You must run adop phase=abort,cleanup cleanup_mode=full to fully clean up before starting a fresh prepare. Do not skip the cleanup after abort.

Full abort and cleanup sequence # Abort the current session adop phase=abort # Run full cleanup to remove the patch edition adop phase=cleanup cleanup_mode=full # Verify no pending session remains adop phase=status # Start fresh when ready adop phase=prepare

Quick Log Location Reference

Log File	Location	What It Contains
`adop.log`	`$APPL_TOP_NE/admin/log/adop/<session>/<timestamp>/`	Main ADOP session log — overall progress and phase transitions
`adwork<N>.log`	Same directory as adop.log	Individual worker log — the actual error will be here
`adzdshowstatus.out`	Same directory as adop.log	Status report output from `adop phase=status`
Patch-specific logs	Under `$PATCH_TOP/<patch_number>/log/`	Logs written by the patch driver itself during execution

Recovery Decision Tree

Situation	Action
Worker failed — root cause identified and fixed	`adop phase=apply patches=... restart=yes`
Worker failed again after restart	Re-read the log — root cause not fully resolved. Fix and restart again.
Prerequisite patch missing	Abort, cleanup full, apply prerequisite, start fresh cycle
Disk space exhausted	Free up space, then restart=yes — do not abort
DB object locked	Kill blocking session, then restart=yes
Oracle Support recommends abort	`adop phase=abort` then `cleanup_mode=full`
Unsure of root cause	Read the log again — do not abort until you understand what failed

Key SQL Queries for ADOP Troubleshooting

All queries — run as APPS user -- Current and recent ADOP sessions SELECT adop_session_id, status, abandon_flag, start_date, end_date FROM ad_adop_sessions ORDER BY adop_session_id DESC FETCH FIRST 5 ROWS ONLY; -- Worker status for current session SELECT worker_id, status, start_date, end_date FROM ad_workers ORDER BY worker_id; -- Find blocking sessions if ORA-00054 SELECT s.sid, s.serial#, s.username, s.status, s.machine, s.last_call_et FROM v$session s WHERE s.status = 'ACTIVE' AND s.username IS NOT NULL ORDER BY s.last_call_et DESC; -- Check invalid objects after failed apply SELECT owner, object_type, COUNT(*) AS cnt FROM dba_objects WHERE status = 'INVALID' GROUP BY owner, object_type ORDER BY owner, object_type; -- Confirm a patch is registered after recovery SELECT bug_number, last_update_date FROM ad_bugs WHERE bug_number = '<patch_number>';

Recovery Checklist

#	Step
1	Run `adop phase=status` to confirm which phase failed
2	Query `AD_WORKERS` to find the failed worker IDs
3	Read the failed worker log — `adwork<N>.log` — from the bottom up
4	Identify the root cause — ORA- error, disk, permission, or prerequisite
5	Fix the root cause — kill locks, free disk, fix permissions
6	Restart using `adop phase=apply patches=... restart=yes`
7	Monitor the restarted session with `adop phase=status`
8	Confirm all workers show Completed in `AD_WORKERS`
9	Continue with `adop phase=finalize` then `cutover`
10	Only abort if restart fails repeatedly and root cause cannot be resolved

Database Consultant (Passionate about Database & Cloud Technologies)

Monday, May 18, 2026