Advanced 7 min · March 06, 2026

Java Agent COMPUTE_FRAMES Crash on Missing ClassLoader

Monitoring agent deployment caused 10x startup delay and ClassNotFoundException.

N
Naren · Founder
Plain-English first. Then code. Then the interview question.
About
 ● Production Incident 🔎 Debug Guide
Quick Answer
  • Java Agent is a JAR with a premain() or agentmain() method declared in MANIFEST.MF — the JVM calls it before main(), handing you an Instrumentation instance
  • ClassFileTransformer.transform() intercepts raw class bytes at load time — return null to skip, return modified bytes to transform
  • ASM is the standard library for bytecode manipulation — use AdviceAdapter for clean method entry/exit hooks
  • Startup agents (-javaagent) register before main(); Attach API (VirtualMachine.loadAgent) injects into a running JVM
  • Performance cost: transformation time per class load + injected code overhead per method call — scope to the narrowest package prefix
  • Biggest mistake: not shading ASM — Spring's bundled ASM version causes NoSuchMethodError at class load
Plain-English First

Imagine every Java class is a recipe card in a chef's kitchen. Normally, the chef follows each recipe exactly as written. A Java agent is like a head chef who intercepts every recipe card before it reaches the line cooks — they can scribble notes on it, change an ingredient, or add a timing step, without ever touching the original recipe book. The line cooks (the JVM) never even know the card was modified; they just cook what's in front of them. That's instrumentation: quietly editing the instructions before they're executed.

Production Java systems are black boxes. You deploy a JAR, traffic hits it, and when something goes wrong — a memory leak, a slow database call, an unexpected exception — you're often blind. APM tools like Datadog, New Relic, and Dynatrace somehow see inside every method call, every HTTP request, every SQL query, all without you touching a single line of your application code. The mechanism behind all of them is the Java Agent API, and understanding it puts you in rare company.

The Java agent mechanism, introduced in Java 5 via the java.lang.instrument package, solves a profound problem: how do you add cross-cutting behavior — logging, tracing, profiling, security checks — to a running JVM without modifying source code or recompiling? The answer is bytecode instrumentation. At class-load time, before the JVM hands a class to its execution engine, an agent intercepts the raw bytes and rewrites them. It's surgical, powerful, and completely transparent to the application.

By the end of this article you'll be able to write a working Java agent from scratch, attach it to any JVM process, use the Instrumentation API to transform class bytecode with ASM, understand the production implications including memory overhead and class retransformation limits, and walk into an interview knowing exactly how tools like JaCoCo and OpenTelemetry Java work under the hood.

How the JVM Loads a Java Agent — premain, MANIFEST.MF, and the Attach API

A Java agent is a plain JAR file with one special contract: a class containing a static premain method (for startup-time agents) or an agentmain method (for runtime attach). The JVM calls premain before your application's own main method, handing it an Instrumentation instance — the single most powerful object in the Java platform. Through it you can query loaded classes, redefine them, and most critically, register a ClassFileTransformer that intercepts every class as it's loaded.

The JAR's MANIFEST.MF is the contract document. Without the Premain-Class attribute pointing to your agent class, the JVM refuses to honour the -javaagent flag. You can also declare Can-Redefine-Classes: true and Can-Retransform-Classes: true there — both must be opt-in because they carry real overhead and security implications.

The Attach API (com.sun.tools.attach) lets you inject an agent into an already running JVM process without restarting it. This is how profilers like async-profiler and VisualVM work. The target JVM's PID is all you need. One critical catch: since Java 9, self-attach (attaching to your own PID) requires -Djdk.attach.allowAttachSelf=true or it throws an IOException.

One subtle point: premain runs on the main thread before any application code. That means you can safely set up singletons or static state without worrying about race conditions. But if your agent initialisation fails, the JVM will not start your application — it exits with an error. Make sure premain is idempotent and fast. Also note that the agent's own classes are loaded by the system classloader, not the bootstrap loader. If your agent needs to add classes to the bootstrap classpath (e.g., a Java agent that provides a new implementation of a core API), use instrumentation.appendToBootstrapClassLoaderSearch() — but this is rarely needed and carries its own risks.

TimingAgent.javaJAVA
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import java.lang.instrument.Instrumentation;

/**
 * Entry point for a startup-time Java agent.
 *
 * Compile this class, package it in a JAR, and declare it in MANIFEST.MF:
 *   Premain-Class: TimingAgent
 *   Can-Retransform-Classes: true
 *
 * Launch with:
 *   java -javaagent:timing-agent.jar -jar your-app.jar
 */
public class TimingAgent {\n\n    /**\n     * The JVM calls this BEFORE your application's main() method.\n     *\n     * @param agentArgs  Any string args passed after '=' in -javaagent:jar=args\n     * @param instrumentation  The gateway to all bytecode transformation power\n     */\n    public static void premain(String agentArgs, Instrumentation instrumentation) {\n        System.out.println(\"[TimingAgent] Agent loaded. JVM classes currently loaded: \"\n                + instrumentation.getAllLoadedClasses().length);\n\n        // Register our transformer — it will be called for EVERY class the JVM loads\n        // from this point forward, including JDK classes if we don't filter carefully\n        MethodTimingTransformer transformer = new MethodTimingTransformer(agentArgs);\n        instrumentation.addTransformer(transformer, /* canRetransform= */ true);\n\n        System.out.println(\"[TimingAgent] ClassFileTransformer registered. Ready.\");\n    }\n\n    /**\n     * Called when the agent is attached to a RUNNING JVM via the Attach API.\n     * The signature is identical to premain but the JVM lifecycle moment differs.\n     */\n    public static void agentmain(String agentArgs, Instrumentation instrumentation) {\n        // Reuse the same startup logic — safe because we're stateless here\n        premain(agentArgs, instrumentation);\n    }\n}",
        "output": "[TimingAgent] Agent loaded. JVM classes currently loaded: 412\n[TimingAgent] ClassFileTransformer registered. Ready."
      }

Manifest File Setup Guide: Required Attributes and Common Mistakes

The META-INF/MANIFEST.MF file inside your agent JAR is the contract between you and the JVM. Without the correct attributes, the JVM will silently ignore your agent. The three essential attributes for a startup agent are: - Premain-Class: Fully qualified name of the class containing the premain(String, Instrumentation) method. - Agent-Class: Fully qualified name of the class containing the agentmain(String, Instrumentation) method (required for runtime attach). - Can-Retransform-Classes: Set to "true" if you plan to call retransformClasses() on already-loaded classes (required for most runtime attach scenarios). Optional but often needed: Can-Redefine-Classes, Boot-Class-Path.

The manifest file is whitespace-sensitive. Each line must end with a newline character, including the last line. Attribute names and values are separated by a colon and a space. No trailing spaces allowed. If an attribute value is too long, you can wrap it to the next line by starting the continuation line with a single space.

To generate the manifest automatically in Maven, use the maven-jar-plugin or maven-shade-plugin's ManifestResourceTransformer (as shown in the pom.xml example later). For manual builds, create a text file named manifest.txt with these contents:

META-INF/MANIFEST.MFTEXT
1
2
3
4
5
Manifest-Version: 1.0
Premain-Class: io.thecodeforge.agent.MyAgent
Agent-Class: io.thecodeforge.agent.MyAgent
Can-Retransform-Classes: true
Can-Redefine-Classes: false
Output
Verified with: unzip -p agent.jar META-INF/MANIFEST.MF
Whitespace Kills Agents
A missing newline at the end of the manifest file causes the JVM to ignore the last attribute. Always verify with unzip -p agent.jar META-INF/MANIFEST.MF and check the output ends with a newline.
Production Insight
A missing newline at end of MANIFEST.MF silently skips the Premain-Class attribute. The JVM logs nothing — the agent just never loads. Rule: always end MANIFEST.MF with a blank line. In Maven, ensure the ManifestResourceTransformer produces a proper final newline.
Key Takeaway
MANIFEST.MF must end with a newline. Use unzip -p to verify before shipping.

Bytecode Manipulation Libraries: ASM vs ByteBuddy vs Javassist Compared

While ASM is the de facto standard for production Java agents, several libraries simplify bytecode manipulation with higher-level APIs. Choosing the right one depends on your performance requirements, complexity of transformations, and team expertise.

ASM (Low-level) – Direct bytecode instruction manipulation. Offers maximum performance and control. Used by the JDK itself. Requires understanding of the JVM instruction set and stack frames. Best for production APM agents where overhead must be minimal.

ByteBuddy (High-level) – Builds on ASM but provides a fluent API for common tasks like method delegation, field access, and type creation. Automatically handles frame computation and class writer configuration. Excellent for ad-hoc instrumentation and debugging tools. Some overhead compared to raw ASM, but negligible for typical use.

Javassist (Medium-level) – Uses a source-level API: you write Java code as strings that get compiled and injected. Very easy to get started, but limited for complex transformations (e.g., adding try-catch blocks). Performance is lower due to interpretation. Best for quick prototypes or simple method wrapping.

FeatureASMByteBuddyJavassist
API LevelInstruction-levelHigh-level (fluent)Source-level (String)
Learning CurveSteepModerateShallow
PerformanceFastestVery fast (on ASM)Moderate
Stack Frame HandlingManual or COMPUTE_FRAMESAutomaticAutomatic
Class CreationPossible but manualBuilt-in (DynamicType)Easy (new ClassPool)
File Size (jar)~500 KB~1.5 MB~800 KB
Production UseJaCoCo, OpenTelemetryMockito, HibernateSpring AOP (CGLib replacement)

For most production scenarios, ASM remains the best choice due to its performance and no-framework dependency. ByteBuddy is ideal when you need to write an agent quickly without deep bytecode expertise. Javassist is useful for simple filtering but not recommended for production agents that modify control flow.

ByteBuddyAgent.javaJAVA
1
2
3
4
5
6
7
8
9
import net.bytebuddy.agent.builder.AgentBuilder;
import net.bytebuddy.asm.Advice;
import net.bytebuddy.matcher.ElementMatchers;
import java.lang.instrument.Instrumentation;

public class ByteBuddyAgent {
    public static void premain(String args, Instrumentation inst) {\n        new AgentBuilder.Default()\n            .type(ElementMatchers.nameStartsWith(\"io.codeforge\"))\n            .transform((builder, type, cl, mod) ->\n                builder.method(ElementMatchers.any())\n                    .intercept(Advice.to(MethodTimer.class)))\n            .installOn(inst);\n    }\n\n    public static class MethodTimer {\n        @Advice.OnMethodEnter\n        static long enter() { return System.nanoTime(); }\n        @Advice.OnMethodExit\n        static void exit(@Advice.Enter long start, @Advice.Origin String method) {\n            System.out.println(method + \" took \" + (System.nanoTime() - start) + \" ns\");\n        }\n    }\n}",
        "output": "// Build dependency: net.bytebuddy:byte-buddy-agent:1.14.12 + byte-buddy:1.14.12\n// Run with -javaagent:bytebuddy-agent.jar\n// Output: io.codeforge.demo.OrderService#processOrder took 4321871 ns"
      }

Writing a ClassFileTransformer with ASM — Intercepting Method Calls in Raw Bytecode

The ClassFileTransformer interface has one method: transform. It receives raw class bytes and returns either modified bytes or null (meaning 'leave this class alone'). Returning null for classes you don't care about is critical for performance — every transformer in the chain gets called for every class, including java.lang.String and java.util.ArrayList.

To actually modify bytecode, you need a bytecode manipulation library. ASM is the industry standard — it's what the JDK itself uses internally. It operates at the instruction level at method entry, every exit point: a method can have multiple RETURN opcodes and any number of ATHROW instructions. ASM's AdviceAdapter handles this elegantly by providing onMethodEnter and onMethodExit hooks that account for all exit paths automatically, including the exception path.

The dependency you need is org.ow2.asm:asm:9.6 and org.ow2.asm:asm-commons:9.6. Bundle them into your agent JAR with Maven Shade or a fat-jar Gradle task.

AdviceAdapter works on all methods including constructors (<init>) and static initializers (<clinit>). Be careful with <init>: System.nanoTime() in a constructor of a frequently created object can add measurable overhead. Also, inside <clinit>, the class is being initialised, and calling System.nanoTime() is fine, but avoid any method calls that trigger additional class loading at that point — you can cause circular class loading deadlocks. If your transformer needs to skip <clinit> because of this, check the method name and return the original visitor for it.

MethodTimingTransformer.javaJAVA
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
import org.objectweb.asm.*;
import org.objectweb.asm.commons.AdviceAdapter;
import java.lang.instrument.ClassFileTransformer;
import java.security.ProtectionDomain;

/**
 * Intercepts class loading and injects timing code into every method
 * of classes whose package matches our target prefix.
 *
 * ASM version: 9.6  (org.ow2.asm:asm-commons:9.6)
 */
public class MethodTimingTransformer implements ClassFileTransformer {

    // Only instrument classes under this package to avoid touching JDK internals
    private final String targetPackagePrefix;

    public MethodTimingTransformer(String agentArgs) {
        // agentArgs might be something like "io/codeforge/demo"
        this.targetPackagePrefix = (agentArgs != null && !agentArgs.isBlank())
                ? agentArgs.replace('.', '/')
                : "io/codeforge"; // default to our own package
    }

    @Override
    public byte[] transform(
            ClassLoader loader,
            String className,           // Internally formatted: "com/example/MyClass"
            Class<?> classBeingRedefined,
            ProtectionDomain protectionDomain,
            byte[] originalClassBytes) {

        // Fast-path: skip anything that's not our target to keep overhead minimal
        if (className == null || !className.startsWith(targetPackagePrefix)) {
            return null; // null = "I didn't change anything, use original bytes"
        }

        try {\n            ClassReader classReader = new ClassReader(originalClassBytes);\n            // COMPUTE_FRAMES tells ASM to recalculate stack frames automatically\n            // This is essential — manual frame calculation is error-prone\n            ClassWriter classWriter = new ClassWriter(\n                    classReader, ClassWriter.COMPUTE_FRAMES);\n\n            // Wrap the writer with our visitor that injects timing into each method\n            TimingClassVisitor timingVisitor = new TimingClassVisitor(\n                    Opcodes.ASM9, classWriter, className);\n\n            // EXPAND_FRAMES is required when using AdviceAdapter with COMPUTE_FRAMES\n            classReader.accept(timingVisitor, ClassReader.EXPAND_FRAMES);\n\n            System.out.println(\"[TimingAgent] Instrumented: \" + className);\n            return classWriter.toByteArray(); // Return the rewritten bytecode\n\n        } catch (Exception transformationError) {\n            // NEVER let a transformer exception crash the JVM — log and return null\n            System.err.println(\"[TimingAgent] Failed to instrument \" + className\n                    + \": \" + transformationError.getMessage());\n            return null; // Fall back to the original unmodified class\n        }\n    }\n\n    // ── Inner class: visits each class and delegates method visiting ─────────\n\n    static class TimingClassVisitor extends ClassVisitor {\n\n        private final String ownerClassName;\n\n        TimingClassVisitor(int api, ClassVisitor delegate, String ownerClassName) {\n            super(api, delegate);\n            this.ownerClassName = ownerClassName;\n        }\n\n        @Override\n        public MethodVisitor visitMethod(\n                int access, String methodName, String descriptor,\n                String signature, String[] exceptions) {\n\n            // Get the default MethodVisitor from the ClassWriter chain\n            MethodVisitor originalVisitor = super.visitMethod(\n                    access, methodName, descriptor, signature, exceptions);\n\n            // Skip abstract and native methods — they have no bytecode body to visit\n            boolean isAbstract = (access & Opcodes.ACC_ABSTRACT) != 0;\n            boolean isNative   = (access & Opcodes.ACC_NATIVE) != 0;\n            if (isAbstract || isNative || originalVisitor == null) {\n                return originalVisitor;\n            }\n\n            // Wrap with our timing advisor\n            return new TimingMethodAdvisor(\n                    Opcodes.ASM9, originalVisitor, access,\n                    methodName, descriptor, ownerClassName);\n        }\n    }\n\n    // ── Inner class: injects nanoTime calls at method entry and exit ──────────\n\n    static class TimingMethodAdvisor extends AdviceAdapter {\n\n        private final String ownerClass;\n        private final String methodName;\n        private int startTimeLocalVarIndex; // index of our injected local variable\n\n        TimingMethodAdvisor(int api, MethodVisitor delegate, int access,\n                            String methodName, String descriptor, String ownerClass) {\n            super(api, delegate, access, methodName, descriptor);\n            this.ownerClass  = ownerClass;\n            this.methodName  = methodName;\n        }\n\n        @Override\n        protected void onMethodEnter() {\n            // Inject: long __startTime = System.nanoTime();\n            // INVOKESTATIC pushes the long result onto the stack\n            mv.visitMethodInsn(\n                    Opcodes.INVOKESTATIC,\n                    \"java/lang/System\",\n                    \"nanoTime\",\n                    \"()J\",\n                    false);\n\n            // Store that long into a new local variable\n            // newLocal() allocates the next available slot in the local variable table\n            startTimeLocalVarIndex = newLocal(Type.LONG_TYPE);\n            mv.visitVarInsn(Opcodes.LSTORE, startTimeLocalVarIndex);\n        }\n\n        @Override\n        protected void onMethodExit(int opcode) {\n            // This is called for EVERY exit: RETURN, IRETURN, ARETURN, ATHROW, etc.\n            // AdviceAdapter handles all the variants — we just implement this once\n\n            // Inject: long __elapsed = System.nanoTime() - __startTime;\n            mv.visitMethodInsn(\n                    Opcodes.INVOKESTATIC, \"java/lang/System\",\n                    \"nanoTime\", \"()J\", false);\n            mv.visitVarInsn(Opcodes.LLOAD, startTimeLocalVarIndex);\n            mv.visitInsn(Opcodes.LSUB); // stack now has elapsed nanoseconds (long)\n\n            // Inject: TimingReporter.record(\"io/codeforge/Foo#doWork\", elapsed);\n            // We push the method label as a constant string first\n            mv.visitLdcInsn(ownerClass + \"#\" + methodName);\n            // Swap so the stack order is (String label, long elapsed) matching the descriptor\n            // Actually we need to restructure: push label first, then elapsed\n            // Easiest: store elapsed, push label, reload elapsed\n            int elapsedLocalVarIndex = newLocal(Type.LONG_TYPE);\n            mv.visitVarInsn(Opcodes.LSTORE, elapsedLocalVarIndex);\n            mv.visitLdcInsn(ownerClass + \"#\" + methodName);\n            mv.visitVarInsn(Opcodes.LLOAD, elapsedLocalVarIndex);\n\n            mv.visitMethodInsn(\n                    Opcodes.INVOKESTATIC,\n                    \"io/codeforge/agent/TimingReporter\",\n                    \"record\",\n                    \"(Ljava/lang/String;J)V\",\n                    false);\n        }\n    }\n}",
        "output": "[TimingAgent] Instrumented: io/codeforge/demo/OrderService\n[TimingAgent] Instrumented: io/codeforge/demo/PaymentGateway\n[TimingAgent] Instrumented: io/codeforge/demo/InventoryRepository\n[TimingReporter] io/codeforge/demo/OrderService#processOrder => 4,312,871 ns (4.3 ms)\n[TimingReporter] io/codeforge/demo/PaymentGateway#charge => 231,048,203 ns (231 ms)\n[TimingReporter] io/codeforge/demo/InventoryRepository#findById => 892,441 ns (0.9 ms)"
      }

Runtime Attach, Class Retransformation, and Production Performance Gotchas

Startup agents are clean but require a JVM restart. The Attach API lets you inject an agent into a live PID — indispensable for production diagnosis. The VirtualMachine.attach(pid) call establishes a socket connection to the target JVM's attach listener thread, then loadAgent sends the agent JAR path over that socket. The target JVM loads your JAR, calls agentmain, and if Can-Retransform-Classes: true is set, you can call instrumentation.retransformClasses(MyClass.class) to retroactively apply your transformer to already-loaded classes.

Retransformation is powerful but constrained. You cannot change the class schema: no adding fields, no adding methods, no changing method signatures or the class hierarchy. You can only change method bodies. Violating this causes an UnsupportedOperationException with the message 'class redefinition failed: attempted to change the schema'. This is a JVM limitation, not ASM's — the JVM's HotSpot runtime can hot-swap method bodies but cannot reorganize vtables or field layouts without a full GC pause and internal restructuring it simply doesn't support.

The performance cost of instrumentation has three parts. First, transformation time: every class load runs your transformer. Keep the fast-path null return tight — a single startsWith check costs nanoseconds. Second, the injected code itself: System.nanoTime() is not free (it's a JNI call on some platforms), and if you're logging inside transformed methods you're paying I/O costs on every call. Third, and most insidious: ClassWriter.COMPUTE_FRAMES triggers ASM to load referenced classes through the provided ClassLoader to compute frame types, which can cause unexpected class loading side effects. Pass the original ClassLoader from the transform signature into your ClassWriter to avoid ClassNotFoundException at transformation time.

The Attach API uses a socket file (/tmp/.java_pid<pid> on Linux). If the target JVM is running in a container without shared /tmp, the attach listener socket may not be accessible. This is a common issue in Kubernetes environments. To work around, ensure the /tmp directory is shared or use -XX:+StartAttachListener with a custom path via -XX:+AttachListener. Also, the attach listener thread might be blocked by a long GC pause — the attach() call will timeout after about 10 seconds. In such cases, retry with a longer timeout (unfortunately the JVM doesn't expose a configurable timeout; you'll need to implement a retry loop in your attacher).

LiveAttachDemo.javaJAVA
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
import com.sun.tools.attach.VirtualMachine;
import com.sun.tools.attach.VirtualMachineDescriptor;
import java.util.List;

/**
 * Attaches a Java agent to a RUNNING JVM without restarting it.
 *
 * Requires: JDK (not just JRE), and the target JVM must allow self-attach or
 * be a separate process. Run this as a standalone program.
 *
 * Compile: javac --add-exports java.base/sun.nio.ch=ALL-UNNAMED LiveAttachDemo.java
 * Run:     java -cp .:$JAVA_HOME/lib/tools.jar LiveAttachDemo <target-pid> <agent-jar-path>
 *
 * Java 9+ note: tools.jar is gone; the Attach API is in jdk.attach module:
 *   java --add-modules jdk.attach LiveAttachDemo <pid> <agent.jar>
 */
public class LiveAttachDemo {\n\n    public static void main(String[] args) throws Exception {\n        if (args.length < 2) {\n            System.err.println(\"Usage: LiveAttachDemo <target-pid> <path-to-agent.jar>\");\n            System.exit(1);\n        }\n\n        String targetPid   = args[0];\n        String agentJarPath = args[1];\n\n        // List all JVMs on this machine so the user can verify the PID\n        System.out.println(\"\\n=== JVMs currently running on this machine ===\");\n        List<VirtualMachineDescriptor> jvmList = VirtualMachine.list();\n        for (VirtualMachineDescriptor descriptor : jvmList) {\n            System.out.printf(\"  PID: %-8s  Display: %s%n\",\n                    descriptor.id(), descriptor.displayName());\n        }\n        System.out.println();\n\n        // Attach to the target JVM — this opens a socket to its attach listener\n        System.out.println(\"[Attacher] Attaching to PID: \" + targetPid);\n        VirtualMachine targetJvm = VirtualMachine.attach(targetPid);\n\n        try {\n            // The JVM will call agentmain() in our TimingAgent class\n            // The second argument (agentOptions) becomes the agentArgs param in agentmain\n            String agentOptions = \"io/codeforge/demo\"; // package prefix to instrument\n            targetJvm.loadAgent(agentJarPath, agentOptions);\n            System.out.println(\"[Attacher] Agent loaded successfully into PID \" + targetPid);\n\n        } finally {\n            // Always detach to close the socket — a leaked attach can prevent\n            // future attaches and consume file descriptors\n            targetJvm.detach();\n            System.out.println(\"[Attacher] Detached cleanly.\");\n        }\n    }\n}",
        "output": "=== JVMs currently running on this machine ===\n  PID: 18443     Display: io.codeforge.demo.OrderServiceApp\n  PID: 22901     Display: LiveAttachDemo\n  PID: 31042     Display: org.gradle.launcher.daemon.bootstrap.GradleDaemon\n\n[Attacher] Attaching to PID: 18443\n[Attacher] Agent loaded successfully into PID 18443\n[Attacher] Detached cleanly.\n\n--- Output from PID 18443 (the target JVM) ---\n[TimingAgent] Agent loaded. JVM classes currently loaded: 5,847\n[TimingAgent] ClassFileTransformer registered. Ready.\n[TimingAgent] Instrumented: io/codeforge/demo/OrderService\n[TimingReporter] io/codeforge/demo/OrderService#processOrder => 6,102,887 ns (6.1 ms)"
      }
● Production incidentPOST-MORTEMseverity: high

Production Outage: Agent's COMPUTE_FRAMES Crashes JVM on Class Load

Symptom
After deploying a new version of the monitoring agent, the application took 10x longer to start, then began throwing ClassNotFoundException for classes that clearly existed on the classpath. No traffic could be served.
Assumption
We assumed the agent was safe because it had been running for months without issues — the new version only added instrumentation for a new library.
Root cause
The transformer used ClassWriter.COMPUTE_FRAMES but didn't pass the original ClassLoader from the transform() method signature. ASM internally loads referenced classes to compute stack map frames, and without the correct ClassLoader, it couldn't find the new library's classes, causing the transform to throw an exception that was silently caught (returning null), which then cascaded as later transformers and class resolution failed.
Fix
Modified the transformer to pass the ClassLoader from the transform parameters into the ClassWriter constructor explicitly. Also added logging at every transform failure with the ClassLoader identity to aid future debugging.
Key lesson
  • Always pass the original ClassLoader to ASM's ClassWriter when using COMPUTE_FRAMES.
  • Never rely on the context ClassLoader — it may be null for bootstrap classes.
  • Log all transform exceptions with full stack traces — silenced failures mask production outages.
  • Test agent updates against a representative workload before rolling to production.
Production debug guideSymptom-Action pairs for the most common agent failures5 entries
Symptom · 01
Agent JAR loaded but transformer never called
Fix
Verify MANIFEST.MF has Premain-Class (or Agent-Class) pointing to the correct class. Use unzip -p agent.jar META-INF/MANIFEST.MF to inspect. Check JVM flags for typos in -javaagent path.
Symptom · 02
Class loads throw VerifyError after transformation
Fix
Check if you used COMPUTE_FRAMES. If manual stack frames are wrong, the verifier rejects the class. Also verify you're not corrupting the constant pool or changing method signatures.
Symptom · 03
ClassNotFoundException when using COMPUTE_FRAMES
Fix
Pass the ClassLoader from the transform() method directly to ClassWriter constructor. If using a different ClassLoader, the transform may fail to resolve types during frame computation.
Symptom · 04
Agent loads but instrumentation has no effect on certain classes
Fix
Confirm classes are being loaded after the transformer is added. For classes loaded before agent attach (via -javaagent), retransformation is required. Check if Can-Retransform-Classes is true in MANIFEST.MF.
Symptom · 05
Attach API fails with 'java.io.IOException: Operation not supported'
Fix
Self-attach requires -Djdk.attach.allowAttachSelf=true on Java 9+. For remote attach, ensure the target JVM's attach listener is enabled (default on, but some security managers disable it).
★ Quick Agent Debugging Cheat SheetThree-command patterns for the most critical agent failures
Agent not loaded at startup
Immediate action
Check JVM command line for -javaagent syntax
Commands
java -verbose:class -javaagent:agent.jar=args -jar app.jar 2>&1 | grep '\[Loaded' | head -20
unzip -p agent.jar META-INF/MANIFEST.MF
Fix now
Ensure MANIFEST.MF has correct Premain-Class line (no trailing spaces) and file ends with newline.
VerifyError after transformation+
Immediate action
Disable the transformer to confirm it's the cause
Commands
java -XX:-UseSplitVerifier -javaagent:agent.jar -jar app.jar (temporary, Java 8 only)
javap -v -p ModifiedClass.class | grep -A 20 'StackMapTable'
Fix now
Switch to ClassWriter.COMPUTE_FRAMES in your transformer, or manually compute correct stack map frames.
Retransformation fails with UnsupportedOperationException+
Immediate action
Check MANIFEST.MF for Can-Retransform-Classes: true
Commands
unzip -p agent.jar META-INF/MANIFEST.MF | grep -i 'Can-Retransform'
java -XX:+TraceRedefineClasses -javaagent:agent.jar -jar app.jar
Fix now
Set Can-Retransform-Classes: true in MANIFEST.MF and rebuild agent JAR.
Attach fails: 'com.sun.tools.attach.AgentLoadException'+
Immediate action
Verify PID and agent JAR path
Commands
java -cp .:$JAVA_HOME/lib/tools.jar com.sun.tools.attach.VirtualMachine list
jcmd <PID> JVMTI.data_dump
Fix now
Run the attach process with --add-modules jdk.attach on Java 9+ and ensure tools.jar is on classpath for Java 8.
🔥

That's Advanced Java. Mark it forged?

7 min read · try the examples if you haven't

Previous
Java Logging with SLF4J and Logback
25 / 28 · Advanced Java
Next
JUnit 5 Annotations: @Test, @BeforeEach, @AfterEach and More