The following simple illustration portrays your test configuration:
Test configuration using other JVM
This illustration is included to help you understand which parts of the test process run on which VMs. This is important to understand when you configure the classpath for each of the VMs. The JavaTest harness and your compilers run in different processes. Processes are started as needed depending on the type of compiler test:
Test Type Description
Negative Only the Java compiler is started, the compiler is expected to fail and produce an error. A new compiler is started for every test.
Positive (Java) A new Java compiler is started for every test -- output of the compiler is always checked using a fresh version of the reference VM.
Positive (RMI) New Java and RMI compilers are started for every test -- output of the compilers is always checked using a fresh version of the reference VM.

The Configuration Editor attempts to set the classpaths correctly, but if you get errors about classes not being found, check that the classpaths are set correctly.