Check out this page for more detailed tutorials, tips, and code samples for creating Java Autograders.

Overview

This is a guide to creating a Java autograder on the CodeHS platform, detailing how to program an autograder and what tests are available. This guide assumes prior knowledge of the basic function of an autograder and provides further details on available parameters, test methods, and test options.

Autograders in Java rely on reading methods and output from the different class files that students use to complete each exercise. In general, to create a Java autograder, exercises need to have the solution code already written. The simplest example relies on students creating a method or function, which is then tested in the main method in the same class file.

For more information on creating/editing autograders, feel free to check out this article: Creating Autograders to Check Student Code

Building a Java Autograder

Autograder Starter Code

When you add a Java Autograder to an assignment, it comes preloaded with starter code containing example test cases. These examples are designed to help you get started quickly.

Autograder.java contains useful methods to help write autograders.
Grader.java is the only file you will edit to add your autograder tests.

💡 A Separate file should be created to host the exercise solution. It is best to name the file similarly to the student's file. For example, if the student file is named ExtractDigits.java, the solution file could be named ExtractDigitsSol.java

Autograder Test Methods

Java autograder tests can be written in two main ways:

grader.addTest takes a boolean value and presents the student with a pass/fail result based on the boolean value:

grader.addTest(
    "This is the test name", 
    true, 
    "This is the student output", 
    "This is the solution output", 
    "This is the message students see!"
);

grader.assertEqual compares two values and determines if they are equivalent:

grader.assertEqual(
    "This is the test name", 
    "This is the student output", 
    "This is the solution output", 
    "Message if passed!", 
    "Message if failed"
);

Getting Student Output

To get output from a student's program, create an object from the student's class and call its methods. Do the same for the solution class, then compare the results.

ExtractDigits student = new ExtractDigits();
student.extractDigits(12345);

ExtractDigitsSol solution = new ExtractDigitsSol();
solution.extractDigits(12345);

For console output, use grader.getOutput, which takes the class name as its parameter and returns a string of all the output for that class:

String output = grader.getOutput("ExtractDigits");

💡 grader.getOutput should be called after the methods that print to the console have run. If you want to run multiple tests, use grader.clearOutput() before new print calls to clear previous output.

Once you have both the student and solution outputs, you can run a comparison:

grader.assertEqual(
    "Extract Digit has the correct output?", 
    studentOutput, 
    solutionOutput, 
    "Great!", 
    "Try again!"
);

Reading Student Code

Java autograders can analyze a student's source code to check for specific variables, keywords, or patterns. For example, you could check whether the student created a variable named myName or used the keyword private to declare a variable.

To read the contents of a student's file, include the following code in your autograder:

import java.io.File;
ArrayList<String> userLines = new ArrayList<String>();

try {
    Scanner input = new Scanner(new File("YourFileName.java"));
    while (input.hasNext()) {
        userLines.add(input.nextLine());
    }
} catch (Exception e) {
    grader.addTest("File Not Found", false, "", "", 
        "Looks like we have a problem on our end");
}

Replace "YourFileName.java" with the name of the file students are working on.

⚠️ When reading student code in this way, comments are not excluded. If a student writes the target phrase inside a comment, the test case may still pass.

Creating Java Autograder Demo

Still have questions? Contact our team at hello@codehs.com to learn more!

Creating a Java Autograder