💡 Check out this page for more detailed tutorials, tips, and code samples for creating JavaScript Autograders.

Overview

This is a guide to programming a JavaScript autograder on the CodeHS platform. This guide assumes prior knowledge of the basic functioning of an autograder, and provides further details on available parameters, test methods, and test options.

For more information on creating/editing autograders, feel free to check out this article: Creating Autograders to Check Student Code

Building a JavaScript Autograder

Autograder Starter Code

When a JavaScript autograder is added to an assignment, it comes preloaded with starter code that includes sample test cases. These examples are designed to help you get started quickly

Understanding the Parameters

beforeRun:

code is an object with keys student and solution

code.student is a String containing all of the student’s code
code.solution is a String containing all of the solution’s code

afterRun:

output is an object mapping {String:object}. The keys are student and solution

output.student is an object mapping {String:array}
- output.student.graphics is an array of all graphic objects created by the student’s program
- output.student.console is an array of Strings. Each element is one line of output from the student’s program
  VERY IMPORTANT: this does not contain any text that was printed using readInt, readLine, etc
- output.student.runnerData contains information about the execution of the program
output.solution is an object mapping {String:array}
- output.solution.graphics is an array of all graphic objects created by the solution program
- output.solution.console is an array of Strings. Each element is one line of output from the solution program
  VERY IMPORTANT: this does not contain any text that was printed using readInt, readLine, etc
- output.solution.runnerData contains information about the execution of the program

Autograder Test Methods

To create a test, use expect(res) and then call one of the following test methods. Let res be the student’s output/code:

toBe(expected)

Checks that res == expected
Code: expect(res).toBe(expected)
Example: checks the first line of student output prints a title
- expect(output.student.console[0]).toBe(“Karel’s To Do List”);

toEqual(expected)

Checks that res.isEqual(expected)
We suggest using this method instead of toBe()
Code: expect(res).toEqual(expected)
Example: checks the first graphics object added to the Canvas is a circle
- expect(output.student.graphics[0].type).toEqual('Circle');

toContain(expected)

Checks that the string expected is contained in res
Code: expect(res).toContain(expected)
Example: checks the student has a while loop
- expect(code.student).toContain(“while”);

toBeGreaterThan (expected)

Checks res > expected
Code: expect(res).toBeGreaterThan (expected)
Example: checks the student printed at least 4 lines of output
- expect(output.student.console.length).toBeGreaterThan(3);

toBeGreaterThanOrEqualTo (expected)

Checks res >= expected
Code: expect(res).toBeGreaterThanOrEqualTo (expected)
Example: checks the student printed at least 4 lines of output
- expect(output.student.console.length).toBeGreaterThanOrEqualTo(4);

toBeLessThan(expected)

Checks res < expected
Code: expect(res).toBeLessThan(expected)

Example: checks the student used no more than 2 for loops

var fors = code.student.match(/for/g); 
// match takes a regular expression 
var numFors = fors ? fors.length:0;
 //make sure fors isn’t null
expect(numFors).toBeLessThan(3);

toBeLessThanOrEqualTo(expected)

Checks res <= expected
Code: expect(res).toBeLessThanOrEqualTo(expected)

Example: checks the student used no more than 2 rectangles

var rects = output.student.graphics.filter(function(elem){ return elem.type == "Rectangle"});// rects will be an array with only Rectangle objectsexpect(rects.length).toBeLessThanOrEqualTo(1);

toBeUndefined()

Checks res === ‘undefined’
Code: expect(res).toBeUndefined()

Example: checks the student did not use any for loops

//str.match returns null if there are no matches
var fors = code.student.match(/for/g);
expect(fors).toBeUndefined();

Autograder Test Options

With each of the previous methods, you can then call withOptions() on the expect object. This determines the message students will see in the Test Cases. The parameter is an object with the following possible keys:

testName is the test name displayed to the student

defaults to a string representation of the test. For example, something like “Expected “print(“hello world’)” to contain “print””

messagePass is the message displayed if the test passed.

defaults to “Success”

messageFail is the message displayed if the test failed.

defaults to “Try Again”

studentOutput is what is shown as the student output (labeled “your result”).

defaults to what’s passed to expect

solutionOutput is what is shown as the solution output.

defaults to the value passed to the expectation function (e.g expected or value from the examples above)

showDiff shows the difference between the student’s output and the solution output

defaults to False
Note: unless the program only passes with very specific formatting, students usually find this more confusing than helpful

Example: this code checks if the student used a for loop and did not use a while loop.

This test produces the following test results. The code shown in Your Result is the student’s program. Note that there is no expected output shown.

image showing the autograder result when a student uses a while loop and the autograder fails

General Tips

We suggest looking for key pieces of code instead of using expect(output.student.console).toEqual(output.solution.console);
To help debug your tests, you can set studentOutput or solutionOutput to see the values of your variables or print them using console.log();
- Remember studentOutput and solutionOutput should be String values
The browser has a built in JavaScript console. You can use the console to work out syntax errors or make sure your array functions are behaving as expected.
String comparisons are case sensitive and whitespace sensitive
- use str.toLowerCase() to make str all lowercase
- use str.replace(‘ ‘, ‘’) to remove spaces (but not all whitespace)
  - Note: the first parameter is a single space and the second is an empty string

Still have questions? Contact our team at hello@codehs.com to learn more!

Creating a JavaScript Autograder