This is an advanced guide to programming an autograder on the CodeHS platform. This guide assumes prior knowledge of the basic functioning of an autograder, and provides further details on available parameters, test methods, and test options.

Configuring an Assignment with an HTML Autograder

From the Create App

Access the Create Assignments tool by clicking Create from the left menu.
1. If you haven't done so yet, Create a Coding Exercise and choose HTML.
2. Add an assignment title, description, and any starter or solution code for the assignment.
3. Save your changes, and close the popup window, and refresh the page.
Click the three dots "..." next to an assignment you created and choose Edit.
Scroll down to the bottom of the window and choose Custom Autograder Edit.
Click on the Add Custom Autograder button. An autograder with example tests and comments will be added. You can also enable any of the other premade autograder checks.
Write your code to create your autograder and click Save Autograder Code.

Animated GIF demonstrating how to access the Autograder configuration window from the Create page in CodeHS

From the Assignments App

Click Assignments from the left menu. Choose the course where you'd like to add or edit an assignment by clicking the dropdown menu in the upper left corner
1. If you haven't done so yet, Create a Coding Exercise and choose HTML.
2. Add an assignment title, description, and any starter or solution code for the assignment.
3. Save your changes, and close the popup window, and refresh the page.
Click [Edit] next to an assignment you created.
Scroll down to the bottom of the window and choose Custom Autograder Edit.
Click on the Add Custom Autograder button. An autograder with example tests and comments will be added. You can also enable any of the other premade autograder checks.
Write your code to create your autograder and click Save Autograder Code.

Animated GIF demonstrating how to access the Autograder configuration window from the Assignments page in CodeHS

Boilerplate Autograder

This is what you see when you first add an autograder to a program. It already contains examples of how to write tests:

Building Your Autograder

Tests That Can Be Run

.find() : Used to search for and select specific elements within the HTML structure of a document. It works by targeting elements based on their tag names, classes, IDs, or other selectors.
.length() : Used to count the number of elements selected by a method like .find(). It is best to set the value to one less than how many tags/text should appear in student's code.
.includes() : Used to verify whether an element's attribute or content includes the expected value.
.attr() : Used to get or set the value of an attribute of an HTML element.
.replace() : Used to replace occurrences of specific strings or pattern with another string.

Editing or Creating Custom Pass/Fail Messages for Test Cases

In the autograder, you can define custom pass/fail messages for each test case by modifying the result object in the this.tests.push structure. This allows you to provide tailored feedback to guide the user based on whether their code meets the test criteria.

Example Code Block:

var hasBodyTag = autograder.$codeTree['index.html']
    .find('body')  // Change 'div' to any tag or text you want to check
    .length > 0;  // Change value to one less than how many tags/text should appear in student's code

this.tests.push({
    testName: 'Your code must have a "body" tag.',  // This is the incorrect statement
    result: {
        // Make sure the variables below match the one used for this test case (line 26 in this file)
        success: hasBodyTag,
        message: hasBodyTag ? 'Great!' : 'Try again!',
    },
});

Explanation:

hasBodyTag:
- This variable checks if the <body> tag exists in the student's HTML code.
- .find('body') searches for <body> tags, and .length > 0 confirms at least one <body> tag is present.
this.tests.push:
- Adds a test case to the autograder.
- The testName field describes what the test is checking.
result:
- success: A boolean that determines if the test passed. In this case, it uses hasBodyTag to confirm if the <body> tag exists.
- message: A customized pass/fail message:
  - If hasBodyTag is true, it displays 'Great!'.
  - If hasBodyTag is false, it displays 'Try again!'.

How to Customize:
To create or edit a test case:

Update the condition in hasBodyTag (or similar variables) to match the element or criteria you want to test.
Change the testName to reflect the purpose of the test.
Adjust the message for both the success and fail scenarios to provide more specific feedback.

Example for Testing <h1> Tags:

var hasH1Tag = autograder.$codeTree['index.html']
    .find('h1')
    .length > 0;

this.tests.push({
    testName: 'Your code must include an <h1> tag.',
    result: {
        success: hasH1Tag,
        message: hasH1Tag ? 'Excellent, you added an <h1>!' : 'Oops, don’t forget an <h1> tag!',
    },
});

By customizing the message field, you can provide clear, actionable feedback that aligns with the specific goals of your test cases.

Examples

Below are examples of different test cases that are most commonly used in current HTML assignments. Feel free to copy and use or edit to your liking:

Check for "img" tag:

var hasImgTag = autograder.$codeTree['index.html']
        .find('img')  // Change 'div' to any tag or text you want to check
        .length > 0;  // Change value to one less than how many tags/text should appear in student's code
    this.tests.push({
        testName: 'Your code must have a "img" tag.',  // This is the incorrect statement
        result: {
            // Make sure the variables below match the one used for this test case (line 26 in this file)
            success: hasImgTag,
            message: hasImgTag ? 'Great!' : 'Try again!',
        },
    });

Check for at least 8 "li" tags inside of a "ol" tag:

var hasOlLiTag = autograder.$codeTree['index.html']
        .find('ol > li')  // This line checks for <li> within a <ul> tag
        .length > 7 ;  // This checks to see if student has at least 4 <li> inside of a <ul>
    this.tests.push({
        testName: 'Your code must have at least 8 "li" tags inside of a "ol" tag.',  // This is the incorrect statement
        result: {
            // Make sure the variables below match the one used for this test case (line 41 in this file)
            success: hasOlLiTag,
            message: hasOlLiTag ? 'Great!' : 'Try again!',
        },
    });

Check that the "body" background color should be #000000:

var madeH1Large = (autograder.$codeTree['index.html']
        .find('body')  // Change 'h1' to whatever tag you want to check
        .attr("style") || '')  // DO NOT EDIT THIS LINE
        .replace(' ', '')  // DO NOT EDIT THIS LINE
        .includes("background-color:#000000" || "background-color:black");  // Change to any style --> No Spaces
    this.tests.push({
        testName: 'The "body" background color should be #000000.',  // This is the incorrect statement
        result: {
            // Make sure the variables below match the one used for this test case (line 54 in this file)
            success: madeH1Large,
            message: madeH1Large ? 'Great!' : 'Try again!',
        },
    });

Check for "p" tag with id "funny":

var hasPWithClass1 = autograder.$codeTree['index.html']
        .find('div#funny')  // Checks for a class in a tag --> Use # instead of . to check for an id in a tag
        .length > 0;  // Change to check for number of instances in the HTML file
    this.tests.push({
        testName: 'Has p tag with id "funny"',  // This is the incorrect statement
        result: {
            // Make sure the variables below match the one used for this test case (line 70 in this file)
            success: hasPWithClass1,
            message: hasPWithClass1 ? 'Great!' : 'Try again!',
        },
    });

Things to Keep in Mind

You only need to edit the htmlAutograder.js file
When checking for a certain number of tags/text, change the value to one less than how many tags/text should appear in the student's code (If looking for 8 "li" tags, the value should be .length > 7).
Use "." to check for a class in a tag or "#" for an id in a tag.
When creating test cases, you only need to edit what is in between lines 52-88 or within the HtmlAutograder.prototype.runHtmlAutograder function in the htmlAutograder.js file.

Demo Video

Still have questions? Contact our team at hello@codehs.com to learn more!