Quick Start: How to set up the first exercise?

Introduction

In the following we will show how to create an exercise and set it up in CodeJudge. We will use Java as language, but this can easily be substituted by another language (see "Quick Start: How to create test data for my exercise?" for examples). In order to do this, we will describe the following steps:

  1. Create an exercise
  2. Prepare a solution
  3. Prepare test cases
  4. Upload to CodeJudge
  5. Test it!

Please note this is a quick start guide, and CodeJudge supports many more features for testing then described here, but this guide will take you through the most common features.

Create an exercise

First we should figure out what the exercise should be about. In the following we will use the following problem:

Create a Java class Calculator that supports the method int sum(int a, int b) that should return the sum of the intgers a and b. Furthermore Calculator should support the method int factorial(int a) that must return 1 * 2 * ... * a = a! (should work for a >= 0).

Suggestions:

  • The problem should be testable, ie. input/output must be well defined.
  • Clearly state what classes/methods/etc. the students should make and what the indended behaviour is.
  • Mark the exercise with "CodeJudge" or something similar, to indicate the exercise can be tested on CodeJudge.
  • For more complex problems, it can be a good idea to include samples of how the methods could be called and what it should return.

Prepare a solution

Next, we create a solution for the problem. This serves multiple purposes; we get a feeling of how difficult the problem is to solve and we will use it to generate the expected output and as "suggested solution" in CodeJudge.

// File: Calculator.java
public class Calculator {
    public int sum(int a, int b) {
        return a + b;
    }

    public int factorial(int a) {
        if (a == 0) {
            return 1;
        }

        return a * factorial(a - 1);
    }
}
        

Suggestions:

  • Use the coding style you want your students to use, ie. remember to do indentation, commenting, etc. since it will be used as a suggested solution.
  • Spend some extra time to verify the solution is correct - otherwise, you will probably get some annoyed students.

Prepare test cases

Let us jump right in to an example of a test file with test cases for our problem, and then discuss how it works afterwards:

// File: Tests.java
public class Tests {
    public static void main(String[] args) {
        Calculator calculator = new Calculator();

        /// Test
        System.out.println(calculator.sum(3, 6));
        /// Test
        System.out.println(calculator.sum(-5, -9));
        /// Test
        System.out.println(calculator.sum(33, 64));

        /// Test
        System.out.println(calculator.factorial(5));
        /// Test
        System.out.println(calculator.factorial(10));
        /// Test
        System.out.println(calculator.factorial(2));
        /// Test
        System.out.println(calculator.factorial(1));
        /// Test
        /// Hint: Remember 0! = 1
        System.out.println(calculator.factorial(0));
        /// EOT
    }
}        

Hopefully, it is relatively clear what the above test cases are, so now we will look at the structure of the file. Note the above test code is a fully correct and runable Java program when combined with the solution, so we can easily develop both the solution and the test file directly in our favorite IDE.

It's important to note that this file must be called Tests.java and thus the class name should be Tests. Note how the beginning of a test should be marked with "/// Test", and after the last test a line with "/// EOT" must be written (EOT = End Of Test). A test can easily consists of multiple code lines. All lines not inside a test are considered common to all tests. Finally a hint can be added with "/// Hint: The hint text here" - this hint will only be shown, when a student fails the test. Especially useful for special/tricky test cases.

When the file is uploaded to CodeJudge it is split into a runable program for each test. For instance the first test will be (the class is automatically renamed):

public class Test01 {
    public static void main(String[] args) {
        Calculator calculator = new Calculator();

        System.out.println(calculator.sum(3, 6));
    }
}

You might now be thinking that we also somehow need to tell what the expected outcome of our test cases are. However, in this example we will let CodeJudge produce the expected output by uploading just the "input" of the test cases and the solution, ie. Tests.java and Calculator.java. (It is possible to specify the expected output instead - see the documentation for tihs)

Test cases can be much more complex than this example; they may involve input from the console, command line arguments, files, multiple correct outputs, etc. For more about this, see the documentation.

Suggestions:

  • Put common case tests first, then special cases and finally larger tests. This will allow for easier debugging for the students.
  • Be thorough with the test cases, students sometimes make very unexpected solutions. In our experience 5-15 test cases will be sufficient for most problems - obviously depending of the difficulty.
  • Try not to make unnecessary long and complex test code, remember the students should be able to understand what is tested.
  • When creating tests for special cases, make sure the correct behaviour is actually described in the problem text.
  • You can test error handling with "invalid" input by surrounding test code with a try-catch block (again, remember to tell students they are supposed to throw an exception in this case).

Upload to CodeJudge

It is now time to setup the exercise on CodeJudge. First, we navigate to our course on CodeJudge. Before setting up the exercise, we have to create an exercise group. Normally one will create an exercise group per week/exercise class/similar. Follow these steps:

  1. Select "Exercises" in the menu.
  2. Click "Add Exercise Group".
  3. Give the exercise group a name (like "Week XX") and choose when the students should be able to see the test group in "Visible From".
  4. Click "Save".

We are now ready to setup the actual exercise. We will now use the files "Calculator.java" and "Tests.java" created before:

  1. Navigate to the exercise group you just created.
  2. Click "Add Exercise".
  3. Fill in the name of the exercise and the description (the type should be left unchanged on "Code Exercise").
  4. Click "Create".
  5. If Java is not the default language for your course, you should select this under "Languages" and press "Save".

We are now ready to upload the solution we have made:

  1. Choose the file "Calculator.java" under "Solution Files".
  2. Choose when your students should be able to see the suggested solution: always, when they have solved it themselves, or never.
  3. Click "Save".

Finally we must upload the test data. This will automatically generate the expected output as well. An exercise can consist of multiple test groups, but simple exercises only need one. See the documentation for more information:

  1. Click "Create New Test Group".
  2. For "Test Data" select the file "Tests.java". Leave the remaining fields unchanged.
  3. Click "Save".

After a short while of processing, the exercise is ready!

Test it!

As a final step, it is a good idea to see if the exercise actually works as expected (especially when it is the first time you create one). In the right side, you can find a link "See Exercise". Click this and try to upload your solution. If you have not tried CodeJudge before, it is also a good idea to experiment with the different kind of mistakes to students might make (compiler errors, programs producing exceptions, wrong output, etc)

CodeJudge.net - Automated Code Judging
28-05-2018 01:09