CodeJudge

Introduction for Users

Introduction

CodeJudge is designed to improve the experience of learning how to program. After coding and testing a program CodeJudge can be used to quickly verify whether the program is correct or not. You just need to upload the program to CodeJudge, then a couple of tests will be run automatically on the program, and the result of the tests will be presented within seconds. You will therefore quickly know if your program is correct, and if it is not, CodeJudge will provide good feedback from the failed test(s) to help the you find the mistake.

Exercises

An exercise in CodeJudge corresponds to a programming exercise from the course. When a solution to the exercise is programmed, the program can be submitted (called a submission) to CodeJudge and automatically be tested.

The exercises in the 'Exercises'-tab are meant to be training exercises. There can be submitted any number of solution at any time to these exercises, and the status of the exercise (solved, attempted or un-attempted, see submission status) is determined by the best submission. Therefore after solving the exercise, other solutions can still be submitted and tested without changing anything.

An exercise can have attached relevant files (such as header files, templates etc), which can be downloaded. The first test case and it's expected output can usually be shown before submitting any solutions. After the exercise is solved a suggested solution from the course adminstrator can be shown (if such one is uploaded).

Submissions

A submission is a single attempt of solving an exercise by uploading code to CodeJudge. You can use as many submissions as needed to solve an exercise, but the lesser the better. After submitting the program it is then 'graded', meaning it is compiled and then a number of tests are run to test its correctness.

How to Submit

To submit a solution, first select the language of the solution (if multiple languages are allowed). Then either copy the source directly into the text area, or upload the source code file. If the solution consists of multiple files, the files must be uploaded. If the solution has a specific dictionary structure, this structure must be zipped together and uploaded as a zip-file. Finally press the 'submit'-button.

To each submission a comment can be written and it will then be shown in the list of submissions. This is only for your own use to keep track of your submissions, for instance a comment could be "solution using ArrayList" or likewise.

Submission Status

After submitting, the submission can have one of the following statuses:

In queue
The submission is waiting for the system to grade it. This should normally only take a few seconds.
Processing
The submission is first compiled, and if compiled succesfully then the solution is tested on the test data.
Succeeded
The submission passed all tests.
Submitted
The submission passed all visible tests, but the exercise has some invisible tests for which the results are unknown.
Test(s) failed
The submission failed one or more tests.
Compilation Error
There was an error when compiling the submission. Look at the error message from the compiler.
Exercise Error
There is a problem with how this exercise was setup. Contact the exercise responsible.
System Error
An unknown error occured in the system. We will look into it as fast as possible. You can try to resubmit your solution.

Test Data

After the program is graded the relevant test data becomes accessible. Each test case can have one of the following results:

Correct
The test was passed, meaning that the program's output matched the expected output and it was produced within the timelimit.
Time limit exceeded
The program was terminated because it ran for too long (whether the program was correct or not cannot be determined since it was terminated). This can be due to it having an infinite loop, so it will never terminate, or that it is too slow to solve its task.
Runtime Error
The program had an error while executing (observe - this has nothing to do with the running time). This is often caused by unhandled exceptions in Java. More details can be seen in the 'Error' section of the test data.
Wrong output
The output of the program was different from the expected output.
System error
An unknown error occured in the system. We will look into it as fast as possible. You can try to resubmit your solution.

A specific test case can then be opened in an overlay, where both the input of the test, your program's output, the expected output and error messages if any are shown. The input/output will be truncated for very large tests, but all the test-data can then be downloaded as a zip-file instead.

Assignments

An assignment consists of a set of exercises similar to the training exercises, but since the solutions to these exercises are to be handed in, the exercises have a due-date and most of the test-data will likely be hidden.

Submitting

Any number of solutions can be submitted to the exercises of an assignment, but only the last submission before the due-time will count as the solution handed in. For the visible test data you will be able to see if the solution solved all tests or failed some. If all visible tests are passed, but the exercise has hidden tests, the solution will get the submitted status (see submission status) to indicate that only the visible test-cases are passed but the solution still might be wrong.

Groups

Assignments can be handed in in groups (if allowed by the course administrator). To form a group to an assignment, one person has to invite all the other group members, who then have to accept the invitation. When working in a group, all submissions to the assignment are shared, so all members of the group can submit solutions and all members can see each others submissions. This means that all group members can submit the handin submission (the last submission), not only the group leader.

Feedback / Comments

The course administrator can write feedback to the handins, which will then be visible on the assignment page. It is also possible to write a comment to the course administrator together with the solution when handing in.