-
Notifications
You must be signed in to change notification settings - Fork 34
Home
vmchecker is a grading platform built by students from University POLITEHNICA of Bucharest. vmchecker requires two systems to run:
- A storer system: where assignments are uploaded, stored, submitted for testing and graded by teachers.
- A tester system: where assignments are tested/checked using a virtual machine infrastructure.
We welcome contributors. If you know a bit of Python and/or JavaScript and a bit on virtualization, please join the mailing list by visiting the mailman page for discussion on vmchecker.
To make contributions fork the vmchecker repository on GitHub.
Make your changes and check Python source code using pylint then push your code. You may request code review on the mailing list or create a pull request. Wait for your code to be pulled into the main repository and a thanks message from the maintainers. Grab yourself a beer or a cup of coffee and celebrate.
We recommend using our virtual machine in order to test your code before submitting a pull request.
-
bin
: A directory that should be contained by the storer machine. It stores the following Python scripts :-
vmchecker-download-external-files
: A script to download files (that are not stored on the storer) required for testing from another system through SSH. -
vmchecker-init-course storer
: Initializes the directory path for a class storer machine. Its actions are:- Creates a sample config file named
config
in the current directory. It gets the sample config fromexamples/config-template
.
- Creates a sample config file named
- Creates the repo for the assignments on the storer.
- Create the implicit db if it does not exist.
- Copy the example auth file from
examples/auth_file.json
. - Copy build and run example scripts into the root path
- Run initialization tasks for the tester machine.
- Run initialization tasks for the storer machine.
-
vmchecker-init-course tester
: Initializes the directory path for a course tester machine. -
vmchecker-queue-manager
: Waits for assignments to appear and invokevmchecker-vm-executor
to handle them. -
vmchecker-submit
: Submit assignments from the command line. -
vmchecker-update-db
: Manually update the db storing info from the repo. -
vmchecker-view-grades
: Generates a HTML table containing student grades. -
vmchecker-vm-executor
: Starts a vm, reverts it to a known snapshot, tests a submission bundle (submission + tests), and closes the vm.
-
The bin
folder must also reside on the tester machine, where vmchecker-queue-manager
runs as a deamon. For each submission vmchecker-vm-executor
is started.
The vmchecker WebUI authentication is achieved through the university LDAP server. If the user isn't on the LDAP server, then the user is searched in a plain text file, where the user names and the passwords are saved.
In order to add a new class on vmchecker, you need to add a corresponding user on the tester system and configure vmchecker for that user. You may then configure assignments, request students to submit assignments and then grade assignments.
We'll use pc
(for "Programarea Calculatoarelor") as the username to describe the steps to follow. The steps are show below. We assume the tester system is elf.cs.pub.ro
.
- First you'll need to create the actual user:
$ ssh [email protected]
elf:~# adduser pc # don't set a password; use ssh keys
elf:~# su - pc
pc@elf:~$ ssh pc@localhost # creates .ssh directory with default permissions
pc@elf:~$ emacs .ssh/authorized_keys # put your id_rsa.pub key here and log-out
- Create and initialize the vmchecker class folder:
$ ssh [email protected]
pc@elf:~$ mkdir vmchecker-storer
pc@elf:~$ cd vmchecker-storer/
pc@elf:~/vmchecker-storer$ vmchecker-init-course storer
- Create ssh key to connect to tester system:
pc@elf:~/vmchecker-storer$ ssh-keygen -f ~/.ssh/id_rsa-vmchecker -N '' # creates a key with no password
- Add the SSH key to the corresponding user on the tester system (we'll assume it is calleed
checker.cs.pub.ro
; you will use whatever you have configured):
youraccount@yourhostname:~$ ssh [email protected]
pc@sanctuary:~$ emacs .ssh/authorized_keys
- Test connection and also save the fingerprint to
.ssh/known_hosts
:
pc@elf:~$ ssh -i .ssh/id_rsa-vmchecker [email protected]
The authenticity of host 'checker.cs.pub.ro (141.85.227.120)' can't be established.
RSA key fingerprint is 8a:0a:51:86:46:85:17:85:b6:84:3d:e1:b5:68:45:01.
Are you sure you want to continue connecting (yes/no)? yes
- Add [email protected]'s public key in the
~.ssh/authorized_keys
file:
pc@elf:~$ emacs .ssh/authorized_keys
# paste-in [email protected]'s password
- Make private key readable by
www-data
(grantswww-data
read access to the file):
pc@elf:~/.ssh$ setfacl -m u:www-data:r id_rsa-vmchecker
- Edit vmchecker config file:
pc@elf:~/vmchecker-storer$ emacs config
You'll find a sample config file in the repository.
-
Add tests to
vmchecker-storer/tests/<asg-name>.zip
files.<asg-name>
is the name of each assignment as specified in theconfig
file. -
Add new class to global list:
$ ssh [email protected]
elf:~# emacs /etc/vmchecker/config.list
Add this to the file:
PC:/home/pc/vmchecker-storer/config
The class folder is generally stored in /home/class_name/vmchecker-storer
(where class_name
is the name of the class: so
, or so2
or programare
) and will look something like this:
/home/class_name/vmchecker-storer
|--tests
|--asg1_tests.zip
|--build.sh
|--run.sh
|--config
|--auth_file.json
|--vmchecker.db
auth_file.json
is a plain text file that contains the user name and password for the users that are not in the ldap server.
vmchecker.db
is an SQLite3 database that contains the paths to the assignment archives that will be graded.
In order to add a new assignment the user must:
- Add the assignment in the config file.
- Upload the assignment tests.
- Upload the
run
script. - Upload the
build
script
Each class specific folder (vmchecker-storer/
) resides on the storer system contains a repository where assignments are saved. The repository follows a hierarchical structure consisting, on each level, of:
- 1st level: assignment name
- 2nd level: student name that submitted assignment for the 1st level
- 3rd level: submission id of submission by student for the given assignment
- 4th level: three items
-
archive.zip
: a compressed archive of the submission -
archive/
: a folder representing the uncompressed submission -
results/
: a folder that stores assignment test results and grading
-
The results/
folder is created after the homework is sent back from the tester machine after being tested/checker. It stores the entries below:
-
submission-config
: contains assignment details. -
grade.vmr
: contains the grades given by the teacher -
run-stdout.vmr
: contains the results of the automatic tests. -
build-stdout.vmr
: contains the results of the build.
In order to upload an assignment a user must log in the WebUI, browse for the assignment archive and click submit.
On the click of the submit button a new directory is created in the course name with the user name and the assignment archive, if it does not exist.
It may happen that due to an error on the tester system side, an assignment needs to be resubmitted; to preserver timestamp and other. You may use the submit.py
script for this, while located in the class folder. A sample run is:
./bin/submit.py 1-minishell-linux Alexandru\ Moșoi path_to_some_archive.zip