Auckland university have a site that students can create various programming questions and then put in the code as well as a testing scheme. This informs students straight away if there program is correct.
The auckland university cs website has a webapp called codewrite
Learning how to program requires lots of practice, and practice requires good exercises. With CodeWrite, you are given the task of defining small programming exercises with which others can practice - and of course, you have access to all of the contributed exercises.This was a great way for students to get involved.
As far as I can gather, it also requires a sandbox to be installed so it can run the programs, this may require another server to be thrown into the mix just to do that processing.
This will be a great opportunity to get students to develop their own questions, answers and testing and will show some of the key competencies now required by our students as well as Curiosity? Problem solving? Decision making? Reliance?
Note: Looking at the way code academy works, this is what I am after for my students, but for python.