We are working on a number of exciting and challenging innovations at Coursera. Recently, we released one such project: Code Executable Blocks.
Code Executable Blocks (as covered by FastCompany) allow course instructors to extend the Coursera platform with a development environment exposed as an embeddable code editor. This editor can then be integrated into quizzes, videos, generic course pages and more.
Code Executable Blocks is powered by three technologies, both internal and external to Coursera:
- Coursera Markup Language (CML)
Coursera Markup Language (CML)
As we started to migrate more of our courses to the new platform, we expanded CML to support links, images, videos and audio.
We built CML from the ground up to be extensible, much like the way HTML can be extended with new tags. Unlike HTML, CML removes the need for instructors to deal with design, layout and responsiveness by focusing directly on the content and the various CML blocks that it supports. Coursera takes care of the rest!
When it came time to explore how to embed code editors into our course, CML was ready for just another block extension. This block exposes an embedded ace editor library that supports syntax highlighting and a code editing user experience.
Running arbitrary code has to be done very cautiously.
We want to generalize Code Executable Blocks so that our instructors can build arbitrary environments and expose them to learners, all without interference from our platform.
At Coursera, we actually have experience with this from a previous project: programming assignments. With programming assignments, we grade incoming submissions from learners against an instructor’s grading script. Just like Code Executable Blocks, we needed sandboxing, scaling and generalizability to various code execution environments.
Docker gave us the solutions we needed for both projects!
Docker is an exciting technology that gives us a turnkey solution for sandboxing processes to a secure environment. With a little bit of SELinux magic, we can run arbitrary code with only a few constraints like memory, CPU time and limitations on network access.
Coursera runs almost exclusively on top of AWS. This allows us to build a lean infrastructure team that spends more time building amazing internal developer tools, exploring graphQL and coming up with our long term architecture.
With programming assignments, we turned to Amazon ECS to help us coordinate our Docker images to run course graders.
However, when designing Code Executable Blocks, we couldn’t reuse the same architecture. With our grader service, submissions are queued and learners are later informed when grading is done. This allows us to coordinate ECS such that a Docker image is turned on at the time of submission request and a grade is later synced to the learners progress store. All of this takes some time and thus we let learners know by email when grading is done.
With Code Executable Blocks, the environment is much different. Learners expect a real time response as if they were using an IDE or a terminal. Coordinating these requests through ECS requires a latency hit (~200-300ms to spin up Docker) that we can’t hide.
Luckily, AWS gives a lot of flexibility in how we architect our services at Coursera. We ended up using ELB in front of auto-scaled EC2 machines (much like we do for our other web services). We built a custom control plane to schedule incoming ECB requests on shared resources. On each request, we spin up the appropriate Docker image, run the arbitrary code, and then return the results as fast as possible!
We are still learning as we scale this out and we would love to share more with you. Luckily both the learning experience team and the infrastructure team have a few open positions, so if you want to learn more, consider joining our engineering team!