COMEM+ Web-Oriented Architecture Course

The goal of this course is to learn about the generic concept of web service, focusing on RESTful APIs as one way to expose such a service. You will:

• Learn the core principles of the REST architectural style.
• Learn how to implement a REST API in JavaScript with Node.js.
• Deploy your REST API on a cloud application platform.
• Write documentation and automated tests for your REST API.

This course is a COMEM+ web development course taught at HEIG-VD.

• Plan
• What you will need
• Useful links
• Evaluation
  • Delivery
• Project Roadmap
  • Week 1
  • Week 2
  • Week 3
• References

Plan

• Introduction

• Basics

  • HTTP & REST introduction
    • Exercise
  • Extra: JavaScript
    • Exercises (solutions)
    • Extra: promises
    • Extra: prototypes
    • Extra: classes
    • Extra: modules
    • Extra: closures
  • Node.js JavaScript runtime
    • Exercises (solutions)
  • npm Node.js package manager

• Creating a web service

  • Express web framework
    • Exercises (solutions)
  • MongoDB document-oriented database
    • Guide: install MongoDB
  • Mongoose Object-Document Mapper
    • Guide: store geospatial data with Mongoose

• Deploying your web service

  • Guide: deploy an Express.js application with Render & MongoDB Atlas

• Creating a REST API

  • REST in depth
    • Exercises
  • Express best practices
  • Utilizing Mongoose in Express (filtering, pagination, aggregation)
  • Express Authentication

• Documenting and testing a REST API

  • Guide: document a REST API
  • Automated Testing
    • Guide: set up automated tests for an Express.js REST API

• Extras

  • Guide: the many worlds of asynchronous JavaScript

What you will need

• A Unix CLI (Git Bash is included with Git on Windows)
• Git
• A free GitHub account
• Google Chrome (recommended, any browser with developer tools will do)
• Node.js 13.2+
• Postman (recommended, any tool that makes raw HTTP requests will do)
• MongoDB
• A free Render account

Useful links

Documentation

• Architecture & source code management diagrams
• Demonstration REST API implemented with Express (OpenAPI documentation, apiDoc documentation)
• Project suggestions

Other

• Command line cheatsheet
• Git cheatsheet

Evaluation

REST API

Your REST API must be developed with the Express framework and use a MongoDB database. It must provide (at least):

• The API must provide user management:
  • New users must be able to register.
  • Existing users must be able to authenticate (to allow users to log in).
• The API must provide at least 2 other types of resources:
  • Both types must be linked together (e.g. aggregation or composition).
  • At least one of the types must be linked to users.
  • The API must provide minimal CRUD operations to manage and use those types in a mobile application.
• The API must use the knowledge learned during the course:
  • At least one resource must be a paginated list.
  • At least one resource must be a list with optional filters.
  • At least one resource must provide aggregated data from other resources using a MongoDB aggregation pipeline (e.g. the number of items created by a user).
  • The API must be developed as a backend to a mobile application using at least 2 mobile hardware features, for example:

    • At least one resource must be geolocated.

      See Store geospatial data with Mongoose for information on how to store this data.

    • At least one resource must have one or multiple pictures.

      It is sufficient to store a picture URL or URLs in the database.

  • Sensitive operations must be protected by requiring valid authentication and performing authorization:

    • Authentication must be provided in the form of a JWT token or an equivalent mechanism.

    • You must define who is authorized to perform which operations on which resources, and enforce these permissions. These restrictions must make sense in the context of your domain model. For example, a user may not be authorized to update a resource created by another user, even when authenticated.

      There must be at least one operation in the API which limits the permissions of authenticated users.

Infrastructure

• The source code of your REST API must be in a repository on GitHub.
• Your REST API must be deployed on Render.

Documentation

• Your REST API must be documented.

  By reading the documentation, a user must know in advance (before testing the API) which requests can be made, what can be sent in each request (URL path parameters, headers and/or body, and their validation constraints, if any), and what responses can be expected from the API (status code, headers and/or body).

  You may but do not have to document the 500 Internal Server Error response, which is considered to always be a possible response to any request.

Automated testing

• You must implement automated tests to test your REST API:

  • You must write tests for at least 4 separate REST operations in your API (for example: create thing, update thing, list things, delete thing). One test for each operation is enough.

    A test for a GET request that retrieves an empty list does not count in the minimal number of tests you must write.

  • Your tests must be reproducible (running npm test several times in a row should always produce the same result).

Quality of the implementation

• You must follow REST best practices:
  • Your REST resources must use appropriate HTTP methods, headers and status codes.
  • Your REST resources must have an appropriate and consistent URL hierarchy and/or naming structure.
• Your asynchronous code must be correct.
• Your Express routes must handle asynchronous operation errors.
• You must avoid excessive code duplication (e.g. using Express middleware).
• Your API must have basic validations on user input (e.g. using Mongoose validations).
• Your API must validate the existence of linked resources (e.g. when creating an item linked to a user).

Bonus

Doing more than is required MAY earn you some bonus points in the evaluation if implemented correctly. Here are some examples:

• Implement a level 3 hypermedia API using a standard format such as JSON:API or HAL+JSON.

• Implement "full" (80-100%) test coverage with automated tests.

  Note that test coverage alone is useless. The tests must also make meaningful assertions.

• Implement role-based authorization, i.e. users may have different roles with fewer or more permissions. For example, an admin user may be authorized to update resources that do not belong to them, whereas a regular user may not.

Delivery

Send an e-mail no later than December 1st 2023 at 23:59:59.999 CET to Simon Oulevay with:

• The list of group members.
• The link to your source code repository on GitHub.
• The link to your deployed REST API on Render.

Project Roadmap

This course is structured in such a way that it will feel very heavy in new theory during the first month. Indeed, there are plenty of concepts which you will need to grasp before fully commiting to the term project. Nevertheless, this is first and foremost a practical course. Our goal is for you to be able to apply the concepts which we will discuss in class. In order to avoid losing sight of this fact during these first few weeks of the semester, we suggest you get going immediately with the project.

Week 1

Get in a team!

The term project will be built in groups of 3 or 4, so try and get together as early as possible. We also suggest you mix it up: try working with people you've never worked with before.

Communicate!

Team formation is your responsibility. Be friendly and respectful during the process. If you feel dissapointed, talk it out with your partners. The best way to resolve conflicts is to avoid them in the first place. A good place to start is to set ground rules. Make sure every team member feels heard by – for example – setting a list of each and everyone's needs:

• Anna: I need to be able to talk. Please do not interrupt me.
• Bijan: I feel insecure about my code. Please be extra gentle when you are reviewing it.
• Charlene: I need to use sarcasm, it's my way of communicating. Please do your best to not feel offended, it's nothing personal.

Brainstorm

By the end of the first class, you should have a rough idea of what an API is and what we are looking for in your projects. On this basis, start having discussions about what could be interesting, useful or fun. Nothing is set in stone at this point, but having a few ideas in the back of your mind will help you add context to the lectures.

Week 2

JavaScript reviews

Make sure you do the assigned exercises. Help each other out so that everyone in your group has these basics covered. Everyone must code!

Consume APIs

To get a better grasp of how APIs work and why they are useful, it's a good idea to try to consume them. There are many open APIs, so explore the Public APIs List and try getting data back from a couple different sources. To do so, you can already get going with Postman, an API testing tool we will be discussing later in the semester. They have an excellent set of video tutorials. You could also use the Fetch API which you might have seen in ProgWeb.

The list is huge, so you might stumble onto something that is broken or poorly designed. If you can't make it work, try again. If that still doesn't work, move on.

Week 3

Decision Time

Now is a good time to really decide what the team's project will be. Remember, the API you design in this course will almost certainly be the foundation for your front-end project in DevMobil, so every team member should feel some ownership for the idea.

Get Specific

A good way to transition from ideation to development is to write down an intention document for the project. This document will serve as a foundation for your work and should address the following:

• General questions
  • What are we building, in one sentence?
  • Why are we building it?
  • Who will use it?
  • What is it called?
• Specific questions
  • How will we build it?
  • How will our team operate? Are there specific roles?
  • What are our different ressources?
  • What endpoints are we going to use?

Divide and conquer

Even if you can't anticipate everything that you will need to complete for the project to work, try dividing the work into manageable tasks.

For example, a task like Implement authentication could be divided in the following way:

• Design the endpoints
• Add password hashing to the user model
• Implement token-generating endpoint
• Implement token-based authorization middleware
• Write tests
• Refactor

You can be as granular as you want in the way you divide and define tasks. You can then assign tasks to people in your team.

Weeks 4 to 10 will be almost wholly dedicated to the developement of your API.

References

These are the main references used throughout this course. More detailed and additional links to various online articles and documentation can be found at the end of each subject.

• Mozilla Developer Network
  • JavaScript
    • A Re-Introduction to JavaScript
    • Inheritance and the Prototype Chain
    • Introducing JavaScript Objects
• Learn Node
• npm CLI Documentation
• Architectural Styles and the Design of Network-based Software Architectures - Roy Thomas Fielding
• Internet Engineering Task Force
  • RFC 2616 - Hypertext Transfer Protocol - HTTP/1.1
  • RFC 5789 - PATCH Method for HTTP
  • RFC 8288 - Web Linking
• HTTP Status Codes
• Express Documentation
• Functional Design Patterns for Express.js - Jonathan Lee Martin
• Standards.REST
• Richardson Maturity Model - Leonard Richardson, Martin Fowler
• MongoDB Documentation
• Mongoose Documentation
• Auth0
  • Introduction to JSON Web Tokens
  • Token Based Authentication Made Easy
• Render Documentation