Skip to content

Latest commit

 

History

History
231 lines (152 loc) · 24.9 KB

SDPR-StepwiseServiceDesign.md

File metadata and controls

231 lines (152 loc) · 24.9 KB
Raw
Scope Phases Roles Input Output Abstraction/Refinement Level
Presentation and business logic layer
Analysis, design
API product owner, integration architects, API designers
(Non-)functional requirements, domain model/layer design
Service contract (with technology mapping and protocol binding), SLA
Both conceptual (platform-independent) and technical (platform-specific)

Activity/Technique: Stepwise Service Design

also known as: Contract-First, Incremental/Top-Down Service Identification, Iterative API Design and Refinement, Evolutionary Integration Architecting

Context

The Microservice API Patterns (MAP) website motivates the need for Application Programming Interface (API) and (micro-)service design as this:

"While much has been said about microservices in general and about supporting infrastructure architectures, the actual service design has received less attention:

  • How many services should be exposed? What is an adequate size for them?
  • How to ensure that services are loosely coupled? How much data should they exchange, and how often does this happen?
  • What are the most suitable message representations? How to agree on the meaning of each message?"

API and service design have high architectural significance, but also have to be implemented, obviously. Hence, software architects and developers collaborate on this activity; API owners initiate and oversee this work.

Goal and Purpose (When to Use and When not to Use)

As an architect and/or product owner, I want to design loosely coupled services that expose business capabilities through well-defined, quality-driven APIs so that clients can satisfy their information needs and initiate server-side processing efficiently.

This activity has the objective to answer the questions raised under 'Context' above. It delivers:

  1. Platform-independent interface specifications, including API Description a.k. a. service contract and Service Level Agreement (SLA).
  2. At least one serialization technology mapping and communication protocol binding for this design (for instance, JSON schemas and HTTP resource contracts).

This activity includes Domain-Driven Design (DDD). It is commonly used when Backend Integrations are realized. It can also be applied in Frontend Integration; in that case, User Interface Mocking is an alternative and complementary activity.

Instructions (Synopsis, Definition)

There is no single path to APIs and service endpoints of quality and style. When "surfing" the Web searching for advice regarding API design and (micro-)service size (or asking the elders), one "rides" at least seven "waves" of analysis and design work (called steps from now on):

Service Design Workflow: Seven Steps from Analysis to Design, Realization, Evolution

  1. Understand the business problem as well as stakeholder wants and needs, including desired system qualities.

    • Before anything can be designed, we ought to know: what should be delivered (on the project, by the software), and why? Which quality properties are desired?
    • Arnaud Lauret suggests the notion of API goals, driven by end user wants and frontend application needs, in "The Design of Web APIs" (@Lauret:2019).

  2. Model the business domain and group related capabilities, for instance by applying Tactic DDD and Strategic DDD (@Vernon:2013).

    • Event storming is a workshop technique supporting this step. Many alternative forms of business-driven forward engineering exist and can be blended in. Two examples are event-driven process chains and use case scenario walkthroughs.
    • If "buy" or "rent" is an option (rather than "build" from scratch only), also reverse engineer the interfaces and domain models of the existing systems to be bought or rented and integrated and model them on the same level of detail as those representing the results from forward engineering.
    • No matter which technique or template you use to elicit the related Non-Functional Requirements (NFRs), do so in a SMART, value- and risk-driven way (@Fairbanks:2010). Quality storming is one option.

  3. Split applications into frontends and backends, again applying Strategic DDD.

    • Use patterns for distributed computing while doing so (@Buschmann:2007, @RenzelKeller:1997).
    • Apply recognized system decomposition techniques, considering coupling criteria, during this step. The method promoted by Service Cutter, for instance, is based on a catalog of such criteria. Context Mapper integrated this approach, see "Context Map Suggestions with Service Cutter". The "integrators" and "disintegrators" from "Software Architecture: The Hard Parts" (@Ford:2021) provide related advice.
    • While designing, capture the architectural decisions made and model the resulting architecture.

  4. Create a Candidate Endpoint List that identifies potential API endpoints and their roles.

    • For each candidate endpoint, foresee a Remote Facade that exposes Data Transfer Objects (DTOs) in the request and response messages of its API operations to decouple the (published) languages of frontends and backends and to optimize the message exchange over the network w.r.t exchange frequency and message size.
    • Consider adding a Service Layer to further encapsulate the business logic, to control transactions and to coordinate responses (@Fowler:2002). Note that the term service is overloaded; it can refer to remote API services (and this is what this activity is all about), but also to supporting local application, domain, and infrastructure services (see description of Tactic DDD).
    • Capture your architectural decisions about these patterns as well as resulting additional decisions.

  5. Specify operation responsibilities and data formats to yield a Refined Endpoint List and map the endpoints to existing or new API providers.

    • If needed, decompose monolithic backends into (micro-)services (@Newman:2015) to promote flexibility and scalability if these are desired qualities and your software engineering (and operations) toolbox is rich and mature enough.
    • Criteria-based decomposition approaches such as those listed under Step 3 can be used. Legacy modernization approaches such as Strangler Fig described by Martin Fowler and techniques from "Working Effectively with Legacy Code" by Michael Feathers can be applied as well.
    • Refactor (@Zimmermann:2017) the preliminary architecture from the previous steps along the way (including the remote facades and DTOs). Document and justify these conceptual and technology-related architectural decisions and add the resulting architectural decision records to the decision log from Steps 3 and 4.

  6. Once the refined endpoint list is somewhat stable, decide for integration technologies (protocols such as plain HTTP, GraphQL, or gRPC; message exchange formats such as JSON and XML) and implement stubs (or a minimum viable API product).

    • Integrate and test these stubs; iterate and revise the list as needed. If the results are good enough, go ahead and specify service contracts including protocol bindings and technology mappings in an API Description also known as service contract.
    • Optionally, establish Service Level Agreements and Rate Plans.
    • Also decide on (micro-)service deployment technologies and infrastructure middleware such as API gateways, load balancers, and container orchestration engines as well as cloud offerings (@Fehling:2014), again capturing your architectural decisions in the log.

  7. Improve and evolve the API design and its implementation, for instance adjust endpoint and operation numbers as well as request and response message structures to meet the desired runtime qualities (for instance, performance and scalability).

Note: The seven-step sequence does not suggest sequential or one-time execution ("Big Design Upfront, BDUF"). Whenever you learn something new in later steps, you can return to previous ones; whenever you believe you need to jump ahead to learn more about the technologies, existing systems, and so on, you can and should do so.

In DPR, the seven top-level steps and activities are supported by one or more artifact templates:

  1. Functional requirements are often communicated and specified as User Stories and/or Use Cases.
  2. Domain Models on different levels of elaboration and refinement capture the results of Object-Oriented Analysis and Design (OOAD) in general and Tactic DDD in particular; Context Maps result from Strategic DDD.
  3. Context Maps can also record the results of systematic system decomposition work. Many templates exist for Architectural Decision Capturing, which has emerged from an unwelcome but important side activity to a full-fledged practice. DPR includes Y-Statements that can be captured in Markdown Architectural Decision Records or directly in the code. Many options for visualizing/diagramming architectures exist; DPR lists a few practices, notations, and tools in the Architecture Modeling activity. Enterprise Architecture Management and frameworks such as TOGAF are out of scope of DPR at present.
  4. The Candidate Endpoint List (CEL) balances flexibility and expressivity. It translates user stories/use cases and domain model elements into API requirements.
  5. The Refined Endpoint List (REL) then is more precise and assertive. It can specify endpoint roles and operation responsibilities by referencing patterns, and also identify data formats and media types as well as provider implementation candidates and related decisions.
  6. Step 6 from above should be "business as usual" for agile full stack developers and integration specialists for the most part, yielding an expressive, understandable API Description. Both the abstract "port" level as well as technology-specific "adapter" bindings should be covered in it; both business and technical information has to be published in it.
  7. Patterns from two categories in Microservice API Patterns (MAP) are eligible here, with the Quality Category and the Evolution Category being particularly relevant.

Note: It cannot be emphasized enough that all these artifacts can be drafted, revised, and completed iteratively and incrementally. And they should only be created if they serve a purpose in the given project context.

Example(s)

DPR Tutorial 1 applies the seven steps to an online shop example. MAP Tutorial 2 provides an additional application example.

In the end-to-end demo for tool-supported API design and service identification "Domain-Driven Service Design with Context Mapper and MDSL", the seven steps are applied, and partially automated with the help of Context Mapper and MDSL tools such as an OpenAPI generator:

Service Design Example (BPMN)

Finally, the microservices in the sample application Lakeside Mutual contain several Remote Facades implemented as HTTP resources and DTOs that are serialized into JSON.

Benefits vs. Effort (Expected Benefits, Skill Levels)

The more clients an API has and the longer it runs and the more mission-critical it is, the more it pays off to invest in contract-first API and service design.

Hints and Pitfalls to Avoid

Let us repeat and emphasize again:

  • Avoid — or spot and overcome — analysis paralysis. Acknowledge the general rules of method engineering, including:
  1. If in doubt, leave it out.
  2. Do not follow templates blindly, but adopt them to your needs.
  3. Context matters.
  • Do not follow the steps sequentially but iterate, refactor, evolve stepwise. Iterate, refactor, evolve again. Iterate, refactor, evolve continuously. The logical ordering of the seven steps by no means implies waterfall or BDUF.

You might want to apply patterns to optimize API qualities when evolving it:

  • Never forget the business vision, stakeholder concerns, and project goals while deciding, designing, coding, debugging, testing, documenting, and so on.
  • Do not over-fetch and do not under-fetch; adding Pagination or Wish Lists to an API operation signature slices the response payload and allows clients to specify what data they need (GraphQL has this design goal).
  • Respect technology- and platform-specific design guidelines, for instance those in the "RESTful Web Services Cookbook" (@Allamaraju:2010).

Origins and Signs of Use

If OpenAPI specifications are provided, either contract-first or code-first API design has been practiced. Usage of DDD patterns such as Published Language also may indicate use.

The DDD DSL tool Context Mapper supports some of the steps with its model transformations, architectural refactorings, and service contract generation.

Related Content

Performing Roles and Related Artifacts (Synopsis)

Roles:

Produced artifacts:

Practices and Techniques (Refinements, Guides)

Code first. Sometimes, a bottom-up approach exposing already existing solution-internal APIs is preferred, in particular when only a few straightforward API calls are required: standardized or framework-specific annotations (or other forms of configuration) call our services, operations, and parameters (and map them to JSON and Web server settings). Such code-first approach is supported well, for instance in Web Frameworks; it runs the risk of not meeting API client requirements and violating API design best practices — unless a dedicated Service Layer and/or Remote Facades are included in the architecture to decouple application and domain logic from integrations and interfaces.

Bottom-up code-first API design can be combined with this top-down contract-first design activity to yield a meet-in-the-middle approach (note that code-first runs the risk of exposing provider-side implementation details in the API contract, which violates the information hiding principle).

More Information

While written with the Web and RESTful HTTP in mind, many of the existing informal "methods" (or design heuristics) can also be applied when other technologies are chosen:

Tutorial 1 in MAP explains how patterns can mitigate quality issues. Tutorial 2 includes more patterns and pattern categories.

A presentation by James Higginbotham talks about messaging and streaming in the context of API design and REST.

The SOAD project 2006 to 2009 compiled a number of architectural decisions that are required when designing service-oriented architectures. Being independent of application genre and architectural style, the meta issues in Table 2 from the SOAD paper "Architectural Decision Identification in Architectural Patterns" can guide the decision making in Steps 3 to 7:

Meta Issues (Decisions Required) in Service and API Design

Note that in microservices architectures, more options for these decisions (in transition from meta issues to actual decisions required) are available, and the decision drivers may vary too. See this blog post and this article for more information on microservices as an implementation approach to service-oriented architectures.

Data Provenance

title: "Design Practice Repository (DPR): Stepwise Service Design"
author: Olaf Zimmermann (ZIO)
date: "01, 16, 2023"
copyright: Olaf Zimmermann, 2020-2023 (unless noted otherwise). All rights reserved.
license: Creative Commons Attribution 4.0 International License