JBI Roadmap - Q4 2023

[bsieber-mozilla]

Q4 Roadmap: Enhancing Observability and Customer Satisfaction for Jira Bugzilla Integration (JBI)

Presented by Services Engineering - SysEng Pod
Date: Q4 '23

Table of Contents

1. Executive Summary
2. Introduction
3. Current State Assessment
4. Goal Definition
5. Roadmap Overview
6. Metrics
7. Collaboration and Communication Plan
8. Conclusion and Commitment

Executive Summary

The Q4 roadmap outlines our plan to enhance observability and improve customer satisfaction for JBI. This document provides a comprehensive overview of our goals, milestones, and resource allocation for this initiative.

Introduction

Jira Bugzilla Integration connects Bugzilla to Jira providing users to continue using existing Bugzilla workflows while getting the benefit of Jira's unique views and displays of the data. Jira has been in use by (Product) Managers to track the forecasting of projects, provide real-time charts in confluence documents for executives, and provide a view of the data/tasks in a way that Bugzilla currently doesn't provide.

Observability is important to understand and resolve questions around the health of the system--questions that should be resolved with a few clicks.

Current State Assessment

Currently, the team and users have a tough time (more time than should be invested) to answer the question: "is [this] working [as expected]?" This informs us that JBI has been missing sufficient visibility/transparency into status of: workflows, the services, and the errors. When something is wrong, often customers are the first to point us to the error.

JBI uses a distributed means for observability: grafana alerts, slack alerts, as well as google cloud monitoring. These distributed patterns exist, but are not at a fidelity that enables customers to resolve the question above themselves.

Goal Definition

Enhance Observability: Improve the ability to monitor and gain insights into system performance.

• Key Results:
  • Increase the coverage of monitoring across critical system components.
  • Implement advanced analytics and visualization tools for better insights.

Implement Resilient Architectures: Enhance system resilience to adapt to changing conditions and loads.

• Key Result:
  • Develop self-healing mechanisms to automatically recover from failures.

Enhance Customer Experience: Improve the overall experience for customers or end-users.

• Key Results:
  • Conduct user surveys and feedback analysis to identify pain points and areas for improvement.
  • Implement customer-centric design principles in product development.
  • Implement Net Promoter Score (NPS) or customer satisfaction rating.

Roadmap Overview

P1: Enhanced Observability

• Assess the state of our logging #645
• The lack of explicit connection between webhooks and workflows makes webhook errors hard to diagnose #567
• Consider migrating verification checks from /heartbeat to Telescope? #343
• How to measure coverage of monitoring system components? (baseline) #714

P2: Resilent Architecture

• System is able to "drop" erring messages without user interference #710
• Create config validation workflow #390

P3: Enhance Customer Experience

• Setting up Webhooks with 'ANY' Product #21
• Document requirements for using different step functions #606
• Clearly document that JBI does not sync private bugs #592
• Document how to test configuration in STAGE #581
• Document application configuration options #212
• Determine Customer Satisfaction -- establish recurring NPS #709

Metrics

NPS metrics should become available after Oct 19; then on a monthly basis this data will be tracked to confirm that we are acting on the interests of the users through an improving score. The github issues associated with this roadmap should be moved to "done" by Dec 15th '23.

Collaboration and Communication Plan

#jbi will be the channel for collaboration and communication
github will be the medium for task tracking

Conclusion and Commitment

JBI has been running quietly... "too quietly" for some. The goal and commitment for this roadmap is to increase the transparency (particularly around errors), so the system's health and status can be easily seen and any concerns around operations mitigated.
Without an engineer or SRE, user's should be able to answer (and understand if), "[this] is working [as expected]?"

[grahamalama]

This is awesome @bsieber-mozilla! Thanks for taking the time to write this plan out

[grahamalama]

Maybe we can try using milestones to group these issues?

[bsieber-mozilla]

Maybe we can try using milestones to group these issues?

Made them! Feel free to edit :)