Skip to content

Commit

Permalink
Processed my review into an updated design. Biggest change: not put t…
Browse files Browse the repository at this point in the history
…he standard text into the CSV as it will be a nuisance to edit. Makes sense? If not: revert that change.

Signed-off-by: Rob van der Veer <[email protected]>
  • Loading branch information
robvanderveer authored Jul 15, 2024
1 parent c66568a commit 2c66bbf
Showing 1 changed file with 27 additions and 18 deletions.
45 changes: 27 additions & 18 deletions docs/designs/my-opencre.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,45 +3,54 @@

## Introduction

Users asked for a way to include their own standards into opencre.
So far importing has been done via developers either writing importers or doing mappings manually and then running import jobs to populate a database from data sources.
Users asked for a way to include their own standards into opencre, sometimes with the added need to run OpenCRE at their own premises because of confidentiality.
Why?
This alllows them to use search, browse, refer, map analysis and chat on an integrated and tailored platform - organizing their guidelines, policies, and requirements according to the OpenCRE catalog and by doing so, linking everything to the key industry security standards.

These jobs are a pain as they are:
* Cli only
* Require intimate knowledge of cre import format
* Require intimate knowledge of existing cres
So far, importing standards has been done via developers either writing importers or doing mappings manually and then running import jobs to populate a database from data sources.

Currently this requires:
* Command-line interactions only
* Intimate knowledge of the cre import format
* Intimate knowledge of the OpenCRE catalog structure

## The problem

Currently users cannot run opencre locally and have the ability to layer their own data and policies on top of cre. The process is complicated and has a number of non user friendly points described above.
Currently users cannot run opencre locally and even if they could, adding their own standards is currently an OpenCRE expert job: difficult.

## The suggestion

In V3 we allowed everyone to run cre locally while also giving them the ability to download upstream data on launch.
Allow OpenCRE to be run locally by providing a container, streamlining some data initialization processes, and providing documentation. In the begining of July 2024 we accomplished this through the release of OpenCRE V3.

In v4 we intend to create 3 more features:

In v4 we should create 3 more features:
* Mapping template export: Export the OpenCRE catalog as mapping template to contribute mappings between the cre catalog and a standard in CSV format
* Initial automated standard matching: Given a CSV containing a standard with a requirement in each row, use embeddings to match the nearest cre for every requirement in the standard and populate the mapping template for review
* Import standard: Given a populated mapping template and a CSV containing a standard, import the standard and calculate gap analysis
* Provide an easy interface to manage imported standards

V5 will feature:
* Change OpenCRE Catalog: Allow an organization to use the mapping template to make changes in the OpenCRE catalog by completely replacing how existing Common Requirements are related, and by adding or removing Common Requirements.

* Export cre structure as CSV
* Given a CSV describing a standard, use embeddings to suggest the nearest cre
* Given a CSV mapping cre to standard clauses, import the standard and calculate gap analysis
Until Version 5, the mapping template can be used to communicate suggestions to the OpenCRE catalog, that will then be processed by the OpenCRE team, and provided as downloadable for local MyOpenCRE implementations.

## Exporting cre structure as CSV
## Mapping template export

This is useful for creating a templated import spreadsheet.
Given a populated opencre application instance users should be able to download a CSV that contains the structure of all CREs loaded in that instance.
Given a populated opencre application instance, users should be able to download a CSV that contains the structure of all CREs loaded in that instance.
The CSV should follow a format that we can use to import.
The cres should be presented stacked to show the hierarchical format.

## Populate CSV describing standard with CREs
## Initial automated standard matching

This is useful for reducing the time it takes for mappings to be created.
The user-facing aspect of this functionality is a page where users can upload a file and in return after processing, they receive a file with the results.
The user-facing aspect of this functionality is a page where users can upload a file and in return after processing, they receive a mapping template file with the results.

Given a csv containing a resource that follows a specific format described below <export format but no CREs, only standard entries with title, section, section id, hyperlink and potentially text>, for every row of the standard, use the embeddings generation functionality and the similarity calculation functionality to suggest the nearest most appropriate cre entry if there is one. If not, leave empty.
Given a csv containing a resource that follows a specific format described below <export format but no CREs, only standard entries with title, section, section id, hyperlink>, for every row of the standard, use the embeddings generation functionality and the similarity calculation functionality to suggest the nearest most appropriate cre entry if there is one. If not, leave empty.


## Import from CSV

Provide a page that when running in client mode, allows users to drag n drop a CSV that contains cre mappings following the format described above. OpenCRE then imports the mappings, calculates embeddings and gap analysis and notifies the user when it's done or on error.
Provide a page that when running in client mode, allows users to drag and drop a CSV that contains cre mappings following the format described above. OpenCRE then imports the mappings, and gap analysis and notifies the user when it's done or on error.
Optionally users can provide the standard file, so that it can be incorporated in the OpenCRE Chat repository (add text, calculate embedding).
When importing users can select to skip calculation of specific gap analysis or embeddings altogether.

0 comments on commit 2c66bbf

Please sign in to comment.