Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UBI docs consolidation #1

Merged
merged 25 commits into from
May 31, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
d100735
gathering potential documentation attempts
RasonJ May 15, 2024
0596e47
considering the dashboard tutorial
RasonJ May 16, 2024
555d72b
place holder for js data structure usage
RasonJ May 16, 2024
a89ae40
data-structures placeholder
RasonJ May 16, 2024
19d1516
Updating index links
RasonJ May 20, 2024
c1085c1
adding old doc to be merged
RasonJ May 20, 2024
cf9e128
Starting to link things together
RasonJ May 22, 2024
e26b8e5
fix broken link
RasonJ May 22, 2024
92d84b1
respond to vale
epugh May 22, 2024
84f9395
more vale violations
epugh May 22, 2024
e390422
name files consistently with docs site and fix links.
epugh May 22, 2024
46cc0d9
vale
epugh May 22, 2024
4e2a617
Minor tweaks. Moved Ubi under SEARCH.
RasonJ May 22, 2024
c251237
add label for versining of spec and OS version
epugh May 23, 2024
2a39c1d
try to sort out vale error
epugh May 23, 2024
982e334
Converting mermaid diagrams to png's
RasonJ May 28, 2024
9402dbd
Updating query_id mermaid code
RasonJ May 28, 2024
db2546e
Better way to ignore the mermaid scripts in the md files
RasonJ May 28, 2024
cec25c9
description updates
RasonJ May 29, 2024
1f0fa78
schema.md updating (still need to update the mermaid diagram)
RasonJ May 30, 2024
ad1b92a
schema updates
RasonJ May 30, 2024
6f08485
chase some more vale warnings.
epugh May 30, 2024
c9edc86
updates
RasonJ May 31, 2024
6626bb1
Merge branch 'ubi-docs-consolidation' of https://github.com/o19s/docu…
RasonJ May 31, 2024
736bf69
Getting there. TODO's indicate a few loose ends to tie
RasonJ May 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/vale/styles/Vocab/OpenSearch/Products/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ Simple Schema for Observability
Tableau
TorchScript
Tribuo
User Behavior Insights
VisBuilder
Winlogbeat
Zstandard
2 changes: 2 additions & 0 deletions _search-plugins/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ OpenSearch provides the following search relevance features:

- [Querqy]({{site.url}}{{site.baseurl}}/search-plugins/querqy/): Offers query rewriting capability.

- [User Behavior Insights]({{site.url}}{{site.baseurl}}/search-plugins/ubi/): Link user application behavior to their queries for improving search relevance insights.

## Search results

OpenSearch supports the following commonly used operations on search results:
Expand Down
93 changes: 93 additions & 0 deletions _search-plugins/ubi/client_datastructures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
layout: default
title: Client Data structures old
parent: User behavior insights
has_children: false
nav_order: 7
---

The datastructures that we use for indexing events adhere to the following nested structure that aligns with the Ubi schemas. See the [schemas](.././schemas.md) for descriptions and examples of the following fields.

Check failure on line 9 in _search-plugins/ubi/client_datastructures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/client_datastructures.md#L9

[OpenSearch.Spelling] Error: datastructures. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output
{"message": "[OpenSearch.Spelling] Error: datastructures. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_search-plugins/ubi/client_datastructures.md", "range": {"start": {"line": 9, "column": 5}}}, "severity": "ERROR"}

Check failure on line 9 in _search-plugins/ubi/client_datastructures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/client_datastructures.md#L9

[OpenSearch.Spelling] Error: Ubi. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output
{"message": "[OpenSearch.Spelling] Error: Ubi. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_search-plugins/ubi/client_datastructures.md", "range": {"start": {"line": 9, "column": 114}}}, "severity": "ERROR"}

Check failure on line 9 in _search-plugins/ubi/client_datastructures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/client_datastructures.md#L9

[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'schemas. See'.
Raw output
{"message": "[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'schemas. See'.", "location": {"path": "_search-plugins/ubi/client_datastructures.md", "range": {"start": {"line": 9, "column": 118}}}, "severity": "ERROR"}
epugh marked this conversation as resolved.
Show resolved Hide resolved

`struct UbiEvent {`
- application
- action_name
- user_id
- query_id
- session_id
- page_id
- message_type
- timestamp
- <details>
<summary>event_attributes {</summary>
<p>

- <details>
<summary>position {</summary>

- ordinal
- x
- y
- trail
}
</details>
- <details>
<summary>object {</summary>

- internal_id
- object_id
- object_type
- description
- object_details /
- object_details.json
}
</details>
}
</details>}
`}`

Typescript versions of these classes can be found in [ts/UbiEvent.ts](./ts/UbiEvent.ts).

Example javascript code:

Check failure on line 50 in _search-plugins/ubi/client_datastructures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/client_datastructures.md#L50

[Vale.Terms] Use 'JavaScript' instead of 'javascript'.
Raw output
{"message": "[Vale.Terms] Use 'JavaScript' instead of 'javascript'.", "location": {"path": "_search-plugins/ubi/client_datastructures.md", "range": {"start": {"line": 50, "column": 9}}}, "severity": "ERROR"}
```js
// basic message format
let e = new UbiEvent('chorus', 'add_to_cart', user_id, query_id);
e.message_type = 'PURCHASE';
e.message = item.title + ' (' + item.primary_ean + ')';
e.session_id = session_id;
e.page_id = window.location.pathname;

// adding custom data fields
e.event_attributes['user_info'] = {user_name:'TheJackal', phone:'555-555-1234'}

// associate the object added to the cart
e.event_attributes.object = new UbiObject('product', item.primary_ean, item.title, item);

// save any click info
e.event_attributes.position = new UbiPosition({x:event.clientX, y:event.clientY});

// index here
console.log(e.toJson());
```

With very similar [python data structures](./py/ubi.py):

Check failure on line 72 in _search-plugins/ubi/client_datastructures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/client_datastructures.md#L72

[Vale.Terms] Use 'Python' instead of 'python'.
Raw output
{"message": "[Vale.Terms] Use 'Python' instead of 'python'.", "location": {"path": "_search-plugins/ubi/client_datastructures.md", "range": {"start": {"line": 72, "column": 20}}}, "severity": "ERROR"}
```python
if __name__ == '__main__':
e = UbiEvent(application='chorus', action_name='add_to_cart',
user_id='USER-' + guuid(), query_id='QUERY-ID-'+ guuid(), session_id='SESSION-' + guuid(),
# object added to the cart
object= UbiObject(
object_type='product',
object_id='ISBN-' + guuid(),
description='a very nice book', details={'product_data':'random product data'}
)
)
e.message_type = 'PURCHASE'

# adding custom attribute fields
e.event_attributes['user_info'] = {'user_name':'TheJackal', 'phone':'555-555-1234'}

# product position information
e.event_attributes.position = UbiPosition(ordinal=3, x=250, y=400)

print(e.to_json())
```
126 changes: 126 additions & 0 deletions _search-plugins/ubi/data-structures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
layout: default
title: Client data structures
parent: User behavior insights
has_children: false
nav_order: 7
---

# Sample data structures
The data structures below can be used to create events that follow the [UBI event schema](schemas.md).

Check warning on line 10 in _search-plugins/ubi/data-structures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/data-structures.md#L10

[OpenSearch.DirectionAboveBelow] Use 'following or later' instead of 'below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.
Raw output
{"message": "[OpenSearch.DirectionAboveBelow] Use 'following or later' instead of 'below' for versions or orientation within a document. Use 'above' and 'below' only for physical space or screen descriptions.", "location": {"path": "_search-plugins/ubi/data-structures.md", "range": {"start": {"line": 10, "column": 21}}}, "severity": "WARNING"}

The developer just needs to decide on implementations for the following functions:

Check warning on line 12 in _search-plugins/ubi/data-structures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/data-structures.md#L12

[OpenSearch.Simple] Don't use 'just' because it's not neutral in tone. If you mean 'only', use 'only' instead.
Raw output
{"message": "[OpenSearch.Simple] Don't use 'just' because it's not neutral in tone. If you mean 'only', use 'only' instead.", "location": {"path": "_search-plugins/ubi/data-structures.md", "range": {"start": {"line": 12, "column": 15}}}, "severity": "WARNING"}
- `getUserId()`
- `getQueryId()`
- `getSessionId()`
- `getPageId()`- e.g.:
```js
function getPageId(){
return location.pathname;
}
```
Other sample implementations can be found [here](#TODO).

```js
/*********************************************************************************************
* Ubi Event data structures
* The following structures help ensure adherence to the UBI event schema
*********************************************************************************************/

export class UbiEventData {
constructor(type, id=null, description=null, details=null) {
this.object_type = type;
this.object_id = id;
this.description = description;
this.object_detail = details;

//override if using key_field's and values
this.key_value = id;
}
}
export class UbiPosition{
constructor({ordinal=null, x=null, y=null, trail=null}={}) {
this.ordinal = ordinal;
this.x = x;
this.y = y;
this.trail = trail;
}
}


export class UbiEventAttributes {
/**
* Attributes, other than `object` or `position` should be in the form of
* attributes['item1'] = 1
* attributes['item2'] = '2'
*
* The object member is reserved for further, relevant object payloads or classes
*/
constructor({attributes={}, object=null, position=null}={}) {
if(attributes != null){
Object.assign(this, attributes);
}
if(object != null && Object.keys(object).length > 0){
this.object = object;
}
if(position != null && Object.keys(position).length > 0){
this.position = position;
}
}
}



export class UbiEvent {
constructor(action_name, {message=null, event_attributes={}, data_object={}}={}) {
this.action_name = action_name;
this.user_id = getUserId();
this.query_id = getQueryId();
this.session_id = getSessionId();
this.page_id = getPageId();
this.timestamp = Date.now();

this.message_type = 'INFO';
if( message )
this.message = message;

this.event_attributes = new UbiEventAttributes({attributes:event_attributes, object:data_object});
}

/**
* Use to suppress null objects in the json output
* @param key
* @param value
* @returns
*/
static replacer(key, value){
if(value == null ||
(value.constructor == Object && Object.keys(value).length === 0)) {
return undefined;
}
return value;
}

/**
*
* @returns json string
*/
toJson() {
return JSON.stringify(this, UbiEvent.replacer);
}
}
```

# Sample Usage

Check failure on line 114 in _search-plugins/ubi/data-structures.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _search-plugins/ubi/data-structures.md#L114

[OpenSearch.HeadingCapitalization] 'Sample Usage' is a heading and should be in sentence case.
Raw output
{"message": "[OpenSearch.HeadingCapitalization] 'Sample Usage' is a heading and should be in sentence case.", "location": {"path": "_search-plugins/ubi/data-structures.md", "range": {"start": {"line": 114, "column": 3}}}, "severity": "ERROR"}

```js
export async function logDwellTime(action_name, page, seconds){
console.log(`${page} => ${seconds}`);
let e = new UbiEvent(action_name, {
message:`On page ${page} for ${seconds} seconds`,
event_attributes:{dwell_seconds:seconds},
data_object:TimeMe
});
logEvent(e);
}
```
Loading
Loading