-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(api): add slug parser in the api requests #6705
Merged
djabarovgeorge
merged 23 commits into
next
from
nv-4494-add-slug-parser-in-the-api-requests
Oct 23, 2024
Merged
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit
Hold shift + click to select a range
bd88fa6
feat(api): allow passing workflow identifier as id in new v2 workflow…
djabarovgeorge c0fa7d5
feat(api): add slug api parser
djabarovgeorge a18552c
feat(api): refactor parsing
djabarovgeorge a3236b5
feat(api): update submodule hash
djabarovgeorge cc61194
Merge remote-tracking branch 'origin/next' into nv-4494-add-slug-pars…
djabarovgeorge 46151b4
Merge branch 'next' into nv-4494-add-slug-parser-in-the-api-requests
djabarovgeorge 7a321ca
fix(api): lock file
djabarovgeorge 347b7fc
fix(api): lock file
djabarovgeorge 541510d
Merge branch 'next' into nv-4494-add-slug-parser-in-the-api-requests
djabarovgeorge 6fd09a6
Merge branch 'next' into nv-4494-add-slug-parser-in-the-api-requests
djabarovgeorge 89606cc
refactor(api): update after pr comments
djabarovgeorge 1ac74bd
fix(api): revert test script
djabarovgeorge 2059789
Merge branch 'next' into nv-4494-add-slug-parser-in-the-api-requests
djabarovgeorge aa9ed71
feat(api): encode workflow and step ids on api response (#6725)
djabarovgeorge f74d73c
Merge remote-tracking branch 'origin/next' into nv-4494-add-slug-pars…
djabarovgeorge 2a64335
fix(api): after next merge
djabarovgeorge 06cbea7
fix(api): after next merge
djabarovgeorge 61c1a5d
fix(api): after next merge
djabarovgeorge 2a15a3b
fix(api): after next merge
djabarovgeorge cf9526b
fix(api): fix id
tatarco bc632b6
Merge remote-tracking branch 'origin/next' into nv-4494-add-slug-pars…
djabarovgeorge d5b30f6
Merge remote-tracking branch 'origin/nv-4494-add-slug-parser-in-the-a…
djabarovgeorge 5d27235
feat: revert hash to next
djabarovgeorge File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Submodule .source
updated
from 604b3e to a32427
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
import basex from 'base-x'; | ||
|
||
const BASE62 = '0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz'; | ||
const { encode, decode } = basex(BASE62); | ||
|
||
export function encodeBase62(value: string): string { | ||
const buffer = Buffer.from(value, 'hex'); | ||
|
||
return encode(buffer); | ||
} | ||
|
||
export function decodeBase62(encoded: string): string { | ||
const uint8Array = decode(encoded); | ||
|
||
return Buffer.from(uint8Array).toString('hex'); | ||
} | ||
|
||
export function isBase62(input: string): boolean { | ||
const base62Regex = /^[0-9A-Za-z]+$/; | ||
|
||
return base62Regex.test(input); | ||
} | ||
djabarovgeorge marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,2 @@ | ||
export * from './utils'; | ||
export * from './base62'; |
65 changes: 65 additions & 0 deletions
65
apps/api/src/app/workflows-v2/pipes/parse-slug-Id.pipe spec.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
import { expect } from 'chai'; | ||
import { ArgumentMetadata } from '@nestjs/common'; | ||
|
||
import { ParseSlugIdPipe } from './parse-slug-Id.pipe'; | ||
import { encodeBase62 } from '../../shared/helpers'; | ||
|
||
describe('ParseSlugIdPipe', () => { | ||
let pipe: ParseSlugIdPipe; | ||
|
||
beforeEach(() => { | ||
pipe = new ParseSlugIdPipe(); | ||
}); | ||
|
||
it('should return the original value for non-slug IDs', () => { | ||
const workflowIdentifier = 'non-slug-id'; | ||
expect(pipe.transform(workflowIdentifier, {} as ArgumentMetadata)).to.equal(workflowIdentifier); | ||
|
||
const internalId = '6615943e7ace93b0540ae377'; | ||
expect(pipe.transform(internalId, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
|
||
it('should handle invalid encoded IDs', () => { | ||
const invalidSlugId = 'my-workflow_invalidEncoding'; | ||
expect(pipe.transform(invalidSlugId, {} as ArgumentMetadata)).to.equal(invalidSlugId); | ||
}); | ||
|
||
it('should not trim or decode internalId', () => { | ||
const internalId = '6615943e7ace93b0540ae377'; | ||
expect(pipe.transform(internalId, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
|
||
it.skip('should trim prefix and decode base62 encoded internalId', () => { | ||
const internalId = '6615943e7ace93b0540ae377'; | ||
const encodedId = encodeBase62(`wf_${internalId}`); | ||
expect(pipe.transform(`wf_${encodedId}`, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
|
||
it('should not trim or decode simple workflow identifier', () => { | ||
const identifier = 'my-workflow'; | ||
expect(pipe.transform(identifier, {} as ArgumentMetadata)).to.equal(identifier); | ||
}); | ||
|
||
it.skip('should trim, decode, and remove prefix for a valid slug ID', () => { | ||
const internalId = '6615943e7ace93b0540ae377'; | ||
const encodedId = encodeBase62(`wf_${internalId}`); | ||
expect(pipe.transform(`my-workflow_${encodedId}`, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
|
||
it('should return original value for invalid encoded ID', () => { | ||
const invalidSlug = 'my-workflow_invalid'; | ||
expect(pipe.transform(invalidSlug, {} as ArgumentMetadata)).to.equal(invalidSlug); | ||
}); | ||
|
||
it('should handle slug IDs without known prefixes', () => { | ||
const internalId = '6615943e7ace93b0540ae377'; | ||
const encodedId = encodeBase62(internalId); | ||
expect(pipe.transform(`my-workflow_${encodedId}`, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
|
||
it.skip('should handle slug IDs with multiple underscores', () => { | ||
const internalId = '6615943e7ace93b0540ae377'; | ||
const encodedId = encodeBase62(`wf_${internalId}`); | ||
expect(pipe.transform(`my_complex_workflow_${encodedId}`, {} as ArgumentMetadata)).to.equal(internalId); | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
import { ArgumentMetadata, Injectable, PipeTransform } from '@nestjs/common'; | ||
import { BaseRepository } from '@novu/dal'; | ||
import { decodeBase62 } from '../../shared/helpers'; | ||
|
||
type InternalId = string; | ||
const INTERNAL_ID_LENGTH = 24; | ||
const ENCODED_ID_LENGTH = 16; | ||
|
||
function isWorkflowId(value: string) { | ||
return value.length < ENCODED_ID_LENGTH; | ||
} | ||
|
||
function isInternalId(value: string) { | ||
return BaseRepository.isInternalId(value) && value.length === INTERNAL_ID_LENGTH; | ||
} | ||
|
||
function lookoutForId(value: string): string | null { | ||
if (isInternalId(value)) { | ||
return value; | ||
} | ||
|
||
if (isWorkflowId(value)) { | ||
return value; | ||
} | ||
|
||
return null; | ||
} | ||
|
||
export function parseSlugId(value: string): InternalId { | ||
const validId = lookoutForId(value); | ||
if (validId) { | ||
return validId; | ||
} | ||
|
||
const encodedValue = value.slice(-16); | ||
let decodedValue: string; | ||
try { | ||
decodedValue = decodeBase62(encodedValue); | ||
} catch (error) { | ||
return value; | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. if not base 62 |
||
} | ||
const validDecodedId = lookoutForId(decodedValue); | ||
if (validDecodedId) { | ||
return validDecodedId; | ||
} | ||
|
||
return value; | ||
} | ||
|
||
@Injectable() | ||
export class ParseSlugIdPipe implements PipeTransform<string, InternalId> { | ||
transform(value: string, metadata: ArgumentMetadata): InternalId { | ||
return parseSlugId(value); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
8 changes: 8 additions & 0 deletions
8
apps/api/src/app/workflows-v2/usecases/get-workflow-by-ids/get-workflow-by-ids.command.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
import { EnvironmentWithUserObjectCommand } from '@novu/application-generic'; | ||
import { IsDefined, IsString } from 'class-validator'; | ||
|
||
export class GetWorkflowByIdsCommand extends EnvironmentWithUserObjectCommand { | ||
@IsString() | ||
@IsDefined() | ||
workflowIdOrIdentifier: string; | ||
} |
34 changes: 34 additions & 0 deletions
34
apps/api/src/app/workflows-v2/usecases/get-workflow-by-ids/get-workflow-by-ids.usecase.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
import { Injectable } from '@nestjs/common'; | ||
|
||
import { NotificationTemplateEntity, NotificationTemplateRepository } from '@novu/dal'; | ||
|
||
import { GetWorkflowByIdsCommand } from './get-workflow-by-ids.command'; | ||
import { WorkflowNotFoundException } from '../../exceptions/workflow-not-found-exception'; | ||
|
||
@Injectable() | ||
export class GetWorkflowByIdsUseCase { | ||
constructor(private notificationTemplateRepository: NotificationTemplateRepository) {} | ||
async execute(command: GetWorkflowByIdsCommand): Promise<NotificationTemplateEntity> { | ||
const isInternalId = NotificationTemplateRepository.isInternalId(command.workflowIdOrIdentifier); | ||
|
||
let workflowEntity: NotificationTemplateEntity | null; | ||
|
||
if (isInternalId) { | ||
workflowEntity = await this.notificationTemplateRepository.findById( | ||
command.workflowIdOrIdentifier, | ||
command.user.environmentId | ||
); | ||
} else { | ||
workflowEntity = await this.notificationTemplateRepository.findByTriggerIdentifier( | ||
command.user.environmentId, | ||
command.workflowIdOrIdentifier | ||
); | ||
} | ||
|
||
if (!workflowEntity) { | ||
throw new WorkflowNotFoundException(command.workflowIdOrIdentifier); | ||
} | ||
|
||
return workflowEntity; | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠ issue: Can we write our own base62 encoder from scratch? The pipe logic is effectively using unvalidated data. I'm concerned that any vulnerabilities (intentionally or maliciously) added to the
base-x
package could expose us to an attack vectorThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you think if will have fixed version 5.0.0? Will it solve the concern for us? Then, we could upgrade the version if needed and only to a stable and secure version.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a know security issue with the package?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The truth is that a custom implementation with two functions, encodeBase62, decodeBase62 would be much much simpler, no need for pipes, no need to worry about potential issues.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i had a hard time implementing the code from scratch, i added up comping the code from base-x so we won't get any vulnerabilities by mistake.
not at the moment.
i implemented encodeBase62, decodeBase62 based on base-x, pipes are actually a nice touch of nest js making the boundaries of controller and use case responsibility cleaner