Use Base58 CID's for CAS Interface #331

BenjaminMoe · 2021-12-22T18:10:24Z

This pull request contains the following changes.

Added CAS tests for ion vectors
Changed Cas module to return ipfs-format base58 CID's
Added force initialization to MockCas to replicate compatibility
Updated code where MockCas interface was not being initialized
Cleaned up base64/58 conversion functions from Encoder

Test run:

Ganache local read/write
Ropsten local read
Ganache multiple nodes read/write
AWS S3 cas module
AWS QLDB ledger module

Transaction Number: 289

Note that the comments marked as hidden on this PR are me taking notes for what needs to be tested and confirmed before marking the PR as ready for review.

packages/cas-s3/src/S3Cas.ts

BenjaminMoe · 2022-01-05T18:37:29Z

Did a test write with did:elem:ropsten:EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A, looks like we are able to resolve it on both existing and updated nodes.

BenjaminMoe · 2022-01-05T18:46:33Z

We might have a problem? It still works, but we get a bunch of errors before the document resolves successfully.

Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11611494. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}
Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11611565. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)\n    at runNextTicks (internal/process/task_queues.js:60:5)\n    at processImmediate (internal/timers.js:437:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}
Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11611994. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}
Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11612000. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}
Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11612061. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}
Skipped bad operation for DID EiCtwD11AV9e1oISQRHnMJsBC3OBdYDmx8xeKeASrKaw6A at time 11612068. Error: {"stack":"Error: create_operation_missing_or_unknown_property\n    at Function.parseObject (webpack-internal:///../core/dist/core.esm.js:2169:13)\n    at _callee$ (webpack-internal:///../core/dist/core.esm.js:2137:49)\n    at tryCatch (webpack-internal:///../core/dist/core.esm.js:216:40)\n    at Generator.invoke [as _invoke] (webpack-internal:///../core/dist/core.esm.js:447:22)\n    at Generator.eval [as next] (webpack-internal:///../core/dist/core.esm.js:272:21)\n    at asyncGeneratorStep (webpack-internal:///../core/dist/core.esm.js:62:24)\n    at _next (webpack-internal:///../core/dist/core.esm.js:84:9)","message":"create_operation_missing_or_unknown_property","code":"create_operation_missing_or_unknown_property"}

And i checked, this seems to be happening with the existing base64 did's as well. So it't not related to the changes I made for thi PR. I'll open a new issue, and then open this PR for review (finally).

Create: #341

OR13 · 2022-01-05T21:27:21Z

packages/cas/src/tests/__fixtures__/index.ts

      '1f8b080000000000000325cdc16e82300080e177e979878103851ba0885388202cb065592ab4a50c282b05bb11de7d18cf7ff2fd13e8381b694f590beb435b20e9d21a259c0213844d70239acf2bb7dc17672c7efcd8cea8c661160ca5a2aadb6acc3c2b4c043f4a3d044f80758843b1503d30279073040502e6c704fa01632ab750c07b28502da007fb7279eca8a39cf6b87c59bfdb69e546e9ee2458fbbae99c2bd173c3be0a2fb588a56ae3f91293e5c151ce46c47f1dd6345434a8150f65e57fa9c5e12db1d9b0d213031f633bd0fd3f435aaa9451a490defb769ed71be37203f3fc39cfff5b850a70fa000000',
      'hex'
    ),
-    json:
+    jsonStr:


I love this name change. much clearer.

OR13 · 2022-01-05T21:27:38Z

packages/cas/src/tests/__fixtures__/index.ts

      '{"provisionalIndexFileUri":"QmNwg5MrjFhGdPftqMTBYi5raYNuh122DjvYHAQUtrKx6Q","operations":{"create":[{"suffixData":{"deltaHash":"EiC1LGfh47ZBXjFRXELtonJ8pCbg6c9BbtHXAgA25vPSTg","recoveryCommitment":"EiC3M_2dIVUBou36U9fKTBN6Mz9xA2xxRR1gsHkC0789Sw"}}]}}',
  },
  recoverOperation: {
    cid: 'QmWA2bmcdTLAVDMjddeFJvw4fdxoNkJSUvUhKJiipPgP7R',
-    data: Buffer.from(
+    content: Buffer.from(


another good name change, well done.

OR13 · 2022-01-05T21:27:59Z

packages/cas/src/tests/__fixtures__/index.ts

+ * 'should pass the writer lock ID to CoreIndexFile.createBuffer() if a value lock exists.'
+ *
+ * By Logging the values passed into the MockCas write function during the test
+ * https://github.com/decentralized-identity/sidetree/blob/master/tests/mocks/MockCas.ts#L34


great comments

OR13 · 2022-01-05T21:28:21Z

packages/cas/src/tests/__fixtures__/index.ts

-export const testBufferHash = 'EiAuYMQ4_ZBO1ckbGpyyDKlcq8EFpG3duG3nkhR79RwFQQ';
-export const testStringHash = 'EiBLkBZiYEmzDAD-CuMapW18cBaQr0un-cG0AjiLfdFhZA';
-export const testObjectHash = 'EiAxghX7H2qN_c_wwt0GCcCVRt0JEhAPw3UoHS9JS-5cRg';
+// Base 58 (Ipfs Content Id)


technically an IPFS v1 CID, it is possible to link to a spec for this i believe, if you wanted to.

BenjaminMoe added 12 commits December 21, 2021 23:14

chore(core): update mock cas and add types

7c18b90

chore(core): add vectors to test suite

d7a7bf2

chore(core): update cas tests with compression

09c2443

chore(core): add hash compat to mock cas and tests

f9631c9

chore(core): update S3 to match mockcas interface

0b28b20

chore(core): update cas-ipfs content id

f15828c

chore(core): rename cid to base58

c08188c

chore(core): initilialize mock cas

49b7695

chore(core): convert anchor string to base64

a37990b

chore(core): fix type for error logging

6561094

chore(core): add initialize to mockcas usage

56b2f86

chore(core): checking for base58 swap condition

6b609ce

This comment has been minimized.

Sign in to view

chore(core): update anchor string to work with cid

78ad8a5

This comment has been minimized.

Sign in to view

chore(element): anchor strings

446f7e4

This comment has been minimized.

Sign in to view

chore(element): remove extra characters from config

2599ebc

This comment has been minimized.

Sign in to view

Merge branch 'main' into fix/cas-tests

4363026

This comment has been minimized.

Sign in to view

BenjaminMoe changed the title ~~Fix/cas tests~~ Use Base58 CID's for CAS (wihtout breaking anything) Jan 3, 2022

BenjaminMoe marked this pull request as ready for review January 3, 2022 17:46

BenjaminMoe requested review from acarnagey, gjgd, ipbyrne and OR13 as code owners January 3, 2022 17:46

BenjaminMoe changed the title ~~Use Base58 CID's for CAS (wihtout breaking anything)~~ Use Base58 CID's for CAS (without breaking anything) Jan 3, 2022

BenjaminMoe changed the title ~~Use Base58 CID's for CAS (without breaking anything)~~ Use Base58 CID's for CAS Interface Jan 3, 2022

OR13 reviewed Jan 3, 2022

View reviewed changes

packages/cas-s3/src/S3Cas.ts Outdated Show resolved Hide resolved

BenjaminMoe marked this pull request as draft January 3, 2022 18:50

chore(core): remove base64 cas compatibility

e84baf1

This comment has been minimized.

Sign in to view

chore(core): remove base64 conversion for cas

ea0b9dd

This comment has been minimized.

Sign in to view

BenjaminMoe added 2 commits January 5, 2022 22:47

chore(core): remove comments

ce852be

chore(core): change base58 function names

0e8dd33

This comment has been minimized.

Sign in to view

chore(core): fix removed function

e1fa253

This comment has been minimized.

Sign in to view

chore(element): update doc example

69e1c5e

BenjaminMoe marked this pull request as ready for review January 5, 2022 18:59

OR13 reviewed Jan 5, 2022

View reviewed changes

OR13 approved these changes Jan 5, 2022

View reviewed changes

BenjaminMoe merged commit 9c23c8f into main Jan 5, 2022

BenjaminMoe deleted the fix/cas-tests branch January 5, 2022 21:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use Base58 CID's for CAS Interface #331

Use Base58 CID's for CAS Interface #331

BenjaminMoe commented Dec 22, 2021 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

BenjaminMoe commented Jan 5, 2022 •

edited

Loading

BenjaminMoe commented Jan 5, 2022 •

edited

Loading

OR13 Jan 5, 2022

OR13 Jan 5, 2022

OR13 Jan 5, 2022

OR13 Jan 5, 2022 •

edited

Loading

Use Base58 CID's for CAS Interface #331

Use Base58 CID's for CAS Interface #331

Conversation

BenjaminMoe commented Dec 22, 2021 • edited Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

BenjaminMoe commented Jan 5, 2022 • edited Loading

BenjaminMoe commented Jan 5, 2022 • edited Loading

OR13 Jan 5, 2022

Choose a reason for hiding this comment

OR13 Jan 5, 2022

Choose a reason for hiding this comment

OR13 Jan 5, 2022

Choose a reason for hiding this comment

OR13 Jan 5, 2022 • edited Loading

Choose a reason for hiding this comment

BenjaminMoe commented Dec 22, 2021 •

edited

Loading

BenjaminMoe commented Jan 5, 2022 •

edited

Loading

BenjaminMoe commented Jan 5, 2022 •

edited

Loading

OR13 Jan 5, 2022 •

edited

Loading