Replies: 2 comments
-
I made some progress in the sense that we can rule out using the Java Module The problem it is supposed to solve is version conflict: when identical transitive dependencies of different versions are in the same classpath, and your application or library cannot choose which one -- and only one -- it loads into the runtime. For example, if you have libx-v1.0 and libx-v2.0 in the classpath, your runtime will load the first one it finds and ignore the other. If libx-2.0 removed methods or moved them to other packages, and another 3rd party lib needs to call that old (ignored) libx-v1 method, you'll get a method-not-found error. Many other hard to debug problems occur when your runtime loads dependencies compiled against lib versions that are not in your class path. (Or they are in your classpath, but the classloader ignores them because they are not the most recent version.) At various times, Bisq devs have tried to flatten the dependency graph using gradle exclusions, to remove duplicate / ignored libs in the distribution. But this effort does not solve the problem of incorrectly aligned dependencies in the runtime. Only purely I'm done investigating I think the best we can achieve is to align dependency versions as best we can. That means accepting some version conflicts, and knowing about them up front. Towards that end, I built a simple tool for loading various gradle dependecy graphs into an RDBMS -- to query it to find out what the best aligned dependency version sets are. For example, the To briefly explain the experiment, I take simple a gradle build with the following compile time dependencies:
I dump the gradle
A java program loads the tree representation into an RBMS schema: I will create canned queries to quickly see how well or badly aligned these versions of In theory, if I can find well aligned |
Beta Was this translation helpful? Give feedback.
-
Here's a brief update so you can digest it before the meeting. The state of play with wallet research: Full node wallets as described previously in the research document are the most promising in terms of full-featuredness but really are only a solution for a small percentage of users (i.e. the ones who run their own full node). It would be impractical to install a full node as part of Misq, the best we can offer is give users the option to connect to their existing full node wallet. I was asked to consider key manager interface separately from protocol interface, and I have done that. It however poses issues from a practical use standpoint given that most existing solutions forcibly combine the use of the two. Consider bitcoind for example, you can access key management via its wallet interface. You can access the protocol functions only via its wallet interface and ONLY for the keys which are present in your wallet. BitcoinJ is the same, both functionalities are combined. BitcoinDevkit - this is a rust project which acts as a tool for working with bitcoin scripts, aka Miniscript. It currently seems to use an interface to the electrum protocol. I have not finished looking into it but a number of things currently give me pause: using it might incur a large overhead of pulling in the entire rust ecosystem to the user's PC. Its Java bindings are described as "unstable" in their github. We would have a lot of work to do if adopting miniscript as it is fundamentally different from our previous wallet interfaces. I cannot remember from looking at the code if it has BIP39 functionality built in. As stated above, projects implemented in entirely different toolchains can theoretically be used but the overhead of pulling in their ecosystem and translating from Java to their native language is prohibitive. Btcd, bitcoin-s, and nbitcoin fall under this category. LightningDevKit - have not yet looked into it in depth. BitcoinJ - I have spent most time since last meeting doing an experiment / research into BitcoinJ; how it works and how it can be adapted to use alternate chains e.g. Liquid/elements. This is time consuming but I think it will reap rewards. |
Beta Was this translation helpful? Give feedback.
-
I did not get much progress. Some larger refactorings and peer exchange seems to work in its basics but did not continue much.
Looked into TLV message format encoding but i am not sure if its worth to go that route. I looked a bit into the Eclair kotlin code base and its quite complex. The basics of TLV are simple but for handling nested and chained messages it ends up building your own serialisation framework, replicated things which are already done in other serialisation frameworks. Determinism will also only be achieved by conventions (e.g. enforce deterministic sorting of lists, use deterministic order of records,....), so those conventions can be applied to protobuf as well (e.g. dont use maps). But of course protobuf does not guaranteee deterministic byte representation, though in reality it seems its stable (we use it for the DAO monitor hashes as well as data store hashmap).
I am not sure anymore if we really need determinism on the byte representation level. The main use case is for the hash in the hashmap in the data store, but I am considering other approaches (data ooriginator provides UID which is authenticated with signature or usage of hmac, but not though enough yet to have a more concrete idea).
I also started to add the signature, encryption and hash classes (not checked in yet) and fix a few minor issues with the old Bisq crypto classes (hmac was not using encrypted data but plain text as input). I just want to add the basic as I need it for my next tasks in the p2p lib, but I would like to leave it to Steven then to define and finalize the crypto classes for prodcution. He is our crypto expert ;-). Would be good also to find ciphers with smaller footprints as the pubKeys and signatures take quite some space in current Bisq.
Also started to look into how to use protobuf in a more modular way. I would like to keep payloads of external modules out from the proto parsing, so each module is responsible for serialisation of its payload types.
I would like to find a better approach as we do with the Resolvers as well as not use the
oneof
for mimicing inheritence but use rather composition instead. But not sure yet as the more complex message data structures in the data storage layer are not started yet.EDIT:
Just checked in the current version. Here is the crypto package:
https://github.com/chimp1984/misq/tree/master/common/src/main/java/misq/common/security
Beta Was this translation helpful? Give feedback.
All reactions