Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Datagen proposal to amend payment for Milestone 2 #1810

Closed
wants to merge 5 commits into from
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 94 additions & 4 deletions applications/Datagen_Project.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ The goal is to achive a fully functional mechanism for the random selection of t

- **Total Estimated Duration:** 5 months (starting date - August 2022) _ Update: Because of changes in the development team M2 delivery date will be ≈15th of March 2023, with relative shift of the whole timeline.
- **Full-Time Equivalent (FTE):** 2
- **Total Costs:** 44,000 USD
- **Total Costs:** 53,000 USD

### Milestone 1 — Implement the randomized substrate pallet `pallet_random_node_selector` and `pallet_check_node_computational_work`

Expand All @@ -203,7 +203,7 @@ The goal is to achive a fully functional mechanism for the random selection of t

- **Estimated Duration:** 1,5 months
- **FTE:** 2
- **Costs:** 12,000 USD
- **Costs:** 21,000 USD

| Number | Deliverable | Specification |
| -----: | ----------- | ------------- |
Expand Down Expand Up @@ -242,6 +242,96 @@ The goal is to achive a fully functional mechanism for the random selection of t

## Future Plans

- Short term, while the development team works on the grant proposal, the management team will continue the private sale of our DataGen token (to ensure more financial resources for our project) and increase our available network;
In- Short term, while the development team works on the grant proposal, the management team will continue the private sale of our DataGen token (to ensure more financial resources for our project) and increase our available network;

- Our long-term plan is to access the Substrate Builder's Program and walk the path to obtain a Parachain.

## Future Proposal to amend the amount for the second Milestone from 12K to 21K (9K USDC increase).

After an insightful conversation with your colleagues of Parity, Coleman Maher and Richard Casey
we thought to apply for an integration of the payment for the second Milestone of our ongoing grant.
We will do a quick recap and explain why we think it is necessary:

-We applied last summer for the grant. While I have always been personally responsible for the general technical design of th
e architecture, including the specifics for the grant, the development was done in synergy with @viac92, which, back then, was our lead dev.

-At the time of the grant application @viac92 expertise was mostly solidity and other languages, not rust or the substrate framework.
For this reason, B-Datagray (the legal entity that is responsible for developing the project) allowed the dev to take some time to be
more familiar with the network, paying him a full-time salary even for the time necessary for his formation.
The actual development started in September.

-In late November we delivered the first milestone.

-As already explained, the developer that completed the first milestone left us before Christmas 2022.

-Due to the bearish market our fundraising with VCs came to a halt in late 2022 and early 2023, preventing us from scaling the team further.

-We started to work with @cuteolaf in late January, part-time, given the financial constraints.

-@cuteolaf left the development in March for a mix of underestimating the technical complexities from his side and (once he correctly
assessed the number of working hours necessary for the development of the Milestone) his unwillingness to develop such a great amount
of working hours for the financial compensation that was possible given the actual Milestone2 payment (the agreement was for him to be
paid once the Milestone code was approved by W3F committee).

-In April we started again to search for substrate devs, but our proposal was totally off-market prices for a full-time substrate dev.
The market monthly salary of one full-time senior substrate dev is ≈10 to 20k per month, with 15K being a common benchmark. When we
asked for the Milestone:
1. We underestimated the technical complexity of the milestone;
2. We thought we could count on @viac92 as an affordable dev. @viac92 was working at a much lower salary than the average substrate
dev, both because he was involved in the project with a native token allocation and because he was very junior, and we paid him a
salary during his formation in the substrate framework. In assessing the Milestone2 payment we were counting on this factor, while,
having to pay substrate devs at the open market price, the second Milestone amount of 12K was totally insufficient.

-Given the above premises, the search of a new dev in April was very difficult, we both explored the road of dev shops with substrate
capabilities, mostly with the help of Richard Casey from Parity, and of independent contractors. The pricing to develop the milestone
was always above 15K. Usually, prices of 15K to 20K could buy ≈200h of development, but no guarantee of developing the milestone in case
the number of hours was exceeded. While one specific dev shop was ready to contractually guarantee the development, but for 30K.


-Similarly, it was impossible to hire full-time devs, both for the insufficient amount from the milestone and because most devs were
requesting (for a full-time job) contracts of 6 months to 1 year, well beyond the scope of the milestone and, until we can secure more
VC funds, we can’t hire a substrate dev for 6 months;

-At the end of May, we started to work with another senior substrate dev, @ysfkel, for the reasons above, our collaboration is part
time;

As so, while the work on the second milestone is progressing, we are aware that we are greatly out of the originally proposed timeline,
and we fell that a payment integration for the milestone will allow us to speed up the milestone development.

I would also like to do some specifications about the deliverable of the second milestone, just to underline the complexity of we
weren’t completely aware when we submitted the original grant proposal.

a) It revolves around the communication between a solo substrate chain and a parachain. Usually in the substrate framework, you have a
Relay Chain and a Parachain speaking with each other (due to the way in which the Polkadot and Kusama ecosystems are built), meaning
that for the communication between a Parachain and a Solo Chain (which is not its own Relay Chain), the standard XCM method can’t be
applied;

b) For the same reasons you need custom RPC bridges;

c) There is very fragmentary previous work; something can be found in Parity’s repo https://github.com/paritytech/solo-para-bridge-poc
and some work has been done by Darwinia’s team, but all those repos are in beta, not ready to use (by us or by other teams);

d) It would be the first time, best of my knowledge, that someone attempts to connect via RPC bridge a Parachain with a Solochain which
is not the Parachain’s Relay;

e) This implies more development;

f) This can benefit a plurality of teams in the Polkadot ecosystem: for example, it could enable direct communication between a
Polkadot’s Parachain and Kusama (or viceversa) or any Polkadot’s/ Kusama’s Parachain and any Solochain; it can also enable Parachain’s
to speak with Solochains which have a different blocktime;

g) Solochain’s use can be beneficial for any use case in which is required a latency time lower than the ≈6 sec of blocktime required
by Polkadot and can greatly expand Polkadot’s/Kusama’s use cases for low latency time applications; however the lack of frameworks for
efficient communications between Solochains and Paracahains (which are the backbone of the Polkadot/Kusama ecosystem), is greatly
reducing the use of Solochains as a complement for Parachains.

h) Besides the importance of the development of the Milestone itself, we remember that Datagen (https://github.com/Datagen-Project/Whitepaper-2.0 ), as a service blockchain for verification of random computational problems, can greatly benefit the Polkadot
network, by adding a tool to randomly verify on-chain off-chain computational workloads and to assign workloads in a decentralized way
(as a long-term solution to the vexing problem of having Polkadot’s and Kusama’s validators all hosted by AWS and similar providers
with censorship capability over the network’s validators); as so, beyond the scope of Milestone2, we think that progressing as fast
as possible with the general roadmap of the Datagen network can be beneficial for the whole ecosystem.

For all the reasons above, I would like to ask for an increase of the amount due for the M2 deliverable of 9K USDC (from 12K to 21K
USDC), to better reflect the real development costs and speed up the delivery.


- Our long term plan is to access the Substrate Builder's Program and walk the path to obtain a Parachain.