Add Validator_Monitoring_Service proposal

[avtishin]

Project Abstract

Validator Monitoring Service is a monitoring platform designed to track the performance of validators in Polkadot and Kusama networks. Validator Monitoring Service has several key features that make it an essential tool for validator operators and nominators in Polkadot and Kusama networks.
We provide a live dashboard that users can access through an easy-to-use telegram bot (@p2pvalidator_monitoring_bot). This live dashboard allows users to track essential metrics such as session/era progress, rewards points, consensus participation, position in the active set, and more. We collect data on all active validators and store it for up to one month, providing users with historical data on validator performance. This allows users to analyse their validator's behaviour over time and identify any potential issues.

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• I have read the application guidelines.
• Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
• The software delivered for this grant will be released under an open-source license specified in the application.
• The initial PR contains only one commit (squash and force-push if needed).
• The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
• I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

[keeganquigley]

Thanks for the application @avtishin, I appreciate you comparing your implementation to the existing solutions, you certainly did a lot of research. A couple of initial questions:

• I'm curious why there is a need to authenticate validators to make sure they own the address. Why not create a read-only functionality similar to the Parity staking dashboard that allows me to monitor any validator address?
• The second milestone seems a bit expensive for optimizing the telegram bot. It also seems very specific to your use case. Would others be able to re-use the code to create telegram bots of their own, as an alternative to using yours?
• If the data is only stored for one month, what happens after it is deleted? Does this mean users can only view the historical performance data for the last month?

[avtishin]

Hey @keeganquigley , thank you for your questions. So,

1. It seems there might be a slight misinterpretation. We’re not authenticating the owners of the validators. The Telegram bot’s authentication is just for establishing your personal dashboard credentials. You’re free to submit any validator addresses you wish to monitor - we simply verify that these are indeed addresses of validators, not who owns them. In fact, in our 2nd Milestone, we will eliminate the need to provide validator addresses to the Telegram bot entirely. Instead, we are transitioning this feature to the Grafana dashboard. That means you can choose your validators directly from the Grafana dashboard. This step is aimed at making the user onboarding process simpler and more intuitive. In essence, the primary role of the Telegram bot will be limited to generating dashboard credentials.

2. Regarding the Telegram bot and code, our goal is to offer a user service rather than just a codebase. We believe it’s significant that users don’t need to host anything; they simply log into the bot and access the dashboard. We’ve also made available a repository containing all the files necessary to run your own monitoring system. In this case, you won’t need the Telegram bot, as you can specify the validators you wish to follow directly (in fact, all validators will be accessible on the dashboard when using our Docker setup, they just need to be filtered). Hence, we haven’t made the bot’s code public because there is no need for it. Additionally, from point 1, we plan to further simplify the bot’s use by transferring validator selection from the bot to the dashboard, so the bot’s role will be limited to generating dashboard credentials.

3. Concerning data storage, you’ll have access to the past 30 days’ worth of data once your dashboard is set up. Each subsequent day, the oldest day’s data is removed, meaning you consistently have a 30-day data window. Our primary objective is to offer a monitoring tool, which doesn’t necessitate extensive data storage. This feature is also useful for those setting up their own monitoring using our repository, as initially, they won’t have access to data until their local monitoring begins to collect it.

Does that answer your questions?

[keeganquigley]

Thanks for your thorough answers @avtishin yes that definitely answers my questions! I'll reply here:

1. Ah thanks for clarifying, that makes a lot more sense.
2. Got it, thanks yep that's what I was getting at - the ability to host your own monitoring system if you wanted to. I also wanted to bring up the bot code specifically because, as specified in our guidelines, all code delivered "must be open-sourced, and it must also not rely on closed-source software for full functionality". So beware that we would likely ask you to make it public upon submitting the first milestone.
3. Makes sense! Thanks for the explanation.

That being said, I will go ahead and mark the application as ready for review and ping the committee internally for questions/comments.

[avtishin]

@keeganquigley thank you again very much for your questions. Regarding 2. rules are the rules, we will prepare a code for deploying a tg bot. However, it's not possible to make deployment fully automized because the tg bot relies on personal tg token and github actions (we described it in the proposal, btw), which are needed to configure manually. So, we will prepare an instruction for all of it.

[Noc2]

Thanks for the grant application. It looks good to me, and I'm happy to support it. You might want to consider applying for treasury funding in the future to maintain and further improve the tool.

[keeganquigley]

@avtishin thanks for your answers and sorry for the delay. I am also happy to go forward with it. I think this will be very helpful for validators and I'm aware of your team's influence in our validator ecosystem. Can't wait to see the results!

[avtishin]

Hey @keeganquigley am I correct that we should receive more more approve?

[Noc2]

Hey @keeganquigley am I correct that we should receive more more approve?

That's correct. You need one more approval. I ping the rest of the team again.

[github-actions]

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at [email protected] and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

[keeganquigley]

Hi @avtishin how is milestone 1 coming along?

[avtishin]

Hey @keeganquigley w3f/Grant-Milestone-Delivery#971 -- we are working on comments by external evaluator

[keeganquigley]

Whoops sorry @avtishin didn't realize you had submitted a delivery. Thanks!

[avtishin]

No problem, it's a good experience. You have a exceptional evaluators, so detailed questions and remarks)) We (as a team) highly appreciate it. We are still committed to deliver everything)

[keeganquigley]

Hi @avtishin how is milestone 2 coming along?

[avtishin]

@keeganquigley hey, planing to publish till the end of the year

[keeganquigley]

Hi @avtishin still planning to submit M2 soon?

[avtishin]

Yes, we are on it

[keeganquigley]

Hi @avtishin are you able to provide an update? If you don't think a delivery will be submitted in the next couple of weeks we would ask for an amendment PR to extend the timeline.

[avtishin]

Let's say end of the next week otherwise we will do PR to extend

[avtishin]

@keeganquigley good news, we are going to publish in a couple of days. We are fixing the last bugs. etc

[avtishin]

@keeganquigley w3f/Grant-Milestone-Delivery#1155 here we are!