Highly improbable slot leadership schedule for epoch 165 suggests randomization bug

[staking4ada]

Describe the bug
In particular with pool 4ADA2 it has happened multiple times that, after big increase in stake, slot leadership assignment is unusually low.

In this most recent example, slot allocation is 57% of what it on average should be (15 slots assigned, 26 expected based on 80M ADA stake).

In itself this is not something to worry about.

However, in the slot leadership schedule, there is not a single slot leadership assignment between slot 165.13208 (05/26/2020 04:33:53) and 165.32488 (05/26/2020 21:13:45), which is 17 hours, whereas all 15 assigned slots are distributed between slot 165.0 and 165.13208 (less than 5 hours).

I consider this slot leadership distribution throughout the epoch extremely improbable to the point that I am suggesting there might be a bug in the randomization algorithm.

Please check from your side. I'd like to hear your opinion on this.

Mandatory Information

1. jcli --full-version output
   jcli 0.8.18 (HEAD-690eb547, release, windows [x86_64]) - [rustc 1.42.0 (b8cedc004 2020-03-09)]
2. jormungandr --full-version output;
   jormungandr 0.8.18 (HEAD-690eb547, release, windows [x86_64]) - [rustc 1.42.0 (b8cedc004 2020-03-09)]

To Reproduce
see attached leaderlogs
N2leaderlogs.zip

[NicolasDP]

Thanks this is definitely something interesting. Did you get this kind of distribution on other epochs? Looking that overall the blockchain is producing a fairly well distributed number of slots across the epoch.

[staking4ada]

Hi Nicolas, thanks for checking.

I've had this impression a few times, seeiing big gaps during the day, but never thoroughly investigated. I did now since especially with 4ADA2, it has happened a few times that, just after nearly doubling the delegated stake, slot leadership was low enough to lose all stake again fairly quickly.

If identical methods are used in the Haskell node, it might be worth asking other operators to submit leadership logs which might confirm the phenomenon.

On a side note but possibly related, when comparing between my own S, M and XL pools, after 6 months of ITN I personally am convinced that the current protocol lets big pools stay big while 'obstructing' most if not all medium-size pools from making the jump to join the big ones as well.

This could be due to a bug with leadership allocation to pools with strong fluctuation of stake, but for sure the current saturation mechanism (rewards cap instead of slot leadership cap) doesn't help either. Anyway, that's a different story.

[staking4ada]

For what it is worth, here's another one -again with 4ADA2- with what I find most peculiar distribution throughout the epoch. This is in chronological order, the attached file contains the original jcli output.

Controlled stake approx. 83.3M ADA
16 blocks (57% of what would normally be expected)
There is a 5 hour empty window from 7:15-12:38 (utc+2)
And a 4 hour empty window from 16:43-epoch end (utc+2)

06/14/2020 21:13:45, 06/14/2020 22:16:37, 184.1890, Pending
06/14/2020 21:13:45, 06/15/2020 00:32:53, 184.5978, Pending
06/14/2020 21:13:45, 06/15/2020 02:25:29, 184.9356, Pending
06/14/2020 21:13:45, 06/15/2020 03:13:41, 184.10802, Pending
06/14/2020 21:13:45, 06/15/2020 03:50:49, 184.11916, Pending
06/14/2020 21:13:45, 06/15/2020 04:58:13, 184.13938, Pending
06/14/2020 21:13:45, 06/15/2020 05:55:17, 184.15650, Pending
06/14/2020 21:13:45, 06/15/2020 06:26:25, 184.16584, Pending
06/14/2020 21:13:45, 06/15/2020 07:17:17, 184.18110, Pending
06/14/2020 21:13:45, 06/15/2020 12:38:59, 184.27761, Pending
06/14/2020 21:13:45, 06/15/2020 12:46:23, 184.27983, Pending
06/14/2020 21:13:45, 06/15/2020 14:03:49, 184.30306, Pending
06/14/2020 21:13:45, 06/15/2020 14:17:51, 184.30727, Pending
06/14/2020 21:13:45, 06/15/2020 14:31:23, 184.31133, Pending
06/14/2020 21:13:45, 06/15/2020 15:08:17, 184.32240, Pending
06/14/2020 21:13:45, 06/15/2020 16:43:01, 184.35082, Pending

N2leaderlogs.zip