Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stop testing with broken upstream/version-2-0 #6554

Merged
merged 1 commit into from
Sep 16, 2024
Merged

stop testing with broken upstream/version-2-0 #6554

merged 1 commit into from
Sep 16, 2024

Conversation

tersec
Copy link
Contributor

@tersec tersec commented Sep 16, 2024

Can re-add once version-2-0 builds Nimbus again

@tersec tersec enabled auto-merge (squash) September 16, 2024 21:01
@tersec
Copy link
Contributor Author

tersec commented Sep 16, 2024

in theory, nim-lang/Nim@fd1d008

will retest with current

@tersec
Copy link
Contributor Author

tersec commented Sep 16, 2024

Still fails, different error:

2024-09-16T22:31:04.7214101Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/beacon_chain/rpc/rest_key_management_api.nim(521, 9) template/generic instantiation of `api2` from here
2024-09-16T22:31:04.7217796Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-presto/presto/route.nim(442, 11) template/generic instantiation of `async` from here
2024-09-16T22:31:04.7221471Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/beacon_chain/rpc/rest_key_management_api.nim(547, 39) template/generic instantiation of `setResult` from here
2024-09-16T22:31:04.7225029Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/beacon_chain/rpc/rest_key_management_api.nim(530, 30) template/generic instantiation of `decodeBody` from here
2024-09-16T22:31:04.7228668Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/beacon_chain/spec/eth2_apis/eth2_rest_serialization.nim(3591, 15) template/generic instantiation of `decode` from here
2024-09-16T22:31:04.7232461Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization.nim(72, 15) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7236192Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(193, 22) template/generic instantiation of `fieldReadersTable` from here
2024-09-16T22:31:04.7240116Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(261, 34) template/generic instantiation of `makeFieldReadersTable` from here
2024-09-16T22:31:04.7244334Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(221, 26) template/generic instantiation of `enumAllSerializedFields` from here
2024-09-16T22:31:04.7248658Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(155, 32) template/generic instantiation of `enumAllSerializedFieldsImpl` from here
2024-09-16T22:31:04.7253101Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(239, 52) template/generic instantiation of `readFieldIMPL` from here
2024-09-16T22:31:04.7256897Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(206, 13) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7260675Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(309, 6) template/generic instantiation of `parseArray` from here
2024-09-16T22:31:04.7264661Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(312, 16) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7268680Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(193, 22) template/generic instantiation of `fieldReadersTable` from here
2024-09-16T22:31:04.7273060Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(261, 34) template/generic instantiation of `makeFieldReadersTable` from here
2024-09-16T22:31:04.7277574Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(221, 26) template/generic instantiation of `enumAllSerializedFields` from here
2024-09-16T22:31:04.7282423Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(155, 32) template/generic instantiation of `enumAllSerializedFieldsImpl` from here
2024-09-16T22:31:04.7286884Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(239, 52) template/generic instantiation of `readFieldIMPL` from here
2024-09-16T22:31:04.7291063Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(206, 13) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7295136Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(309, 6) template/generic instantiation of `parseArray` from here
2024-09-16T22:31:04.7299543Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(312, 16) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7304018Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-json-serialization/json_serialization/reader_impl.nim(193, 22) template/generic instantiation of `fieldReadersTable` from here
2024-09-16T22:31:04.7308577Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(261, 34) template/generic instantiation of `makeFieldReadersTable` from here
2024-09-16T22:31:04.7313164Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(221, 26) template/generic instantiation of `enumAllSerializedFields` from here
2024-09-16T22:31:04.7317806Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(155, 32) template/generic instantiation of `enumAllSerializedFieldsImpl` from here
2024-09-16T22:31:04.7321868Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(239, 52) template/generic instantiation of `readFieldIMPL` from here
2024-09-16T22:31:04.7325798Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization/object_serialization.nim(206, 13) template/generic instantiation of `readValue` from here
2024-09-16T22:31:04.7329768Z /github-runner/workspace/nimbus-eth2/nimbus-eth2/vendor/nim-serialization/serialization.nim(31, 20) Error: for a 'var' type a variable needs to be passed; but 'uint64(result)' is immutable
2024-09-16T22:31:04.8763943Z make: *** [Makefile:447: nimbus_validator_client] Error 1

https://github.com/status-im/nimbus-eth2/actions/runs/10884264375/job/30227039171?pr=6553

so automerge back on

@tersec tersec enabled auto-merge (squash) September 16, 2024 22:36
Copy link

Unit Test Results

         9 files  ±0    1 349 suites  ±0   53m 1s ⏱️ + 3m 17s
  5 143 tests ±0    4 795 ✔️ ±0  348 💤 ±0  0 ±0 
21 201 runs  ±0  20 797 ✔️ ±0  404 💤 ±0  0 ±0 

Results for commit c1a01a1. ± Comparison against base commit b3c8c71.

@tersec tersec merged commit 205dff3 into unstable Sep 16, 2024
11 checks passed
@tersec tersec deleted the ysv branch September 16, 2024 23:14
@tersec tersec mentioned this pull request Sep 16, 2024
@tersec
Copy link
Contributor Author

tersec commented Sep 17, 2024

Repro for this particular regression

import
  std/[typetraits, macros]

type
  DefaultFlavor = object

template serializationFormatImpl(Name: untyped) {.dirty.} =
  type Name = object

template serializationFormat(Name: untyped) =
  serializationFormatImpl(Name)

template setReader(Format, FormatReader: distinct type) =
  when arity(FormatReader) > 1:
    template Reader(T: type Format, F: distinct type = DefaultFlavor): type = FormatReader[F]
  else:
    template ReaderType(T: type Format): type = FormatReader
    template Reader(T: type Format): type = FormatReader

template useDefaultReaderIn(T: untyped, Flavor: type) =
  mixin Reader

  template readValue(r: var Reader(Flavor), value: var T) =
    mixin readRecordValue
    readRecordValue(r, value)

from stew/shims/macros import field, isTuple, recordFields, skipPragma

type
  FieldTag[RecordType: object; fieldName: static string] = distinct void

func declval*(T: type): T {.compileTime.} =
  default(ptr T)[]

macro enumAllSerializedFieldsImpl(T: type, body: untyped): untyped =
  var typeAst = getType(T)[1]
  var typeImpl: NimNode
  let isSymbol = not typeAst.isTuple

  if not isSymbol:
    typeImpl = typeAst
  else:
    typeImpl = getImpl(typeAst)
  result = newStmtList()

  var i = 0
  for field in recordFields(typeImpl):
    let
      fieldIdent = field.name
      realFieldName = newLit($fieldIdent.skipPragma)
      fieldName = realFieldName
      fieldIndex = newLit(i)

    let fieldNameDefs =
      if isSymbol:
        quote:
          const fieldName {.inject, used.} = `fieldName`
          const realFieldName {.inject, used.} = `realFieldName`
      else:
        quote:
          const fieldName {.inject, used.} = $`fieldIndex`
          const realFieldName {.inject, used.} = $`fieldIndex`

    let field =
      if isSymbol:
        quote do: declval(`T`).`fieldIdent`
      else:
        quote do: declval(`T`)[`fieldIndex`]

    result.add quote do:
      block:
        `fieldNameDefs`

        template FieldType: untyped {.inject, used.} = typeof(`field`)

        `body`

  # echo repr(result)

template enumAllSerializedFields(T: type, body): untyped =
  enumAllSerializedFieldsImpl(T, body)

type
  FieldReader[RecordType, Reader] = tuple[
    fieldName: string,
    reader: proc (rec: var RecordType, reader: var Reader)
                 {.gcsafe, nimcall.}
  ]

proc totalSerializedFieldsImpl(T: type): int =
  mixin enumAllSerializedFields
  enumAllSerializedFields(T): inc result

template totalSerializedFields(T: type): int =
  (static(totalSerializedFieldsImpl(T)))

template GetFieldType(FT: type FieldTag): type =
  typeof field(declval(FT.RecordType), FT.fieldName)

proc makeFieldReadersTable(RecordType, ReaderType: distinct type,
                           numFields: static[int]):
                           array[numFields, FieldReader[RecordType, ReaderType]] =
  mixin enumAllSerializedFields, handleReadException
  var idx = 0

  enumAllSerializedFields(RecordType):
    proc readField(obj: var RecordType, reader: var ReaderType)
                  {.gcsafe, nimcall.} =

      mixin readValue

      type F = FieldTag[RecordType, realFieldName]
      field(obj, realFieldName) = reader.readValue(GetFieldType(F))

    result[idx] = (fieldName, readField)
    inc idx

proc fieldReadersTable(RecordType, ReaderType: distinct type): auto =
  mixin readValue
  type T = RecordType
  const numFields = totalSerializedFields(T)
  var tbl {.threadvar.}: ref array[numFields, FieldReader[RecordType, ReaderType]]
  if tbl == nil:
    tbl = new typeof(tbl)
    tbl[] = makeFieldReadersTable(RecordType, ReaderType, numFields)
  return addr(tbl[])

proc readValue(reader: var auto, T: type): T =
  mixin readValue
  reader.readValue(result)

template decode(Format: distinct type,
                 input: string,
                 RecordType: distinct type): auto =
  mixin Reader
  block:  # https://github.com/nim-lang/Nim/issues/22874
    var reader: Reader(Format)
    reader.readValue(RecordType)

template readValue(Format: type,
                    ValueType: type): untyped =
  mixin Reader, init, readValue
  var reader: Reader(Format)
  readValue reader, ValueType

template parseArrayImpl(numElem: untyped,
                        actionValue: untyped) =
  actionValue

serializationFormat Json
template createJsonFlavor(FlavorName: untyped,
                           skipNullFields = false) {.dirty.} =
  type FlavorName = object

  template Reader(T: type FlavorName): type = Reader(Json, FlavorName)
type
  JsonReader[Flavor = DefaultFlavor] = object

Json.setReader JsonReader

template parseArray(r: var JsonReader; body: untyped) =
  parseArrayImpl(idx): body

template parseArray(r: var JsonReader; idx: untyped; body: untyped) =
  parseArrayImpl(idx): body

proc readRecordValue[T](r: var JsonReader, value: var T) =
  type
    ReaderType {.used.} = type r
    T = type value

  discard T.fieldReadersTable(ReaderType)

proc readValue[T](r: var JsonReader, value: var T) =
  mixin readValue

  when value is seq:
    r.parseArray:
      readValue(r, value[0])

  elif value is object:
    readRecordValue(r, value)

type
  RemoteSignerInfo = object
    id: uint32
  RemoteKeystore = object

proc readValue(reader: var JsonReader, value: var RemoteKeystore) =
  discard reader.readValue(seq[RemoteSignerInfo])

createJsonFlavor RestJson
useDefaultReaderIn(RemoteSignerInfo, RestJson)
proc readValue(reader: var JsonReader[RestJson], value: var uint64) =
  discard reader.readValue(string)

discard Json.decode("", RemoteKeystore)
block:  # https://github.com/nim-lang/Nim/issues/22874
  var reader: Reader(RestJson)
  discard reader.readValue(RemoteSignerInfo)

Araq pushed a commit to nim-lang/Nim that referenced this pull request Sep 18, 2024
…24130)

split again from #24038, fixes
status-im/nimbus-eth2#6554 (comment)

`var`/pointer types are no longer implicitly convertible to each other
if their element types either:

* require an int conversion or another conversion operation as long as
it's not to `openarray`,
* are subtypes with pointer indirection,

Previously any conversion below a subrange match would match if the
element type wasn't a pointer type, then it would error later in
`analyseIfAddressTaken`.

Different from #24038 in that the preview define that made subrange
matches also fail to match is removed for a simpler diff so that it can
be backported.
narimiran pushed a commit to nim-lang/Nim that referenced this pull request Sep 18, 2024
…24130)

split again from #24038, fixes
status-im/nimbus-eth2#6554 (comment)

`var`/pointer types are no longer implicitly convertible to each other
if their element types either:

* require an int conversion or another conversion operation as long as
it's not to `openarray`,
* are subtypes with pointer indirection,

Previously any conversion below a subrange match would match if the
element type wasn't a pointer type, then it would error later in
`analyseIfAddressTaken`.

Different from #24038 in that the preview define that made subrange
matches also fail to match is removed for a simpler diff so that it can
be backported.

(cherry picked from commit 1660ddf)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant