Dyno: Type constructors for inheriting generic classes (chapel-lang#2…

…5778)

This PR implements type constructors for inheriting generic classes,
which requires some substantial changes to the way type constructors are
implemented in dyno.

Prior to this PR, we used a compiler-generated signature that re-used
the uAST and IDs for fields in place of real formals. This breaks down
when inheriting classes are involved because the parent fields will have
different symbol paths from the child's fields, which can confuse the
rest of dyno's infrastructure. This especially causes issues for the way
we store resolution results, which cannot account for IDs from different
symbol paths.

This PR addresses this problem by generating dedicated uAST for the type
constructor, rather than trying to fake formals with the uAST of fields.
Generating uAST on the fly like this is new for dyno, and has required
some additional infrastructure. The notable changes are as follows:
- Ability to copy uAST
- Adds ``ID::FabricatedKind::Generated`` to indicate non-specific
compiler-generated IDs
- Storing/fetching a BuilderResult for compiler-generated uAST based on
a symbol path of a compiler-generated module
- Updating "id to ast" code to handle compiler-generated IDs

With these capabilities in place, typeConstructorInitialQuery now builds
a small module to store a uAST Function that serves as the type
constructor for a given type. This module pretends to be an included
module living under the module in which the type was declared. For
example, if a type 'R' was defined in module 'foo', the generated module
would have the symbol path of ``foo.chpl__internal_foo_R``.

This module also includes a 'use' statement for the original module,
which is necessary in the event that fields' type/init expressions refer
to other symbols.

Once finished building the uAST, we store the BuilderResult based on the
module path.

This PR updates the implementation of IDs to treat postOrderId values
less than ``-2`` as compiler-generated IDs, but returning said IDs as a
positive 0-based value so as not to confuse the rest of resolution.
Internally, a value of ``-3`` represents a compiler-generated symbol,
and anything less than ``-3`` is a child of a compiler-generated symbol,
and will have its value translated into the proper positive value upon a
call to ``ID::postOrderId()``. This change was made to more easily
support id-to-ast lookups in which we want to quickly determine whether
an ID has an actual file behind it, or if it requires special handling
because it was compiler-generated.

Other changes:
- Updated CompositeType::stringify to print generic parent substitutions
- A new resolution query ``findFieldIDByName`` is added to support these
changes

[reviewed-by @DanilaFe]

frontend/include/chpl/framework/ID.h

@@ -26,6 +26,7 @@
 namespace chpl {
 class Context;
 
+#define ID_GEN_START -3
 
 /**
   This class represents an ID for an AST node.
@@ -38,6 +39,7 @@ class ID final {
   enum FabricatedIdKind {
     // all of these need to be <= -2
     ExternBlockElement = -2,
+    Generated = ID_GEN_START,
   };
 
  private:
@@ -101,14 +103,27 @@ class ID final {
     of a symbol's nodes. When the AST node defines a new ID symbol scope,
     (as with Function or Module) this will return -1.
    */
-  int postOrderId() const { return postOrderId_; }
+  int postOrderId() const {
+    if (postOrderId_ < -2) {
+      if (postOrderId_ == ID_GEN_START) {
+        // generated symbol
+        return -1;
+      } else {
+        // generated uAST nodes
+        return (postOrderId_ * -1) - ID_GEN_START - 1;
+      }
+    } else {
+      return postOrderId_;
+    }
+  }
 
   /**
     Returns 'true' if this symbol has a 'postOrderId()' value of == -1,
     which means this is an ID for something that defines a new symbol
     scope.
    */
-   inline bool isSymbolDefiningScope() const { return postOrderId_ == -1; }
+   inline bool isSymbolDefiningScope() const { return postOrderId_ == -1 ||
+                                                      postOrderId_ == ID_GEN_START; }
 
   /**
     Some IDs are introduced during compilation and don't represent
@@ -119,7 +134,11 @@ class ID final {
 
   FabricatedIdKind fabricatedIdKind() const {
     CHPL_ASSERT(isFabricatedId());
-    return (FabricatedIdKind) postOrderId_;
+    if (postOrderId_ <= ID_GEN_START) {
+      return FabricatedIdKind::Generated;
+    } else {
+      return (FabricatedIdKind) postOrderId_;
+    }
   }
 
   /**

frontend/include/chpl/parsing/parsing-queries.h

@@ -110,6 +110,20 @@ introspectParsedFiles(Context* context);
 const uast::BuilderResult*
 parseFileContainingIdToBuilderResult(Context* context, ID id);
 
+/**
+  Fetch the BuilderResult storing compiler-generated uAST based on the given
+  symbolPath.
+ */
+const uast::BuilderResult&
+getCompilerGeneratedBuilder(Context* context, UniqueString symbolPath);
+
+/**
+  Set the BuilderResult storing compiler-generated uAST based on the given
+  symbolPath.
+ */
+void setCompilerGeneratedBuilder(Context* context, UniqueString symbolPath,
+                                 uast::BuilderResult result);
+
 /**
   A function for counting the tokens when parsing
 */

frontend/include/chpl/resolution/resolution-queries.h

@@ -505,6 +505,15 @@ reportInvalidMultipleInheritance(Context* context,
  */
 const std::vector<const uast::Function*>& getTestsGatheredViaPrimitive(Context* context);
 
+/**
+  Returns the field in 'ad' (or its parent) that matches 'name'.
+*/
+const uast::Decl* findFieldByName(Context* context,
+                                  const uast::AggregateDecl* ad,
+                                  const types::CompositeType* ct,
+                                  UniqueString name);
+
+
 
 } // end namespace resolution
 } // end namespace chpl

frontend/include/chpl/resolution/resolution-types.h

@@ -161,6 +161,9 @@ class UntypedFnSignature {
   // this will not be present for compiler-generated functions
   const uast::AstNode* whereClause_;
 
+  // The ID that this compiler-generated function is based on
+  ID compilerGeneratedOrigin_;
+
   UntypedFnSignature(ID id,
                      UniqueString name,
                      bool isMethod,
@@ -170,7 +173,8 @@ class UntypedFnSignature {
                      uast::asttags::AstTag idTag,
                      uast::Function::Kind kind,
                      std::vector<FormalDetail> formals,
-                     const uast::AstNode* whereClause)
+                     const uast::AstNode* whereClause,
+                     ID compilerGeneratedOrigin = ID())
     : id_(id),
       name_(name),
       isMethod_(isMethod),
@@ -180,7 +184,8 @@ class UntypedFnSignature {
       idTag_(idTag),
       kind_(kind),
       formals_(std::move(formals)),
-      whereClause_(whereClause) {
+      whereClause_(whereClause),
+      compilerGeneratedOrigin_(compilerGeneratedOrigin) {
     CHPL_ASSERT(idTag == uast::asttags::Function ||
            idTag == uast::asttags::Class    ||
            idTag == uast::asttags::Record   ||
@@ -200,7 +205,8 @@ class UntypedFnSignature {
                         uast::asttags::AstTag idTag,
                         uast::Function::Kind kind,
                         std::vector<FormalDetail> formals,
-                        const uast::AstNode* whereClause);
+                        const uast::AstNode* whereClause,
+                        ID compilerGeneratedOrigin = ID());
 
  public:
   /** Get the unique UntypedFnSignature containing these components */
@@ -213,7 +219,8 @@ class UntypedFnSignature {
                                        uast::asttags::AstTag idTag,
                                        uast::Function::Kind kind,
                                        std::vector<FormalDetail> formals,
-                                       const uast::AstNode* whereClause);
+                                       const uast::AstNode* whereClause,
+                                       ID compilerGeneratedOrigin = ID());
 
   /** Get the unique UntypedFnSignature representing a Function's
       signature from a Function uAST pointer. */
@@ -234,7 +241,8 @@ class UntypedFnSignature {
            idTag_ == other.idTag_ &&
            kind_ == other.kind_ &&
            formals_ == other.formals_ &&
-           whereClause_ == other.whereClause_;
+           whereClause_ == other.whereClause_ &&
+           compilerGeneratedOrigin_ == other.compilerGeneratedOrigin_;
   }
   bool operator!=(const UntypedFnSignature& other) const {
     return !(*this == other);
@@ -251,6 +259,7 @@ class UntypedFnSignature {
       context->markPointer(elt.decl);
     }
     context->markPointer(whereClause_);
+    compilerGeneratedOrigin_.mark(context);
   }
 
 
@@ -275,6 +284,10 @@ class UntypedFnSignature {
     return isCompilerGenerated_;
   }
 
+  ID compilerGeneratedOrigin() const {
+    return compilerGeneratedOrigin_;
+  }
+
   /** Returns true if id() refers to a Function */
   bool idIsFunction() const {
     return idTag_ == uast::asttags::Function;

frontend/include/chpl/uast/AstNode.h

@@ -138,6 +138,14 @@ class AstNode {
       deserialize the AstNode fields other than the children */
   AstNode(AstTag tag, Deserializer& des);
 
+  AstNode(const AstNode& node) {
+    this->tag_ = node.tag_;
+    this->attributeGroupChildNum_ = node.attributeGroupChildNum_;
+    for (auto child : node.children()) {
+      children_.push_back(child->copy());
+    }
+  }
+
   /** Completes the deserialization process for an AstNode
       by deserializing the children. */
   void deserializeChildren(Deserializer& des);
@@ -554,6 +562,8 @@ class AstNode {
       #undef CASE_OTHER
     }
   }
+
+  owned<AstNode> copy() const;
 };
 } // end namespace uast
 

frontend/include/chpl/uast/Builder.h

@@ -77,6 +77,8 @@ class Builder final {
   // These are removed in the astToLocation_ map.
   AstLocMap notedLocations_;
 
+  bool generatedCode_ = false;
+
   // These map AST to additional locations while the builder is building.
   // This is an equivalent to notedLocations for the additional locations.
   // The key type is just 'AstNode' so that we can use generic functions.
@@ -120,6 +122,10 @@ class Builder final {
                                                 const char* filepath,
                                                 UniqueString parentSymbolPath);
 
+  static owned<Builder> createForGeneratedCode(Context* context,
+                                               const char* filepath,
+                                               UniqueString parentSymbolPath);
+
   /** Construct a Builder for use when reading uAST from a library file. */
   static owned<Builder> createForLibraryFileModule(
                                           Context* context,

frontend/lib/parsing/parsing-queries.cpp

@@ -192,19 +192,56 @@ introspectParsedFiles(Context* context) {
   return toReturn;
 }
 
+static const BuilderResult&
+compilerGeneratedBuilderQuery(Context* context, UniqueString symbolPath) {
+  QUERY_BEGIN(compilerGeneratedBuilderQuery, context, symbolPath);
+
+  BuilderResult ret;
+
+  return QUERY_END(ret);
+}
+
 // parses whatever file exists that contains the passed ID and returns it
 const BuilderResult*
 parseFileContainingIdToBuilderResult(Context* context, ID id) {
-  UniqueString path;
-  UniqueString parentSymbolPath;
-  bool found = context->filePathForId(id, path, parentSymbolPath);
-  if (found) {
-    const BuilderResult& p = parseFileToBuilderResult(context, path,
-                                                      parentSymbolPath);
-    return &p;
+  if (id.isFabricatedId() &&
+      id.fabricatedIdKind() == ID::FabricatedIdKind::Generated) {
+    // Find the generated module's symbol path
+    UniqueString symbolPath;
+    if (id.symbolName(context).startsWith("chpl__generated")) {
+      symbolPath = id.symbolPath();
+    } else {
+      symbolPath = ID::parentSymbolPath(context, id.symbolPath());
+
+      // Assumption: The generated module goes only one symbol deep.
+      CHPL_ASSERT(ID::innermostSymbolName(context, symbolPath).startsWith("chpl__generated"));
+    }
+
+    const BuilderResult& br = getCompilerGeneratedBuilder(context, symbolPath);
+    assert(br.numTopLevelExpressions() != 0);
+    return &br;
+  } else  {
+    UniqueString path;
+    UniqueString parentSymbolPath;
+    bool found = context->filePathForId(id, path, parentSymbolPath);
+    if (found) {
+      const BuilderResult& p = parseFileToBuilderResult(context, path,
+                                                        parentSymbolPath);
+      return &p;
+    }
+
+    return nullptr;
   }
+}
 
-  return nullptr;
+const BuilderResult&
+getCompilerGeneratedBuilder(Context* context, UniqueString symbolPath) {
+  return compilerGeneratedBuilderQuery(context, symbolPath);
+}
+
+void setCompilerGeneratedBuilder(Context* context, UniqueString symbolPath,
+                                 BuilderResult result) {
+  QUERY_STORE_RESULT(compilerGeneratedBuilderQuery, context, result, symbolPath);
 }
 
 void countTokens(Context* context, UniqueString path, ParserStats* parseStats) {

frontend/lib/resolution/Resolver.cpp

@@ -1016,8 +1016,10 @@ void Resolver::resolveTypeQueries(const AstNode* formalTypeExpr,
 
         // get the substitution for that field from the CompositeType
         // and recurse with the result to set types for nested TypeQuery nodes
-        const uast::Decl* field = fa->formal();
+        auto formal = fa->formal()->toNamedDecl();
         const SubstitutionsMap& subs = actualCt->substitutions();
+        auto ad = parsing::idToAst(context, actualCt->id())->toAggregateDecl();
+        auto field = findFieldByName(context, ad, actualCt, formal->name());
         auto search = subs.find(field->id());
         if (search != subs.end()) {
           QualifiedType fieldType = search->second;
@@ -2388,11 +2390,13 @@ QualifiedType Resolver::typeForId(const ID& id, bool localGenericToUnknown) {
 
   if (id.isFabricatedId()) {
     switch (id.fabricatedIdKind()) {
-      case ID::ExternBlockElement:
-      // TODO: resolve types for extern block
-      // (will need the Identifier name for that)
-      auto unknownType = UnknownType::get(context);
-      return QualifiedType(QualifiedType::UNKNOWN, unknownType);
+      case ID::ExternBlockElement: {
+        // TODO: resolve types for extern block
+        // (will need the Identifier name for that)
+        auto unknownType = UnknownType::get(context);
+        return QualifiedType(QualifiedType::UNKNOWN, unknownType);
+      }
+      case ID::Generated: break;
     }
   }
 
@@ -3933,7 +3937,8 @@ void Resolver::exit(const Dot* dot) {
     // Try to resolve a it as a field/parenless proc so we can resolve 'this' on
     // it later if needed.
     if (!receiver.type().isUnknown() && receiver.type().type() &&
-        receiver.type().type()->isCompositeType()) {
+        receiver.type().type()->isCompositeType() &&
+        dot->field() != "init") {
       std::vector<CallInfoActual> actuals;
       actuals.push_back(CallInfoActual(receiver.type(), USTR("this")));
       auto ci = CallInfo(/* name */ dot->field(),