feat: @type intrinsic

[Chriscbr]

Closes #7150

Adds for the new @type intrinsic function. This function can be passed a type, and it will give you back an object (std.reflect.Type) with information about it -- for example, whether it's a struct or class, what properties or fields it has, etc. You can use this to validate request data structures at runtime, generate schemas for APIs or databases, or for simply asserting on the structure of your code.

Future work:

• Support doc reflection
• Fix support for variadic functions
• Open discussion about accessing private fields or methods via reflection
• Open issue about explicit "preflight" annotation
• Open issue about explicit TypeId enum

Checklist

• Title matches Winglang's style guide
• Description explains motivation and solution
• Tests added (always)
• Docs updated (only required for features)
• Added pr/e2e-full label if this feature requires end-to-end testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

[github-actions]

Thanks for opening this pull request! 🎉
Please consult the contributing guidelines for details on how to contribute to this project.
If you need any assistance, don't hesitate to ping the relevant owner over Discord.

Topic	Owner
Wing SDK and standard library	@chriscbr
Wing Console	@skyrpex
Wing compiler and language design	@chriscbr
VSCode extension and language server	@chriscbr
Wing CLI	@chriscbr
Documentation	@boyney123
Examples	@boyney123
Wing Playground	@skyrpex

[eladb]

Beautiful!

[Chriscbr]

It looks like we need to be smart about how we emit these reflection classes, otherwise we'll run into code generation issues. For example, supposing you have structs that refer to each other like these:

struct S1 {
  child: S2?;
}
struct S2 {
 child: S1?;
}

Then the naive way of generating meta-objects in JavaScript will look something like this:

new std.StructType("S1", {
  child: new std.OptionalType(
    new std.StructType("S2", {
      child: new std.OptionalType(
        new std.StructType("S1", <infinity>)
      ),
    }),
  ),
});

The example above is contrived, but in practice there are many recursive relationships between classes in Wing libraries and the SDK via method parameters and return types etc.

[Chriscbr]

The way I'm thinking of solving this is to "collect" the list of types referenced through helper functions in jsify.rs, assigning names to each of them, and then to emit somewhere around the beginning of preflight.js looking something like:

// prelude: we instantiate classes for all types, without their contents filled
let $s1_type = new std.StructType("S1");
let $s2_type = new std.StructType("S2");

// after: emit code to fill in type information, with cycles allowed
$s1_type.fields = { child: new std.OptionalType($s2_type) };
$s2_type.fields = { child: new std.OptionalType($s1_type) };

This code that defines reflection data for named types (classes, structs, interfaces, enums) can go into some kind of prelude section of preflight.js. Or even in a separate file. Then, wherever @type(MyStruct) is actually called, it will just emit a reference to the variable $s1_type.

[Chriscbr]

I've been making a little bit more progress. To do the approach above, we basically need to do some recursion on types. When we generate the javascript reflection code for S1, it needs to emit the reflection code for S2, and to emit that reflection code, we'd need to emit reflection code for S1. To avoid recursing indefinitely, we need to have a base case where if a type has already been explored (we've generated code for it), then we stop and reuse the existing result.

But we can't do that easily today because Type / TypeRef do not implement the Eq/Hash traits. Put another way, we have no way to tell if two types are equal. Part of the reason why it hasn't been easy to define equality on types is because we create duplicate type objects even for identical function signatures - there's no interning process going on. It's possible we can add a stub implementation of Eq/Hash that handles enough of the cases and works around this, but it might be worthwhile to see if we can change how we're storing types to avoid.

[Chriscbr]

Wow - this recursion problem really punches back!

So it seems like I fixed all of the issues with generating types that may have cyclic references in preflight. I've copied an example of a types.cjs file generated with my local build:

const std = require("@winglang/sdk").std;
const $types = {};
$types.t1_interface_MyInterface = std.Type._ofInterface(new std.InterfaceType("MyInterface", "examples-valid.MyInterface"));
$types.t2_interface_BaseInterface = std.Type._ofInterface(new std.InterfaceType("BaseInterface", "examples-valid.BaseInterface"));
$types.t3_interface_IResource = std.Type._ofInterface(new std.InterfaceType("IResource", "@winglang/sdk.std.IResource"));
$types.t4_interface_IConstruct = std.Type._ofInterface(new std.InterfaceType("IConstruct", "constructs.IConstruct"));
$types.t5_interface_IDependable = std.Type._ofInterface(new std.InterfaceType("IDependable", "constructs.IDependable"));
$types.t6_interface_IHostedLiftable = std.Type._ofInterface(new std.InterfaceType("IHostedLiftable", "@winglang/sdk.std.IHostedLiftable"));
$types.t7_interface_ILiftable = std.Type._ofInterface(new std.InterfaceType("ILiftable", "@winglang/sdk.std.ILiftable"));
$types.t8_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t9_interface_IInflightHost = std.Type._ofInterface(new std.InterfaceType("IInflightHost", "@winglang/sdk.std.IInflightHost"));
$types.t10_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t11_str = std.Type._ofStr();
$types.t12_void = std.Type._ofVoid();
$types.t13_array_str = std.Type._ofArray(undefined, false);
$types.t14_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t15_class_MyClass = std.Type._ofClass(new std.ClassType("MyClass", undefined));
$types.t16_class_Resource = std.Type._ofClass(new std.ClassType("Resource", "@winglang/sdk.std.Resource"));
$types.t17_class_Construct = std.Type._ofClass(new std.ClassType("Construct", "constructs.Construct"));
$types.t18_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t19_any = std.Type._ofAny();
$types.t20_bool = std.Type._ofBool();
$types.t21_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t22_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t23_array_str = std.Type._ofArray(undefined, false);
$types.t24_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t25_array_str = std.Type._ofArray(undefined, false);
$types.t26_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t27_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t28_fn = std.Type._ofFunction(new std.FunctionType(std.Phase.PREFLIGHT));
$types.t29_enum_MyEnum = std.Type._ofEnum(new std.EnumType("MyEnum", "examples-valid.MyEnum", {"VARIANT1": new std.EnumVariant("VARIANT1"), "VARIANT2": new std.EnumVariant("VARIANT2"), }));;
$types.t30_struct_MyStruct = std.Type._ofStruct(new std.StructType("MyStruct", "examples-valid.MyStruct"));
$types.t31_struct_Base1 = std.Type._ofStruct(new std.StructType("Base1", "examples-valid.Base1"));
$types.t32_struct_Base2 = std.Type._ofStruct(new std.StructType("Base2", "examples-valid.Base2"));
$types.t33_num = std.Type._ofNum();
$types.t34_array_opt_json = std.Type._ofArray(undefined, false);
$types.t35_opt_json = std.Type._ofOptional(undefined);
$types.t36_json = std.Type._ofJson();
$types.t1_interface_MyInterface.data.bases.push($types.t2_interface_BaseInterface.data);
$types.t1_interface_MyInterface.data.methods["method1"] = new std.Method("method1", false, $types.t14_fn.data);
$types.t2_interface_BaseInterface.data.bases.push($types.t3_interface_IResource.data);
$types.t3_interface_IResource.data.bases.push($types.t4_interface_IConstruct.data);
$types.t3_interface_IResource.data.bases.push($types.t6_interface_IHostedLiftable.data);
$types.t3_interface_IResource.data.methods["onLift"] = new std.Method("onLift", false, $types.t8_fn.data);
$types.t4_interface_IConstruct.data.bases.push($types.t5_interface_IDependable.data);
$types.t6_interface_IHostedLiftable.data.bases.push($types.t7_interface_ILiftable.data);
$types.t6_interface_IHostedLiftable.data.methods["onLift"] = new std.Method("onLift", false, $types.t8_fn.data);
$types.t8_fn.data.params.push($types.t9_interface_IInflightHost.data);
$types.t8_fn.data.params.push($types.t13_array_str.data);
$types.t8_fn.data.returns = $types.t12_void;
$types.t9_interface_IInflightHost.data.bases.push($types.t3_interface_IResource.data);
$types.t9_interface_IInflightHost.data.methods["addEnvironment"] = new std.Method("addEnvironment", false, $types.t10_fn.data);
$types.t10_fn.data.params.push($types.t11_str.data);
$types.t10_fn.data.params.push($types.t11_str.data);
$types.t10_fn.data.returns = $types.t12_void;
$types.t13_array_str.data.child = $types.t11_str;
$types.t14_fn.data.returns = $types.t12_void;
$types.t15_class_MyClass.data.base = $types.t16_class_Resource.data;
$types.t15_class_MyClass.data.interfaces.push($types.t1_interface_MyInterface.data);
$types.t15_class_MyClass.data.properties["field1"] = new std.Property("field1", $types.t11_str);
$types.t15_class_MyClass.data.methods["method1"] = new std.Method("method1", false, $types.t27_fn.data);
$types.t15_class_MyClass.data.methods["method2"] = new std.Method("method2", true, $types.t28_fn.data);
$types.t15_class_MyClass.data.methods["onLift"] = new std.Method("onLift", false, $types.t22_fn.data);
$types.t15_class_MyClass.data.methods["onLiftType"] = new std.Method("onLiftType", true, $types.t24_fn.data);
$types.t15_class_MyClass.data.methods["toInflight"] = new std.Method("toInflight", true, $types.t26_fn.data);
$types.t15_class_MyClass.data.methods["isConstruct"] = new std.Method("isConstruct", true, $types.t18_fn.data);
$types.t15_class_MyClass.data.methods["toString"] = new std.Method("toString", false, $types.t21_fn.data);
$types.t16_class_Resource.data.base = $types.t17_class_Construct.data;
$types.t16_class_Resource.data.interfaces.push($types.t3_interface_IResource.data);
$types.t16_class_Resource.data.methods["onLift"] = new std.Method("onLift", false, $types.t22_fn.data);
$types.t16_class_Resource.data.methods["onLiftType"] = new std.Method("onLiftType", true, $types.t24_fn.data);
$types.t16_class_Resource.data.methods["toInflight"] = new std.Method("toInflight", true, $types.t26_fn.data);
$types.t16_class_Resource.data.methods["isConstruct"] = new std.Method("isConstruct", true, $types.t18_fn.data);
$types.t16_class_Resource.data.methods["toString"] = new std.Method("toString", false, $types.t21_fn.data);
$types.t17_class_Construct.data.interfaces.push($types.t4_interface_IConstruct.data);
$types.t17_class_Construct.data.methods["isConstruct"] = new std.Method("isConstruct", true, $types.t18_fn.data);
$types.t17_class_Construct.data.methods["toString"] = new std.Method("toString", false, $types.t21_fn.data);
$types.t18_fn.data.params.push($types.t19_any.data);
$types.t18_fn.data.returns = $types.t20_bool;
$types.t21_fn.data.returns = $types.t11_str;
$types.t22_fn.data.params.push($types.t9_interface_IInflightHost.data);
$types.t22_fn.data.params.push($types.t23_array_str.data);
$types.t22_fn.data.returns = $types.t12_void;
$types.t23_array_str.data.child = $types.t11_str;
$types.t24_fn.data.params.push($types.t9_interface_IInflightHost.data);
$types.t24_fn.data.params.push($types.t25_array_str.data);
$types.t24_fn.data.returns = $types.t12_void;
$types.t25_array_str.data.child = $types.t11_str;
$types.t26_fn.data.params.push($types.t3_interface_IResource.data);
$types.t26_fn.data.returns = $types.t11_str;
$types.t27_fn.data.returns = $types.t12_void;
$types.t28_fn.data.returns = $types.t12_void;
$types.t30_struct_MyStruct.data.bases.push($types.t31_struct_Base1.data);
$types.t30_struct_MyStruct.data.bases.push($types.t32_struct_Base2.data);
$types.t30_struct_MyStruct.data.fields["base1"] = new std.Property("base1", $types.t20_bool);
$types.t30_struct_MyStruct.data.fields["base2"] = new std.Property("base2", $types.t20_bool);
$types.t30_struct_MyStruct.data.fields["field1"] = new std.Property("field1", $types.t33_num);
$types.t30_struct_MyStruct.data.fields["field2"] = new std.Property("field2", $types.t34_array_opt_json);
$types.t31_struct_Base1.data.fields["base1"] = new std.Property("base1", $types.t20_bool);
$types.t32_struct_Base2.data.fields["base2"] = new std.Property("base2", $types.t20_bool);
$types.t34_array_opt_json.data.child = $types.t35_opt_json;
$types.t35_opt_json.data.child = $types.t36_json;
module.exports = $types;

All of this works fine. We define the types first, then fill in the types' contents.

But it appears that separately, there's a problem with lifting std.Type instances from preflight to inflight. Namely, calling toInflight() sometimes blows the stack frame.

Let's look back at the example before:

struct S1 {
  child: S2?;
}

struct S2 {
 child: S1?;
}

And suppose we lift a type reflection:

let s1 = @type(S1);

inflight () => {
  log(s1.kind); // should print "struct"
}

In preflight, s1 is an instance of std.Type storing inside a std.StructType. To use it inflight, we have to generate some JS code that will get included in the inflight function, through toInflight(). What does the toInflight() implementation look like? The current naive implementation looks approximately like this:

  /** @internal */
  public _toInflight(): string {
    const args = [
      `"${this.name}"`,
      this.fqn ? `"${this.fqn}"` : "undefined",
      arrayToInflight(this.bases), // helper function that calls _toInflight() on each parent struct reflection
      mapToInflight(this.fields), // helper function that calls _toInflight() on each field reflection
    ];
    return `new std.StructType(${args.join(", ")})`;
  }

But as you can imagine, this means s1._toInflight() will end up calling s2._toInflight() which will then call s1._toInflight() and so on.

The solution isn't as obvious to me. One idea could be to do some kind of runtime tree traversal -- so we can find all unique types -- then define some variables for them, then call an altered method like toInflightWithContext() on each type so that it can refer to this map of variables (stopping the endless recursion) when it sees another type.

Another idea could be to totally revise how we're code generating these helper classes. But I'm not sure where I'd start with that to be honest.

[eladb]

Another idea could be to totally revise how we're code generating these helper classes. But I'm not sure where I'd start with that to be honest.

A sketch of a thought: instead of only generating types that are accessed, perhaps we could generate a single in-memory data structure that includes the entire app type system. Basically a big-ass JSON dump of the application's type info that the compiler holds in-memory. Then @type(x) will just access this at runtime (preflight-time).

So this:

class Foo {}

let x = @type(Foo);

Becomes this:

const $types = require("./types.json");

const x = std.Type._of($types["fqn.of.Foo"]);

[MarkMcCulloh]

instead of only generating types that are accessed, perhaps we could generate a single in-memory data structure that includes the entire app type system.

Depending on how this is done, this could be a huge performance hit with JSII. Unless the original jsii manifest could be reused somehow.

But it appears that separately, there's a problem with lifting std.Type instances from preflight to inflight. Namely, calling toInflight() sometimes blows the stack frame.
But as you can imagine, this means s1._toInflight() will end up calling s2._toInflight() which will then call s1._toInflight() and so on.

@Chriscbr Maybe this is too far, but what if the std.Type API were changed so that all access to child type information is done through a preflight method instead of a property? iirc this is how the Type class in C# reflection works https://learn.microsoft.com/en-us/dotnet/api/system.type?view=net-8.0 (perhaps due to similar recursion problems?)

There's a hit to ergonomics, but I think it avoids the eager/cyclic lifting

struct S1 {
  child: S2?;
}

struct S2 {
 child: S1?;
}

let s1 = @type(S1);
let s2 = s1.getField("child");
let allFields = s1.getFields();

inflight () => {
  log(s1.kind); // should print "struct"
  log(s2.kind); // should print "struct"
}

[monadabot]

Console preview environment is available at https://wing-console-pr-7151.fly.dev 🚀

Last Updated (UTC) 2024-09-27 20:40

[monadabot]

Benchmarks

Comparison to Baseline ⬜🟥⬜⬜⬜⬜⬜⬜⬜🟥⬜⬜⬜

Benchmark	Before	After	Change
version	57ms±0.61	57ms±0.59	+1ms (+1.11%)⬜
hello_world.test.w -t sim	421ms±4.31	439ms±24.57	+19ms (+4.4%)🟥
hello_world.test.w -t tf-aws	1535ms±14.86	1519ms±9.52	-16ms (-1.03%)⬜
jsii_small.test.w -t sim	390ms±5.33	396ms±13.6	+6ms (+1.47%)⬜
jsii_small.test.w -t tf-aws	625ms±9.79	631ms±6.32	+6ms (+1.02%)⬜
jsii_big.test.w -t sim	3049ms±19.68	3075ms±22.1	+25ms (+0.83%)⬜
jsii_big.test.w -t tf-aws	3236ms±10.67	3256ms±22.84	+21ms (+0.64%)⬜
empty.test.w -t sim	382ms±4.28	374ms±5.36	-7ms (-1.96%)⬜
empty.test.w -t tf-aws	614ms±3.4	611ms±4.08	-3ms (-0.54%)⬜
functions_10.test.w -t sim	484ms±15.59	510ms±12.85	+26ms (+5.47%)🟥
functions_10.test.w -t tf-aws	2224ms±16.73	2234ms±12.46	+10ms (+0.44%)⬜
functions_1.test.w -t sim	415ms±5.03	414ms±3.33	-2ms (-0.4%)⬜
functions_1.test.w -t tf-aws	865ms±3.87	857ms±4.12	-8ms (-0.91%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results

name	mean	min	max	moe	sd
version	57ms	57ms	59ms	1ms	1ms
hello_world.test.w -t sim	439ms	406ms	516ms	25ms	34ms
hello_world.test.w -t tf-aws	1519ms	1502ms	1545ms	10ms	13ms
jsii_small.test.w -t sim	396ms	386ms	448ms	14ms	19ms
jsii_small.test.w -t tf-aws	631ms	617ms	646ms	6ms	9ms
jsii_big.test.w -t sim	3075ms	3021ms	3117ms	22ms	31ms
jsii_big.test.w -t tf-aws	3256ms	3220ms	3315ms	23ms	32ms
empty.test.w -t sim	374ms	362ms	384ms	5ms	7ms
empty.test.w -t tf-aws	611ms	602ms	619ms	4ms	6ms
functions_10.test.w -t sim	510ms	467ms	532ms	13ms	18ms
functions_10.test.w -t tf-aws	2234ms	2207ms	2271ms	12ms	17ms
functions_1.test.w -t sim	414ms	408ms	425ms	3ms	5ms
functions_1.test.w -t tf-aws	857ms	848ms	864ms	4ms	6ms

Last Updated (UTC) 2024-09-27 20:45

[mergify]

Thanks for contributing, @Chriscbr! This PR will now be added to the merge queue, or immediately merged if rybickic/type-intrinsic is up-to-date with main and the queue is empty.

[monadabot]

Congrats! 🚀 This was released in Wing 0.85.18.