feat: resolve relations with lateral joins

[Weakky]

Overview

• closes https://github.com/prisma/team-orm/issues/450
• closes https://github.com/prisma/team-orm/issues/453
• closes https://github.com/prisma/team-orm/issues/452
• closes https://github.com/prisma/team-orm/issues/451

This PR enables the QueryEngine to resolve relations using LATERAL JOINs. This feature is currently only available on Postgres & CockroachDB and under the relationJoins preview feature flags. To minimize damage, we have, as much as possible, created new code paths for this new feature.

To minimize data duplicates traditionally caused by joins, we're using JSON aggregations. Here's what a query may look like on a one2m relation:

model user {
  userId Int     @id @default(autoincrement()) @map("user_id")
  email    String?

  posts post[]
}

model post {
  postId     Int       @id @default(autoincrement()) @map("post_id")
  title String?
  createdAt DateTime? @map("created_at")

  userId Int?  @map("user_id")
  user   user? @relation(fields: [userId], references: [userId])
}

{
  findManyuser(where: { email: "[email protected]" }) {
    email
    posts(take: 10, orderBy: { createdAt: desc }) {
      title
    }
  }
}

SELECT
  "t1"."user_id",  -- selected scalar
  "t1"."email", -- selected scalar
  "user_posts"."__prisma_data__" AS "posts" -- selected relation aggregated as JSON
FROM
  "a_join_repro"."user" AS "t1"
  LEFT JOIN LATERAL (
    SELECT
      COALESCE(JSON_AGG("__prisma_data__"), '[]') AS "__prisma_data__" -- JSON aggregation
    FROM
      (
        SELECT
          "t4"."__prisma_data__" -- nested query to enable ordering and pagination to work
        FROM
          (
            SELECT
              JSON_BUILD_OBJECT('title', "t3"."title") AS "__prisma_data__", -- Actual JSON object building per records
              "t3"."created_at" -- this field is queried so that it can be ordered by on the outer query
            FROM
              (
                SELECT
                  "t2".*
                FROM
                  "a_join_repro"."post" AS "t2"
                WHERE
                  "t1"."user_id" = "t2"."user_id" -- most inner select, where the table is joined with the parent
              ) AS "t3" /* root select */
          ) AS "t4" /* inner select */
        ORDER BY
          "t4"."created_at" DESC -- relation ordering
        LIMIT 10 -- relation pagination
      ) AS "t5" /* middle select */
  ) AS "user_posts" ON true /* outer select */
WHERE
  "t1"."email" = '[email protected]' -- top-level filter

This is roughly achieved thanks to the addition/update of some core abstractions:

1. FieldSelection: A Relation variant is added. Previously, relations were always collected as separate ReadQuery. Please note that, for now, separate read queries are still collected even when relationJoins is enabled.
2. RelationLoadStrategy { Query, Join }: A new enum that let's the core and the connectors decide how to load relations. While I would very much like to keep the way relations are loaded entirely to the connectors' responsibility, we, unfortunately, have to handle results differently. So for now, the core passes down to the connectors how to resolve relations, and the result is handled according to that strategy.
3. RecordSelectionWithRelation: A new type of RecordSelection (eventually passed to the serializer) was added to serialize data differently (given that relations are now part of the ManyRecords when joins are used, as opposed to before where relations data were independently stored as results of nested RelatedRecordsQuery)

[codspeed-hq]

CodSpeed Performance Report

Merging #4509 will degrade performances by 8.26%

_{⚠️ No base runs were found}

_{Falling back to comparing integration/join-support (d6d5262) with main (0d724e3)}

Summary

❌ 1 regressions
✅ 10 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	main	integration/join-support	Change
❌	medium_read	1.3 ms	1.4 ms	-8.26%

[miguelff]

This PR is big enough and outside of my full area of knowledge as for me alone not being able to provide a meaningful review. For that, @Weakky kindly walked me through the code changes and we discussed some of the decisions made.

The conclusion, after that conversation is that the behavior is correct (as expressed by the tests) and that there is no reason to block shipping the Lateral Join strategy for relation fetching as a preview feature.

[miguelff]

We discussed how this could trigger by a user exceeding the number of bind params artificially (f.i. by dynamically composing wrong a query) and we agreed this wouldn't be subject of an issue on our end. As such this message can be misleading and frame as if we did something wrong with the implementation. A more appropriate tactic might be ignoring batching altogether and letting the code reach the DB, bubbling up to the user any potential query parameter limit exceeded.