Feature: Enable Multiple Connections

[carneijp]

Driver can now open and handle multiple connections to MongoDB.

Description

We have changed the ⁠ Cluster.swift, Connection.swift ⁠, ⁠ Connection+Execute.swift ⁠ and ⁠ IsMaster.swift ⁠ in order to utilize MongoKitten's already existing multiple connections set up which wasn't fully implemented.

IsMaster:

• Created a new variable called isHandshake: Bool. It's used to determine whether the connection is doing a handshake and thus decide whether it can be reused.
• Included the new variable in the init with a default value of false.

⁠ Connection:

• ⁠Added ⁠ isInUse ⁠ boolean flag, as a way to know if the connection is busy or available.
• Included timeout configuration, based on timeout variable from ⁠ ConnectionSettings ⁠.
• Function ⁠ doHandshake ⁠ now uses new constructor of ⁠ IsMaster ⁠ class.

⁠ Connection+Execute ⁠:

• command: Document argument on function execute now includes the isHandshake flag (it will be cleaned up after).
• ⁠Function ⁠ execute ⁠ now checks if it's a handshake connection.
  • If it is false (it isn't a handshake connection), the connection is marked as not available (connection's isInUse variable is set to true).
  • If it is true(it is a handshake connection), the connection will remain available to use by another query.
  • Then it will remove the flag from the command variable. (We didn't want to change execute's signature as to not provoke any unexpected behaviour).
• ⁠Function execute sets the connection's completion (result.whenComplete) to set the connection's isInUse flag back to false respecting the isHandshake condition (if it was a handshake, it won't set the flag to true as it didn't set it to false before).

Cluster:

• Created support variables for the connection pool:
  • poolBalancerCounter: NIOAtomic<Int64> - Used to distribute queries throughout the connections in the pool.
  • poolRequestedCounter: NIOAtomic<Int8> - Used to control how many connections were requested to be created and are still alive in order to create connections respecting the ConnectionSettings's maximumNumberOfConnections variable.
  • In pool: [PooledConnection] variable we configured it's didSet property observer to subtract from poolRequestedCounter when pool.count is smaller than oldValue (when a connection is removed from the pool, the counter updates).
• Function findMatchingConnection now has a new logic:
  • If the number of connections to be created or alive (poolRequestedCounter) is less than the settings' maximumNumberOfConnections:
    - If there is an available connection, it will return it.
    - If there is no available connection, it will return nil and increment poolRequestedCounter.
  • Else it will return a connection from the pool respecting a "queue" (we store the poolBalancerCounter to determine what was the last connection to receive a query and then try the next one).
• Function _getConnection changed to create connections for hosts that have already been created (currently we resort to first creating connections to all undiscovered hosts and if there are none, we use the first that is stored in the discoveredHosts list. Ideally, it should be a queue, but that's out of scope for these changes).

Motivation and Context

We are building an app and needed an API. We have some experience with other API frameworks in other languages (SpringBoot with Java and .NET with C#) and were able to identify some bottlenecks with our API when connecting to MongoDB using Fluent (which has the Fluent-Mongo-Driver that depends on MongoKitten).

The bottleneck:

Our API was slowing down waiting for queries to complete and when we looked the performance tab in Mongo Compass it showed that there was only one connection being made. We searched for this issue and found people saying to increase the maximumNumberOfConnections when creating Fluent's DatabaseConfigurationFactory and it didn't increased our connections.

We started to read all Fluent's abstractions and it seemed ok, it was passing the "request" for new connections all the way down.
After a while we arrived on MongoKitten and read through everything until we understood that MongoKitten had everything set up to handle multiple connections, but it wasn't being used for some reason. We changed a few pieces of code (only 4 files) and now we are creating connections as needed. Our problem was solved and we hope we can help other people who may face this problem as well.

How Has This Been Tested?

We validated the increase on throughput capability:

• Checking Mongo Compass' performance tab it listed all the connections we created.
• Our API got significantly faster.

We tested edge cases:

• Disconnecting the database didn't affect the connection's lifetime in unexpected ways.
• When restarting the DB process, the connections were recreated successfully and the counter updated along with it.
• The counter reflected the connections spawning and dying accordingly.

Checklist:

•⁠ ⁠[ ] If applicable, I have updated the documentation accordingly.
•⁠ ⁠[ ] If applicable, I have added tests to cover my changes.

Contributors:

Myself
Gabriel M

Our thanks

Rafael and Marina who gave us many ideas and helped us with hard concepts we were unfamiliar with.

[Joannis]

Hello @carneijp , thanks for the PR! I really appreciate the thought and effort that went into this, but I don't think this PR is appropriate for MongoKitten, but should rather be targeting the MongoDB Fluent driver.

MongoDB and MongoKitten can run many concurrent requests over a single connection thanks to MongoDB's wire protocol supporting multiplexing. Most SQL databases, and in fact none of the SQL databases I've seen so far, support multiplexing.

This likely means that Fluent is trying to artificially limit the concurrent queries on a single MongoDB connection, which isn't necessary. So I'd recommend looking at lifting this limit in the MongoDB Fluent driver instead.

If you'd like, let's hop on a chat through one of the various Discord servers, or we can set up a call so I can walk you through it all.

[Joannis]

I'm closing this for now. Feel free to re-open the discussion

[tayloraswift]

fwiw, multiplexing is explicitly disallowed according to the MongoDB connection pooling specification

Single Track: A Connection MUST limit itself to one request / response at a time. A Connection MUST NOT multiplex/pipeline requests to an Endpoint.