When use Min Pool Size = 10 app hangs due to connection timeout

[jjavierdguezas]

Steps to reproduce

Use this project as example

MySql.Hang.Test.zip

I am using this Dockerfile:

FROM mcr.microsoft.com/dotnet/aspnet:8.0-jammy AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:8.0-jammy AS build

ARG BUILD_CONFIGURATION=Release

RUN apt update 

COPY . /source

WORKDIR /source/MySql.Hang.Test.Api

RUN dotnet restore

RUN dotnet build "MySql.Hang.Test.Api.csproj" -c $BUILD_CONFIGURATION -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish "MySql.Hang.Test.Api.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false

FROM base AS final

RUN ln -s /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libdl.so
RUN apt-get update && apt-get upgrade -y
RUN apt-get -y install libgdiplus libc6-dev fontconfig libfreetype6 libjpeg-turbo8 libpng16-16 libx11-6 libxcb1 libxext6 libxrender1 xfonts-75dpi xfonts-base xvfb libfontconfig libfontconfig1 libfontconfig1-dev nfs-common
RUN ln -s /usr/lib/libgdiplus.so /lib/x86_64-linux-gnu/libgdiplus.so


WORKDIR /app
COPY --from=publish /app/publish .

ENTRYPOINT ["dotnet", "MySql.Hang.Test.Api.dll"]

and this docker-compose.yml:

version: "3.3"
name: "test-app"

services:

  test-api:
    image: test-api:latest
    container_name: "test-api"
    depends_on:
      - mysql
    ports:
      - "80:80"
      - "443:443"
    environment:
      ASPNETCORE_HTTP_PORTS: 80
      # ConnectionStrings__DefaultConnection: server=mysql,3306;user=root;password=p@ssw0rd.;database=mysql_hang_db;;
      ConnectionStrings__DefaultConnection: server=mysql,3306;user=root;password=p@ssw0rd.;database=mysql_hang_db;Min Pool Size=10
      Logging__LogLevel__Microsoft.EntityFrameworkCore: Debug
      Logging__LogLevel__Microsoft.EntityFrameworkCore.Database.Connection: Debug
      Logging__LogLevel__Microsoft.EntityFrameworkCore.Database.Command: Debug
      Logging__LogLevel__Default: Debug
      Logging__LogLevel__Microsoft.AspNetCore: Debug

  mysql:
    image: mysql:8.0.35
    container_name: "test-db"
    restart: always
    ports:
      - "3306:3306"
    environment:
      MYSQL_ROOT_PASSWORD: p@ssw0rd.
      MYSQL_DATABASE: mysql_hang_db

The issue

On ubuntu 22.04 If I use a connection string like:

"ConnectionStrings": {
    "DefaultConnection": "server=127.0.0.1,3306;user=root;password=p@ssw0rd.;database=mysql_hang_test"
  }

it works but if I include Min Pool Size=10

"ConnectionStrings": {
    "DefaultConnection": "server=127.0.0.1,3306;user=root;password=p@ssw0rd.;database=mysql_hang_test;Min Pool Size=10"
  }

it doesnt. A timeout error is thrown

test-api  | fail: Microsoft.EntityFrameworkCore.Database.Connection[20004]
test-api  |       An error occurred using the connection to database 'mysql_hang_db' on server 'mysql,3306'.
test-api  | Unhandled exception. MySqlConnector.MySqlException (0x80004005): Connect Timeout expired.
test-api  |  ---> MySqlConnector.MySqlException (0x80004005): Connect Timeout expired.
test-api  |    at MySqlConnector.Core.ServerSession.OpenTcpSocketAsync(ConnectionSettings cs, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 1063
test-api  |    at MySqlConnector.Core.ServerSession.ConnectAsync(ConnectionSettings cs, MySqlConnection connection, Int64 startingTimestamp, ILoadBalancer loadBalancer, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ServerSession.cs:line 422
test-api  |    at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 427
test-api  |    at MySqlConnector.Core.ConnectionPool.ConnectSessionAsync(MySqlConnection connection, Action`4 logMessage, Int64 startingTimestamp, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 432
test-api  |    at MySqlConnector.Core.ConnectionPool.CreateMinimumPooledSessions(MySqlConnection connection, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 405
test-api  |    at MySqlConnector.Core.ConnectionPool.GetSessionAsync(MySqlConnection connection, Int64 startingTimestamp, Int32 timeoutMilliseconds, Activity activity, IOBehavior ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/Core/ConnectionPool.cs:line 33
test-api  |    at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1056
test-api  |    at MySqlConnector.MySqlConnection.CreateSessionAsync(ConnectionPool pool, Int64 startingTimestamp, Activity activity, Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 1090
test-api  |    at MySqlConnector.MySqlConnection.OpenAsync(Nullable`1 ioBehavior, CancellationToken cancellationToken) in /_/src/MySqlConnector/MySqlConnection.cs:line 556
test-api  |    at MySqlConnector.MySqlConnection.Open() in /_/src/MySqlConnector/MySqlConnection.cs:line 518
test-api  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenDbConnection(Boolean errorsExpected)
test-api  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.OpenInternal(Boolean errorsExpected)
test-api  |    at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Open(Boolean errorsExpected)
test-api  |    at Pomelo.EntityFrameworkCore.MySql.Storage.Internal.MySqlRelationalConnection.Open(Boolean errorsExpected)
test-api  |    at Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteScalar(RelationalCommandParameterObject parameterObject)
test-api  |    at Microsoft.EntityFrameworkCore.Migrations.HistoryRepository.Exists()
test-api  |    at Microsoft.EntityFrameworkCore.Migrations.Internal.Migrator.Migrate(String targetMigration)
test-api  |    at Microsoft.EntityFrameworkCore.RelationalDatabaseFacadeExtensions.Migrate(DatabaseFacade databaseFacade)
test-api  |    at MySql.Hang.Test.Data.AppDbContext..ctor(DbContextOptions`1 options) in /source/MySql.Hang.Test.Data/AppDbContext.cs:line 18
test-api  |    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
test-api  |    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
test-api  |    at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite callSite, RuntimeResolverContext context)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass2_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(ServiceIdentifier serviceIdentifier, ServiceProviderEngineScope serviceProviderEngineScope)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
test-api  |    at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
test-api  |    at MySql.Hang.Test.Api.Program.InitializeDb(IServiceProvider provider) in /source/MySql.Hang.Test.Api/Program.cs:line 85
test-api  |    at MySql.Hang.Test.Api.Program.Main(String[] args) in /source/MySql.Hang.Test.Api/Program.cs:line 73
test-api  |    at MySql.Hang.Test.Api.Program.<Main>(String[] args)

Further technical details

MySQL version: mysql:8.0.35
Operating system: Ubuntu 22.04.2 LTS
Pomelo.EntityFrameworkCore.MySql version: 8.0.2
Microsoft.AspNetCore.App version: 8.0.5

Other details about my project setup:

I reduced the test scenario to an aspnet project (see attached .zip file) with a method that inserts data into the db when starting the app and a mysql all in a docker container

// ...
            await InitializeDb(((IApplicationBuilder)app).ApplicationServices);

            app.Run();
        }

        private static async Task InitializeDb(IServiceProvider provider)
        {
            using var scope = provider.GetService<IServiceScopeFactory>().CreateScope();
            var serviceProvider = scope.ServiceProvider;

            var dbContextFactory = serviceProvider.GetService<IDbContextFactory<AppDbContext>>();
            using var db = await dbContextFactory.CreateDbContextAsync();
            //using var db = serviceProvider.GetService<AppDbContext>();
            //await db.Database.MigrateAsync();


            var userInfos = new List<UserInfo>(1000);

            for (int i = 0; i < 1000; i++)
            {
                var name = $"User {i:000}";

                if (!await db.Set<UserInfo>().AnyAsync(x => x.Name == name))
                {
                    userInfos.Add(new UserInfo() { Id = Guid.NewGuid(), Name = name, CreatedOn = DateTime.Now });
                }
            }

            if (userInfos.Count > 0)
            {
                await db.AddRangeAsync(userInfos);
                await db.SaveChangesAsync();
            }

        }
    }

--
PS: I dont really know if this is related to Pomelo, mysql o what.. so any help is welcome. I test this exact docker compose on windows and it works I run the attached project from visual studio on windows and a mysql on docker desktop and it works. A coworker tested it on windows wsl and it works too. it seems related to linux/ubuntu

[mcuervoe]

I have the same issue. It looks like it is related to where the app is running.

• In Windows 11 Pro + WSL Ubuntu 22.04 + docker desktop it works without problems
• In Windows 11 Pro + VM Ubuntu 22.04 + docker, it works without problems
• In bare metal Ubuntu 22.04 + docker, it fails
• In bare metal Ubuntu 22.04, running locally without docker, it fails
• In AWS EC2 Ubuntu 22.04 or Amazon Linux 2023 + docker, it fails
• In AWS ECS Fargate it fails

The same behavior is observed with MySQL 5.7, MySQL 8, Aurora DB, and MariaDB

I also tried with MYSQL running directly on the operating system, on a docker image, AWS RDS, and I got the same behavior

This issue is blocking the adoption of the .NET solution in our project