[gear] Change deadlock logs from warning to info (#14251)

As discussed in #14240, we emit warnings on database deadlocks, which
there are enough of to trigger noisy alerts. Since there's nothing to be
done operationally (and there's no current work underway to get rid of
them), these alerts only contribute to alert fatigue and hide potential
problems in the system that could be addressed. This demotes a deadlock
to the `info` level so we can still see how often they occur but are not
alerted by them. In the future when we resolve the current deadlock we
can re-escalate this error so that we can catch new deadlocks that are
introduced.

gear/gear/database.py

@@ -28,6 +28,7 @@
 # 2003 - Can't connect to MySQL server on ...
 # 2013 - Lost connection to MySQL server during query ([Errno 104] Connection reset by peer)
 operational_error_retry_codes = (1040, 1213, 2003, 2013)
+operational_error_log_level = {1213: logging.INFO}
 # 1205 - Lock wait timeout exceeded; try restarting transaction
 internal_error_retry_codes = (1205,)
 
@@ -54,7 +55,8 @@ async def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
                     raise
             except pymysql.err.OperationalError as e:
                 if e.args[0] in operational_error_retry_codes:
-                    log.warning(
+                    log.log(
+                        operational_error_log_level.get(e.args[0], logging.WARNING),
                         f'encountered pymysql error, retrying {e}',
                         exc_info=True,
                         extra={'full_stacktrace': '\n'.join(traceback.format_stack())},