Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace Proxool with HikariCP #618

Merged
merged 7 commits into from
Dec 6, 2023
Merged

Replace Proxool with HikariCP #618

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
399156a
Replace Proxool with HikariCP
the-thing Feb 27, 2023
f040b78
Update quickfixj-core/src/main/doc/usermanual/usage/configuration.html
the-thing Mar 3, 2023
256476c
Update quickfixj-core/src/main/doc/usermanual/usage/configuration.html
the-thing Mar 3, 2023
707bb5c
Update quickfixj-core/src/main/doc/usermanual/usage/configuration.html
the-thing Mar 3, 2023
9545fb6
Update quickfixj-core/src/main/java/quickfix/JdbcSetting.java
the-thing Mar 3, 2023
d0e6391
Merge branch 'master' into 614/replace_proxool_with_hikari_cp
the-thing Nov 2, 2023
980929b
Merge branch 'master' into 614/replace_proxool_with_hikari_cp
chrjohn Dec 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quickfixj-all/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@
com.sleepycat*;resolution:=optional,
org.apache.maven*;resolution:=optional,
org.codehaus.plexus*;resolution:=optional,
org.logicalcobwebs.proxool*;resolution:=optional,
com.zaxxer*;resolution:=optional,
org.dom4j*;resolution:=optional,
*
</Import-Package>
Expand Down
46 changes: 3 additions & 43 deletions quickfixj-core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,50 +66,10 @@
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>

<dependency>
<groupId>com.cloudhopper.proxool</groupId>
<artifactId>proxool</artifactId>
<version>0.9.1</version>
<optional>true</optional>
<exclusions>
<!-- bug in proxool pom - should be test scope -->
<exclusion>
<groupId>avalon-framework</groupId>
<artifactId>avalon-framework-api</artifactId>
</exclusion>
<!-- we use slf4j which provides this API -->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.cloudhopper.proxool</groupId>
<artifactId>proxool-cglib</artifactId>
<version>0.9.1</version>
<optional>true</optional>
<exclusions>
<!-- bug in proxool pom - should be test scope -->
<exclusion>
<groupId>avalon-framework</groupId>
<artifactId>avalon-framework-api</artifactId>
</exclusion>
<!-- we use slf4j which provides this API -->
<exclusion>
<groupId>commons-logging</groupId>
<artifactId>commons-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<!-- slf4j facade for JCL which is required by proxool -->
<groupId>org.slf4j</groupId>
<artifactId>jcl-over-slf4j</artifactId>
<version>${slf4j.version}</version>
<scope>runtime</scope>
<optional>true</optional>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
<version>4.0.3</version>
</dependency>
<dependency>
<groupId>com.sleepycat</groupId>
Expand Down
8 changes: 1 addition & 7 deletions quickfixj-core/src/main/doc/usermanual/installation.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,16 +81,10 @@ <h4>Optional run-time libraries:</h4>
use Log4J logging.</td>
</tr>
<tr>
<td nowrap="nowrap">proxool.jar</td>
<td nowrap="nowrap">HikariCP.jar</td>
<td>This JAR provided database connection pooling capabilities. It is required
if you are using the JDBC store or log.</td>
</tr>
<tr>
<td nowrap="nowrap">jcl104-over-slf4j.jar</td>
<td>Adapts Jakarta Commons Logging to SLF4J. Required if you are using an optional
library that depends on Jakarta Commons Logging. Currently, this includes Proxool
(needed by JDBC store and log for connection pooling).</td>
</tr>
<tr>
<td nowrap="nowrap">sleepycat-je.jar</td>
<td>Needed if the SleepyCat JE message store is used.</td>
Expand Down
76 changes: 53 additions & 23 deletions quickfixj-core/src/main/doc/usermanual/usage/configuration.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -970,7 +970,7 @@ <H3>QuickFIX Settings</H3>
<TD><I>JdbcDriver</I></TD>
<TD>JDBC driver for JDBC logger. Also used for JDBC log.</TD>
<TD>Class name for the JDBC driver. Specify driver properties directly will cause the
creation of a Proxool data source that supports connection pooling. If you are using a
creation of a HikarCP data source that supports connection pooling. If you are using a
the-thing marked this conversation as resolved.
Show resolved Hide resolved
database with it's own pooling data source (e.g., Oracle) then use the <code>setDataSource()</code>
the-thing marked this conversation as resolved.
Show resolved Hide resolved
method on the Jdbc-related factories to set the data source directly.</TD>
<TD>&nbsp;</TD>
Expand Down Expand Up @@ -1031,39 +1031,69 @@ <H3>QuickFIX Settings</H3>
<TD>Any nonempty string.</TD>
<TD>"" (empty string)</TD>
</TR>

<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcMaxActiveConnection</I></TD>
<TD>Specifies the maximum number of connections to the database.</TD>
<TD>Any number</TD>
<TD>Positive number</TD>
<TD>32</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcMaxActiveTime</I></TD>
<TD>Specifies if the housekeeper comes across a thread that has been active for longer than
this (milliseconds) then it will kill it. So make sure you set this to a number bigger than your
slowest expected response!</TD>
<TD>Any number</TD>
<TD>5000</TD>
<TD><I>JdbcMinIdleConnection</I></TD>
<TD>Controls the minimum number of idle connections that HikariCP tries to maintain in
the pool, including both idle and in-use connections. If the idle connections dip
below this value, HikariCP will make the best effort to restore them quickly and
efficiently.
</TD>
<TD>[0, JdbcMaxActiveConnection]</TD>
<TD>Same as JdbcMaxActiveConnection</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcMaxConnectionLifeTime</I></TD>
<TD>Specifies the maximum amount of time that a connection exists for before
it is killed (milliseconds).</TD>
<TD>Any number</TD>
<TD>28800000</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcSimultaneousBuildThrottle</I></TD>
<TD>Specifies the maximum number of connections we can be building at any one time.
That is, the number of new connections that have been requested but aren't yet
available for use. Because connections can be built using more than one thread
(for instance, when they are built on demand) and it takes a finite time between
deciding to build the connection and it becoming available we need some way of
ensuring that a lot of threads don't all decide to build a connection at once.
(We could solve this in a smarter way - and indeed we will one day)</TD>
<TD>Any number</TD>
<TD>32</TD>
<TD>Positive</TD>
<TD>28800000 ms (8 hours)</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcConnectionTimeout</I></TD>
<TD>Set the maximum number of milliseconds that a client will wait for a connection from the
pool. If this time is exceeded without a connection becoming available, a SQLException
the-thing marked this conversation as resolved.
Show resolved Hide resolved
will be thrown from javax.sql.DataSource.getConnection().</TD>
<TD>Non-negative number</TD>
<TD>250 ms</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcConnectionIdleTimeout</I></TD>
<TD>Controls the maximum amount of time that a connection is allowed to sit idle in the pool.
Whether a connection is retired as idle or not is subject to a maximum variation of +30 seconds, and
average variation of +15 seconds. A connection will never be retired as idle before this timeout.
A value of 0 means that idle connections are never removed from the pool.</TD>
<TD>Non-negative number</TD>
<TD>600000 ms (10 minutes)</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcConnectionKeepaliveTime</I></TD>
<TD>Controls the keepalive interval for a connection in the pool. An in-use connection will never be
tested by the keepalive thread, only when it is idle will it be tested.</TD>
<TD>Non-negative number</TD>
<TD>0 ms</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcConnectionKeepaliveTime</I></TD>
<TD>Controls the keepalive interval for a connection in the pool. An in-use connection will never be
tested by the keepalive thread, only when it is idle will it be tested.</TD>
<TD>Non-negative number</TD>
<TD>0 ms</TD>
</TR>
<TR ALIGN="left" VALIGN="middle">
<TD><I>JdbcConnectionTestQuery</I></TD>
<TD>Set the SQL query to be executed to test the validity of connections. Using the JDBC4
Connection.isValid() method to test connection validity can be more efficient on some
databases and is recommended. If your driver supports JDBC4 we strongly recommend not
setting this property.</TD>
<TD>Valid SQL query</TD>
<TD></TD>
</TR>
</TR>

<TR ALIGN="center" VALIGN="middle">
Expand Down
6 changes: 3 additions & 3 deletions quickfixj-core/src/main/java/quickfix/JdbcLog.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ class JdbcLog extends AbstractLog {
private final Map<String, String> deleteItemsSqlCache = new HashMap<>();

public JdbcLog(SessionSettings settings, SessionID sessionID, DataSource ds)
throws SQLException, ClassNotFoundException, ConfigError, FieldConvertError {
throws SQLException, ConfigError, FieldConvertError {
this.sessionID = sessionID;
dataSource = ds == null
? JdbcUtil.getDataSource(settings, sessionID)
Expand Down Expand Up @@ -109,8 +109,8 @@ private void createCachedSql() {
}

private void createInsertItemSql(String tableName) {
insertItemSqlCache.put(tableName, "INSERT INTO " + tableName + " (time, "
+ getIDColumns(extendedSessionIdSupported) + ", text) " + "VALUES (?,"
insertItemSqlCache.put(tableName, "INSERT INTO " + tableName + " (time,"
+ getIDColumns(extendedSessionIdSupported) + ",text) " + "VALUES (?,"
+ getIDPlaceholders(extendedSessionIdSupported) + ",?)");
}

Expand Down
70 changes: 47 additions & 23 deletions quickfixj-core/src/main/java/quickfix/JdbcSetting.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@

package quickfix;

import javax.sql.DataSource;

/**
* Class for storing JDBC setting constants shared by both the log and message
* store classes.
Expand Down Expand Up @@ -103,40 +105,62 @@ public class JdbcSetting {
public static final String SETTING_JDBC_SESSION_ID_DEFAULT_PROPERTY_VALUE = "JdbcSessionIdDefaultPropertyValue";

/**
* Specifies the maximum number of connections to the database
*
* @see <a href="http://proxool.sourceforge.net/properties.html">http://proxool.sourceforge.net/properties.html</a>
* Controls the maximum size that the pool is allowed to reach, including both idle and in-use connections.
* Basically this value will determine the maximum number of actual connections to the database backend.
* A reasonable value for this is best determined by your execution environment. When the pool reaches this size,
* and no idle connections are available, calls to {@link DataSource#getConnection()} will block for up to
* {@link JdbcSetting#SETTING_JDBC_CONNECTION_TIMEOUT} milliseconds before timing out.
*/
public static final String SETTING_JDBC_MAX_ACTIVE_CONNECTION = "JdbcMaxActiveConnection";

/**
* Specifies if the housekeeper comes across a thread that has been active for longer than
* this then it will kill it. So make sure you set this to a number bigger than your
* slowest expected response!
*
* @see <a href="http://proxool.sourceforge.net/properties.html">http://proxool.sourceforge.net/properties.html</a>
* Controls the minimum number of idle connections that HikariCP tries to maintain in the pool.
* If the idle connections dip below this value and total connections in the pool are less than
* {@link JdbcSetting#SETTING_JDBC_MAX_ACTIVE_CONNECTION}, HikariCP will make the best effort to add
* additional connections quickly and efficiently. However, for maximum performance and responsiveness
* to spike demands, we recommend not setting this value and instead allowing HikariCP to act as a fixed
* size connection pool.
*/
public static final String SETTING_JDBC_MAX_ACTIVE_TIME = "JdbcMaxActiveTime";
public static final String SETTING_JDBC_MIN_IDLE_CONNECTION = "JdbcMinIdleConnection";

/**
* Specifies the maximum amount of time that a connection exists for before
* it is killed (milliseconds).
*
* @see <a href="http://proxool.sourceforge.net/properties.html">http://proxool.sourceforge.net/properties.html</a>
* Controls the maximum lifetime of a connection in the pool. An in-use connection will never be retired, only when
* it is closed will it then be removed. On a connection-by-connection basis, minor negative attenuation is applied to
* avoid mass-extinction in the pool. We strongly recommend setting this value, and it should be several seconds shorter
* than any database or infrastructure imposed connection time limit. A value of 0 indicates no maximum lifetime (infinite
* lifetime), subject of course to the {@link JdbcSetting#SETTING_JDBC_CONNECTION_IDLE_TIMEOUT} setting.
*/
public static final String SETTING_JDBC_MAX_CONNECTION_LIFETIME = "JdbcMaxConnectionLifeTime";

/**
* Specifies the maximum number of connections we can be building at any one time.
* That is, the number of new connections that have been requested but aren't yet
* available for use. Because connections can be built using more than one thread
* (for instance, when they are built on demand) and it takes a finite time between
* deciding to build the connection and it becoming available we need some way of
* ensuring that a lot of threads don't all decide to build a connection at once.
* (We could solve this in a smarter way - and indeed we will one day)
*
* @see <a href="http://proxool.sourceforge.net/properties.html">http://proxool.sourceforge.net/properties.html</a>
* Controls the maximum number of milliseconds that a client (that's you) will wait for a connection from the pool. If this time
* is exceeded without a connection becoming available, a SQLException will be thrown. Lowest acceptable connection timeout is 250 ms.
the-thing marked this conversation as resolved.
Show resolved Hide resolved
*/
public static final String SETTING_JDBC_SIMULTANEOUS_BUILD_THROTTLE = "JdbcSimultaneousBuildThrottle";
public static final String SETTING_JDBC_CONNECTION_TIMEOUT = "JdbcConnectionTimeout";

/**
* Controls the maximum amount of time that a connection is allowed to sit idle in the pool.
* This setting only applies when {@link JdbcSetting#SETTING_JDBC_MIN_IDLE_CONNECTION} is defined to be less than
* {@link JdbcSetting#SETTING_JDBC_MAX_ACTIVE_CONNECTION}. Idle connections will not be retired once the pool
* reaches {@link JdbcSetting#SETTING_JDBC_MIN_IDLE_CONNECTION} connections.
*/
public static final String SETTING_JDBC_CONNECTION_IDLE_TIMEOUT = "JdbcConnectionIdleTimeout";

/**
* Controls how frequently HikariCP will attempt to keep a connection alive, in order to prevent it from being timed out by the
* database or network infrastructure. This value must be less than the {@link JdbcSetting#SETTING_JDBC_MAX_CONNECTION_LIFETIME} value.
* A "keepalive" will only occur on an idle connection. When the time arrives for a "keepalive" against a given connection, that connection
* will be removed from the pool, "pinged", and then returned to the pool. The 'ping' is one of either: invocation of the JDBC4
* {@link java.sql.Connection#isValid(int)} method, or execution of the {@link JdbcSetting#SETTING_JDBC_CONNECTION_TEST_QUERY}. Typically, the duration
* out-of-the-pool should be measured in single digit milliseconds or even sub-millisecond, and therefore should have little or no noticeable
* performance impact. The minimum allowed value is 30000ms (30 seconds), but a value in the range of minutes is most desirable.
*/
public static final String SETTING_JDBC_CONNECTION_KEEPALIVE_TIME = "JdbcConnectionKeepaliveTime";

/**
* If your driver supports JDBC4 we strongly recommend not setting this property. This is for "legacy" drivers that do not support the
* JDBC4 {@link java.sql.Connection#isValid(int)} API. This is the query that will be executed just before a connection is given to you
* from the pool to validate that the connection to the database is still alive.
*/
public static final String SETTING_JDBC_CONNECTION_TEST_QUERY = "JdbcConnectionTestQuery";
}
Loading