Version 0.10.3

• Fixed an issue where the Database unnecessarily loaded data from disk when performing a read for a key in a record after a previous read for the entire record made the desired data available in memory.
• Fixed a minor bug that caused the Database to create unnecessary temporary directories when performing a reindex.
• The Criteria builder now creates a NavigationKeySymbol for navigation keys instead of a KeySymbol.
• Fixed a bug that caused Convert#stringToJava to throw an NumberFormatException when trying to convert numeric strings that appeared to be numbers outside of Java's representation range. As a result of this fix, those kinds of values will remain as strings.
• Added a ForwardingConcourse wrapper that can be extended by subclasses that provide additional functionality around a subset of Concourse methods.
• Fixed a bug that prevent a custom ConnectionPool using a custom Concourse instance (e.g. one that extends ForwardingConcourse) from returning a connection of the correct class. As a result of this change, the ConnectionPool constructors that accept explicit Concourse connection parameters have been deprecated in favor of one that takes a Supplier of Concourse connections.
• Fixed a bug that caused TObject#compareTo to return logically inconsistent results relative to TObject#equals. Previously, comparing TObjects with type STRING occurred in a case insensitive manner whereas the equals evaluation was case sensitive. Now, the compareTo method is case sensitive.
• Added the ability to compare TObjects in a case insensitive manner.
• Fixed a bug that made it possible for storage engine to return inaccurate results for REGEX and NOT_REGEX queries if matching values had different case formats.
• Fixed a bug that caused historical queries to incorrectly return logically different results compared to present state queries if matching values had different case formats.
• Fixed a bug that made it possible for reads within the Buffer to cause write lock starvation and resource exhaustion; preventing any further writes from occurring and generating a backlog of reads that never terminated.

Version 0.10.2

• Fixed a bug that caused an error to be thrown when creating a Criteria containing a navigation key using the Criteria#parse factory.
• Added an option to limit the length of substrings that are indexed for fulltext search. It is rare to add functionality in a patch release, but this improvement was needed to alleviate Concourse deployments that experience OutOfMemory exceptions because abnormally large String values are stuck in the Buffer waiting to be indexed. This option is turned off by default to maintain consistency with existing Concourse expectations for fulltext indexing. The option can be enabled by specifying a positive integer value for the newly added max_search_substring_length preference. When a value is supplied, the Storage Engine won't index any substrings longer than the provided value for any word in a String value.
• Made internal improvements to the search indexing algorithm to reduce the number of itermediary objects created which should decrease the number of garbage collection cycles that are triggered.
• Fixed a bug that caused the Engine to fail to accumulate metadata stats for fulltext search indices. This did not have any data correctness or consistency side-effects, but had the potential to make some search-related operations inefficient.
• Fixed a bug that made it possible for the Database to appear to lose data when starting up after a crash or unexpected shutdown. This happened because the Database ignored data in Block files that erroneously appeared to be duplicates. We've fixed this issue by improving the logic and workflow that the Database uses to test whether Block files contain duplicate data.
• Fixed a regression introduced in version 0.10.0 that made it possible for the Database to ignore errors that occurred when indexing writes. When indexing errors occur, the Database creates log entries and stops the indexing process, which is the behaviour that existed prior to version 0.10.0.
• Fixed a bug that made it possible for fatal indexing errors to occur when at least two writes were written to the same Buffer page for the same key with values that have different types (i.e. Float vs Integer) but are essentially equal (i.e. 18.0 vs 18). In accordance with Concourse's weak typing system, values that are essentially the same will be properly indexed and queryable across associated types. This change requires a reindex of all block files which is automatically done when upgrading from a previous version.
• Fixed a bug that causes the ManagedConcourseServer in the concourse-ete-test-core framework to take longer than necessary to allow connections in ClientServerTest cases.

Version 0.10.1

• Fixed a regression that caused an error when attempting an action with a CCL statement containing an unquoted string value with one or more periods.

Version 0.10.0

BREAKING CHANGES

There is only PARTIAL COMPATIBILITY between

• an 0.10.0+ client and an older server, and
• a 0.10.0+ server and an older client.

Due to changes in Concourse's internal APIs,

• Older client will receive an error when trying to invoke any navigate or calculation methods on a 0.10.0+ server.
• Older servers will throw an error message when any navigate or calculation methods are invoked from an 0.10.0+ client.

New Features

Sorting

Concourse Server now (finally) has the ability to sort results!

• A result set that returns data from multiple records (across any number of keys) can be sorted.
• Concourse drivers now feature an Order framework that allows the client to specify how Concourse Server should sort a result set. An Order can be generated using an intuitive builder chain.
• An Order can be specified using values for any keys; regardless of whether those keys are explictly fetched or not.
• A timestamp can optionally be associated with each order component.
  • NOTE: If no timestamp is specified with an order component, Concourse Server uses the value that is explicitly fetched in the request OR it looks up the value using the selection timestamp of the request. If there is no selection timestamp, the lookup returns the current value(s).
• A direction (ascending or descending) can optionally be associated with each order component.
  • NOTE: The default direction is ascending.
• An Order can contain an unlimited number of components. If, after evaluating all the order components, two records still have the same sort order, Concourse Server automatically sorts based on the record id.
• If a null value is encountered when evaluating an order components, Concourse Server pushes the record containing that null value to the "end" of the result set regardless of the order component's direction.

Pagination

Concourse Server now (finally) has the ability to page through results!

• A result set that returns data from multiple records (across any number of keys) can be paginated.
• Concourse drviers now feature a Page object that allows the client to specify how Concourse Server should paginate a result set. A Page is an abstraction for offset/skip and limit parameters. The Page class contains various factory methods to offset and limit a result set using various intuitive notions of pagination.

Navigable Data Reads

• You can now traverse the document graph by specifying one ore more navigation keys in the following read methods:
  • browse
  • calculate
  • get
  • select
• Reading a navigation key using get or select is intended to repleace the navigate methods.
• When reading a navigation key in the context of one or more records, the root record (e.g the record from which the document-graph traversal starts) is mapped to the values that are retrieved from the destination records. In the navgiate methods, the destination record is associated with the destination value(s).
  • For example, assume record 1 is linked to record 2 on the friends key. Record 2 contains the value Jeff for the name key...
  • if you select("friends.name", 1), the return value will map 1 to [Jeff] whereas the return value of navigate("friends.name", 1) maps 2 to [Jeff].

Navigable Criteria

• You can now use navigation keys in Criteria objects or ccl statements that are passed to the find, get and select methods.

ETL

• Added the com.cinchapi.concourse.etl package that contains data processing utilities:
  • A Strainer can be used to process a Map<String, Object> using Concourse's data model rules. In particular, the Strainer encapsulates logic to break down top-level sequence values and process their elements individually.
  • The Transform class contains functions for common data transformations.

Miscellaneous

• Added an iterative connection builder that is accessible using the Concourse.at() static factory method.
• Added the com.cinchapi.concourse.valididate.Keys utility class which contains the #isWritable method that determines if a proposed key can be written to Concourse.
• Added Parsers#create static factory methods that accept a Criteria object as a parameter. These new methods compliment existing ones which take a CCL String and TCriteria object respectively.
• Upgraded the ccl dependency to the latest version, which adds support for local criteria evaluation using the Parser#evaluate method. The parsers returned from the Parsers#create factories all support local evaluation using the function defined in the newly created Operators#evaluate utility.
• Added support for remote debugging of Concourse Server. Remote debugging can be enabled by specifying a remote_debugger_port in concourse.prefs.
• The calculate methods now throw an UnsupportedOperationException instead of a vague RuntimeException when trying to run calculations that aren't permitted (i.e. running an aggregation on a non-numeric index).

Improvements

• Refactored the concourse-import framework to take advantage of version 1.1.0+ of the data-transform-api which has a more flexible notion of data transformations. As a result of this change, the Importables utility class has been removed. Custom importers that extend DelimitedLineImporter can leverage the protected parseObject and importLines methods to hook into the extraction and import logic in a manner similar to what was possible using the Importables functions.
• Refactored the Criteria class into an interface that is implemented by any language symbols that can be immediately transformed to a well-built criteria (e.g. ValueState and TimestampState). The primary benefit of this change is that methods that took a generic Object parameter and checked whether that object could be built into a Criteria have now been removed from the Concourse driver since that logic is automatically captured within the new class hiearchy. Another positive side effect of this change is that it is no longer necessary to explicitly build a nested Criteria when using the group functionality of the Criteria builder.
• Improved the performance of querying certain Concourse Server methods from a plugin via ConcourseRuntime by eliminating unnecessary server-side calculations meant to facilitate analytics. These calculations will continue to be computed when the plugin receives the data from ConcourseRuntime.

Bug Fixes

• Fixed a bug that caused data imported from STDIN to not have a __datasource tag, even if the --annotate-data-source flag was included with the CLI invocation.
• Fixed a bug that allowed Concourse Server to start an environment's storage engine in a partially or wholly unreadable state if the Engine partially completed a block sync while Concourse Server was going through its shutdown routine. In this scenario, the partially written block is malformed and should not be processed by the Engine since the data contained in the malformed block is still contained in the Buffer. While the malformed block files can be safely deleted, the implemented fix causes the Engine to simply ignore them if they are encountered upon initialization.
• Added checks to ensure that a storage Engine cannot transport writes from the Buffer to the Database while Concourse Server is shutting down.
• Fixed a bug that allow methods annotated as PluginRestricted to be invoked if those methods were defined in an ancestor class or interface of the invokved plugin.
• Fixed a bug introduced by JEF-245 that caused Concourse Server to fail to start on recent versions of Java 8 and Java 9+ due to an exploitation of a JVM bug that was used to allow Concourse Server to specify native thread prioritization when started by a non-root user. As a result of this fix, Concourse Server will only try to specify native thread prioritization when started by a root user.

Deprecated and Removed Features

• Removed the Strings utility class in favor of AnyStrings from accent4j.
• Removed the StringSplitter framework in favor of the same from accent4j.
• Deprecated Criteria#getCclString in favor of Criteria#ccl.
• The navigate methods in the client drivers have been deprecated in favor of using select/get to traverse the document-graph.

Version 0.9.6

• Fixed a bug that caused a ParseException to be thrown when trying to use a Criteria object containing a string value wrapped in single or double quotes out of necessity (i.e. because the value contained a keyword). This bug happened because the wrapping quotes were dropped by Concourse Server when parsing the Criteria.
• Fixed a bug where the CCL parser failed to handle some Unicode quote characters.

Version 0.9.5

• Fixed a bug where some of the ManagedConcourseServer#get methods in the concourse-ete-test-core package called the wrong upstream method of the Concourse Server instance under management. This had the effect of causing a runtime ClassCastException when trying to use those methods.
• Fixed a bug that caused ambiguity and erroneous dispatching for query methods (e.g. #get, #navigate and #select) that have different signatures containing a long (record) or generic Object (criteria) parameter in the same position. This caused issues when a Long was provided to the method that expected a long because the dispatcher would route the request that expected an Object (criteria) parameter.
• Fixed a bug in the concourse-ete-test-core framework where Timestamp objects returned from the managed server were not translated to Timestamp type from the test application's classpath.

Version 0.9.4

• Context has been added exceptions thrown from the v2 ccl parser which makes it easier to identify what statements are causing issues.

Version 0.9.3

• Fixed a bug that caused a NullPointerException to be thrown when trying to set configuration data in .prefs files.

Version 0.9.2

• Deprecated the forGenericObject, forCollection, forMap and forTObject TypeAdapter generators in the TypeAdapters utility class in favor of primitiveTypesFactor, collectionFactory and tObjectFactory in the same class, each of which return a TypeAdapterFactory instead of a TypeAdapter. Going forward, please register these type adapter factories when building a Gson instance for correct Concourse-style JSON serialization semantics.
• Upgraded to ccl version 2.4.1 to capture fix for an bug that caused both the v1 and v2 parsers to mishandle numeric String and Tag values. These values were treated as numbers instead of their actual type. This made it possible for queries containing those values to return inaccurate results.
• Added the associated key to the error message of the Exception that is thrown when attempting to store a blank value.
• Fixed a regression that caused programmatic configuration updates to no longer persist to the underlying file.
• The Timestamp data type now implements the Comparable interface.

Version 0.9.1

Enhancements

• Upgraded client and server configuration management and added support for incremental configuration overrides. Now
  • Server preferences defined in conf/concourse.prefs can be individually overriden in a conf/concourse.prefs.dev file (previously override preferences in the .dev file had to be done all or nothing).
  • Both server and client preferences can be individually overriden using environment variables that are capitalized and prefixed with CONCOURSE_. For example, you can override the heap_size preference with an environment variable named CONCOURSE_HEAP_SIZE.
• Added Docker image https://hub.docker.com/r/cinchapi/concourse. See https://docs.cinchapi.com/concourse/quickstart for more information.

Bug Fixes

• Fixed a bug that caused Concourse to improperly interpret values written in scientific notation as a STRING data type instead of the appropriate number data type.
• Fixed a bug in the logic that Concourse Server used to determine if the system's data directories were inconsistent. Because of the bug, it was possible to put Concourse in an inconsistent state from which it could not automatically recover by changing either the buffer_directory or database_directory and restarting Concourse Server. The logic has been fixed, so Concourse now accurately determines when the data directories are inconsistent and prevents the system from starting.
• Fixed a bug that caused an exception to be thrown when trying to jsonify a dataset that contained a Timestamp.