Add helper method for platform independent file path building #467
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c-schuler
requested changes
Oct 6, 2023
There was a problem hiding this comment.
Thank you for using the Apache commons library!
One thing I would recommend is to make the code cleaner by adding a utility function that takes a list/array of strings and concatenates them.
For example:
public static String concat(String basePath, String ... filesToAdd)
Then call it like so:
String outputFilePath = IOUtils.concat(outputPath, "input", "resources", "library");
Does that make sense?
Chris0296
changed the title
Chris0296
changed the title
-This uses FilenameUtils.concat for underlying path concatenation
JPercival
pushed a commit
that referenced
this pull request
Nov 28, 2023
* Initial lift - added annotation pattern for operations and some base tests * Adding Bundle operations, tests, and docs * Adding dateroller operation, tests, and docs * Adding accelerator kit operations and docs * Adding library generation operation, tests, and docs - some cleanup for other operations * Adding extract MAT bundle operation, tests, and docs * Cleaned up utils, added descriptions to operation params, enabled help menu for operations, and int/bool type mapping for operations * Added terminology generation tooling, tests, and docs * Added LibraryRefresh operation, tests, and documents. Stubbed out RefreshIG operation * Added DataProfileConformance operation, tests, and documentation. Updated dependencies * Updated dependencies, logic, and tests - added StructureMap tooling and tests * Replace use of System.out/System.err with SLF4J Logging (#465) -Clean up log message formatting -Operation outputs have been unchanged as these should be visible via CLI * Add helper method for platform independent file path building (#467) Add helper method to build platform independent file paths -This uses FilenameUtils.concat for underlying path concatenation Co-authored-by: c-schuler <[email protected]> * Remove comments which are inconsistent with actual dictionary & remove unused entry (#472) * Added UI for LOINC hierarchy valueset generation tooling * Prep for PR * Update javafx version * Update spring-boot version * Update translator dependency versions --------- Co-authored-by: Chris0296 <[email protected]>
JPercival
pushed a commit
that referenced
this pull request
Nov 29, 2023
* Initial lift - added annotation pattern for operations and some base tests * Adding Bundle operations, tests, and docs * Adding dateroller operation, tests, and docs * Adding accelerator kit operations and docs * Adding library generation operation, tests, and docs - some cleanup for other operations * Adding extract MAT bundle operation, tests, and docs * Cleaned up utils, added descriptions to operation params, enabled help menu for operations, and int/bool type mapping for operations * Added terminology generation tooling, tests, and docs * Added LibraryRefresh operation, tests, and documents. Stubbed out RefreshIG operation * Added DataProfileConformance operation, tests, and documentation. Updated dependencies * Added processing for transaction bundle test files during post * Using Mockito.mockStatic to capture HttpClientUtils.post calls. Test now includes fhir server entry in arguments. * Updated dependencies, logic, and tests - added StructureMap tooling and tests * Executor approach, take 2 * Replace use of System.out/System.err with SLF4J Logging (#465) -Clean up log message formatting -Operation outputs have been unchanged as these should be visible via CLI * Add helper method for platform independent file path building (#467) Add helper method to build platform independent file paths -This uses FilenameUtils.concat for underlying path concatenation Co-authored-by: c-schuler <[email protected]> * Remove comments which are inconsistent with actual dictionary & remove unused entry (#472) * Introduced caching where paths are same * Reverting IGProcessor & IGBundleProcessor * Removed system out calls * Reverting change * Reverting change * Added UI for LOINC hierarchy valueset generation tooling * Added some better caching and test directory handling. Updated HttpClientUtils to collect POST calls and execute them at the end. * Added custom ConcurrentLinkedHashMap and ConcurrentLinkedHashSet classes for thread safety. Declaring certain variables outside of loops for stability. * Reduced performance issues within for loops, using StringBuilder where needed. * Processing POST calls sequentially instead of in paralell, for security reasons (ddos false flag) * Restoring ResourceUtils valueset handling to older straightforward approach. * Restoring ResourceUtils * Found good balance in post settings to not overload cqf-ruler * Backing out changes to IGProcessor * Reverting some files * Using List as argument instead of ArrayList for compatibility. * Removing unused repository entries, setting firm versions * ConcurrentLinkedHashMap and ConcurrentLinkedHashSet appear to not be needed due to the nature of modifications of each by the process. * Using buffered readers and writers in ResourceUtils and IOUtils for better performance. * More accurate caching for getFilePaths and getDirectoryFilePaths. Adjusted IOUtils.copyFile to return false if the file wasn't copied, true if it was, to accurately track how many files get copied during TestCaseProcessor.bundleTestCaseFiles * Streamlined exception handling during valueset checks, beefed up ExtractMatBundleOperation with multithreading, updated matching test accordingly * Prep for PR * Added ThreadUtils to handle common handling of executing multiple tasks. * Stabilized the ExtractMatBundle process which had varying results for file count due to old approach to eliminating duplicate resource processing * Cleaning up diff view * Reverting LibraryProcessor to sequential processing due to concurrency issues in tests. IGBundleProcessor checking fhirUri for null before reference. RefreshIGOperationTest using wiremock to monitor post calls to mock fhir server. pom.xml updated for wiremock. Will possibly try different libraries for similar approach. * Updating to latest wiremock for security * Removed unused dependencies for RefreshIGOperationTest * Fixed logic behind retrying failed posts * Maven runs all tests sequentially, so member variables within these classes should be reset after the process. cleanUp() added to various classes and called from IGBundleProcessor at the end of the process so subsequent tests start fresh like the app restarted. TODO: Move away from static! * Rolling back BufferedOutput changes as they don't appear to be needed. * Restoring LibraryProcessor multhreading. Attempting to fix tooling/src/test/java/org/opencds/cqf/tooling/operation/StripGeneratedContentOperationTest.java as it is failing in Windows but not linux. * Fixed pom entry for wiremock, abandoning multhreading on LibraryProcessor as it's too unstable at the server during ci checks, and added conditional path processing to tooling/src/test/java/org/opencds/cqf/tooling/operation/StripGeneratedContentOperationTest.java * Broke up process into smaller methods, fixed issue with task focount * using ConcurrentLinkedQueue for LibraryProcessor * Adjusted logic for shouldPersist boolean in LibraryProcessor's bundleLibraryDependenciesTasks. Adjusted error handling for clear information in console when shouldPersist finds an error. * Returning from execute if no tasks exist in ThreadUtils. restoring previous approach to IOUtils.getFilePaths (caching retained) because readability exceeds any performance benefit from Files.walk * Update javafx version * Update spring-boot version * Update translator dependency versions * Refactored code to use a parent abstract class housing common methods to reduce duplication of code * Organizing imports * Organized imports for all touched files * Restoring correct TestCaseProcessor * Restoring IGProcessor * Code cleanup * Fixed tests * Merged from multithreading pr * Moving from LogUtils to logger * Adjusted IOUtils.translate calls that resulted in Exception * Grouping warnings based on library name since these warnings don't exit the loop. * Cleaned up messaging a bit, added warning collection, added CQLTranslatorException giving us the ability to pass back the list of errors as as a Set (to easily avoid duplicate warnings.) * Transient variable assigned to errors list in CQLTranslatorException to avoid warning (and failure in tests.) Fixed typo in moveAndRename * Resolving conflicts. * revert settings.json * Added javadoc to various methods for clarity * Code cleanup with fixed merge of LibraryProcessor * Better reporting of missing values in getDepValueSetResources * Cleaned up some warnings. Restored final keyword in favor of calling clear() on specific variables as there's a slight performance improvment over initializing new instances. cleanUp() is useful because ci tests don't start a new instance of our app with each new test and RefreshIG vs IGProcessor can leave data in memory that throws eachother off. Future tests could see similar issues without this due to static nature of member variables. TODO: Move to instance-based IOUtils. * cleanUp called from RefreshIGOperationTest --------- Co-authored-by: c-schuler <[email protected]> Co-authored-by: Evan Chicoine <[email protected]> Co-authored-by: Chris0296 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Throughout the project "/" is used as a delimiter for file paths. This is specific to Linux/Mac OS. This PR adds a helper method which uses Apache Commons IO FilenameUtils.concat function to build platform independent file paths
Github Issue: #187
By creating this PR you acknowledge that your contribution will be licensed under Apache 2.0