Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Os::FileSystem and Os::Directory #2871

Merged
merged 45 commits into from
Sep 27, 2024
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
bf11671
Add FileSystem Interface
thomas-bc Aug 16, 2024
c70ced0
Add OS::FileSystem interface and implementations
thomas-bc Aug 16, 2024
eea790f
static interface + global Os::init()
thomas-bc Aug 20, 2024
34c9f38
Add test harness
thomas-bc Aug 20, 2024
2cdca89
Implement Os::Directory interfaces and use it in FileSystem::readDire…
thomas-bc Aug 20, 2024
e1bce3c
Add Stub tests for Directory and FileSystem
thomas-bc Aug 22, 2024
12a655a
wip: modify the interface from Michael feedback
thomas-bc Aug 29, 2024
8e11926
wip: move functions around and rework interface a little more
thomas-bc Aug 30, 2024
8b6899a
Add Directory rule-based tests
thomas-bc Aug 30, 2024
af0223a
Refactored Directory for open-create + WIP: FileSystem tests
thomas-bc Sep 9, 2024
645a771
Update state tracking mechanism in FileSystem test implementation + a…
thomas-bc Sep 10, 2024
8f2c5ba
Replace bounded while-loop with for-loop
thomas-bc Sep 10, 2024
e7e8383
Code cleanup
thomas-bc Sep 10, 2024
d89a27f
fix typo
thomas-bc Sep 10, 2024
a25d748
Rename FileTracker and code reorganization / clarification
thomas-bc Sep 11, 2024
96894f1
Code style and add CWD rule
thomas-bc Sep 11, 2024
709b4ff
Clean up tester code
thomas-bc Sep 12, 2024
1b4afc0
More cleanup
thomas-bc Sep 12, 2024
5f9da79
Add Posix tests to Os_ut_exe
thomas-bc Sep 12, 2024
7ae9db4
Fix DpCatalog readDirectory usage
thomas-bc Sep 13, 2024
dd29323
Appease CI checks
thomas-bc Sep 13, 2024
3929031
Spelling and CI warnings
thomas-bc Sep 13, 2024
930aa97
Spelling + removing FppConstants include
thomas-bc Sep 13, 2024
a0aba85
Add review changes and todos
thomas-bc Sep 16, 2024
aa5592b
Add Directory::read(Fw::StringBase&) overload
thomas-bc Sep 16, 2024
520beb6
Refactor copyFileData to iterate over copied size
thomas-bc Sep 17, 2024
ae7edf2
Remove FppConstants.hpp include
thomas-bc Sep 17, 2024
815a27a
Rework moveFile into rename/copy-remove
thomas-bc Sep 17, 2024
7f54573
Add majestic comments
thomas-bc Sep 17, 2024
d2c2e59
Add option to error if dir exist in Fs::createDirectory
thomas-bc Sep 18, 2024
32be989
spelling
thomas-bc Sep 18, 2024
eef6499
include <algorithm> for std::find in tests
thomas-bc Sep 18, 2024
2518010
Fix leftover ::CREATE modes in comments and stub tests
thomas-bc Sep 18, 2024
f202ec4
Fix CmdSequencer test case
thomas-bc Sep 18, 2024
ac14b33
Fix memory leak in testing by conditioning the Open rule
thomas-bc Sep 18, 2024
d62b5d6
Fix FileManager tests
thomas-bc Sep 18, 2024
c8eed05
Add Directory and FileSystem FPP type definitions; update CMakeLists …
thomas-bc Sep 18, 2024
f83d57f
spelling
thomas-bc Sep 18, 2024
6aa7b75
Reoder header guards
thomas-bc Sep 19, 2024
6301a45
Implement changes from peer-review
thomas-bc Sep 26, 2024
fe1efa9
read into StringBase with const_cast
thomas-bc Sep 26, 2024
96cda71
Fix resetSer() call
thomas-bc Sep 26, 2024
6da1d39
Fix spelling
thomas-bc Sep 26, 2024
63f9955
fix spelling again i guess
thomas-bc Sep 26, 2024
9c348b3
Fix DpCatalog unchecked return value
thomas-bc Sep 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 17 additions & 8 deletions Os/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,12 @@ set(SOURCE_FILES
"${CMAKE_CURRENT_LIST_DIR}/ValidatedFile.cpp"

# Refactored common files
"${CMAKE_CURRENT_LIST_DIR}/Os.cpp"
"${CMAKE_CURRENT_LIST_DIR}/File.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Task.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Mutex.cpp"
"${CMAKE_CURRENT_LIST_DIR}/FileSystem.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Directory.cpp"
)
# Check for default logger
if (NOT FPRIME_DISABLE_DEFAULT_LOGGER)
Expand All @@ -56,8 +59,6 @@ if (FPRIME_USE_POSIX)
"${CMAKE_CURRENT_LIST_DIR}/Linux/InterruptLock.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Linux/WatchdogTimer.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Posix/IntervalTimer.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Linux/FileSystem.cpp"
"${CMAKE_CURRENT_LIST_DIR}/Linux/Directory.cpp"
)
endif()
# Darwin IPC queue implementation
Expand Down Expand Up @@ -97,26 +98,34 @@ endif()
register_fprime_module()

require_fprime_implementation(Os/File)
require_fprime_implementation(Os/FileSystem)
require_fprime_implementation(Os/Directory)
require_fprime_implementation(Os/Mutex)
# require_fprime_implementation(Os/Task) # should be added in
require_fprime_implementation(Os/Mutex)
# require_fprime_implementation(Os/Console) # should be added in


### UTS ### Note: 3 separate UTs registered here.
set(UT_SOURCE_FILES
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsQueueTest.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsTestMain.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/IntervalTimerTest.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsFileSystemTest.cpp"
# "${CMAKE_CURRENT_LIST_DIR}/test/ut/OsFileSystemTest.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsValidateFileTest.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsSystemResourcesTest.cpp"
"${CMAKE_CURRENT_LIST_DIR}/test/ut/OsMutexBasicLockableTest.cpp"
)
register_fprime_ut()
if (BUILD_TESTING)
foreach (TEST IN ITEMS StubFileTest PosixFileTest StubMutexTest PosixMutexTest) # add? : StubTaskTest PosixTaskTest
if (TARGET "${TEST}")
add_dependencies("Os_ut_exe" "${TEST}")
endif()
foreach (TEST IN ITEMS
thomas-bc marked this conversation as resolved.
Show resolved Hide resolved
StubFileTest PosixFileTest
StubMutexTest PosixMutexTest
StubDirectoryTest PosixDirectory
StubFileSystemTest PosixFileSystemTest
) # add? : StubTaskTest PosixTaskTest
if (TARGET "${TEST}")
add_dependencies("Os_ut_exe" "${TEST}")
endif()
endforeach()
endif()
# Second UT Pthreads
Expand Down
126 changes: 126 additions & 0 deletions Os/Directory.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
// ======================================================================
// \title Os/Directory.cpp
// \brief common function implementation for Os::Directory
// ======================================================================
#include <Fw/Types/Assert.hpp>
#include <Os/Directory.hpp>

namespace Os {

Directory::Directory() : m_handle_storage(), m_delegate(*DirectoryInterface::getDelegate(m_handle_storage)) {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
}

Directory::~Directory() {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
m_delegate.~DirectoryInterface();
}

DirectoryHandle* Directory::getHandle() {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
return this->m_delegate.getHandle();
}

Directory::Status Directory::open(const char* path, OpenMode mode) {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
return this->m_delegate.open(path, mode);
}

bool Directory::isOpen() {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
return this->m_delegate.isOpen();
}
Directory::Status Directory::rewind() {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
if (!this->m_delegate.isOpen()) {
return Status::NOT_OPENED;
}
return this->m_delegate.rewind();
}

Directory::Status Directory::read(char * fileNameBuffer, U32 bufSize) {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
if (!this->m_delegate.isOpen()) {
return Status::NOT_OPENED;
}
return this->m_delegate.read(fileNameBuffer, bufSize);
}

void Directory::close() {
FW_ASSERT(&this->m_delegate == reinterpret_cast<DirectoryInterface*>(&this->m_handle_storage[0]));
return this->m_delegate.close();
}

// ------------ Common Directory Functions (non-OS-specific) ------------

Directory::Status Directory::getFileCount(FwSizeType& fileCount) {
Fixed Show fixed Hide fixed
Fixed Show fixed Hide fixed
Dismissed Show dismissed Hide dismissed
if (this->isOpen() == false) {
return Status::NOT_OPENED;
}
// REVIEW NOTE: should getFileCount rewind before counting?
// Likely yes?? Otherwise it's just weird
// TODO: check return value (how to handle if not OP_OK?)
this->rewind();

const U32 loopLimit = std::numeric_limits<U32>::max();
FwSizeType count = 0;
FwSizeType unusedBufferSize = 1;
char unusedBuffer[unusedBufferSize];
Status readStatus = Status::OP_OK;
for (U32 iter = 0; iter < loopLimit; ++iter) {
readStatus = this->read(unusedBuffer, unusedBufferSize);
if (readStatus == Status::NO_MORE_FILES) {
break;
} else if (readStatus != Status::OP_OK) {
return Status::OTHER_ERROR;
}
++count;
}
fileCount = count;
// and after??
this->rewind();
return Status::OP_OK;
}


Directory::Status Directory::readDirectory(Fw::String filenameArray[], const FwSizeType filenameArraySize, FwSizeType& filenameCount) {

Check notice

Code scanning / CodeQL

Long function without assertion Note

All functions of more than 10 lines should have at least one assertion.

Check notice

Code scanning / CodeQL

No raw arrays in interfaces Note

Raw arrays should not be used in interfaces. A container class should be used instead.
FW_ASSERT(filenameArray != nullptr);
FW_ASSERT(filenameArraySize > 0);
if (this->isOpen() == false) {
return Status::NOT_OPENED;
}
// same thing here - should we rewind before?
this->rewind();

Status readStatus = Status::OP_OK;
Status returnStatus = Status::OP_OK;

FwIndexType index;
constexpr FwIndexType loopLimit = std::numeric_limits<FwIndexType>::max();

char fileName[FPP_CONFIG_FILENAME_MAX_SIZE]; // should be FW_FIXED_LENGTH_STRING_SIZE instead??

for (index = 0; ((index < loopLimit) && (index < static_cast<FwIndexType>(filenameArraySize))); index++) {
readStatus = this->read(fileName, FPP_CONFIG_FILENAME_MAX_SIZE);
if (readStatus == Status::NO_MORE_FILES) {
break;
} else if (readStatus != Status::OP_OK) {
return Status::OTHER_ERROR;
}

filenameArray[index] = Fw::String(fileName);
}

filenameCount = index;

if (index == loopLimit) {
returnStatus = Status::FILE_LIMIT;
}
this->rewind();

return returnStatus;

}


} // namespace Os
Fixed Show fixed Hide fixed
134 changes: 106 additions & 28 deletions Os/Directory.hpp
Original file line number Diff line number Diff line change
@@ -1,44 +1,122 @@
#ifndef _Directory_hpp_
#define _Directory_hpp_
#ifndef _OS_DIRECTORY_HPP_
#define _OS_DIRECTORY_HPP_

#include <FpConfig.hpp>
#include <Os/Os.hpp>
#include <config/FppConstantsAc.hpp>
#include <Fw/Types/String.hpp>

namespace Os {

// This class encapsulates a very simple directory interface that has the most often-used features
struct DirectoryHandle {};

class Directory {
public:
class DirectoryInterface {
public:
static constexpr FwSizeType FPP_CONFIG_FILENAME_MAX_SIZE = FppConstant_FileNameStringSize::FileNameStringSize;
typedef enum {
OP_OK, //!< Operation was successful
DOESNT_EXIST, //!< Directory doesn't exist
NO_PERMISSION, //!< No permission to read directory
NOT_OPENED, //!< Directory hasn't been opened yet
NOT_DIR, //!< Path is not a directory
NO_MORE_FILES, //!< Directory stream has no more files
FILE_LIMIT, //!< Directory has more files than can be read
BAD_DESCRIPTOR, //!< Directory stream descriptor is invalid
NOT_SUPPORTED, //!< Operation is not supported by the current implementation
OTHER_ERROR, //!< A catch-all for other errors. Have to look in implementation-specific code
} Status;

typedef enum {
OP_OK, //!< Operation was successful
DOESNT_EXIST, //!< Directory doesn't exist
NO_PERMISSION, //!< No permission to read directory
NOT_OPENED, //!< Directory hasn't been opened yet
NOT_DIR, //!< Path is not a directory
NO_MORE_FILES, //!< Directory stream has no more files
OTHER_ERROR, //!< A catch-all for other errors. Have to look in implementation-specific code
} Status;
typedef enum {
READ, //!< Error if directory doesn't exist
CREATE, //!< Create directory if it doesn't exist
} OpenMode;

Directory(); //!< Constructor
virtual ~Directory(); //!< Destructor. Will close directory if still open
Status open(const char* dirName); //!< open directory. Directory must already exist
bool isOpen(); //!< check if file descriptor is open or not.
Status rewind(); //!< rewind directory stream to the beginning
//! \brief default constructor
DirectoryInterface() = default;

Check notice

Code scanning / CodeQL

More than one statement per line Note

This line contains 2 statements; only one is allowed.
Status read(char * fileNameBuffer, U32 bufSize); //!< get next filename from directory
Status read(char * fileNameBuffer, U32 bufSize, I64& inode); //!< get next filename and inode from directory
void close(); //!< close directory
//! \brief default virtual destructor
virtual ~DirectoryInterface() = default;

NATIVE_INT_TYPE getLastError(); //!< read back last error code (typically errno)
const char* getLastErrorString(); //!< get a string of the last error (typically from strerror)
//! \brief copy constructor is forbidden
DirectoryInterface(const DirectoryInterface& other) = delete;

private:
//! \brief assignment operator is forbidden
DirectoryInterface& operator=(const DirectoryInterface& other) = delete;

POINTER_CAST m_dir; //!< Stored directory pointer (type varies across implementations)
NATIVE_INT_TYPE m_lastError; //!< stores last error
//! \brief return the underlying Directory handle (implementation specific)
//! \return internal Directory handle representation
virtual DirectoryHandle* getHandle() = 0;

};
//! \brief provide a pointer to a Directory delegate object
static DirectoryInterface* getDelegate(HandleStorage& aligned_new_memory);


//------------ Os-specific Directory Functions ------------

virtual Status open(const char* path, OpenMode mode) = 0; //!< open/create a directory
virtual bool isOpen() = 0; //!< check if file descriptor is open or not.
virtual Status rewind() = 0; //!< rewind directory stream to the beginning
virtual Status read(char * fileNameBuffer, U32 bufSize) = 0; //!< get next filename from directory
virtual void close() = 0; //!< close directory

};

class Directory final : public DirectoryInterface {
public:
Directory(); //!< Constructor (private because singleton pattern)
~Directory() final; //!< Destructor

//! \brief return the underlying Directory handle (implementation specific)
//! \return internal Directory handle representation
DirectoryHandle* getHandle() override;

//------------ Os-specific Directory Functions ------------

//! \brief Open or create a directory
//! \param path: path of directory to open
//! \param mode: enum (READ, CREATE). READ will return an error if directory doesn't exist
//! \return status of the operation
Status open(const char* path, OpenMode mode) override;

//! \brief Check if Directory is open or not
//! \return true if Directory is open, false otherwise
bool isOpen() override;

//! \brief Rewind directory stream to the beginning
//! \return status of the operation
Status rewind() override;

//! \brief Get next filename from directory stream
//! \param fileNameBuffer: buffer to store filename
//! \param bufSize: size of fileNameBuffer
//! \return status of the operation
Status read(char * fileNameBuffer, U32 bufSize) override;

//! \brief Close directory
void close() override;

// ------------ Common Directory Functions (non-OS-specific) ------------

//! \brief Read the contents of the directory and store filenames in filenameArray of size arraySize
//! \param filenameArray: array to store filenames
//! \param arraySize: size of filenameArray
//! \param filenameCount: number of filenames written to filenameArray (output)
//! \return status of the operation
Status readDirectory(Fw::String filenameArray[], const FwSizeType arraySize, FwSizeType& filenameCount);

//! \brief Get the number of files in the directory
//! \param fileCount: number of files in the directory (output)
//! \return status of the operation
Status getFileCount(FwSizeType& fileCount);

private:
// This section is used to store the implementation-defined Directory handle. To Os::Directory and fprime, this type is
// opaque and thus normal allocation cannot be done. Instead, we allow the implementor to store then handle in
// the byte-array here and set `handle` to that address for storage.
//
alignas(FW_HANDLE_ALIGNMENT) HandleStorage m_handle_storage; //!< Directory handle storage
DirectoryInterface& m_delegate;
};

}

Expand Down
Loading
Loading