Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration of SpecializedHandler for Enhanced Archive Processing #1625

Merged
merged 13 commits into from
Aug 15, 2023

Conversation

ahrav
Copy link
Collaborator

@ahrav ahrav commented Aug 15, 2023


This description provides an overview of the changes, the rationale behind them, the impact, and mentions the need for comprehensive testing. It's written in a way that should be accessible to other developers reviewing the PR, giving them the necessary context to understand the changes made.

Description:

This PR introduces the new SpecializedHandler interface to our existing archive handling system, enabling the application to recognize and process specialized archives like Debian (.deb) and RPM (.rpm) packages.

  1. Introduction of SpecializedHandler:
    A new interface SpecializedHandler has been added, which encapsulates the logic to handle specialized archives, including examining files and determining their specialized nature.

  2. Modification to Existing Handler:
    The HandleFile function now utilizes the SpecializedHandler, allowing specialized handling before falling back to regular file type handling. It efficiently leverages both standard and specialized handlers within the existing processing flow.

  3. Support for Debian and RPM Files:
    Implemented methods extractDebContent and extractRpmContent to extract the contents of Debian and RPM files, respectively. These functions utilize command-line tools like 'ar', 'rpm2cpio', and 'cpio'.

  4. Tool Verification and MIME Type Handling:
    A map mimeTools has been defined, mapping MIME types to the corresponding tools needed for handling. Functions isToolInstalled and ensureToolsForMimeType ensure that the required tools are installed.

  5. Optimization for Nested Files:
    Specialized handling for nested files has also been implemented, managing temporary files and directories and recursive handling of nested archives.

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

@ahrav ahrav changed the title Add Ability to Handle .deb Archive Files Add Ability to Handle .deb and .rpm Archive Files Aug 15, 2023
@ahrav ahrav marked this pull request as ready for review August 15, 2023 18:20
@ahrav ahrav requested review from a team as code owners August 15, 2023 18:20
pkg/handlers/handlers.go Outdated Show resolved Hide resolved
pkg/handlers/handlers.go Outdated Show resolved Hide resolved
pkg/sources/filesystem/filesystem.go Outdated Show resolved Hide resolved
pkg/sources/filesystem/filesystem.go Outdated Show resolved Hide resolved
pkg/handlers/handlers.go Outdated Show resolved Hide resolved
@ahrav ahrav requested a review from bill-rich August 15, 2023 20:26
Copy link
Collaborator

@bill-rich bill-rich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Cool use of io.MultiReader to deal with the filetyping using the Reader.

@ahrav
Copy link
Collaborator Author

ahrav commented Aug 15, 2023

Looks good. Cool use of io.MultiReader to deal with the filetyping using the Reader.

I actually learned about it today, I also thought that was a neat addition to the std lib given how often we wanna peek but then use the entire file.

@ahrav ahrav changed the title Add Ability to Handle .deb and .rpm Archive Files Integration of SpecializedHandler for Enhanced Archive Processing Aug 15, 2023
@ahrav ahrav merged commit 6ad5659 into main Aug 15, 2023
7 of 8 checks passed
@ahrav ahrav deleted the add-deb-archive-handler branch August 15, 2023 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants