Skip to content
@ivdatahub

IV - Data Hub

Building Data and Software Engineering solutions
README.md

IV - Data Hub

ivdatahub is the central repository for software engineering projects focused on end-to-end data solutions, covering data manipulation, processing, and analysis.

Our Technology Stack

About me

IV - Data Hub focuses on building software engineering solutions for data, from API development to complex data pipelines. Our projects cover the entire data lifecycle, from ingestion to visualization, leveraging modern technologies like Google Cloud Platform, BigQuery, Apache AirFlow, Docker, and more.

Our data projects go beyond simple automation, aiming for performance and scalability to create robust and innovative solutions.

Key Areas of Focus:

  • Data Engineering
  • Data Pipelines
  • Automation and Orchestration
  • Data Modeling and Analysis
  • Data-Centric APIs

Feel free to explore our repositories and contribute to the evolution of our data solutions.

Pinned Loading

  1. data-consumer-api data-consumer-api Public

    ETL Process for Currency Quotes Data" project is a complete solution dedicated to extracting, transforming and loading (ETL) currency quote data. This project uses several advanced techniques and a…

    Python 1 1

  2. data-producer-api data-producer-api Public

    FastAPI application for sending data to Pub/Sub, used for load testing and triggering pipelines

    Python 1

  3. api-to-dataframe api-to-dataframe Public

    Python library that simplifies obtaining data from API endpoints by converting them directly into Pandas DataFrames. This library offers robust features, including retry strategies for failed reque…

    Python

  4. currency-quote currency-quote Public

    Complete solution for extracting currency pair quotes data with comprehensive testing, parameter validation, flexible configuration management, Hexagonal Architecture, CI/CD pipelines, code quality…

    Python

  5. pypi-package-stats pypi-package-stats Public

    Project for ingest pypi packages data from BigQuery and send to DataDog for analysis and insights with dashboards, monitors and more

    Python

  6. data-consumer-pipeline data-consumer-pipeline Public

    Pipeline for processing and consuming streaming data from Pub/Sub, integrating with Dataflow for real-time data processing

    Python

Repositories

Showing 7 of 7 repositories
  • pypi-package-stats Public

    Project for ingest pypi packages data from BigQuery and send to DataDog for analysis and insights with dashboards, monitors and more

    ivdatahub/pypi-package-stats’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Oct 4, 2024
  • data-producer-api Public

    FastAPI application for sending data to Pub/Sub, used for load testing and triggering pipelines

    ivdatahub/data-producer-api’s past year of commit activity
    Python 1 MIT 0 0 0 Updated Oct 2, 2024
  • currency-quote Public

    Complete solution for extracting currency pair quotes data with comprehensive testing, parameter validation, flexible configuration management, Hexagonal Architecture, CI/CD pipelines, code quality tools, and detailed documentation.

    ivdatahub/currency-quote’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Oct 2, 2024
  • api-to-dataframe Public

    Python library that simplifies obtaining data from API endpoints by converting them directly into Pandas DataFrames. This library offers robust features, including retry strategies for failed requests.

    ivdatahub/api-to-dataframe’s past year of commit activity
    Python 0 MIT 0 0 0 Updated Oct 2, 2024
  • data-consumer-api Public

    ETL Process for Currency Quotes Data" project is a complete solution dedicated to extracting, transforming and loading (ETL) currency quote data. This project uses several advanced techniques and architectures to ensure the efficiency and robustness of the ETL process.

    ivdatahub/data-consumer-api’s past year of commit activity
    Python 1 MIT 1 1 0 Updated Oct 2, 2024
  • data-consumer-pipeline Public

    Pipeline for processing and consuming streaming data from Pub/Sub, integrating with Dataflow for real-time data processing

    ivdatahub/data-consumer-pipeline’s past year of commit activity
    Python 0 MIT 0 1 0 Updated Sep 20, 2024
  • .github Public
    ivdatahub/.github’s past year of commit activity
    0 MIT 0 0 0 Updated Sep 18, 2024

Top languages

Python

Most used topics

Loading…