Skip to content

Commit

Permalink
Modification (#26)
Browse files Browse the repository at this point in the history
* Updating AboutMe

* testing

* update

* pics update

* more thinhgs

* minor

* REAL BI
  • Loading branch information
BaharF authored Oct 13, 2023
1 parent 065dd49 commit 8681c01
Show file tree
Hide file tree
Showing 9 changed files with 62 additions and 30 deletions.
14 changes: 10 additions & 4 deletions projects/authService/intro.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,18 @@
---
title: HPCC Auth Service
shortDescription: At its core, Auth-service relies on JWT tokens for robust access control, employing digital signatures and key pair verification to ensure that only authorized users can enter protected areas. Beyond token management, Auth-service offers an intuitive user interface for seamless user administration, enabling organizations to easily manage user accounts and permissions.
title: HPCC Systems® Auth Service
shortDescription: Auth-service is a robust access control system, ensuring secure access to protected areas through digital signatures and key pair verification. It simplifies user administration with an intuitive interface, allowing organizations to effortlessly manage user accounts and permissions, enhancing overall security and access management.
gitHubRepo: https://github.com/hpcc-systems/Auth-Service
imageName: auth-service
---

In the realm of digital security and user authentication, Auth-service shines as a trusted solution, utilized by instances of Tombolo and REAL-BI, and tested with HPCC clusters for user authentication and authorization.
In the realm of digital security and user authentication, Auth-service shines as a trusted solution, utilized by instances of Tombolo and REAL BI (Roxie Enabled Business Intelligence), and tested with HPCC clusters for user authentication and authorization.

At its core, Auth-service relies on JWT tokens for robust access control, employing digital signatures and key pair verification to ensure that only authorized users can enter protected areas. Beyond token management, Auth-service offers an intuitive user interface for seamless user administration, enabling organizations to easily manage user accounts and permissions.

Furthermore, Auth-service extends its capabilities through APIs, allowing developers to integrate token generation and verification seamlessly into their applications. This enhances security and simplifies the authentication process. The adoption of Auth-service by instances of Tombolo and REAL-BI, coupled with its thorough testing with HPCC clusters, underscores its reputation for reliability, security, and user-friendliness. Auth-service is the preferred solution for organizations seeking to strengthen their security measures without compromising operational efficiency, making it a valuable asset in the quest for data protection in today's digital age.
Furthermore, Auth-service extends its capabilities through APIs, allowing developers to integrate token generation and verification seamlessly into their applications. This enhances security and simplifies the authentication process. The adoption of Auth-service by instances of Tombolo and Real BI, coupled with its thorough testing with HPCC clusters, underscores its reputation for reliability, security, and user-friendliness. Auth-service is the preferred solution for organizations seeking to strengthen their security measures without compromising operational efficiency, making it a valuable asset in the quest for data protection in today's digital age.


</br>
For more information and inquiries, please contact us at . <span style="color:blue">[email protected]</span>.


Binary file added projects/images/ClusterUsage.JPG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added projects/images/RealbiMap.JPG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added projects/images/Workflow.JPG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
29 changes: 21 additions & 8 deletions projects/realBi/intro.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,28 @@
---
title: HPCC REAL-BI
shortDescription: Real BI, a dynamic and powerful data tool, plays a pivotal role in bridging the gap between data stored in HPCC clusters and insightful visualizations. This innovative solution provides organizations with the means to harness the wealth of information within their HPCC data repositories effectively.
link: REAL-BI
title: HPCC Systems® REAL BI
shortDescription: REAL BI (Roxie Enabled Business Intelligence), a dynamic and powerful data tool, plays a pivotal role in bridging the gap between data stored in HPCC clusters and insightful visualizations. This innovative solution provides organizations with the means to harness the wealth of information within their HPCC data repositories effectively.
link: real-bi
gitHubRepo: https://github.com/hpcc-systems/REAL-BI
imageName: REAL-BI
imageName: real-bi
---

Real BI, a dynamic and powerful data tool, plays a pivotal role in bridging the gap between data stored in HPCC clusters and insightful visualizations. This innovative solution provides organizations with the means to harness the wealth of information within their HPCC data repositories effectively.
REAL BI (Roxie Enabled Business Intelligence) is a potent data tool that bridges the gap between HPCC Systems clusters and insightful visualizations. It efficiently taps into HPCC Systems's extensive data resources, allowing users to create visualizations that unveil hidden data patterns and insights. With the ability to source data from various HPCC Systems facets, including Roxie queries, logical files, and ECL scripts, REAL BI empowers organizations to tailor their visualizations to specific needs, making data-driven decisions easier and more informed. Ultimately, REAL BI transforms complex data into actionable insights, facilitating smarter decision-making and driving business success.

At its core, Real BI serves as a versatile conduit to connect with HPCC, tapping into its extensive data resources. By doing so, it enables users to create compelling visualizations that unveil hidden patterns, trends, and insights locked within the data. These visualizations serve as a valuable asset for decision-makers, offering a clear and intuitive representation of complex data sets.
Key features of REAL BI include:

One of the standout features of Real BI is its ability to source data from multiple facets of HPCC. Whether it's drawing data from the output of ROXIE queries, logical files, or custom-executed ECL scripts, Real BI seamlessly integrates these data sources into its visualization engine. This flexibility empowers organizations to tailor their visualizations to specific needs, extracting meaningful insights from a variety of data types and structures.
1. **Deep Integration with HPCC Systems:** REAL BI is seamlessly integrated with HPCC Systems clusters, allowing users to access and manipulate data directly without the need to transfer it between environments. This not only ensures data security but also eliminates additional costs associated with data transfer.

2. **Embedded ECL Coding:** ECL developers can leverage the Embedded ECL plugin to perform data manipulations while creating charts. This feature empowers users to design custom data transformations, queries, and reporting logic with precision and efficiency.

3. **Custom Data Transformations:** REAL BI enables users to create custom data transformations, making it easier to adapt data for specific visualization needs. This flexibility is particularly valuable when dealing with diverse data types and structures.

4. **Visual Data Insights:** The platform allows users to turn raw data into clear and intuitive visualizations, helping organizations uncover hidden patterns and trends within their data. These visualizations provide valuable insights for data-driven decision-making.

5. **Streamlined Data Analysis:** By connecting directly to HPCC Systems data clusters, REAL BI simplifies the process of extracting actionable insights from complex data. This streamlining of data analysis supports more informed and efficient decision-making.

</br>
For more information and inquiries, please contact us at . <span style="color:blue">[email protected]</span>.

<img src="../images/RealbiMap.JPG" alt="" title="REAL BI Map" border= "5px solid #191919;"/>
<figcaption>REAL BI Map</figcaption>

In essence, Real BI goes beyond mere data visualization; it transforms raw data into a visual narrative that empowers organizations to make data-driven decisions with confidence. By connecting to HPCC's data clusters and offering a wide array of data sources, Real BI is an indispensable tool for organizations seeking to unlock the true potential of their data resources. It streamlines the process of turning complex data into actionable insights, facilitating smarter decision-making and ultimately driving business success.
43 changes: 28 additions & 15 deletions projects/tombolo/intro.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,41 @@
---
title: HPCC Tombolo
shortDescription: The Tombolo Data Lake Curation System is the first open-source Data Lake Curation system for the HPCC Systems Platform. It allows creation of documentation along with the data and analyses that provides a roadmap into all aspects (assets) of the Data Lake - Data Files, Data Providers and Consumers, Data Ingestion and Analytics, and User Queries. Its global find facility allows users to rapidly locate any asset, or browse hierarchically to get the lay-of-the-land.
title: HPCC Systems® Tombolo
shortDescription: Revolutionizing HPCC Systems Cluster Management with Data Catalog Power. Tombolo serves as an an open-source robust data catalog tool, transforming HPCC Systems cluster management into a seamless experience. This innovative solution is your gateway to effortless workflow creation, asset monitoring, version control, and more, all with a data-centric focus.
link: tombolo
gitHubRepo: https://github.com/hpcc-systems/Tombolo
imageName: tombolo
---

The Tombolo Data Lake Curation System 1.0 is the first open-source Data Lake Curation system for the HPCC Systems Platform. It allows creation of documentation along with the data and analyses that provides a roadmap into all aspects (assets) of the Data Lake: Data Files, Data Providers and Consumers, Data Ingestion and Analytics, and User Queries. Its global find facility allows users to rapidly locate any asset, or browse hierarchically to get the lay-of-the-land.

Tombolo can be used to design a new Data Lake or new portion of an existing Data Lake, or can automatically import information from an existing HPCC Systems Data Lake. In a design capacity, a developer can lay out the files, processes, and queries that will comprise the Data Lake, and later, when the files have been obtained, or analytics have been written, the “design” items can be attached to their real counterparts within the Data Lake. When that occurs, any available information from the Data Lake is automatically imported into Tombolo, and made available as part of the documentation. This information includes the data definitions, processing code, and the relationships between files and processing jobs or queries. In an add-on capacity, Tombolo can attach to an existing Data Lake and import all of the assets directly.

Tombolo provides a graphical way to map the workflows that keep the data lake running. Like the asset management capacity, Workflows can be developed in either a design or add-on function. They also have the ability to import data directly from the Data Lake. When a process (e.g. job) is added to the workflow, any input and output files are also pulled into the workflow and attached with arrows to the job icon. This greatly simplifies the task of keeping workflow diagrams up to date. Pressing the refresh button will automatically refresh the diagram, adding any new files or relations.
The open source [Tombolo Data Lake Curation](https://github.com/hpcc-systems/Tombolo) offers a comprehensive data catalog solution, primarily designed to optimize the functionality and performance of HPCC Systems clusters. This innovative tool simplifies data management for both technical and non-technical users by providing an intuitive web application interface.

Another important Tombolo capability is Automation. This allows workflows to be created from within Tombolo by scheduling job executions. Jobs can be scheduled on a recurring time basis, scheduled upon the completion of another job, or manually executed from within Tombolo. This allows entire workflows to be built from within Tombolo without coding. Once these workflows are built and attached to the Data Lake, Tombolo monitors the execution of each workflow and notifies designated users if there was a failure. This automation capacity is expected to expand further in future releases of Tombolo, ultimately providing a complete Data Engineering workbench for HPCC Systems Data Lakes.
Key features of Tombolo include:

The current version of Tombolo provides limited governance support, allowing tracking of privacy, proprietary, and contractual constraints on the uses of data assets as well as provider / consumer relationship information. Future versions will significantly enhance these abilities, evolving Tombolo to become a full-featured Data Lake Governance system.

Tombolo has built-in multi-tenant support. Different groups of users (tenants) can be given access to different partitions of a Data Lake, or separate Data Lakes. Each Data Lake may encompass multiple HPCC-Systems clusters. Tombolo’s Access Control currently allows three levels of access for each tenant: read-only, read-write, or tenant administrator. Future versions will allow much more sophisticated access control, including constraint-based permissions.
**Key features of Tombolo include:**

## The Tombolo Vision
The vision for Tombolo is to become the central console for Data Lake developers and operators, providing all of the facilities needed for designing, developing, automating, documenting, and governing Data Lakes. The Tombolo vision encompasses a number of capabilities:
1. **Design and Import Data Lakes:** Tombolo empowers users to design new Data Lakes or seamlessly import assets from existing HPCC Systems Data Lakes. Whether creating from scratch or enhancing an established structure, Tombolo simplifies documentation by automatically importing data definitions, processing code, and the intricate relationships between files and processing jobs or queries.

2. **Graphical Workflow Mapping:** Tombolo provides a user-friendly interface for visually mapping workflows, suitable for both design and add-on functions. Workflow diagrams effortlessly import data from the Data Lake, and any added processes or jobs are automatically integrated into the diagram, simplifying the process of keeping diagrams up to date. A single refresh button ensures real-time accuracy.

3. **Automation Capabilities:** Automation within Tombolo enables the creation of workflows through scheduled job executions. Users can schedule jobs to run on a recurring basis, trigger them upon the completion of other jobs, or manually execute them within Tombolo. This feature allows entire workflows to be constructed without extensive coding. Tombolo continuously monitors workflow execution, providing notifications in case of failure, and its automation capacity is poised for further expansion in upcoming releases.

4. **Data Governance Support:** While currently offering limited governance support, Tombolo facilitates tracking privacy, proprietary, and contractual constraints on data asset usage, as well as provider-consumer relationship information. Future iterations are set to elevate Tombolo into a comprehensive Data Lake Governance system, enhancing data management and compliance.

5. **Multi-Tenant Support:** Tombolo boasts built-in multi-tenant support, allowing different user groups (tenants) access to distinct Data Lake partitions or separate Data Lakes. Each Data Lake can encompass multiple HPCC-Systems clusters. Tombolo's Access Control currently provides three levels of access for each tenant: read-only, read-write, or tenant administrator. Future releases promise advanced access control, including constraint-based permissions, catering to organizations of varying complexity and scale.

For more information and inquiries, please contact us at **[email protected]**.


</br>
For more information and inquiries, please contact us at . <span style="color:blue">[email protected]</span>.

### Tombolo Images

<img src="../images/ClusterUsage.JPG" alt="Tombolo Cluster Usage" title="Tombolo Storage Usage" border= "5px solid #191919;"/>
<figcaption>Tombolo Storage Usage</figcaption>

<img src="../images/Workflow.JPG" alt="Tombolo Workflow" title="Tombolo workflow" border= "5px solid #191919;"/>
<figcaption>Tombolo Workflow</figcaption>

- Curation — Tracking, Documenting, and providing visibility into all aspects of the Data Lake.
- Development — Serving as an integration platform for developer tools.
- Automation — Setting up and tracking work-flows for various applications, from Data Ingestion to Customer Delivery
- DevOps — Automating the process of moving data and code from development / QA to Production
- Governance — Tracking of data restrictions (e.g. legislative, contractual, proprietary) at the source and propagating these restrictions to all derived data throughout the life-cycle. Detecting issues requiring governance review and inserting review requirements into the DevOps process.
4 changes: 2 additions & 2 deletions solutionsLab/01_intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
title: hpcc-intro
---

# HPCC Systems
# HPCC Systems®

HPCC Systems is a mature platform that has been heavily used in commercial applications for almost two decades, predating the development of Hadoop. Created by LexisNexis® Risk Solutions, an innovative pioneer in big data processing, and open source for nearly a decade now, HPCC Systems features a vibrant development community that continues to push the boundaries of Big Data.
[HPCC Systems®](https://hpccsystems.com) is a mature platform that has been heavily used in commercial applications for almost two decades, predating the development of Hadoop. Created by LexisNexis® Risk Solutions, an innovative pioneer in big data processing, and open source for nearly a decade now, HPCC Systems features a vibrant development community that continues to push the boundaries of Big Data.

This powerful, versatile platform makes it easier for developers to see the data they’re working with and manipulate it as needed. Flexible information delivery makes it easier for your clients to query and find the data they need — and it runs analysis and queries faster than other platforms such as SQL or Hadoop.
2 changes: 1 addition & 1 deletion solutionsLab/02_learnECL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@ title: learn-ecl-intro
buttonText: Start Learning ECL
---

HPCC Systems Solutions Lab is proud to provide and support ECL tutorials.Here we introduce basics of HPCC Systems, our big data platform, and a complete tutorial on ECL (Enterprise Control Language).
LearnECL offers interactive tutorials and sample code for mastering ECL, a language for big data processing with HPCC Systems. Ideal for beginners and experts, it covers data handling and analysis, making it a valuable resource for ECL learners.
Binary file removed videos/SomeTest.mp4
Binary file not shown.

0 comments on commit 8681c01

Please sign in to comment.