From 65bc0b34a7ba41e878615cddd5dc4f0d3fee4b89 Mon Sep 17 00:00:00 2001 From: 2pk03 Date: Fri, 22 Sep 2023 13:08:03 +0200 Subject: [PATCH] first draft module overview and module / class documentation --- .../executor/process_feeder.md | 43 +++++++++++++++ .../executor/process_receiver.md | 35 ++++++++++++ .../executor/python_process_caller.md | 35 ++++++++++++ .../executor/python_worker_manager.md | 40 ++++++++++++++ .../executor/reader_iterator.md | 42 ++++++++++++++ .../executor/wayang_api_python_executor.md | 40 ++++++++++++++ .../function/function_overview.md | 36 ++++++++++++ .../wayang-api-python/function/python_code.md | 32 +++++++++++ .../function/python_function_wrapper.md | 33 +++++++++++ .../wayang-api-python/function/python_udf.md | 28 ++++++++++ .../function/wayang_api_python_function.md | 31 +++++++++++ .../wayang_api_python_summary.md | 29 ++++++++++ .../wayang-api-rest/WayangController.md | 38 +++++++++++++ .../wayang-api-rest/WayangPlanBuilder.md | 55 +++++++++++++++++++ .../wayang-api-rest/wayang_api_rest.md | 34 ++++++++++++ .../wayang-api-sql/WayangApiSql_Overview.md | 36 ++++++++++++ .../calcite/CalcitePackage_Overview.md | 46 ++++++++++++++++ .../WayangProjectVisitor_MapFunctionImpl.md | 41 ++++++++++++++ .../WayangProjectVisitor_apply_Method.md | 39 +++++++++++++ .../WayangProjectVisitor_overview.md | 33 +++++++++++ .../WayangProjectVisitor_visit_Method.md | 44 +++++++++++++++ 21 files changed, 790 insertions(+) create mode 100644 documentation/wayang-api/wayang-api-python/executor/process_feeder.md create mode 100644 documentation/wayang-api/wayang-api-python/executor/process_receiver.md create mode 100644 documentation/wayang-api/wayang-api-python/executor/python_process_caller.md create mode 100644 documentation/wayang-api/wayang-api-python/executor/python_worker_manager.md create mode 100644 documentation/wayang-api/wayang-api-python/executor/reader_iterator.md create mode 100644 documentation/wayang-api/wayang-api-python/executor/wayang_api_python_executor.md create mode 100644 documentation/wayang-api/wayang-api-python/function/function_overview.md create mode 100644 documentation/wayang-api/wayang-api-python/function/python_code.md create mode 100644 documentation/wayang-api/wayang-api-python/function/python_function_wrapper.md create mode 100644 documentation/wayang-api/wayang-api-python/function/python_udf.md create mode 100644 documentation/wayang-api/wayang-api-python/function/wayang_api_python_function.md create mode 100644 documentation/wayang-api/wayang-api-python/wayang_api_python_summary.md create mode 100644 documentation/wayang-api/wayang-api-rest/WayangController.md create mode 100644 documentation/wayang-api/wayang-api-rest/WayangPlanBuilder.md create mode 100644 documentation/wayang-api/wayang-api-rest/wayang_api_rest.md create mode 100644 documentation/wayang-api/wayang-api-sql/WayangApiSql_Overview.md create mode 100644 documentation/wayang-api/wayang-api-sql/calcite/CalcitePackage_Overview.md create mode 100644 documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_MapFunctionImpl.md create mode 100644 documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_apply_Method.md create mode 100644 documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_overview.md create mode 100644 documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_visit_Method.md diff --git a/documentation/wayang-api/wayang-api-python/executor/process_feeder.md b/documentation/wayang-api/wayang-api-python/executor/process_feeder.md new file mode 100644 index 000000000..feea90b98 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/process_feeder.md @@ -0,0 +1,43 @@ + +### Class Definition +The `ProcessFeeder` class feeds input data to a Python process for execution within the Wayang framework. It ensures proper serialization of data and manages socket communication for data transmission. + +### Attributes: +1. **socket (Socket)**: Socket for communication with the Python process. +2. **udf (PythonUDF)**: The Python user-defined function to be executed by the process. +3. **serializedUDF (PythonCode)**: Serialized representation of the Python UDF. +4. **input (Iterable)**: Iterable over the input data for the Python UDF. + +### Constructor: +- `ProcessFeeder(Socket socket, PythonUDF udf, PythonCode serializedUDF, Iterable input)`: + - Initializes the feeder with socket, UDF, serialized UDF, and input data. + - Throws a `WayangException` if input is null. + +### Methods: +1. **send()**: + - Sends serialized UDF and input data to the Python process over a socket. + - Writes `END_OF_DATA_SECTION` value to indicate the end of data. + +2. **writeUDF(PythonCode serializedUDF, DataOutputStream dataOut)**: + - Writes serialized UDF to the provided data output stream. + +3. **writeIteratorToStream(Iterator iter, DataOutputStream dataOut)**: + - Writes each item from the iterator to the data output stream. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/executor/process_receiver.md b/documentation/wayang-api/wayang-api-python/executor/process_receiver.md new file mode 100644 index 000000000..f21528147 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/process_receiver.md @@ -0,0 +1,35 @@ + +### Class Definition +The `ProcessReceiver` class is responsible for receiving data from a Python process within the Wayang framework. It manages socket communication for data reception and provides methods to retrieve and print the received data. + +### Attributes: +1. **iterator (ReaderIterator)**: Iterator over the data being read from the Python process. + +### Constructor: +- `ProcessReceiver(Socket socket)`: + - Sets up a data input stream attached to the socket's input stream and initializes the `iterator` attribute. + +### Methods: +1. **getIterable()**: + - Returns an iterable that uses the `iterator` attribute. + +2. **print()**: + - Prints the received data to the console. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/executor/python_process_caller.md b/documentation/wayang-api/wayang-api-python/executor/python_process_caller.md new file mode 100644 index 000000000..057f219ba --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/python_process_caller.md @@ -0,0 +1,35 @@ + +### Class Definition +The `PythonProcessCaller` class manages the execution of Python code within a separate process in the Wayang framework. It sets up socket communication for interaction with the Python process and provides methods to manage the process lifecycle. + +### Attributes: +1. **process (Thread)**: Represents the Python process or worker being managed. +2. **socket (Socket)**: Socket for communication with the Python process. +3. **serverSocket (ServerSocket)**: Server socket to listen for connections from the Python process. +4. **ready (boolean)**: Indicates the readiness state of the Python process. +5. **configuration (Configuration)**: Configuration settings, potentially related to executing Python tasks. + +### Constructor: +- `PythonProcessCaller(PythonCode serializedUDF)`: + - Initializes the process caller with a serialized Python UDF. + - Sets up a configuration, loads it from a properties file (`wayang-api-python-defaults.properties`). + - Initializes a server socket bound to a local address (127.0.0.1) and a dynamic port. + - Launches a separate thread to start a Python worker process, with the required environment set up, and expects it to connect back via the server socket. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/executor/python_worker_manager.md b/documentation/wayang-api/wayang-api-python/executor/python_worker_manager.md new file mode 100644 index 000000000..d73baa40d --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/python_worker_manager.md @@ -0,0 +1,40 @@ + +### Class Definition +The `PythonWorkerManager` class manages the execution of Python UDFs within the Wayang framework. It handles Python worker subprocesses, feeding them input data, and potentially receiving results. + +### Attributes: +1. **udf (PythonUDF)**: The Python user-defined function that this worker manager is set to execute. +2. **serializedUDF (PythonCode)**: Holds a serialized representation of the Python UDF. +3. **inputIterator (Iterable)**: Represents an iterable over the input data for the Python UDF. + +### Constructor: +- `PythonWorkerManager(PythonUDF udf, PythonCode serializedUDF, Iterable input)`: + - Initializes the worker manager with a Python UDF, its serialized version, and input data. + - Assigns the provided parameters to the respective class attributes. + +### Methods: +1. **execute()**: + - Creates an instance of the `PythonProcessCaller` class with the serialized UDF. + - If the worker (Python process) is ready, it does the following: + - Sets up a `ProcessFeeder` to feed input data to the Python process in a separate thread. + - Sets up a `ProcessReceiver` to receive the results from the Python process. + - Returns an iterable over the received data. + - If the worker is not ready, throws a `WayangException`. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/executor/reader_iterator.md b/documentation/wayang-api/wayang-api-python/executor/reader_iterator.md new file mode 100644 index 000000000..bd8fe159f --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/reader_iterator.md @@ -0,0 +1,42 @@ + +### Class Definition +The `ReaderIterator` class provides iterator-based access to objects read from a data input stream. It efficiently processes data received from the Python process in a streaming manner. + +### Attributes: +1. **nextObj (Output)**: The next object to be returned by the iterator. +2. **eos (boolean)**: Flag indicating the end of the data stream. +3. **fst (boolean)**: Unclear from the provided content; possibly a flag for some state. +4. **stream (DataInputStream)**: The input stream from which data is read. + +### Constructor: +- `ReaderIterator(DataInputStream stream)`: + - Initializes the iterator with the data input stream. + +### Methods: +1. **read()**: + - Reads and decodes an object from the stream. + - Handles end of stream and relevant exceptions. + +2. **hasNext()**: + - Checks if there's another object to read. + +3. **next()**: + - Returns the next object or throws an exception if there isn't one. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/executor/wayang_api_python_executor.md b/documentation/wayang-api/wayang-api-python/executor/wayang_api_python_executor.md new file mode 100644 index 000000000..fbec87e41 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/executor/wayang_api_python_executor.md @@ -0,0 +1,40 @@ + +## Executor - wayang-api-python + +### PythonWorkerManager Class +- Manages the execution of Python UDFs within the Wayang framework. +- Handles Python worker subprocesses, feeding them input data, and potentially receiving results. + +### PythonProcessCaller Class +- Manages the execution of Python code within a separate process. +- Sets up socket communication for interaction with the Python process and provides methods to manage the process lifecycle. + +### ProcessFeeder Class +- Feeds input data to a Python process for execution. +- Ensures proper serialization of data and manages socket communication for data transmission. + +### ProcessReceiver Class +- Handles receipt of data from the Python process. +- Reads data over a socket connection and provides an iterator for accessing the received data. + +### ReaderIterator Class +- An iterator over a data stream, specifically designed to read serialized data objects. +- Facilitates reading data sent from the Python process to the Java environment. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/function/function_overview.md b/documentation/wayang-api/wayang-api-python/function/function_overview.md new file mode 100644 index 000000000..eeb5097d8 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/function/function_overview.md @@ -0,0 +1,36 @@ + +## function + +### PythonFunctionWrapper Class +- Acts as a Java wrapper for Python functions. +- Implements `FunctionDescriptor.SerializableFunction, Iterable>`. +- Attributes: + - `PythonUDF myUDF`: Stores the Python UDF the wrapper class handles. + - `PythonCode serializedUDF`: Stores a serialized version of the Python UDF. + +### PythonUDF Interface +- Represents Python functions to be used with the Wayang framework. +- Extends `FunctionDescriptor.SerializableFunction, Iterable>`. +- Does not define any additional methods. + +### PythonCode Class +- Represents serialized Python code snippets or blocks. +- Implements the `Serializable` interface for efficient data storage or transmission. +- Attribute `byte[] code`: Stores the serialized Python code. + +--- diff --git a/documentation/wayang-api/wayang-api-python/function/python_code.md b/documentation/wayang-api/wayang-api-python/function/python_code.md new file mode 100644 index 000000000..dffe9ca6f --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/function/python_code.md @@ -0,0 +1,32 @@ + +### Class Definition +The `PythonCode` class represents Python code in a serialized form within the Wayang Java framework. By implementing the `Serializable` interface, it ensures that the class can be serialized and deserialized, facilitating the transfer of code between Java and Python environments. + +### Attributes: +1. **code (byte[])**: Serialized form of the Python code. + +### Constructor: +- `PythonCode(byte[] code)`: + - Initializes the object with the serialized Python code. + +### Methods: +1. **toByteArray()**: + - Returns the serialized Python code. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/function/python_function_wrapper.md b/documentation/wayang-api/wayang-api-python/function/python_function_wrapper.md new file mode 100644 index 000000000..b47cf0956 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/function/python_function_wrapper.md @@ -0,0 +1,33 @@ + +### Class Definition +The `PythonFunctionWrapper` class serves as a bridge between the Java and Python UDFs in the Wayang framework. It ensures that Python functions can be seamlessly integrated into the Java environment of Wayang. + +### Attributes: +1. **myUDF (PythonUDF)**: Represents the Python user-defined function. +2. **serializedUDF (PythonCode)**: Serialized form of the Python UDF. + +### Constructor: +- `PythonFunctionWrapper(PythonUDF myUDF, PythonCode serializedUDF)`: + - Initializes the wrapper with a Python UDF and its serialized form. + +### Methods: +1. **apply(Iterable input)**: + - Executes the Python UDF on the input data and returns the results. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/function/python_udf.md b/documentation/wayang-api/wayang-api-python/function/python_udf.md new file mode 100644 index 000000000..c035345a3 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/function/python_udf.md @@ -0,0 +1,28 @@ + +### Interface Definition +The `PythonUDF` interface represents Python user-defined functions (UDFs) within the Wayang framework. It is designed to represent functions that handle iterable input and produce iterable output, ensuring seamless integration of Python functions with the Wayang Java framework. + +### Inherits: +- `FunctionDescriptor.SerializableFunction, Iterable>`: Represents serializable functions that operate on iterables. + +### Key Points: +- The `PythonUDF` interface enforces a specific function signature on the classes that implement it. +- It plays a vital role in the bridge between Wayang's Java framework and Python UDFs. + +--- + diff --git a/documentation/wayang-api/wayang-api-python/function/wayang_api_python_function.md b/documentation/wayang-api/wayang-api-python/function/wayang_api_python_function.md new file mode 100644 index 000000000..3627b7592 --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/function/wayang_api_python_function.md @@ -0,0 +1,31 @@ + +# wayang-api-python (Java Components) + +## PythonAPI.java +- This class appears to be a placeholder or an entry point for the Python API functionalities from the Java side. +- Currently contains an empty method called `empty()`, suggesting it might be in the early stages of development or serves as a stub. + +## function directory +- Likely contains Java classes related to Python functions or function interfaces. +- A deeper dive into this directory will provide insights into specific functionalities and interactions. + +## executor directory +- Might house Java classes responsible for executing or managing Python tasks. +- Further analysis of this directory will shed light on its components and their roles. + +These classes and interfaces in the `function` directory facilitate the interaction between Java and Python within the Wayang framework, particularly for executing Python functions and code snippets. diff --git a/documentation/wayang-api/wayang-api-python/wayang_api_python_summary.md b/documentation/wayang-api/wayang-api-python/wayang_api_python_summary.md new file mode 100644 index 000000000..473a7865d --- /dev/null +++ b/documentation/wayang-api/wayang-api-python/wayang_api_python_summary.md @@ -0,0 +1,29 @@ + +# wayang-api-python (Java Components) + +## PythonAPI.java +- This class appears to be a placeholder or an entry point for the Python API functionalities from the Java side. +- Currently contains an empty method called `empty()`, suggesting it might be in the early stages of development or serves as a stub. + +## function directory +- Likely contains Java classes related to Python functions or function interfaces. +- A deeper dive into this directory will provide insights into specific functionalities and interactions. + +## executor directory +- Might house Java classes responsible for executing or managing Python tasks. +- Further analysis of this directory will shed light on its components and their roles. diff --git a/documentation/wayang-api/wayang-api-rest/WayangController.md b/documentation/wayang-api/wayang-api-rest/WayangController.md new file mode 100644 index 000000000..9ef091bf4 --- /dev/null +++ b/documentation/wayang-api/wayang-api-rest/WayangController.md @@ -0,0 +1,38 @@ + +## Overview + +The `WayangController` class is a central controller for the RESTful API of the Wayang platform. It is responsible for handling HTTP requests related to Wayang plans and their executions. + +## Method: `planFromFile` + +**Endpoint**: `@GetMapping("/plan/create/fromfile")` +- This annotation suggests that this method handles GET requests to the `/plan/create/fromfile` endpoint. + +**Functionality**: +- The method reads a file named `wayang_message` from the `protobuf` directory. +- It then initializes a `WayangPlanBuilder` with the content of this file, which presumably contains a serialized Wayang plan. +- The Wayang plan is executed using the `WayangContext` obtained from the `WayangPlanBuilder`. + +This method seems to be responsible for executing Wayang plans provided as files. However, the direct file usage (instead of a passed file parameter) suggests that it might be in a developmental or testing phase. + +## Other Methods + +### @GetMapping + +- This method handles HTTP requests. + diff --git a/documentation/wayang-api/wayang-api-rest/WayangPlanBuilder.md b/documentation/wayang-api/wayang-api-rest/WayangPlanBuilder.md new file mode 100644 index 000000000..9d466fca5 --- /dev/null +++ b/documentation/wayang-api/wayang-api-rest/WayangPlanBuilder.md @@ -0,0 +1,55 @@ + +## Overview + +The `WayangPlanBuilder` class is responsible for building or decoding Wayang plans. It likely serves as an intermediary to convert serialized Wayang plans into Wayang's internal representation and vice-versa. + +## Attributes + +- **`wayangPlan`**: Private attribute of type `WayangPlan`. Represents the primary Wayang plan being managed by the class. +- **`wayangContext`**: Private attribute of type `WayangContext`. Holds essential configurations and settings for executing the Wayang plan. + +## Constructor + +```java +public WayangPlanBuilder(FileInputStream planFile) { + // Constructor details ... +} +``` + +This constructor accepts a `FileInputStream` representing a serialized Wayang plan. It parses the plan and sets up the necessary execution context and plan instance. + +## Methods + +### buildContext + +```java +Method details follow. +``` + +### buildPlan + +```java +Method details follow. +``` + +The `buildContext` and `buildPlan` methods are invoked by the constructor and are essential for setting up the Wayang context and plan, respectively. + +## Summary + +The `WayangPlanBuilder` class plays a pivotal role in managing Wayang plans, especially when they are received or sent in a serialized format, potentially over the REST API. + diff --git a/documentation/wayang-api/wayang-api-rest/wayang_api_rest.md b/documentation/wayang-api/wayang-api-rest/wayang_api_rest.md new file mode 100644 index 000000000..9105c677e --- /dev/null +++ b/documentation/wayang-api/wayang-api-rest/wayang_api_rest.md @@ -0,0 +1,34 @@ + +## Overview + +The `wayang-api-rest` module provides RESTful interfaces for the Wayang platform, likely allowing users to interact with Wayang through HTTP requests. + +## Directory Structure + +The module follows a standard Java project structure with its main source code located under: +`src/main/java/org/apache/wayang/api/rest` + +Within the `rest` directory, the primary package is `org.apache.wayang.api.rest`. + +## Key Directories + +- **server**: This directory suggests that the module might be responsible for setting up and running a REST server for Wayang. + +## Summary + +This module provides RESTful services related to Wayang. The `server` directory implies functionalities related to setting up and managing the REST server. diff --git a/documentation/wayang-api/wayang-api-sql/WayangApiSql_Overview.md b/documentation/wayang-api/wayang-api-sql/WayangApiSql_Overview.md new file mode 100644 index 000000000..f5c1ecb0e --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/WayangApiSql_Overview.md @@ -0,0 +1,36 @@ + + +# `wayang-api-sql` Module - Overview + +The `wayang-api-sql` module appears to integrate SQL functionalities into the Wayang platform. This integration might enable users to write or process SQL-like queries, interact with databases, or use SQL-based optimizations. + +## Main Packages + +1. **calcite**: + - **Description**: This package likely contains classes related to Apache Calcite, a dynamic data management framework. Calcite is known to provide SQL-like query capabilities to various data sources. + +2. **context**: + - **Description**: This package might have classes that deal with the SQL context or environment setup. It could be responsible for setting up connections, managing SQL-related configurations, or any other context-related tasks. + +3. **sources**: + - **Description**: The `sources` package is probably dedicated to handling various data sources in a SQL context. It might contain classes related to database connectors, table configurations, or query handlers. + +Further exploration can delve into each package to understand the classes and their interactions more deeply. + +--- + diff --git a/documentation/wayang-api/wayang-api-sql/calcite/CalcitePackage_Overview.md b/documentation/wayang-api/wayang-api-sql/calcite/CalcitePackage_Overview.md new file mode 100644 index 000000000..202d51616 --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/calcite/CalcitePackage_Overview.md @@ -0,0 +1,46 @@ + + +# `calcite` Package - Overview + +The `calcite` package within the `wayang-api-sql` module appears to be central to integrating SQL functionalities, particularly those related to Apache Calcite, into the Wayang platform. The package comprises several sub-packages that encapsulate specific functionalities. + +## Sub-packages: + +1. **converter**: + - **Description**: Contain classes related to converting different data types or formats. + +2. **optimizer**: + - **Description**: Classes responsible for optimizing SQL queries. + +3. **utils**: + - **Description**: Utility package with helper or utility classes. + +4. **jdbc**: + - **Description**: Classes related to Java Database Connectivity (JDBC) for SQL. + +5. **convention**: + - **Description**: Classes defining certain standards or protocols. + +6. **schema**: + - **Description**: Classes defining the schema or structure of databases or tables. + +7. **rel**: + - **Description**: Relational algebra or relational database operations. + +8. **rules**: + - **Description**: Classes defining certain rules or guidelines for SQL operations. \ No newline at end of file diff --git a/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_MapFunctionImpl.md b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_MapFunctionImpl.md new file mode 100644 index 000000000..4978635dd --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_MapFunctionImpl.md @@ -0,0 +1,41 @@ + +The `MapFunctionImpl` inner class resides within the `WayangProjectVisitor` class in the `converter` sub-package of the `calcite` package in the `wayang-api-sql` module. This class serves as the mapping function for the `MapOperator` used in SQL projections. + +## Class Functionality: + +1. **Implementation**: + - Implements the `FunctionDescriptor.SerializableFunction` interface. + - Allows the class to be serialized and used within Wayang's distributed processing model. + +2. **Fields**: + - Contains a field named `fields`. + - Holds the indices of the projected fields. + +3. **Constructor**: + - Initializes the `fields` based on the provided projects. + +4. **Utility Method - `getProjectFields`**: + - A static method. + - Extracts the field indices from a list of `RexNode` projects. + +5. **Mapping Functionality - `apply` Method**: + - Takes a `Record` as input. + - Constructs a new `Record` based on the projected fields. + - Retrieves values from the input record and sets them in the new record based on the specified projections. + +This class is pivotal in the projection process, ensuring that the specified fields in a SQL projection are retained in the output while all other fields are discarded. \ No newline at end of file diff --git a/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_apply_Method.md b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_apply_Method.md new file mode 100644 index 000000000..b960d42ca --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_apply_Method.md @@ -0,0 +1,39 @@ + +This documentation provides an in-depth understanding of the `apply` method within the `MapFunctionImpl` inner class of the `WayangProjectVisitor` class. + +## Method Functionality: + + +The `apply` method of the `MapFunctionImpl` inner class is pivotal in the projection process. Its primary function is to transform an input `Record` based on the specified projection fields. Here's a detailed understanding: + +1. **Method Signature**: `Record apply(Tuple2 record)` + - **Input**: A `Tuple2` object named `record`. + - **Output**: A transformed `Record` object that contains only the projected fields. + +2. **Creating a New Record for Projection**: + - The method initiates by creating a new `Record` object with a size equal to the number of projected fields. This new record will store the values of the projected fields from the input record. + +3. **Populating the New Record**: + - The method iterates over the `fields` array, which contains the indices of the projected fields. + - For each index in the `fields` array, it retrieves the corresponding value from the input record and sets it in the new record at the respective position. + - This operation effectively filters out any field that is not part of the projection. + +4. **Returning the Projected Record**: + - After populating the new record with the projected fields, the method returns this newly constructed record. + +This method plays a vital role in ensuring that only the specified fields in a SQL projection are retained in the output. All other fields from the input record are discarded. \ No newline at end of file diff --git a/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_overview.md b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_overview.md new file mode 100644 index 000000000..15823b778 --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_overview.md @@ -0,0 +1,33 @@ + +# WayangProjectVisitor Class - Overview + +**Class Description**: +The `WayangProjectVisitor` class is part of the `converter` sub-package within the `calcite` package of the `wayang-api-sql` module. While there isn't class-level documentation available, this class likely serves as a visitor for projections in SQL queries. + +## Methods: + +1. **visit**: + - **Return Type**: `Operator` + - **Description**: Appears to be the primary method for this visitor class, likely processing or visiting projections in SQL queries. + +2. **apply**: + - **Return Type**: `Record` + - **Description**: This method seems to apply some operation on a given record. + +3. **MapFunctionImpl**: + - **Description**: This might be an inner class or a method that constructs instances of `MapFunctionImpl`. Multiple instances of this method were observed. diff --git a/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_visit_Method.md b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_visit_Method.md new file mode 100644 index 000000000..df1342e56 --- /dev/null +++ b/documentation/wayang-api/wayang-api-sql/calcite/converter/WayangProjectVisitor/WayangProjectVisitor_visit_Method.md @@ -0,0 +1,44 @@ + +The `visit` method is part of the `WayangProjectVisitor` class in the `converter` sub-package within the `calcite` package of the `wayang-api-sql` module. This method processes projections in SQL queries. + +## Method Functionality: + +1. **Method Signature**: + - `Operator visit(WayangProject wayangRelNode)` + - **Input**: A `WayangProject` object named `wayangRelNode`. + - **Output**: An object of type `Operator`. + +2. **Converting the Input Node**: + - The method converts the input of `wayangRelNode` using the `wayangRelConverter`. + - The result is stored in the `childOp` variable. + +3. **Extracting and Checking Projects**: + - The method retrieves the projects from the `wayangRelNode`. + - If any `RexNode` in the projects is not a direct field reference, an exception is thrown indicating that generalized projections are not supported yet. + +4. **Creating a Map Operator for Projection**: + - A new `MapOperator` instance named `projection` is created. + - The mapping function for this operator is an instance of the `MapFunctionImpl` inner class. + +5. **Connecting the Operators**: + - The `childOp` is connected to the `projection` operator. + +6. **Returning the Projection Operator**: + - The method returns the `projection` operator. + +