Merge pull request #706 from nevalang/docs3

feat(docs:components)

.vscode/settings.json

@@ -18,6 +18,7 @@
     "Field",
     "gqlgen",
     "graphql",
+    "indeterministic",
     "introspection",
     "intrprtr",
     "irprotosdk",

docs/about.md

@@ -130,14 +130,18 @@ Nevalang is designed for general-purpose programming, expecting entire programs
 
 Nevalang is garbage-collected with immutable data, avoiding ownership concepts. This prevents data races but may impact performance. Mutations are possible via unsafe packages (WIP) but are discouraged. FBP, in contrast, uses ownership and allows mutations.
 
-#### Node Behavior
+#### Node State Control
 
 Nevalang's nodes are always running, automatically starting, suspending, and restarting as needed. FBP processes have explicit states (start, suspend, restart, shutdown) that can be manipulated.
 
 #### Static Typing
 
 Nevalang features a static type system with generics and structural sub-typing, improving IDE support and reducing runtime validations. FBP is dynamically typed in its dataflow part.
 
+#### Buffered Queues
+
+FBP uses buffered queues for connections by default. Nevalang connection-queues are unbuffered, with optional middleware buffer nodes for configurable message buffering.
+
 #### Similarities
 
 Both paradigms support dataflow and implicit parallelism, sharing much terminology.

docs/components.md

@@ -1,49 +1,191 @@
 # Components
 
-Components have a signature (interface), optional compiler directives, nodes, and network. They can be normal or native.
+Components are blueprints for nodes, which are computational units in Nevalang. Components can have multiple instances, each potentially differing in name, type-arguments, and dependencies. Unlike other languages with various computational constructs, Nevalang uses only components for all computations, including data transformations and side-effects. There are 2 types of components: native and normal.
 
-## Main Component
+## Native Components
 
-Executable packages must have a private `Main` component with:
+Native components have only a signature (interface) and no implementation in Nevalang. They use the `#extern` directive to indicate implementation in the runtime's language. There's no way to tell if a component is native by its usage, only by its implementation. For example, the `And` boolean component uses a runtime function `and`. Native components exist only in the standard library and are not user-definable.
 
-- One `start` inport and one `stop` outport, both of type `any`
-- No interface nodes
-- Both nodes and network
-- Not public
+```neva
+#extern(and)
+flow And(a bool, b bool) (res bool)
+```
 
-## Native Components
+> Native components may enable future Go-interop, allowing Go functions to be called from Neva using #extern.
+
+### Overloading
+
+Native components can be overloaded, allowing multiple implementations with the same signature for different data types. The compiler chooses the appropriate implementation based on the given data type. Overloading is limited to native components in the standard library and not available for user-defined components.
+
+Overloaded native components use a modified extern directive: `#extern(t1 f1, t2 f2, ...)`. These components must have exactly one type parameter with a union constraint. Example:
+
+```neva
+#extern(int int_add, float float_add, string string_add)
+pub flow Add<T int | float | string>(acc T, el T) (res T)
+```
+
+Usage:
+
+```
+Add<int> // int_add will be used
+Add<float> // float_add will be used
+Add<string> // string_add will be used
+```
+
+## Normal Components
+
+Normal components are implemented in Nevalang source code. As a Nevalang programmer, you'll primarily work with these components, which are also found in the standard library alongside native ones. Normal components don't use the `#extern` directive and include an implementation consisting of a required network and optional nodes section. The network must use all of the component's inports and outports, enabling at least basic routing.
+
+Minimal nevalang program is one normal component `Main` without any nodes and with network of a single connection:
+
+```neva
+flow Main(start) (stop) {
+   :start -> :stop
+}
+```
+
+### Nodes
+
+Components however typically perform data transformations or side-effects using nodes, which are instances of other components. This means normal components usually depend on other components, directly or indirectly (through interfaces).
+
+Normal component `Main` with a `println` node (instance of `Println`):
+
+```neva
+flow Main(start) (stop) {
+   Println
+   ---
+   :start -> (42 -> println -> :stop)
+}
+```
+
+As you can see, we refer to the instance of `Println` as `println`. The compiler implicitly assigns a lowercase version of the component name to its instance. However, with multiple instances of the same component, this leads to name collisions. To avoid this, the compiler requires explicit naming for nodes in such cases. Example:
+
+```neva
+flow Main(start) (stop) {
+   p1 Println
+   p2 Println
+   ---
+   :start -> (42 -> p1 -> p2 -> :stop)
+}
+```
+
+#### IO (Implicit) Nodes
+
+Normal components actually have implicit `in` and `out` nodes. Even in our `Main` example with a single connection `:start -> :stop`, the compiler interprets this as `in:start -> out:stop`. Interestingly, `in` only has outports (you can only send _from_ it) while `out` only has inports (you can only send _to_ it). The compiler automatically generates these in/out nodes with necessary ports based on the component's interface.
+
+#### Interface Nodes (Dependency Injection)
+
+Consider an application that performs some business logic with logging:
+
+```neva
+flow App(data) (sig) {
+   Logic, Log
+   ---
+   :data -> logic -> log -> :sig
+}
+```
+
+Now imagine we want to replace `Logger` with another component based on a condition. Let's say we want to use real logger in production and a mock in testing. Without dependency injection (DI), we'd have to extend the interface with a `flag bool` inport and check it.
+
+```neva
+flow App(data any, prod bool) (sig any) {
+   Cond, Logic, ProdLogger, MockLogger
+   ---
+   :data -> businessLogic -> cond:data
+   :prod -> cond:if
+   cond:then -> prodLogger
+   cond:else -> mockLogger
+   [prodLogger, mockLogger] -> :sig
+}
+```
+
+This not only makes the code more complex but also means we have to initialize both implementations: `ProdLogger` in the test environment and `MockLogger` in the production environment, even though they are not needed in those respective contexts. What if you need to read environment variables to initialize a component? For example, your logger might need to send requests to a third-party service to collect errors. And finally, imagine if it were not a boolean flag but an enum with several possible states. The complexity would increase dramatically.
+
+> As you can see it's possible to write nodes in a single line, separated by comma: `Cond, Logic, Println, Mock`. Don't abuse this style - Nevalang is not about clever one-liners.
+
+Let's implement this using dependency injection. First, define an interface:
+
+```neva
+type ILog(data) (sig)
+```
+
+Next, define the dependency (interface node):
+
+```neva
+flow App(data) (sig) {
+   Logic, ILog
+   ---
+   :data -> logic -> iLog -> :sig
+}
+```
+
+Our network is again a simple pipeline. Finally, provide the dependency.
+
+Before dependency injection:
+
+```neva
+// cmd/app
+flow Main(start) (stop)
+   App
+   ...
+}
+```
+
+After dependency injection:
+
+```neva
+flow Main(start) (stop)
+   App{ProdLogger}
+   ...
+}
+
+flow Test(start) (stop)
+   App{MockLogger}
+   ...
+}
+```
+
+`App{ProdLogger}` syntax sugar for `App{iLog: MockLogger}`, same for `App{MockLogger}`. Compiler is able to infer name of the dependency we provide if there's only one dependency. Syntax for providing several dependencies looks like a structure or dictionary initialization: `Component{dep1: nodeExpr1, dep2: nodeExpr2, ..., depN: nodeExprN}`.
+
+**Component and Interface Compatibility**
 
-Components without implementation use `#extern` directive to refer to runtime functions. Typically found in `std` module.
+Component `C1` implements interface `I1` if:
 
-## Runtime Function Overloading
+- Type parameters are compatible: same amount and each parameter of `C1` is compatible with corresponding `I1` parameter
+- Inports are compatible: full match by name plus each inport of `C1` is compatible with corresponding `I1` inport
+- Outports are compatible
+  - `C1` has full match by name or superset of `I1` outports. Example: `C1(a) (b, c)` is compatible with `I1(a) (b)`, but not with `I1(a) (b, d)`. Types must be compatible too.
 
-Native components can use overloading with `#extern(t1 f1, t2 f2, ...)`. Requires one type parameter of type `union`.
+### Networks
 
-## Normal Component
+Each normal component must contain a network. If a component is a box with inports and outports as data entry and exit points, the network is what's inside. Like examining an electronic component's internals, networks describe how the component processes input, uses internal nodes, and produces output.
 
-Implemented in source code with network and maybe nodes. Must not use `#extern` directive.
+Networks are complex due to the variety of ways to combine connections. This complexity stems from Nevalang's minimalistic approach, using only dataflow components and message passing for all computations. While keeping the language core small, this necessitates more intricate network designs to maintain expressiveness and compensate for the absence of traditional programming constructs.
 
-## Nodes
+See the [networks page](./networks.md) for detailed information.
 
-Nodes are instances of other components that can be used in component's network to perform computation.
+### Main Component
 
-If entity that node refers to (component or interface) have type-parameters, then type-arguments must be provided with node instantiation. If node is instantiated from component that requires dependencies, then other node instantiations must be provided as those dependencies.
+Main component is the entry point of a nevalang program. A package containing this component is called a "main package" and serves as the compilation entry point. Each main package must have exactly one non-public `Main` component with no interface nodes, implementing the `(start any) (stop any)` interface. Nevalang's runtime sends a message to `Main:start` at startup and waits for a message from `Main:stop`. Upon receiving the stop signal, it terminates the program.
 
-There are 2 types of nodes:
+## Type Parameters
 
-1. IO Nodes - `in` and `out`, created implicitly, you omit node name when you refer to them (e.g. `:start`, `:stop` instead of `in:start` and `in:stop`)
-2. Computational nodes - explicitly created by user, instances of entities:
-   - Components (concrete/component nodes)
-   - Interfaces (abstract/interface nodes)
+Components contain interfaces, which may have type parameters. Type arguments must be provided during initialization. For interfaces, this occurs when initializing an interface node, while for components, it happens when initializing concrete nodes.
 
-## Dependency Injection (DI)
+Component uses type-parameters for its IO:
 
-Normal components can have interface nodes, requiring DI. `Main` component cannot use DI.
+```neva
+flow Foo<T>(data T) (sig any)
+```
 
-## Component and Interface Compatibility
+Components can pass type parameters from their interface to node expressions:
 
-A component implements an interface if:
+```neva
+flow Bar<T>(data T) (sig any) {
+   Println<T>
+   ---
+   :data -> println -> :sig
+}
+```
 
-- Type parameters are compatible (count, order, names, constraints)
-- Inports are compatible (equal amount, same names/kind, compatible types)
-- Outports are compatible (equal or more amount, same names/kind, compatible types)
+This means when we initialize `Bar` with a type argument, it replaces `T` in `Println<T>`. For example, if a parent component initializes `Bar<int>`, then inside Bar, `Println<T>` becomes `Println<int>`.

docs/constants.md

@@ -15,9 +15,13 @@ const f dict<float> = { one: 1.0, two: 2.0 }
 const g struct { b int, c float } = { a: 42, b: 42.0 }
 ```
 
-## Constant References as Network Senders
+## As Network Senders
 
-To create a component that increments a number, we can use an addition component with 2 inports: `:acc` and `:el`. We'll use a constant `$one` for the `:acc` input, while `:el` will receive dynamic values:
+This section briefly outlines how constants are used in networks. For detailed semantics, see the [network page](./networks.md).
+
+### Constant References
+
+It's possible to use constant as a network-sender by refering to it with `$` prefix.
 
 ```neva
 const one int = 1
@@ -31,9 +35,7 @@ flow Inc(data int) (res int) {
 }
 ```
 
-Only primitive data-types are allowed to be used like this: `bool`, `int`, `float` and `string`. You can't use `struct`, `list` or `dict` literals in the network.
-
-## Message Literals as Network Senders
+### Message Literals
 
 You can omit explicit constants; the compiler will create and refer to them implicitly.
 
@@ -47,32 +49,6 @@ flow Inc(data int) (res int) {
 }
 ```
 
-## Semantics
-
-Constants are sent in an infinite loop. Imagine a `SendOne` component that constantly sends ones. Its speed is limited by the receiver: `sendOne -> add:acc` sends a message each time `add:acc` can receive. In this example, `add:acc` and `add:el` are synchronized. When `:data -> add:el` has a message, `add:acc` can receive. If the parent of `Inc` sends `1, 2, 3`, `add` will receive `acc=1 el=1; acc=1 el=2; acc=1 el=3` and produce `2, 3, 4` respectively.
-
-### Internal Implementation (`New` and `#bind`)
-
-> You can skip this section. The above explanation is enough for writing programs. Here we discuss the implementation details of constant sending.
-
-Both forms are syntax sugar. Here's the desugared form:
-
-```neva
-const one int = 1
-
-flow Inc(data int) (res int) {
-    #bind(one)
-    New
-    Add
-    ---
-    new -> add:acc
-    :data -> add:el
-    add -> :res
-}
-```
-
-`New` with `#bind(one)` binds the `one` constant to the component instance, making it an emitter node that infinitely sends `1` to the receiver. This is similar to `SendOne` from the previous section. Constants cannot be rebound, and just a few components need `#bind`. The compiler usually handles this automatically.
-
 ## Nesting and Referensing
 
 Non-primitive constants can to other constants and implement infinite nesting. Examples:

docs/interfaces.md

@@ -69,7 +69,7 @@ interface IAdd(acc int, el int) (res int)
 It's suitable for combining 2 sources, but what if we need to combine any number of sources? Chaining multiple `IAdd` instances is tedious and sometimes impossible. Let's look at the array-inports solution:
 
 ```neva
-interface IAdd([el] int) (res int)
+interface IAdd([data] int) (res int)
 ```
 
 In a component using `IAdd` in its network, we can do this:
@@ -82,7 +82,7 @@ add IAdd
 3 -> add[2]
 ```
 
-Syntax `add[i]` is shorthand for `add:el[i]`. The compiler infers the port name since there's only one.
+Syntax `add[i]` is shorthand for `add:data[i]`. The compiler infers the port name since there's only one.
 
 Another example of a component that benefits from array-ports is `Switch`. It's used for routing - imagine we have message `data` and need to route it to different destinations based on its value. For example, if it's `a` we send it to the first destination, if `b` to the second, and `c` to the third. Otherwise, we send it to a default destination to handle unknown values. An adhoc solution with a fixed number of ports wouldn't scale. We need a generic component with dynamic port support. Here's the Switch signature: