mdbook for operations, ...

book/src/tfgrid4_architecture/.collection

@@ -0,0 +1 @@
+name:architecture

book/src/tfgrid4_architecture/architecture/architecture.md

@@ -0,0 +1,96 @@
+
+
+## Circle + Actor
+
+```mermaid
+
+
+graph TB
+    user1[Member of Circle OurAdmin] .->|3script\ne.g. through chat| actor1
+    subgraph Circle1 [A Circle with name *OurAdmin*]
+        actor1[Actor]
+        modeldsl[Model DSL]
+        worlddsl[World DSL]
+        systemdsl[System DSL]
+        modeldb[Actor Mem DB]
+        actor1-->worlddsl
+        worlddsl-->modeldsl
+        modeldsl-->modeldb
+        worlddsl-->systemdsl
+    end
+
+
+```
+
+A Circle has a main actor which will have recipe's calling world DLS based on context.
+
+The World DSL abstracts access to model dsl as well as system dsl.
+
+The user which is a member of the circle, communicate to the circle actor in this case using chat and 3script.
+
+## Circle DB + Backend
+
+All structured data is stored in actor memory db and version controlled optionally through quantum safe storage system or a git based system.
+
+```mermaid
+graph TB
+    subgraph Circle1 [A Circle with name *OurAdmin*]
+        actor1[Actor]
+        modeldsl[Model DSL]
+        worlddsl[World DSL]
+        systemdsl[System DSL]
+        modeldb[Actor Mem DB]
+        actor1-->worlddsl
+        worlddsl-->modeldsl
+        modeldsl-->modeldb
+        worlddsl-->systemdsl
+    end
+
+    subgraph GitSystem1 [GitSystem 1]
+        git1[version controlled storage of <br>structured data and content]
+    end
+
+    subgraph GitSystem2 [GitSystem 2]
+        git2[version controlled storage of <br>structured data and content]
+    end
+
+    modeldb -->|git over mycelium| git1
+    modeldb -->|git over mycelium| git2
+    
+
+```
+
+The redundant git stor at backend makes sure data data cannot be lost, multiple actors can work together, full version control is being achieved
+
+Mycelium is our secure P2P network.
+
+
+```mermaid
+graph TB
+    subgraph Circle1 [Circle 1]
+        actor1[Actor 1]
+        modeldb[Actor Mem DB]
+    end
+    subgraph Circle2 [Circle 1]
+        actor2[Actor 2]
+        modeldb2[Actor Mem DB]
+    end
+
+    subgraph GitSystem1 [GitSystem 1]
+        git1[version controlled storage of <br>structured data and content]
+    end
+
+    subgraph GitSystem2 [GitSystem 2]
+        git2[version controlled storage of <br>structured data and content]
+    end
+
+    modeldb -->|git over mycelium| git1
+    modeldb -->|git over mycelium| git2
+    modeldb2 -->|git over mycelium| git1
+    modeldb2 -->|git over mycelium| git2
+    actor1 <--> actor2
+    
+
+```
+
+

book/src/tfgrid4_architecture/architecture/concepts.md

@@ -0,0 +1,37 @@
+
+
+
+
+## Actor DNA
+
+Is knowledge to teach actors how to deal with incoming requests.
+
+Its a collection of Recipes which are the registration of knowledge as is needed for the Actor.
+
+### How to give knowledge to an Actor
+
+The Actor DSL is a language allowing an expert to define Actor Recipe's which are teachings of knowledge around specific topic. E.g. an Actor can learn how to deal with a customer if a customer has an issue and is looking for support.
+
+The Actor DSL will use the Model DSL and System DSL to define the knowledge in.
+
+Think about it like a recipe for a cook in a kitchen.
+
+## Circle
+
+
+- A group of people working together on a project (set of tasks, milestones, governance area, IT environment, …).
+- Each Circle has a unique id, called CID = Circle ID
+- Data models are often stored in mem and linked to a circle.
+
+## Actors & Circles
+
+Actors are linked to Circles.
+
+One actor can never operate in an external circle.
+
+Actors can send messages to other actors of other circles, this happens by default using 3 script messages.
+
+## 3script messages
+
+Each Circle Actor has a message queue, and incoming messages (3script) are parsed to actions which are given to right components in crystallib or other v library.
+

book/src/tfgrid4_architecture/architecture/dsl.md

@@ -0,0 +1,73 @@
+
+## DSL's
+
+A Domain Specific Language.
+We believe a DSL's need to be human readable and easy to expand.
+
+We use 3script and vlang to define our DSL's in.
+
+There are different types of DSLs
+
+- Model DSL = manage structured data, in a consistent way (Vlang & 3script)
+    - the memory of our digital world, uses rootobjects at the base
+- System DSL = translate from a specific system/machine/tool to our vlang language (Vlang only)
+  - is a tool
+  - e.g. docker.machine_start would be interface to docker to get a machine started
+  - e.g. smtp.message_send is sending message in specific way
+- World DSL = translate from a world usecase (can be tech or non tech), to the specific System / Model DSL (Vlang & 3script)
+  - is like an abstraction layer of the world, each actionr results in a recipe which is using the different DSL's
+  - e.g. cloud.machine_start would be world dsl, does not have to know about docker or KVM
+  - e.g. communication.message_send would be independent of implementation e.g. mail, sms, ... 
+- Circle DSL 
+  - Each Circle can expose parts of a world DSL, rights can be defined on that world DSL.
+  - e.g. Admin's of Circle can do cloud.* but not a normal member, e.g. Consensus of Members needed to do ...
+  - Each Circle has 1 actor, this actor gets incoming messages (based on 3script) and will look for the right World DSL to execute on.
+  - Each Circle has its own memory structure in which the data can be kept, this data is organize din such a way that only the circle actor can access and modify the data.
+
+### Model DSL 
+
+Is a language (3script or V) which allows actors to manipulate a data model.
+All data is organized following rootobjects, the rootobjects can be loaded in memory and linked to a circle.
+
+- A root object is a complex data type, each rootobject as a unique ID (OID - Object ID).
+- Each rootobject is linked to a Circle.
+- 3Script or VLang can be used to manipulate (CREATE, DELETE, UPDATE) the Root Objects.
+
+Each RootObject has a factory through which we can do our crud actions
+
+- ingest 3script
+- get, create, delete, update (will return copy)
+- object to 3script
+- ...
+
+#### Example root objects
+
+- User
+- Contact
+- Event (calendar)
+- Story
+- Milestone
+- …
+
+### System DSL = SAL
+
+SAL = System Abstraction Layer
+
+Is like set of tools to manage a system e.g. manage virtual machines in a computer. The language used to manipulate a system is a DSL by itself but with specific purpose.
+
+SAL’s DON’T keep state, they are a tool, it's a language to be able to automate or work with systems.
+
+Examples would be
+
+- docker.container.stop
+- docker.container.create
+
+### World DSL = WAL
+
+WAL = World Abstraction Layer
+
+Make a language which is independent of the physical implementation, will call the underlying model and system DSL's
+
+
+
+!!def_define name:'dsl,system_dsl,sal,dal,wal,model_dsl,actor_dsl'

book/src/tfgrid4_architecture/architecture/tfgrid_actors.md

@@ -0,0 +1,104 @@
+
+
+# TFGrid main actors
+
+A Circle actor has the capability to communicate wiht 3Nodes, which are computers running on the TFGrid.
+
+Each 3Node has an actor inside who has as function to manage the provisioning of compute, storage and network capacity.
+
+The 3Node actor will report back to a farming actor (see below)
+
+```mermaid
+
+
+graph TB
+    subgraph Circle1 [A Circle with name *OurAdmin*]
+        actor1[Actor]
+    end
+
+    subgraph 3Node1 [TFGrid 3Node]
+        actor_3node1[Actor for 3Node]
+        actor_3node1 .- compute1 
+        actor_3node1 .- storage1
+        actor_3node1 .- network1
+        compute1[Compute]
+        storage1[Storage]
+        network1[Network] 
+    end
+
+    subgraph 3Node2 [TFGrid 3Node]
+        actor_3node2[Actor for 3Node]
+        actor_3node2 .- compute2 
+        actor_3node2 .- storage2
+        actor_3node2 .- network2
+        compute2[Compute]
+        storage2[Storage]
+        network2[Network] 
+    end
+
+    subgraph 3Node3 [TFGrid 3Node]
+        actor_3node3[Actor for 3Node]
+        actor_3node3 .- network3
+        actor_3node3 .- compute3 
+        actor_3node3 .- storage3
+        compute3[Compute]
+        storage3[Storage]
+        network3[Network] 
+    end
+
+    network2 .-|Mycelium P2P Encrypted Network| network1
+    network2 .-network3
+    network1 .- network3
+    actor1 ---|3script rpc over mycelium \nreliable message bus| actor_3node1
+    actor1 --- actor_3node2
+    actor1 --- actor_3node3
+
+```
+
+## farming actors
+
+
+```mermaid
+
+
+graph TB
+    subgraph Circle1 [A Circle with name *OurAdmin*]
+        actor1[Actor]
+    end
+
+    subgraph 3Node1 [TFGrid 3Node]
+        actor_3node1[Actor for 3Node]
+        actor_3node1 .- compute1 
+        actor_3node1 .- storage1
+        actor_3node1 .- network1
+        compute1[Compute]
+        storage1[Storage]
+        network1[Network] 
+    end
+
+    subgraph 3Node2 [TFGrid 3Node]
+        actor_3node2[Actor for 3Node]
+        actor_3node2 .- compute2 
+        actor_3node2 .- storage2
+        actor_3node2 .- network2
+        compute2[Compute]
+        storage2[Storage]
+        network2[Network] 
+    end
+
+    subgraph FarmingActor [Farming Actor]
+        farmingactor[Actor for Farmer]
+        
+    end
+
+    actor1 ---|3script rpc over mycelium \nreliable message bus| actor_3node1
+    actor1 --- actor_3node2
+    actor_3node1 -->|capacity utilization\nand availability| farmingactor
+    actor_3node2 --> farmingactor
+    actor1 -->|ask information of availability\nby means of 3script| farmingactor
+
+```
+
+The Farming Actor will stor information of utilization & capacity in an internal DB (can be queried), so that every actor can find out how much capacity there is, is being used...
+
+The farming actor will also do monitoring.

book/src/tfgrid4_architecture/concepts/build_sandbox.md

@@ -0,0 +1,23 @@
+# Build Sandbox
+
+We build in a chroot environment, which is driven by SAL in V Crystallib.
+
+The Chroot environment is run like an OCI compatible container.
+
+## process how to build
+
+- build inside the sandbox
+- the result sandbox gets cleaned up (remove non wanted files)
+- export the sandbox to flist4 format
+- store the flist4 format on chosen S3 server
+- store the fileparts to chosen ZDB (future S3 server)
+
+Once 
+
+## OCI = container specification
+
+Is a JSON spec in how to deploy a container, is low level but we have SAL for it in V.
+
+
+!!tree.def_set name:'sandbox,build_sandbox,oci' description:'sandbox in which builds are done.'
+

book/src/tfgrid4_architecture/concepts/build_stack.md

@@ -0,0 +1,7 @@
+
+
+## installers
+
+- v scripts using our [System DSL's](crystallib:dsl.md)
+- the v scripts run in a [sandbox](build_sandbox.md)
+- V templates are used to generate the required configuration scripts.

book/src/tfgrid4_architecture/concepts/flist4.md

@@ -0,0 +1,20 @@
+# Flist 4
+
+Our latest version of flists.
+
+The metadata of a filesystem is in SQLITE DB format and can easily be manipulated.
+The data are deduped parts of files, which are stored on ZDB compatible backend (in future also S3 backends)
+
+There is a command line written in rust which allows us to
+
+- merge flist (TODO)
+- import flist (dir to flist)
+- export flist (flist to dir)
+
+remarks
+
+- the flist can be stored on any S3 or HTTP compatible webserver
+- ZOS knows how to download the flist and mount the flist as filesystem and give to a virtual machine.
+
+
+!!tree.def_set name:'flist4,flist'

book/src/tfgrid4_operations/.collection

@@ -0,0 +1 @@
+name:operations