docs updates (#917)

Signed-off-by: Vigith Maurice <[email protected]> Co-authored-by: Vigith Maurice <[email protected]>
numaproj · Aug 3, 2023 · 0f8f7a1 · 0f8f7a1
1 parent b55566b
commit 0f8f7a1
Show file tree

Hide file tree

Showing 3 changed files with 63 additions and 24 deletions.
diff --git a/docs/quick-start.md b/docs/quick-start.md
@@ -1,38 +1,51 @@
 # Quick Start
 
-Install Numaflow and run a couple of example pipelines.
+In this page, we will guide you through the steps to:  
+1. Install Numaflow.  
+2. Create and run a simple pipeline.
+3. Create and run an advanced pipeline.
 
-## Prerequisites
+## Before you begin: prerequisites
 
-A Kubernetes cluster is needed to try out Numaflow. A simple way to create a local cluster is using Docker Desktop.
+To try Numaflow, you will first need to create a local cluster. This can be done simply using [Docker Desktop](https://www.docker.com/blog/how-kubernetes-works-under-the-hood-with-docker-desktop). Follow the steps on [Docker's documentation](https://docs.docker.com/desktop/kubernetes/) to learn how to create a local Kubernetes cluster.
 
-- [Docker](https://www.docker.com)
-- [Docker Desktop](https://www.docker.com/blog/how-kubernetes-works-under-the-hood-with-docker-desktop)
+You will also need `kubectl` to manage the cluster. [Follow these steps to install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/). In case you need a refresher, all the `kubectl` commands used in this quick start guide can be found in the [kubectl Cheat Sheet](https://kubernetes.io/docs/reference/kubectl/cheatsheet/).
 
-You will also need `kubectl` to manage the cluster.
+## Installing Numaflow
 
-- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)
-
-## Installation
-
-Run the following command lines to install Numaflow and start the Inter-Step Buffer Service that handles communication between vertices.
+Once you have completed all the prerequisites, run the following command lines to install Numaflow and start the [Inter-Step Buffer Service](./core-concepts/inter-step-buffer-service.md) that handles communication between vertices.
 
 ```shell
 kubectl create ns numaflow-system
 kubectl apply -n numaflow-system -f https://raw.githubusercontent.com/numaproj/numaflow/stable/config/install.yaml
 kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/stable/examples/0-isbsvc-jetstream.yaml
 ```
 
-## A Simple Pipeline
+## Creating a simple pipeline
 
-Create a `simple pipeline`, which contains a source vertex to generate messages, a processing vertex that echos the messages, and a sink vertex that logs the messages.
+As an example, we will create a `simple pipeline` that contains a source vertex to generate messages, a processing vertex that echos the messages, and a sink vertex that logs the messages. 
+
+Run the command below to create a simple pipeline.
 
 ```shell
 kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/stable/examples/1-simple-pipeline.yaml
+```
+To view a list of pipelines you've created, run:  
+```shell
 kubectl get pipeline # or "pl" as a short name
+```
+
+This should create a response like the following, with `AGE` indicating the time elapsed since the creation of your simple pipeline.
+```shell
+NAME              PHASE     MESSAGE   VERTICES   AGE
+simple-pipeline   Running             3          9s
+```
 
+To inspect the status of the pipeline, use `kubectl get pods`. Note that the pod names will be different from the sample response: 
+```shell
 # Wait for pods to be ready
 kubectl get pods
+
 NAME                                         READY   STATUS      RESTARTS   AGE
 isbsvc-default-js-0                          3/3     Running     0          19s
 isbsvc-default-js-1                          3/3     Running     0          19s
@@ -41,9 +54,14 @@ simple-pipeline-daemon-78b798fb98-qf4t4      1/1     Running     0          10s
 simple-pipeline-out-0-xc0pf                  1/1     Running     0          10s
 simple-pipeline-cat-0-kqrhy                  2/2     Running     0          10s
 simple-pipeline-in-0-rhpjm                   1/1     Running     0          11s
+```
 
-# Watch the log for the `output` vertex
-kubectl logs -f simple-pipeline-out-0-xxxx
+Now you can watch the log for the `output` vertex. Run the command below and remember to replace `xxxxx` with the appropriate pod name above. 
+```shell
+kubectl logs -f simple-pipeline-out-0-xxxxx
+```
+This should generate an output like the sample below:
+```shell
 2022/08/25 23:59:38 (out) {"Data":"VT+G+/W7Dhc=","Createdts":1661471977707552597}
 2022/08/25 23:59:38 (out) {"Data":"0TaH+/W7Dhc=","Createdts":1661471977707615953}
 2022/08/25 23:59:38 (out) {"Data":"EEGH+/W7Dhc=","Createdts":1661471977707618576}
@@ -54,32 +72,45 @@ kubectl logs -f simple-pipeline-out-0-xxxx
 2022/08/25 23:59:39 (out) {"Data":"iUonN/a7Dhc=","Createdts":1661471978707962505}
 2022/08/25 23:59:39 (out) {"Data":"mkwnN/a7Dhc=","Createdts":1661471978707963034}
 2022/08/25 23:59:39 (out) {"Data":"jk4nN/a7Dhc=","Createdts":1661471978707963534}
+```
 
+Numaflow also comes with a built-in user interface. To port forward the UI,  run the following command.
+
+```shell
 # Port forward the UI to https://localhost:8443/
 kubectl -n numaflow-system port-forward deployment/numaflow-server 8443:8443
 ```
+This renders the following UI on https://localhost:8443/.
 
 ![Numaflow UI](assets/numaflow-ui-simple-pipeline.png)
 
-The pipeline can be deleted by
+The pipeline can be deleted by issuing the following command:
 
 ```shell
 kubectl delete -f https://raw.githubusercontent.com/numaproj/numaflow/stable/examples/1-simple-pipeline.yaml
 ```
 
-## An Advanced Pipeline
+## Creating an advanced pipeline
 
-In this example, there are five vertices in a pipeline. An [HTTP](./user-guide/sources/http.md) source vertex which serves an HTTP endpoint to receive numbers as source data, a [UDF](user-guide/user-defined-functions/map/map.md) vertex to tag the ingested numbers with the key `even` or `odd`, three [Log](./user-guide/sinks/log.md) sinks, one to print the `even` numbers, one to print the `odd` numbers, and the other one to print both the even and odd numbers.
+Now we will walk you through creating an advanced pipeline. In our example, this is called the `even-odd` pipeline, illustrated by the following diagram:
 
 ![Pipeline Diagram](assets/even-odd.png)
 
-Create the `even-odd` pipeline.
+There are five vertices in this example of an advanced pipeline. An [HTTP](./user-guide/sources/http.md) source vertex which serves an HTTP endpoint to receive numbers as source data, a [UDF](user-guide/user-defined-functions/map/map.md) vertex to tag the ingested numbers with the key `even` or `odd`, three [Log](./user-guide/sinks/log.md) sinks, one to print the `even` numbers, one to print the `odd` numbers, and the other one to print both the even and odd numbers.
+
+
+
+Run the following command to create the `even-odd` pipeline.
 
 ```shell
 kubectl apply -f https://raw.githubusercontent.com/numaproj/numaflow/main/examples/2-even-odd-pipeline.yaml
+```
+You may opt to view the list of pipelines you've created so far by running `kubectl get pipeline`. Otherwise, proceed to inspect the status of the pipeline, using `kubectl get pods`.
 
+```shell
 # Wait for pods to be ready
 kubectl get pods
+
 NAME                               READY   STATUS    RESTARTS   AGE
 even-odd-daemon-64d65c945d-vjs9f   1/1     Running   0          5m3s
 even-odd-even-or-odd-0-pr4ze       2/2     Running   0          30s
@@ -90,28 +121,35 @@ even-odd-odd-sink-0-2736r          1/1     Running   0          15s
 isbsvc-default-js-0                3/3     Running   0          10m
 isbsvc-default-js-1                3/3     Running   0          10m
 isbsvc-default-js-2                3/3     Running   0          10m
+```
 
-# Port-forward the HTTP endpoint so we can post data from the laptop
+Next, port-forward the HTTP endpoint, and make a `POST` request using `curl`
+```shell
 kubectl port-forward even-odd-in-0-xxxx 8444:8443
 
 # Post data to the HTTP endpoint
 curl -kq -X POST -d "101" https://localhost:8444/vertices/in
 curl -kq -X POST -d "102" https://localhost:8444/vertices/in
 curl -kq -X POST -d "103" https://localhost:8444/vertices/in
 curl -kq -X POST -d "104" https://localhost:8444/vertices/in
+```
 
+Now you can watch the log for the `even` and `odd` vertices by running the command below. Remember to replace `xxxxx` with the appropriate pod names.
+
+```shell
 # Watch the log for the even vertex
-kubectl logs -f even-odd-even-sink-0-nf2ql
+kubectl logs -f even-odd-even-sink-0-xxxxx
 2022/09/07 22:29:40 (even-sink) 102
 2022/09/07 22:29:40 (even-sink) 104
 
 # Watch the log for the odd vertex
-kubectl logs -f even-odd-odd-sink-0-a6p0n
+kubectl logs -f even-odd-odd-sink-0-xxxxx
 2022/09/07 22:30:19 (odd-sink) 101
 2022/09/07 22:30:19 (odd-sink) 103
 ```
 
 View the UI for the advanced pipeline at https://localhost:8443/
+
 ![Numaflow UI](assets/numaflow-ui-advanced-pipeline.png)
 
 The source code of the `even-odd` [User Defined Function](user-guide/user-defined-functions/map/map.md) can be found [here](https://github.com/numaproj/numaflow-go/tree/main/pkg/function/examples/even_odd). You also can replace the [Log](./user-guide/sinks/log.md) Sink with some other sinks like [Kafka](./user-guide/sinks/kafka.md) to forward the data to Kafka topics.
@@ -130,6 +168,6 @@ kubectl delete -f https://raw.githubusercontent.com/numaproj/numaflow/main/examp
 
 Try more examples in the [`examples`](https://github.com/numaproj/numaflow/tree/main/examples) directory.
 
-After exploring how Numaflow pipeline run, you can check what data [Sources](./user-guide/sources/generator.md) 
+After exploring how Numaflow pipelines run, you can check what data [Sources](./user-guide/sources/generator.md) 
 and [Sinks](./user-guide/sinks/kafka.md) Numaflow supports out of the box, or learn how to write 
 [User Defined Functions](user-guide/user-defined-functions/user-defined-functions.md).
diff --git a/docs/user-guide/reference/multi-partition.md b/docs/user-guide/reference/multi-partition.md
@@ -1,6 +1,6 @@
 # Multi-partitioned Edges
 
-To achieve higher throughput(> 10K but < 25K), users can create multi-partitioned edges.
+To achieve higher throughput(> 10K but < 30K tps), users can create multi-partitioned edges.
 Multi-partitioned edges are only supported for pipelines with JetStream as ISB. Please ensure
 that the JetStream is provisioned with more nodes to support higher throughput.
 

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -84,6 +84,7 @@ nav:
           - user-guide/reference/pipeline-tuning.md
           - user-guide/reference/autoscaling.md
           - user-guide/reference/conditional-forwarding.md
+          - user-guide/reference/multi-partition.md
           - Configuration:              
             - user-guide/reference/configuration/container-resources.md
             - user-guide/reference/configuration/volumes.md