Getting Started

This step-by-step tutorial will help you get Drasi up and running quickly and show you how easy it is to create Sources, Continuous Queries, and Reactions.

After completing this tutorial, which should take around 30 minutes, you will have created a simple end-to-end Drasi-based change-driven solution, and you will have a fully functional Drasi environment suitable for further experimentation on your own. You will then be able to continue to explore the capabilities of the Drasi platform creating Sources, Continuous Queries, and Reactions.

Solution Overview

In this sample Drasi solution, the source of data (and change) will be a Message table in a PostgreSQL database, which holds the content of messages sent by people. The Message table contains these three fields:

You will create three Continuous Queries that observe the Message table to answer the following questions in real-time:

1. Which people have sent the message “Hello World”?
2. How many times has the same message been sent?
3. Which people haven’t sent a message in the last 20 seconds?

Initially, the Message table will contain the following messages:

During the tutorial, you will modify the Message table, adding and removing messages, and observing the effect of those changes on your Continuous Query results using the Debug Reaction. The Debug Reaction subscribes to a set of Continuous Queries and provides a dynamic view of each query’s result set in a Web browser. It is common to use the Debug Reaction to test your Continuous Queries prior to setting up your actual Reactions.

Solution Architecture

When complete, the Hello World solution will have the component architecture shown in this diagram:

Although an intentionally simple example, the Hello World solution contains all the components of any Drasi-based solution, including:

• Drasi Sources that provide access to external data sources
• Continuous Queries that observe Sources and detect change
• Reactions that take action on the Continuous Query result set changes

To complete the tutorial, you will be guided through the following steps:

1. Setup a Drasi Environment
2. Create the PostgreSQL Source
3. Create the Continuous Queries
4. Create the Debug Reaction
5. Test the Solution

Step 1 - Setup a Drasi Environment

To complete this tutorial, you need a Drasi environment and a PostgreSQL database. The quickest and easiest way is to use the GitHub Codespace created for the tutorial, which allows you to complete the tutorial without the need to setup or install anything. Alternatively, you can use the Visual Studio Code Dev Container created for the tutorial, which also avoids the need for you to setup infrastructure, but will run in VS Code on your machine. Finally, you can deploy a full Drasi environment on one of the supported platforms and setup a PostgreSQL database.

Complete one of the options described below before continuing to Step 2:

• GitHub Codespace
• VS Code Dev Container
• Install Drasi

Step 2 - Create the PostgreSQL Source

The following YAML is the content of the hello-world-source.yaml file, which you will use to create a Source that connects to your PostgreSQL database.

apiVersion: v1
kind: Source
name: hello-world
spec:
  kind: PostgreSQL
  properties:
    host: postgres.default.svc.cluster.local
    port: 5432
    user: test
    password: test
    database: hello-world
    ssl: false
    tables:
      - public.Message

This table describes the most important configuration settings in this Source definition.

Use the drasi CLI to create the Source by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-source.yaml

It may take a minute or two for the new Source to startup and become available. You can inspect the status of all deployed sources by running the command:

drasi list source

You should expect to see a response like this until the Source is ready (AVAILABLE = true):

      ID      | AVAILABLE |            MESSAGES             
--------------+-----------+---------------------------------
  hello-world |  false    | query-api - daprd: waiting:     
              |           | ContainerCreating  query-api:   
              |           | waiting: ContainerCreating      
              |           |  reactivator - daprd:           
              |           | waiting: ContainerCreating      
              |           |  reactivator: waiting:          
              |           | ContainerCreating               
              |           | change-dispatcher -             
              |           | change-dispatcher: waiting:     
              |           | ContainerCreating  daprd:       
              |           | waiting: ContainerCreating      
              |           |  change-router -                
              |           | change-router: waiting:         
              |           | ContainerCreating  daprd:       
              |           | waiting: ContainerCreating      
              |           |  proxy - daprd: waiting:        
              |           | ContainerCreating  proxy:       
              |           | waiting: ContainerCreating  

If your Source is not yet available (AVAILABLE = false), you can use the drasi wait command to wait for it to complete its startup:

drasi wait source hello-world -t 120

When drasi wait returns, your Drasi Source for PostgreSQL is created and ready to use.

Step 3 - Create the Continuous Queries

The following YAML is the content of the hello-world-queries.yaml file, which you will use to create the Continuous Queries you need.

apiVersion: v1
kind: ContinuousQuery
name: hello-world-from
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: > 
    MATCH 
      (m:Message {Message: 'Hello World'})
    RETURN 
      m.MessageId AS MessageId,
      m.From AS MessageFrom
---
apiVersion: v1
kind: ContinuousQuery
name: message-count
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: > 
    MATCH 
      (m:Message)
    RETURN 
      m.Message AS Message,
      count(m.Message) AS Frequency
---
apiVersion: v1
kind: ContinuousQuery
name: inactive-people
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: >
      MATCH
        (m:Message)
      WITH
        m.From AS MessageFrom,
        max(drasi.changeDateTime(m)) AS LastMessageTimestamp
      WHERE
        LastMessageTimestamp <= datetime.realtime() - duration({ seconds: 20 })
      OR
        drasi.trueLater(LastMessageTimestamp <= datetime.realtime() - duration({ seconds: 20 }), LastMessageTimestamp + duration({ seconds: 20 }))
      RETURN
        MessageFrom,
        LastMessageTimestamp      

Notice that the YAML describes three Continuous Queries. You can define any number of Drasi Sources, Continuous Queries, and Reactions in a single YAML file as long as you separate each definition with a line containing ---.

This table describes the most important configuration settings in these Continuous Query definitions.

The following table describes the Cypher Query used by each of the Continuous Queries you are about to create:

Use the drasi CLI to create the Continuous Queries by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-queries.yaml

To verify the status of the Continuous Queries, execute the following command:

drasi list query

You should expect to see the following output:

         ID        | CONTAINER | ERRORMESSAGE |          HOSTNAME          |  STATUS   
-------------------+-----------+--------------+----------------------------+-----------
  hello-world-from |  default  |              | default-query-host-xxx-xxx |  Running  
  message-count    |  default  |              | default-query-host-xxx-xxx |  Running  
  inactive-people  |  default  |              | default-query-host-xxx-xxx |  Running   

Step 4 - Create the Debug Reaction

In order to view the results of the Continuous Queries you will deploy an instance of the Debug Reaction. The Debug Reaction provides a simple Web-based UI that lets you see the current result of a Continuous Query as a table, and to view the query results updating dynamically as the source data changes.

The following YAML is the content of the hello-world-reaction.yaml file, which you will use to create the Debug Reaction.

apiVersion: v1
kind: Reaction
name: hello-world-debug
spec:
  kind: Debug
  queries:
    hello-world-from:
    message-count:
    inactive-people:

This table describes the most important configuration settings in this Reaction definition:

Use the drasi CLI to create the Debug Reaction by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-reaction.yaml

To verify the status of the Reaction, execute the following command:

drasi list reaction

You should expect to see the following response:

         ID         | AVAILABLE | MESSAGES  
--------------------+-----------+-----------
  hello-world-debug |  true     |           

If your Reaction is not yet available (AVAILABLE = false), you can use the drasi wait command to wait for it to complete its startup:

drasi wait reaction hello-world-debug -t 120

When drasi wait returns, your Debug Reaction is created and ready to use.

Once the Debug Reaction is working (AVAILABLE = true), the Drasi Hello World solution is fully deployed and ready to test.

Because the Debug Reaction is running in Kubernetes, in order to connect its Web UI you must forward the container port to a local port. In the Dev Container terminal, run the following command:

kubectl port-forward services/hello-world-debug-gateway 8080:8080 -n drasi-system

Now open your browser and navigate to http://localhost:8080, where you will see the Debug Reaction UI shown here:

Step 5 - Test the Solution

To test the Hello World solution, you will need to add/update/delete data in the Message table of the PostgreSQL database, so you will need a way to run SQL commands. The Dev Container is pre-configured with psql, the PostgreSQL CLI, which will connect to the pre-installed PostgreSQL database. If you run the following command from a Dev Container terminal, it will create an interactive terminal session with the database in which you can enter SQL commands and see the results:

psql

If you prefer to use a GUI interface, you can install pgAdmin on your local machine and use the following connections settings:

On the left hand side is a menu listing the three Continuous Queries created earlier. Select hello-world-from entry and the right hand pane will show the current results of the hello-world-from query. Initially, there is only one result, because only Brian Kernighan is associated with the “Hello World” message.

If you add another Message to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (5, 'Allen', 'Hello World');

You will see a second record for Allen appear dynamically in the query result:

In the list of Continuous Queries select the message-count entry and the right hand pane will show the current results to the message-count query. There are three results, as shown below. Note that because of the “Hello World” message you just added for Allen, both “Hello World” and “I am Spartacus” have a Frequency of 2.

If you add another Message to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (6, 'Allen', 'I am Spartacus');

You will see the “I am Spartacus” Frequency increase dynamically to 3:

In the list of Continuous Queries select the inactive-people entry and the right hand pane will show the current results to the inactive-people query. Assuming you issued the last database change more than 20 seconds ago, you will see Allen on the list of inactive people.

If you add another Message from Allen to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (7, 'Allen', 'Goodbye');

Allen will disappear from the list of inactive people, because he just sent a message:

But, if you wait 20 seconds, Allen will reappear on the list of inactive people, because he has not sent a message in the allowed 20 second time interval:

Finally, if you return to the message-count Continuous Query and delete both “Hello World” messages using the following SQL statement:

DELETE FROM public."Message" WHERE "Message" = 'Hello World';

The Debug Reaction updates to show that there are no messages with the text “Hello World”:

And if you switch back to the hello-world-from Continuous Query, the current result is empty as expected:

This tutorial is now complete. The Dev Container is running a fully functional version of Drasi that you can use for further exploration, development, and testing.

Reflection

In completing the Getting Started Tutorial, you were able to answer questions like “Which people have sent the message Hello World”, “How many times has each unique message been sent”, and “Which people haven’t sent messages in the last 20 seconds” using Continuous Queries. Using the Continuous Queries RESULT clause, you were able to describe those changes to best meet your needs. And then you could distribute those changes to Reactions for further processing or integration into a broader solution. You did this with no custom code and a minimal amount of configuration information.

Although the data and queries in the tutorial where trivial, the process is exactly the same for richer and more complex scenarios, only the Continuous Query increases in complexity and this depends totally on what question you are trying to answer.

Without Drasi, to achieve what you just did in the tutorial, you would need to write code to process change logs or periodically poll the database for changes. You would need to maintain your own state to track which data had changed and to calculate the aggregates across the changing data. And you would need to implement a timer and callback mechanism to create notifications when changes had not happened as expected. You would need to implement unique solutions for each type of source you wanted to support.

Hopefully, from this simple tutorial you can see the efficiencies and time saving Drasi offers, and the opportunities it presents for improving and simplifying the ability to detect and react to change in dynamic system as well as its ability to power solutions that are more dynamic and responsive.

Cleanup

Cleanup steps depend on how you ran the Getting Started Tutorial:

• GitHub Codespace
• VS Code Dev Container
• Installed Drasi

drasi delete -f ./resources/hello-world-reaction.yaml

drasi delete -f ./resources/hello-world-queries.yaml

drasi delete -f ./resources/hello-world-source.yaml

drasi uninstall -y