Getting Started

This step-by-step tutorial will help you get Drasi up and running quickly and show you how easy it is to create Sources, Continuous Queries, and Reactions.

After completing this tutorial, which should take around 30 minutes, you will have created a simple end-to-end Drasi-based solution, and you will have a fully functional Drasi environment suitable for further experimentation on your own. You will then be able to continue to explore the capabilities of the Drasi platform creating Sources, Continuous Queries, and Reactions.

Solution Overview

In this sample Drasi solution, the source of data (and change) will be a Message table in a PostgreSQL database, which holds the content of messages sent by people. The Message table contains these three fields:

You will create three Continuous Queries that observe the Message table to answer the following questions in real-time:

1. Which people have sent the message “Hello World”?
2. How many times has the same message been sent?
3. Which people haven’t sent a message in the last 20 seconds?

Initially, the Message table will contain the following messages:

During the tutorial, you will modify the Message table, adding and removing messages, and observing the effect of those changes on your Continuous Query results using the Debug Reaction. The Debug Reaction subscribes to a set of Continuous Queries and provides a dynamic view of each query’s result set in a Web browser. It is common to use the Debug Reaction to test your Continuous Queries prior to setting up your actual Reactions.

Solution Architecture

When complete, the Hello World solution will have the component architecture shown in this diagram:

Although an intentionally simple example, the Hello World solution contains all the components of any Drasi-based solution, including:

• Drasi Sources that provide access to external data sources
• Continuous Queries that observe Sources and detect change
• Reactions that take action on the Continuous Query result set changes

To complete the tutorial, you will be guided through the following steps:

1. Deploy Drasi
2. Create the PostgreSQL Source
3. Create the Continuous Queries
4. Create the Debug Reaction
5. Test the Solution

Step 1 - Install Drasi

To complete the Hello World tutorial, you need a Drasi environment. The quickest and easiest way to get one suitable for the tutorial is to use a Visual Studio Code Dev Container we have created for the tutorial.

If you cannot or do not want to use a Dev Container to run this tutorial, take a look at the alternatives described at the bottom of the page, then continue with Step 2 - Create the PostgreSQL Source.

To use the Drasi Dev Container, you will need to install:

• Visual Studio Code (or Insiders Edition)
• Visual Studio Code Dev Containers extension
• docker

Once you have these prerequisites installed:

1. Download the Drasi Getting Started Tutorial ZIP file, which contains the files you will need during the tutorial.
2. Unzip the Drasi Getting Started Tutorial ZIP file to a suitable location on your computer
3. Run VS Code and open the tutorial/getting-started folder from the Drasi Getting Started Tutorial files you just unzipped

If you have opened the correct folder, in the VS Code Explorer panel you will see two folders:

• .devcontainer contains files that VS Code requires to configure the Dev Container
• resources contains files you will use later in the tutorial to create the Drasi Sources, Continuous Queries, and Reactions

Run the Dev Container as follows:

1. Open the Command Palette using Ctrl + Shift + P (Win/Linux) or Cmd + Shift + P (Mac)
2. Type “dev containers:”
3. Select “Dev Containers: Rebuild and Reopen in Container”

The Drasi Dev Container will take a few minutes to initialize depending on how many images it needs to download and the speed of your internet connection. The first time you run the Dev Container, it could take around 10 minutes because VS Code needs to download multiple images, install PostgreSQL, and install Drasi and its dependencies.

When you see the following message in the Dev Container terminal, it is ready to use and you can proceed with the rest of the tutorial.

Done. Press any key to close the terminal.

If the Dev Container startup fails, it is usually due to a problem with Docker resources. The following link contains instructions for cleaning out unused containers and images. If this doesn’t resolve your problem, you can contact the Drasi Team.

Step 2 - Create the PostgreSQL Source

The following YAML is the content of the hello-world-source.yaml file, which you will use to create a Source that connects to your PostgreSQL database.

apiVersion: v1
kind: Source
name: hello-world
spec:
  kind: PostgreSQL
  properties:
    host: postgres.default.svc.cluster.local
    port: 5432
    user: test
    password: test
    database: hello-world
    ssl: false
    tables:
      - public.Message

This table describes the most important configuration settings in this Source definition.

Use the drasi CLI to create the Source by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-source.yaml

It may take a minute or two for the new Source to startup and become available. You can inspect the status of all deployed sources by running the command:

drasi list source

You should expect to see a response like this until the Source is ready (AVAILABLE = true):

      ID      | AVAILABLE  
--------------+------------
  hello-world | false     

If your Source is not yet available (AVAILABLE = false), you can use the drasi wait command to wait for it to complete its startup:

drasi wait source hello-world -t 120

When drasi wait returns, your Drasi Source for PostgreSQL is created and ready to use.

Step 3 - Create the Continuous Queries

The following YAML is the content of the hello-world-queries.yaml file, which you will use to create the Continuous Queries you need.

apiVersion: v1
kind: ContinuousQuery
name: hello-world-from
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: > 
    MATCH 
      (m:Message {Message: 'Hello World'})
    RETURN 
      m.MessageId AS MessageId,
      m.From AS MessageFrom
---
apiVersion: v1
kind: ContinuousQuery
name: message-count
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: > 
    MATCH 
      (m:Message)
    RETURN 
      m.Message AS Message,
      count(m.Message) AS Frequency
---
apiVersion: v1
kind: ContinuousQuery
name: inactive-people
spec:
  mode: query
  sources:    
    subscriptions:
      - id: hello-world
  query: >
      MATCH
        (m:Message)
      WITH
        m.From AS MessageFrom,
        max(drasi.changeDateTime(m)) AS LastMessageTimestamp
      WHERE
        LastMessageTimestamp <= datetime.realtime() - duration({ seconds: 20 })
      OR
        drasi.trueLater(LastMessageTimestamp <= datetime.realtime() - duration({ seconds: 20 }), LastMessageTimestamp + duration({ seconds: 20 }))
      RETURN
        MessageFrom,
        LastMessageTimestamp      

Notice that the YAML describes three Continuous Queries. You can define any number of Drasi Sources, Continuous Queries, and Reactions in a single YAML file as long as you separate each definition with a line containing ---.

This table describes the most important configuration settings in these Continuous Query definitions.

The following table describes the Cypher Query used by each of the Continuous Queries you are about to create:

Use the drasi CLI to create the Continuous Queries by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-queries.yaml

To verify the status of the Continuous Queries, execute the following command:

drasi list query

You should expect to see the following output:

         ID        | STATUS  | CONTAINER | ERRORMESSAGE |              HOSTNAME                
-------------------+---------+-----------+--------------+--------------------------------------
  hello-world-from | Running | default   |              | default-query-host-xxx-xxx
  message-count    | Running | default   |              | default-query-host-xxx-xxx
  inactive-people  | Running | default   |              | default-query-host-xxx-xxx

Step 4 - Create the Debug Reaction

In order to view the results of the Continuous Queries you will deploy an instance of the Debug Reaction. The Debug Reaction provides a simple Web-based UI that lets you see the current result of a Continuous Query as a table, and to view the query results updating dynamically as the source data changes.

The following YAML is the content of the hello-world-reaction.yaml file, which you will use to create the Debug Reaction.

apiVersion: v1
kind: Reaction
name: hello-world-debug
spec:
  kind: Debug
  queries:
    hello-world-from:
    message-count:
    inactive-people:

This table describes the most important configuration settings in this Reaction definition:

Use the drasi CLI to create the Debug Reaction by running the following command in a terminal window:

drasi apply -f ./resources/hello-world-reaction.yaml

To verify the status of the Reaction, execute the following command:

drasi list reaction

You should expect to see the following response:

         ID         | AVAILABLE  
--------------------+------------
  hello-world-debug | false  

If your Reaction is not yet available (AVAILABLE = false), you can use the drasi wait command to wait for it to complete its startup:

drasi wait reaction hello-world-debug -t 120

When drasi wait returns, your Debug Reaction is created and ready to use.

Once the Debug Reaction is working (AVAILABLE = true), the Drasi Hello World solution is fully deployed and ready to test.

Because the Debug Reaction is running in Kubernetes, in order to connect its Web UI you must forward the container port to a local port. In the Dev Container terminal, run the following command:

kubectl port-forward services/hello-world-debug-gateway 8080:8080 -n drasi-system

Now open your browser and navigate to http://localhost:8080, where you will see the Debug Reaction UI shown here:

Step 5 - Test the Solution

To test the Hello World solution, you will need to add/update/delete data in the Message table of the PostgreSQL database, so you will need a way to run SQL commands. The Dev Container is pre-configured with psql, the PostgreSQL CLI, which will connect to the pre-installed PostgreSQL database. If you run the following command from a Dev Container terminal, it will create an interactive terminal session with the database in which you can enter SQL commands and see the results:

psql

If you prefer to use a GUI interface, you can install pgAdmin on your local machine and use the following connections settings:

On the left hand side is a menu listing the three Continuous Queries created earlier. Select hello-world-from entry and the right hand pane will show the current results of the hello-world-from query. Initially, there is only one result, because only Brian Kernighan is associated with the “Hello World” message.

If you add another Message to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (5, 'Allen', 'Hello World');

You will see a second record for Allen appear dynamically in the query result:

In the list of Continuous Queries select the message-count entry and the right hand pane will show the current results to the message-count query. There are three results, as shown below. Note that because of the “Hello World” message you just added for Allen, both “Hello World” and “I am Spartacus” have a Frequency of 2.

If you add another Message to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (6, 'Allen', 'I am Spartacus');

You will see the “I am Spartacus” Frequency increase dynamically to 3:

In the list of Continuous Queries select the inactive-people entry and the right hand pane will show the current results to the inactive-people query. Assuming you issued the last database change more than 20 seconds ago, you will see Allen on the list of inactive people.

If you add another Message from Allen to the table using the following SQL insert statement:

INSERT INTO public."Message" VALUES (7, 'Allen', 'Goodbye');

Allen will disappear from the list of inactive people, because he just sent a message:

But, if you wait 20 seconds, Allen will reappear on the list of inactive people, because he has not sent a message in the allowed 20 second time interval:

Finally, if you return to the message-count Continuous Query and delete both “Hello World” messages using the following SQL statement:

DELETE FROM public."Message" WHERE "Message" = 'Hello World';

The Debug Reaction updates to show that there are no messages with the text “Hello World”:

And if you switch back to the hello-world-from Continuous Query, the current result is empty as expected:

This tutorial is now complete. The Dev Container is running a fully functional version of Drasi that you can use for further exploration, development, and testing.

Reflection

In completing the tutorial, you were able to answer questions like “Which people have sent the message Hello World”, “How many times has each unique message been sent”, and “Which people haven’t sent messages in the last 20 seconds” using Continuous Queries. Using the Continuous Queries RESULT clause, you were able to describe those changes to best meet your needs. And then you could distribute those changes to Reactions for further processing or integration into a broader solution. You did this with no custom code and a minimal amount of configuration information.

Although the data and queries in the tutorial where trivial, the process is exactly the same for richer and more complex scenarios, only the Continuous Query increases in complexity and this depends totally on what question you are trying to answer.

Without Drasi, to achieve what you just did in the tutorial, you would need to write code to process change logs or periodically poll the database for changes. You would need to maintain your own state to track which data had changed and to calculate the aggregates across the changing data. And you would need to implement a timer and callback mechanism to create notifications when changes had not happened as expected. You would need to implement unique solutions for each type of source you wanted to support.

Hopefully, from this simple tutorial you can see the efficiencies and time saving Drasi offers, and the opportunities it presents for improving and simplifying the ability to detect and react to change in dynamic system as well as its ability to power solutions that are more dynamic and responsive.

Cleanup

If you no longer need the Dev Container and want to cleanup, you can

1. Click the Dev Container connection status box in the bottom left corner of VS Code
2. Select Close Remote Connection from the list of options that appear
3. Run the VS Code command `Dev Containers: Clean Up Dev Containers…"
4. Select the getting-started image and click ok to remove the unused image
5. Select the unused volumes and click ok to remove the unused volumes

Alternatives to the Drasi Dev Container

As an alternative to completing this tutorial using a Dev Container you can install Drasi on one of the other platforms described in the Drasi Installation Guides.

You will also need a PostgreSQL database where you can load the dataset used in the tutorial. The Getting Started Tutorial Dataset page describes a way to easily setup a PostgreSQL server and to load the required data.

Finally, download the Drasi Getting Started Tutorial ZIP file, which contains the files you will need during the tutorial. Unzip the file to a suitable location on your computer and return to the tutorial.