Data, Dialogs, and Databases

Watson Dialog Specification

I recently wrote about managing IBM Watson dialogs from the command line and that I wanted to bring database records into a dialog, combining different IBM Bluemix services. I succeeded and here is the follow-up on how I perform dialog-driven lookups in dashDB / DB2 and update the conversation for Watson to return user-specific answers. The application is written in Python and available on GitHub again and builds on the experience and code from the watson-dialog-client I published earlier.

The user interacts with the Python app which basically is a loop, waiting for the user to end the dialog. The script uses the dialog API to send the user input and obtain a response from Watson. It also checks the dialog profile variables to determine whether there are changes or any variable of interest is set. Depending on state information obtained from the profile variables dashDB is queried. Database results are either directly returned to the user or are used to update the dialog profile.

Data and Dialog Combined

To demonstrate (read: experiment with) data-driven dialogs I created a simple table "dialogdata" in dashDB and populated it with few records. The dialog prompts the user for the name. When entered it is used to look up the corresponding database record which then is fed into the dialog variables. As an example the dialog allows to ask for the birthday or age. The answer is composed of a template with the user-related information filled in.
Another example for combining dialog and database is to enter a service name, i.e., to request an action. This is detected by the Python script again by checking the profile variables. Then, dashDB is queried and the result is directly returned to the user. This way it would be possible to code up a dialog-driven database client ("Ok DB2" :).


As mentioned above, the code for the combination of dialog and database is available on GitHub along with some instructions. The README contains the output from a sample dialog including lots of debugging information which should help you understanding the data-driven dialog.

Bluemix: Where Python and Watson are in a Dialog

Right now I am working on a side project to hook up the Watson Dialog Service on Bluemix with dashDB and DB2. The idea is to dynamically feed data from DB2 into a conversation bot. To register and manage dialogs with the Watson Dialog Service, there is a web-based dialog tool available. But there is also a dialog API and a Python SDK for the Watson services available. So why not manage the dialogs from the command line...?

Converse with Watson from the Command Line

Here is a small overview of my Python client that helps to register, update, delete and list dialogs and that can even drive a dialog (converse with Watson) from the shell window on your machine. The code and for now some short documentation is available on GitHub as watson-dialog-client.

In order to use the tool, you need to have the Watson Dialog Service provisioned on IBM Bluemix. The service credentials need to be stored in a file config.json in the same directory as the tool "henriksDialog". The credentials look like shown here:

{
    "credentials": {
        "url": "https://gateway.watsonplatform.net/dialog/api",
        "password": "yourServicePassword",
        "username": "yourUserIDwhichIsALongString"
    }
}

The credentials are read by the tool to "chat" with the dialog service. The following commands are available:

• register a dialog by providing a new dialog name and the XML definition file
  "henriksDialog -r -dn dialogName -f definitionFile"
• update a dialog by identifying it by its ID and providing a definition file
  "henriksDialog -u -id dialogID -f definitionFile"
• delete a dialog identified by its ID
  "henriksDialog -d -id dialogID"
• list all registered dialogs
  "henriksDialog -l"
• converse, i.e., test out a registered dialog which is identified by its ID
  "henriksDialog -c -id dialogID"

Sample invocations and their output is available in the GitHub repository for this dialog tool. Let me know if something is missing or you had success chatting with Watson from the command line.

Store and Query XML Data with dashDB on Bluemix

XML Column in dashDB

I recently got asked whether it is possible to process XML data with dashDB on IBM Bluemix. The answer to that is that it is possible. dashDB is based on DB2 with its industry-leading pureXML support which I wrote many blog entries about. In the following I give you a quick start into preparing dashDB to store XML data and how to query it.

If you are using the regular dashDB service plans which are tailored to analytics, then by default all tables use columnar storage. That format provides deep compression and high performance query processing capabilities for analytic environments, but it is not suited for natively storing XML data. That is the reason why tables need to be created by explicitly stating ORAGNIZE BY ROW in the "Run SQL" dialog (see screenshot above):

CREATE TABLE myTable(id INT, doc XML) ORGANIZE BY ROW

The above statement creates the table "myTable" with two columns, the second of type XML, and in the classic row-oriented table format.

SQL/XML Query with dashDB

Once the table is created, data can be inserted. This can be done by using INSERT statements in the "Run SQL" dialog or by connecting other tools to dashDB. The "Load Hub" is designed for analytic data sets and does not support XML-typed columns. An introduction to inserting XML data can be found in the pureXML tutorial in the DB2 documentation.
After the XML data is in, the "Run SQL" dialog can be used again to query the documents. Queries can be either in SQL (SQL/XML) or in XQuery, see the screenshots with examples.

I hope that gives you a rough idea how to utilize the pureXML feature in dashDB, even though its main focus is analytics.

XQuery with dashDB