Monday, August 22, 2016

Notes on Notebooks, Data, DB2, and Bluemix

Weather Graph in Jupyter Notebook
Some time ago I shared with you how I used open data like weather data and CeBIT statistics to have some fun with Jupyter Notebooks. Notebooks are an old, but now - in the age of Cloud Computing - trendy way of collaborative data exploration (see this article for some background reading). Notebooks consist of so-called cells, places to put code, instructions, text and more. Cells can hold Markdown-formatted text, code written in Python, Scala and other languages. It is possible to fetch data from DB2, dashDB and other database systems and process it in the notebook, creating stunning graphics. And with extensions such as RISE (reveal.js IPython Slideshow Extension) those notebooks replace Powerpoint & Co as source for great data-driven presentations. How to use notebooks and DB2 is what I plan to present at the IDUG EMEA 2016 Conference in Brussels later this year.

If you can't wait until then I recommend to take a look at these recent blog posts on how to get started with Notebooks and data on Bluemix:
In case you have trouble programming your notebooks, head over to Stack Overflow and search for "ipython-notebook" or "jupyter-notebook".

That's it for today with my notes on notebooks.

Wednesday, July 13, 2016

Data, Dialogs, and Databases

Watson Dialog Specification
I recently wrote about managing IBM Watson dialogs from the command line and that I wanted to bring database records into a dialog, combining different IBM Bluemix services. I succeeded and here is the follow-up on how I perform dialog-driven lookups in dashDB / DB2 and update the conversation for Watson to return user-specific answers. The application is written in Python and available on GitHub again and builds on the experience and code from the watson-dialog-client I published earlier.

The user interacts with the Python app which basically is a loop, waiting for the user to end the dialog. The script uses the dialog API to send the user input and obtain a response from Watson. It also checks the dialog profile variables to determine whether there are changes or any variable of interest is set. Depending on state information obtained from the profile variables dashDB is queried. Database results are either directly returned to the user or are used to update the dialog profile.

Data and Dialog Combined
To demonstrate (read: experiment with) data-driven dialogs I created a simple table "dialogdata" in dashDB and populated it with few records. The dialog prompts the user for the name. When entered it is used to look up the corresponding database record which then is fed into the dialog variables. As an example the dialog allows to ask for the birthday or age. The answer is composed of a template with the user-related information filled in.
Another example for combining dialog and database is to enter a service name, i.e., to request an action. This is detected by the Python script again by checking the profile variables. Then, dashDB is queried and the result is directly returned to the user. This way it would be possible to code up a dialog-driven database client ("Ok DB2" :).

As mentioned above, the code for the combination of dialog and database is available on GitHub along with some instructions. The README contains the output from a sample dialog including lots of debugging information which should help you understanding the data-driven dialog.

Thursday, July 7, 2016

Bluemix: Where Python and Watson are in a Dialog

Right now I am working on a side project to hook up the Watson Dialog Service on Bluemix with dashDB and DB2. The idea is to dynamically feed data from DB2 into a conversation bot. To register and manage dialogs with the Watson Dialog Service, there is a web-based dialog tool available. But there is also a dialog API and a Python SDK for the Watson services available. So why not manage the dialogs from the command line...?
Converse with Watson from the Command Line

Here is a small overview of my Python client that helps to register, update, delete and list dialogs and that can even drive a dialog (converse with Watson) from the shell window on your machine. The code and for now some short documentation is available on GitHub as watson-dialog-client.

In order to use the tool, you need to have the Watson Dialog Service provisioned on IBM Bluemix. The service credentials need to be stored in a file config.json in the same directory as the tool "henriksDialog". The credentials look like shown here:

    "credentials": {
        "url": "",
        "password": "yourServicePassword",
        "username": "yourUserIDwhichIsALongString"


The credentials are read by the tool to "chat" with the dialog service. The following commands are available:
  • register a dialog by providing a new dialog name and the XML definition file
    "henriksDialog -r -dn dialogName -f definitionFile"
  • update a dialog by identifying it by its ID and providing a definition file
    "henriksDialog -u -id dialogID -f definitionFile"
  • delete a dialog identified by its ID
    "henriksDialog -d -id dialogID"
  • list all registered dialogs
    "henriksDialog -l"
  • converse, i.e., test out a registered dialog which is identified by its ID
    "henriksDialog -c -id dialogID"
Sample invocations and their output is available in the GitHub repository for this dialog tool. Let me know if something is missing or you had success chatting with Watson from the command line.


Related Posts with Thumbnails