Thursday, July 13, 2017

Chatbots: Testing Contexts

Watson Conversation Tool in action
Some weeks ago I blogged about a tool I wrote for the IBM Watson Conversation Service (WCS). It allows you to manage your conversation workspaces from the command line and to test dialogs. Yesterday, I added a new feature to it that helps me (and you) to examine and modify the dialog context. Here is how you can test contexts with my watson conversation tool.

Friday, July 7, 2017

Best practices for lively chatbots

TJBot as lively chatbot
More and more chatbots are being developed and there are good reasons for it. Not all chatbot projects succeed. Often, missing user acceptance is stated. The dialog system might not have hit the nerve, might not have fitted into the target environment. Would you talk with a friend who does not remember your name is repeating the same five phrases over and over again? I would not. So what can be done to make chatbots more lively, more human-like? Here are some best practices and ideas on how to implement them.


I started my series on chatbots with lessons and tips from a chatbot hackathon. In that blog I focused on general aspects of building dialogs and designing a conversation system. The language needs to fit the audience. It is something we will look at again today. In a recent blog post I shared tips and tricks for building chatbots. It is possible to carry context throughout a conversation and embed conditions and advanced expressions into the dialog flow and single reponses. We will use that to implement some of the best practices found below. Building lively chatbots could also mean to give the bot a face. The open source project TJBot (pictured) is an example for that. The TJBot can listen, speak and see, give additional feedback and interact through its arm and its light. We won’t cover those aspects, e.g., hardware design or user interfaces, in this blog entry.

Monday, July 3, 2017

IBM Marketing and DB2, Db2 and dashDB

IBM Cloud with Bluemix: DB2, Db2, dashDB
This week I am going to acknowledge that I have been with IBM for 16 years. Looking back, everything might seem brighter and better than it was. However, I remember working in great teams, interesting and challenging projects, many successes and some failures to learn from, and the constant changes. During the time with IBM I got used to those many changes to product names, the constant rebranding. When I first heard about a change to DB2 that was announced last week, I scratched my head and moved on. The product itself does not change, it's a name.

Here are some of the changes, see the Db2 website for details:
  • DB2 for Linux, UNIX, and Windows (DB2 LUW) is now names just "Db2".
  • DB2 for z/OS is "Db2 for z/OS".
  • dashDB TX (dashDB for Transactions) is referred to as "Db2 on Cloud". It reflects that it basically was and is a fully managed Db2 (LUW) database system.
  • The former "IBM DB2 on Cloud", the cloud-hosted version of DB2 LUW, is now named "Db2 Hosted" to correctly imply what it is.
To keep you mentally flexible and to stimulate your brain cells, the uppercase B in "DB2" is now lowercase. "Db2" is what you will see more and more on the outside. But the product itself, as stated above, will remain "DB2". Therefore, you don't have to rewrite your test procedures. And if you have to prepare slides, e.g. for one of the upcoming IDUG conferences, there is one benefit with the new naming: Autocorrect is correct now...

A great Summer and a sane week!

Monday, June 26, 2017

More Tips and Tricks for Building Chatbots

Chatbot Architecture
You build your first chatbot and it is working ok. Did you know that you can make chatbots even more interactive? That you can access conversation metadata and application variables inside the dialog nodes? You can even use predicates to tailor output to the usage scenario. As a follow up from our “Lessons and Tips from a Chatbot Hackathon“, let’s dig deeper into important features of the IBM Watson Conversation service on the IBM Cloud with Bluemix.

Wednesday, June 14, 2017

DB2 Security: Privilege(d) Insight

DB2 Security
Today's blog entry is about an old topic: Who I am, why I am and what I am (allowed) to do? Users and their privileges are at the core of database security. Information about them are stored in the IBM DB2 catalog (or IBM ashDB catalog) which can be queried. Thus, it is good to know your way around and being able to extract that information. Kind of as a reference for myself, here is a summary.

I already covered parts of this topic in the past:

What is important to know is how to extract the information from the catalog:

Friday, June 2, 2017

EgoBot: Fun with a Slightly Mutating ChatBot

Fun with the Bluemix EgoBot
Over the past day and evening I had some fun with a slightly mutating chatbot. The API for the IBM Watson Conversation service offers REST calls to query and change the workspace, the parts that make up a chat. So why not try writing a chatbot that is egocentric? A chatbot that answers questions about itself, that is happy as long as everything is related to itself? Well, let me tell you about this fun project I call EgoBot.

The EgoBot is at an early stage right now. It supports queries about some of its metadata and adding new intents. And it has both an English and a German version (does language change its character...?). You can see a sample session below.

Chatting with the Bluemix EgoBot
To find out more about this chatbot head over to the EgoBot GitHub repository. The bot is written in Python and has everything to get you started with either an English or German conversation. Let me know about your Friday fun.

Thursday, June 1, 2017

How to Manage Bluemix Service Keys via CLI

You probably know that CLI stands for Command Line Interface. And you are aware that IBM Bluemix and Cloud Foundry offer a CLIs. Did you know that you can manage service keys from the command line? Adding new credentials, obtaining keys, and deleting service entries is really simple and fast. In the following, I will show you the commands and use my chatbot project and the IBM Watson Conversation service on Bluemix as example. And I will be using Bluemix in Frankfurt, Germany. So brace yourself for a quick tour through managing service keys from the command line.
manage Bluemix service keys from the command line
Sample Session Managing Keys


With IBM Bluemix Cloud Foundry you have the choice of using either the Cloud Foundry CLI (“cf”) or the Bluemix CLI (“bluemix” or “bx”). The “bx” command has an option for the “cf” commands. Both CLIs can be downloaded from the same page in the Bluemix documentation. The CLIs have many options to manage apps, services, organizations, spaces, and much more. Both can also be extended through plugins. You can even write and integrate your own plugins. I would recommend using the Bluemix CLI because it offers more features, including a handy option to update itself. For the example I am going to use the Cloud Foundry CLI to demonstrate the general case.
Bluemix offers many services, big and small, in its catalog. Most of those service can be used by more than just a single user, a single app, and not just from within Bluemix. Therefore, creating several credentials for a service, so-called service keys, is essential to consuming a service. The keys can be managed from the browser-based Bluemix console or on the command line via CLI.

Manage Service Keys

Using the Cloud Foundry or Bluemix CLI, the first step is to login. As shown in my example, I am using the API endpoint for Bluemix Public in Frankfurt, Germany:
>> cf login -a
I am prompted for my email address as username and the password. Depending on my account usage I might also need to select the organization and space I want to work with. As next step, I am looking for the instance of my IBM Watson Conversation service. This is used for my chatbots and I would like to create new credentials for some tests. The “services” command returns all services, on Unix systems “grep” helps to filter the result:
>> cf services | grep -i conversation Conversation-er conversation free hlred create succeeded
The name “Conversation-er” is the name of my Conversation service instance. Now I want to list the existing service keys. It can be done with the “service-keys” or “sk” command:
>> cf service-keys Conversation-er Getting keys for service instance Conversation-er as…
name Credentials-1
Only one service key labeled “Credentials-l” is present. To add new credentials I can use the “create-service-key” or “csk” parameter:
>> cf csk Conversation-er Conv-DE-user2 Creating service key Conv-DE-user2 for service instance Conversation-er as… OK
I chose the name “Conv-DE-user2” for the service key. Let’s see if it was added.
>> cf service-keys Conversation-er Getting keys for service instance Conversation-er as…
name Credentials-1 Conv-DE-user2
To take a look at the actual credentials, the “service-key” is the right option. It fetches the username, password and everything else making up the credentials. For IBM Watson services the gateway URL is part of it:
>> cf service-key Conversation-er Conv-DE-user2 Getting key Conv-DE-user2 for service instance Conversation-er as…
{  “password”: “BFyyHxxxGnO”,  “url”: “”,  “username”: “ffffffff-458f-4111-9dd4-03xx610xxbxx” }
Existing service keys can be deleted with the “delete-service-key” or “dsk” command. Recreating keys is one way of implementing rotating passwords (credentials).
>> cf dsk Conversation-er Conv-DE-user2
Really delete the service key Conv-DE-user2?> yes Deleting key Conv-DE-user2 for service instance Conversation-er as… OK


As shown above, it is pretty simple to manage service keys from the command line.Both the Bluemix and Cloud Foundry CLIs can be used. But not only the credentials can be administrated from the command line. Create services, bind them to apps, request billing and usage information, and more. And, as mentioned above, you can even extend the functionality through plugins and create your own.

Note that this blog entry was first published at

Wednesday, May 31, 2017

DB2 Security Mysteries, Surrogates and Trusted Contexts

DB2 Security Question
Recently, I was contacted regarding an older blog entry discussing the DB2 security feature of surrogates. During an audit a strange entry was found in the catalog table SYSCAT.SURROGATEAUTHIDS. Whether I could take a look. So, let us take the security-themed tour through some DB2 catalog tables together.

What is the strange entry? In a new database, created as regular or restrictive database, the following entry is found. What does it mean and what is SYSATSCONTEXT (highlighted below)?

DB: HLTEST => select * from syscat.surrogateauthids

GRANTOR                                                                                                                          TRUSTEDID                                                                                                                        TRUSTEDIDTYPE SURROGATEAUTHID                                                                                                                  SURROGATEAUTHIDTYPE AUTHENTICATE CONTEXTROLE                                                                                                                      GRANT_TIME               
-------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- ------------- -------------------------------------------------------------------------------------------------------------------------------- ------------------- ------------ -------------------------------------------------------------------------------------------------------------------------------- --------------------------
HLOESER                                                                                                                          SYSATSCONTEXT                                                                                                                    C             PUBLIC                                                                                                                           G                   N            -                                                                                                                                2016-10-18-

  1 record(s) selected.

The DB2 Knowledge Center provides an overview of all security-related catalog views. As can be seen, SURROGATEAUTHIS "lists the authorization IDs for which another authorization ID can act as a surrogate". The table is not mentioned in the document of default privileges granted on creating a new database. So let's go to the roadmap to catalog views and from there to the entry for SURROGATEAUTHIDS. SURROGATEAUTHIDS is found in the category of "protected tables".

The catalog entry above indicates that I, as database and instance owner, have granted something to "SYSATSCONTEXT" when the database was created. The TRUSTEDIDTYPE is "C" and means the record belongs to a trusted context. Thus, as next step, the catalog views SYSCAT.CONTEXTS and SYSCAT.CONTEXTATTRIBUTES should be visited:

DB: HLTEST => select * from syscat.contexts

CONTEXTNAME                                                                                                                      CONTEXTID   SYSTEMAUTHID                                                                                                                     DEFAULTCONTEXTROLE                                                                                                               CREATE_TIME                ALTER_TIME                 ENABLED AUDITPOLICYID AUDITPOLICYNAME                                                                                                                  AUDITEXCEPTIONENABLED REMARKS                                                                                                                                                                                                                                                      
-------------------------------------------------------------------------------------------------------------------------------- ----------- -------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- -------------------------- -------------------------- ------- ------------- -------------------------------------------------------------------------------------------------------------------------------- --------------------- --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SYSATSCONTEXT                                                                                                                            100 SYSATS                                                                                                                           -                                                                                                                                2016-10-18- 2016-10-18- Y                   - -                                                                                                                                N                     -                                                                                                                                                                                                                                                            

  1 record(s) selected.

DB: HLTEST => select * from syscat.contextattributes

CONTEXTNAME                                                                                                                      ATTR_NAME                                                                                                                        ATTR_VALUE                                                                                                                       ATTR_OPTIONS                                                                                                                   
-------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------------------------------
SYSATSCONTEXT                                                                                                                    ENCRYPTION                                                                                                                       NONE                                                                                                                             -                                                                                                                              

  1 record(s) selected.

The above entries show details for the trusted context. The "ATS" indicates it is part of the DB2 Administrative Task Scheduler. In a non-restrictive database, by default, any user could add a task to the scheduler. Those tasks are later executed as the specific user, i.e., using the authorization ID of that user. Hence, the trusted context is used. They allow to execute SQL statements on behalf of an user (see my old blog entry on "power of attorney and trusted contexts").

Conclusion: The strange catalog entry belongs to the infrastructure of the DB2 Administrative Task Scheduler and seems to be used run scheduled task on behalf of DB2 users.

Wednesday, May 24, 2017

Updates to Chatbot API and Tooling

Build Chatbots with IBM Bluemix
If you have been coding up your own chatbot using the IBM Watson Conversation service on IBM Bluemix, then you might be aware of the new features in Conversation API. Earlier this year I wrote a small tool to manage chatbot workspaces (read here the blog about the management tool). I updated the tool to the newest API version and added basic support for one of the new API functions.

You can now invoke my "Watson Conversation Tool" with the "-logs" parameter. It retrieves recent interactions, i.e., processed messages with all the details. The logs are great to see what's going on (of course), to understand why a specific response was generated by Watson Conversation and hence to improve a chatbot.

[henrik@mymachine] python -logs -id "123123b8-123b-1234-a656-6dxx5a2xxf15"

  "pagination": {},
  "logs": [
      "log_id": "fba37b8c-a4a8-4ec1-a2d0-7c361d24f441",
      "response_timestamp": "2017-05-24T08:02:20.882Z",
      "request": {
        "input": {
          "text": "@db2bm Was liegt heute an, wie wird das Wetter?"


      "request_timestamp": "2017-05-24T08:02:18.494Z"

The API allows to filter, sort and paginate the output. If you want that added to my tool, let me know by leaving feedback or opening an issue.

Friday, May 12, 2017

Chatbot Hackathon: Lessons Learned and Tips

Recently, I was a coach at a hackathon in Germany. Students were tasked to build solutions for a given topic and the solution had to include a chatbot based on the IBM Watson Conversation service. It was impressive how broad the solutions were, what other services were integrated with a chatbot and how easy it was to build a solution using IBM Bluemix. The following “lessons learned” and tips are from my work as coach. They hopefully help you design your own great chatbot.
Chatbots with IBM Watson

Lessons Learned

Hackathons are intense events and require sharp focus and quick decisions. They pack everything from finding the right idea (brainstorming), defining a solution, its users and development phases (using Design Thinking) to building a prototype / minimal viable product (MVP) and pitching it. The first lessons learned and tips deal with the conversation, the dialog, the language itself:

Wednesday, April 12, 2017

DB2 Easter Egg Quiz - Kind of

Are you ready for a new DB2 quiz? Here is a tricky question for you. It is about DB2 security, it is (kind of) hardware-related and deals with a relatively new DB2 feature. Curious...?

The following shows the output of a DB2 tool, shipped with your regular DB2 LUW these days:

Keystore Password:
Password successfully stashed to db2_pkcs11_pwd.sth

xxxxx completed sucessfully.

Have a clue? (The "xxxxx" is replacing the name of the db2 tool). If you know the answer, comment and also paste the link to the page in DB2 Knowledge Center describing that tool. I will publish the comments in the next few days.

Wednesday, April 5, 2017

Aero Expo, Drones and the IBM Cloud

The Aero Expo, the Global Show for General Aviation, is running in my hometown Friedrichshafen from today until the weekend. One of the expo and conference topics is drones of the future (AERODrones UAS Expo). Drones or UAV (Unmanned Aerial Vehicles) have been and are a hot topic for IBM and its customers. Let me give a brief overview of some interesting work where drones, artificial intelligence, analytics, database systems, Internet of Things (IoT) and the IBM Cloud come together.

Saturday, April 1, 2017

The 10 Top Stories at April Fools' Day

Top 10 Stories
Today is April Fools' Day and many websites have made up funny and unbelievable stories. What are the best of them? What are practical jokes done online? Read on to find my collection of the best stories found on April Fools' Day.

Tuesday, March 28, 2017

Chatbots: Manage Your Watson Conversations from the Command Line or App

Manage Watson Conversation Workspaces
I am a big fan of using the command line as most of you know by now. This applies to interacting with IBM Bluemix cloud and its services and for database systems like DB2 or dashDB. Thus, I was excited when the IBM Watson Conversation service added API functions to manage workspaces. To test the new API I wrote a small Python-based tool to manage my Conversation workspaces. It both demonstrates the API usage as well as gives you a nice command line tool to list your workspaces, update them, save local copies or even create or delete workspaces. Read on and learn how to manage your Conversation workspaces.

Monday, March 20, 2017

IBM Bluemix in Germany, includes dashDB and Cloudant

IBM Bluemix in Germany, in German
Today, I wanted to share some exciting news with you. Most of you know that I am German. Thus, it is terrific to have IBM Bluemix available from Frankfurt, Germany, today. As can be seen on the screenshot on the right, the new Bluemix region is labeled "eu-de".

Having Bluemix Public in Germany is a big step for the IBM Cloud and customers alike. Being located next to DE-CIX means low network latency for German and European customers. Utilizing Bluemix Public in London ("eu-gb") it is possible to deploy applications with high-availability requirements redundantly within Europe. The database-as-a-service offerings dashDB ("DB2") and Cloudant are already available in the Bluemix catalog. More database and analytics services are to follow. You can check out the list of initial services here in the Bluemix Catalog for Germany.

That's all for today. I am back to MY German Bluemix...

Friday, February 24, 2017

Securing Workloads on IBM Cloud - Some Resources

Security Guides for IBM Cloud
Security Guides for IBM Cloud
Recently, I provided you with an overview of security and compliance resouces for IBM dashDB and Cloudant. Today, I want to take a broader view and point you to some good introductory material on security for cloud-based workloads. It consists of an overview of different cloud deployment models and their components. Then it digs into each of those categories and takes a look at how to secure those components and the data.

Thursday, February 23, 2017

Location and Intent Matter: Data Privacy vs. US Government

Data is locked away from the US authorities
Some data is locked away
Earlier this month and last Summer two interesting cases related to data privacy were decided. Both concern US search warrants for email data stored outside the United States of America. In July 2016 the United States Court of Appeals for the Second Circuit ruled that Microsoft does not need to hand over email data stored in Ireland. This February, the United States District Court for the Eastern District of Pennsylvania decided that Google must produce the emails which were stored outside the USA. The last case is not finally decided because Google plans to appeal the ruling. Independent of that, what is the take-away from these rulings? Let's take a look.

Monday, February 20, 2017

Write Your Own CLI Plugins for Bluemix Cloud Foundry

Screenshot showing README for my plugin
README for my Plugin
Last year I blogged about how I am using plugins to extend the Bluemix Cloud Foundry command line interface (CLI). The CLI has a set of commands to manage plugin repositories and to install and uninstall plugins. It is pretty easy to use and there are some useful plugins available from Cloud Foundry and IBM Bluemix. Having mastered the first step, I wanted to know how plugins work and what it takes to write my own plugin. Here is what I learned.

Friday, February 17, 2017

Carnival: Even DB2 Wears a Mask (Database Security)

Word cloud for data privacy and security
Data Privacy and Security
Right now we are in the hot phase of the carnival season. Many people are wearing masks. Some move into other characters, some just hide their real identity. Did you know that DB2 is also in the mood for carnival and wears a mask? Here is what my DB2 is doing these days...

Friday, February 3, 2017

Security and Compliance for IBM dashDB and Cloudant

Database Security and Compliance
Database Security & Compliance
I often get asked about the security features of IBM dashDB and Cloudant. Both are database services ("DBaaS") offered on IBM Bluemix. Once the security topic is dealt with, compliance-related questions are next. A good chunk of questions can be answered by going over the provided product documentation. Here are the links to get you started on database security and compliance.

Tuesday, January 31, 2017

Improve Security for your Domains on IBM Bluemix

The key to security
Secure Your Apps

Do you use your own domain names with IBM Bluemix? Then you probably know that you can secure access by adding SSL certificates for your domain. During my recent my work with so-called Context Path Routes for Bluemix Cloud Foundry apps I stumbled over a great project, bluemix-letsencrypt.

The project bluemix-letsencrypt (available on GitHub) provides a Python script and Bluemix app that automate generation and upload of SSL certificates. It uses Let's Encrypt as Certificate Authority (CA). The only thing you need to do is to specify your domain name and email address. Thereafter, the script is run. It uses the Bluemix app to generate the SSL certificates. For the details head over to the Bluemix blog and read the entry "Securing Custom Domains with Let’s Encrypt"

Wednesday, January 18, 2017

Context Path Routing of Apps and Services in Bluemix

Context Paths for Bluemix Apps
As I mentioned in my post yesterday about simplified deployment of complex apps, I have been working on a sample for Context Path Routing to be used with IBM Bluemix. But what are context path routes and what does the sample do? Here are the details.

Cloud Foundry introduced Context Path Routing last year. Until then there was the requirement that each app (or service) was served from its own hostname. Now, apps can share a host with each app being served from a specific path on that host. Here are two examples:

  1. When building a larger website, there could be several so-called microsites embedded. With Context Path Routing it is possible to serve, e.g., from one web app and or from other apps. All these apps could be written in different programming languages such as Node.js, Python, Java and others. 
  2. For a more complex microservice-based app, following the principles of the Twelve Factor App, there could be several (backing) services involved. The app and each would require their own hostname. With Context Path Routing the app could use and services could be served from,, etc.
So how could you use the new routing feature? There are several Cloud Foundry CLI commands related to routing. They allow to specify an additional path for application routes. Manifest files also have optional route properties that could be set. To get started, take a look at my Context Path Routing sample on GitHub for details. It has two small apps written in Python and Node.js that share a host using specific paths. The apps can be deployed with a single command and allow to experiment with context paths. The "What this samples does" section gives you some ideas.

Tuesday, January 17, 2017

Bluemix: Simplified Deployment of Complex Cloud Foundry Apps

Two apps from single manifest
Recently, I was looking over a microservice-based app to be deployed to IBM Bluemix. There app consisted of several pieces, the app itself and multiple services. Fortunately, all could be deployed with a single "push". Here is how.

Cloud Foundry allows multiple apps to be described with a single manifest file. That is, properties for several apps (or services) can be put together. For each app its name and the location where its code is found need to be specified. They are shown in blue in my sample manifest file. Each app can be deployed to a specific machine, identified by the host and domain name. For the example I chose a different approach. It is the relatively new "routes" property. It allows to combine those properties and even add paths information. The routing is highlighted in yellow below. All I needed to do is to execute a simple "cf push" command and the entire application with its multiple pieces got deployed.

Here is the sample manifest.yml file:

# This manifest deploys two applications.
# Both use the same host and domain name as defined
# by their respective route(s) property. The first app
# uses the root path, the second the "sub" and
# "lower" paths.

# The Python app starts here
- name: yourname-myapp
  memory: 256M
  command: python
  - route:
  path: ./top/ 
# The Node.js app starts here 
 - name: yourname-myapp-node
  - route:
  - route:
  path: ./lower/
If you wonder how the entire project looks like, visit for the source code and a more detailed description. I put this repository together to showcase Context Path Routing on IBM Bluemix which I will discuss in an upcoming blog post.

Tuesday, January 10, 2017

DB2 Quiz for the Resource-Minded & IDUG EMEA 2017

Did you know...?
I hope you had a good start into 2017. I am already up and running and accomplished some important tasks. I submitted my presentation proposals for the IDUG DB2 Tech Conference 2017 in Lisbon, Portugal. The Call for Papers is still open until February 20th. The IDUG Conference is always a great place to learn new stuff and meet great people.

Speaking of learning new stuff. Do you know which DB2 function or procedure produced the following output on my system? As you may notice, I checked some DB2 system processes using SQL. The feature I am using is around since DB2 version 9.7.
If you have a guess, leave a comment or send an email.

------ ----------------- -------------------- ------------ ------------
0      db2fmp                           15234            6            7
0      db2vend (PD Vendo                15064            5            3
0      db2ckpwd 0                       15060            0            0
0      db2ckpwd 0                       15061            0            0

0      db2ckpwd 0                       15062            0            0
0      db2sysc 0                        15054           91          128


  8 record(s) selected.


Related Posts with Thumbnails