Combining Bluemix, Open Data on Tourism and Watson Analytics for some Friday Insight

Inbound and Outbound Tourism, Watson Analytics

Yup, it is Friday again and the weekend is coming closer and closer. The Carnival or Fasnet/Fastnacht season is close to its peak and some school holidays and inofficial holidays are coming up late next week. Tourists are pouring into carnival strongholds. Why not take some time today to test drive the Bluemix Analytics Exchange and Watson Analytics with tourism data and try to get some insight?

My DB2 pureScale / Parallel Sysplex Moment

Last week, members of the German union for train operators/engineers were on strike and only a fraction of trains were operating. I had to go to Zurich, Switzerland, but most of the trains on my usual route were out of service. That's when I had my DB2 pureScale or DB2 Parallel Sysplex moment.

What is special about a DB2 Parallel Sysplex or DB2 pureScale configuration? It is a data sharing cluster. If one node in the cluster or a network connection goes down and is inaccessible, the other components can pick up the work. All this is usually transparent to the application. In my case I was the application and data at the same time: Try to get me to Zurich, discuss some issues in person, get back home.

(Updated) Special hybrid, object-capable, all purpose database

Recently I had to deal again with a special database. Let me share my observations.

This special database seems to have excellent compression capabilities as a ot of stuff can be cramped into it. From the outside the database looks small, but it seems to be able to store 10 times its advertised capacity.

This database can handle relational data, large objects, unstructured data, etc. and has several storage areas and storage types. For the experienced user data access is really fast and it seems index-only. For the infrequent user the only way of data access is through repeated table scans.

The database is equipped to deal with "situations" and act in "emergencies". Its capabilities include having candy up for distribution exactly when needed, a first aid kid with the right number and size of bandaids, chapstick, hand lotion, and some hand sanitizer.When fully loaded, the database can be used as a self-defense device, can be thrown as projectile even on longer distances.

I could go on with more features (you can in the comments), but do you know what database I described above...?

Update: My wife's handbag is one of the described special databases.

Winter at the lake

A Katamaran and the car ferry seen in front of the Swiss Alps on Lake Constance (as seen from Friedrichshafen).

Where are the soap bars...? - Database Appliances

Do you remember the days when there were soap bars in the hotel bathrooms? The days when you began washing the hands and then realizing that the soap bar had to be unwrapped? The room service would restock soap, clean the soap dish, and empty the waste basket. Now enter the age of appliances. Many hotels have switched over to "appliances", wall-mounted soap dispensers. They allow for savings for the hotels due to less effort (service, restocking, ...) and are "greener", i.e., the dispensers cause significantly less waste.

The same transition is also happening for database servers. Instead of purchasing hardware (server, storage, network equipments, etc.) and software separately and assemble and test them on-site in a lengthly process, appliances often are delivered ready-to-use (see IBM PureData System). The key to success is to have the right ingredients. Would you use a soap dispenser that says "good for hands, hair, and dishes"? You would expect to have cream soap, shampoo and conditioner, and other cleaning liquids in separate appliances. The same is true for database appliances. That is the reason why IBM (other than some competition ("supports both OLTP and OLAP" )) is offering PureData System for Transactions, PureData System for Operational Analytics, and PureData System for Analytics. Different workloads and requirements need database appliances with different ingredients.

BTW: I still like a nicely shaped, slightly scented bar of soap. It makes the right gift...

How the iPad changed our vacation

After moving to Germany in 2008, our family vacation this Summer was finally (see here why not sooner) in San Jose and San Francisco. The kids were able to use "their" playgrounds of the past, see their preschool and take a look at the school our older son was a student at. What was interesting for all of us was to watch how mobile technology, here the iPad, changes lifes. Note that I am a frequent traveler myself (and sometimes could be one of the cases described below...), but below is what stuck out to the rest of the family.

• While in the past, breakfast rooms were dominated by big screen TVs and the usual morning news, now many people are eating and "smartphoning" or "iPadding" at the same time, occasionally turning heads towards the TV.
• While walking in San Francisco (not much walking in San Jose) special care needs to be taken of people walking and talking. When someone is yelling next to you, the person on the other end of the phone is usually meant. At first this was irritating to the kids.
• iPads are the new "pocket cameras". People are taking pictures using "big screens"...
• Beware of people holding an iPad in front of them and trying to navigate the streets. It was kind of funny and disturbing to see that lamp poles are not on the maps... (and this was before iOS 6 ;-).
• On crowded buses (e.g. line 30) we noticed people trying to locate the position using an iPad to decide when and where to get off.

After noticing both lots of iPad adverts around Union Square where we stayed and then the iPads in use, the focus naturally was on them. This was fueled by the ongoing Apple vs. Samsung trial and the updates in the morning news. Thus, even without owning an iPad (only a Lenovo IdeaPad A1) our family vacation was changed by the iPad...

Welcome to Switzerland! Choose your database territory wisely...

One of the many international aspects of living in the Lake Constance region is that you encounter Swiss, Austrian, and German people, and even those from Lichtenstein on a daily basis. Language (mostly the dialect) is the most obvious differentiator in direct contact as well as a license plate on car. Sometimes, there are also the small and funny struggles with technology which show country-specific preferences.

Depending on the weather, it often happens that you walk in downtown Friedrichshafen and suddenly receive a text message "Welcome to Switzerland!" and your mobile phone started roaming using a Swiss mobile phone company instead of your German mobile provider. Similary, depending on the weather again, you can easily pick up a German provider along the Swiss shore or in Romanshorn waiting for a ferry to Germany. Always make sure that your phone has the right provider, else it will be expensive even receiving a call.

Then, there also funny annoyances like Google Maps. It is showing the Swiss-German border in a very liberal way. The German Reichenau island is, according to Google, Swiss territory. Fortunately, the borderline is clear to everyone and the countries are cooperating and helping each other across the lake.

When working with a database system like DB2, you also have to select the territory, codepage and collation for a database. You could have German, Swiss, Austrian (though no Lichtensteinian) databases side by side in a single DB2 instance. One thing you cannot do is to change the territory as we can do on the mobile phone when picking our home provider. So it would continue to say "Welcome to Switzerland!"


Larger Map

Two whales in the air: Zeppelins over Friedrichshafen

Yesterday I got the sudden chance to climb up a church tower in Friedrichshafen. It was also the rare opportunity to see two Zeppelins side by side (taken with my mobile phone). One had just departed with new passengers, the other was approaching the Zeppelin landing site.
What can also be seen close to the center is the runway of Friedrichshafen Airport, on the left of it the Zeppelin hangar and the Friedrichshafen Messe (expo & fair). The open space in the foreground on the right will be home to the new headquarters of ZF.

Travel and Security Concepts

When I am teaching a DB2 class, or data management in general, security is always on the agenda. Authentication and authorization are basic terms, but often they are mixed up. That is the moment when I bring the following analogy.

Authentication is about making sure that I am really the person I say I am. This can be done during travel through a "government-issued identification card" (passport, driver's license, etc.) or when working with a computer through a userid and password (or keycard, token generator, a PIN, or a fingerprint). Often a special token is then issued which can be used as simplified identicator for a limited time. During air travel this is the (in)famous bording pass. On the computer it could be setting a cookie, a session identifier or something else.

Authorization gives me (after I have been identified) access to specific resources, it is the access control part of security. I am allowed only to board a specific flight, I don't have access to the lounge, I am allowed to sit in economy class, but not in business class, the pilot is allowed to use electronic devices during take-off and landing, I don't.

Next on the agenda: Groups and roles in DB2 (groups are controlled outside the database, roles inside).

Business Travel and DB2 SQL Compatibilty

As part of my job I travel a lot within Europe. Most countries accept the Euro and participate in the Schengen Agreement (forming the Schengen Area) which means borderless travel and few, if any, controls (more that in a future blog post). What remains are the different languages and the different culture, including food. During one of the recent journeys I came up with a nice comparison of the DB2 SQL (Oracle) Compatibility and understanding the different European languages.

For the performance of PL/SQL and the Oracle SQL dialect in DB2 we often have to point out that statements are compiled to regular DB2 runtime code (so-called sections and packages). At runtime DB2 basically doesn't know whether DB2's SQL or the Oracle-like syntax was used. Both perform equally well as there is only one runtime infrastructure, no emulation. The same is true for SQL vs. XQuery, there is the same single runtime infrastructure.

In school I learned English, Latin, and French, later at university Spanish. I don't speak Latin, French, or Spanish fluently, but I understand enough to get around. The same for Dutch, Danish, etc. What that means is that I don't need a translator, but can directly understand local road signs, hotel employees, newspapers. This gives a "performance advantage", again, as I (my "runtime infrastructure") can process the languages directly, not all ("compatibility rate"), but enough for most common situations. Here DB2 and I differ as DB2 has a higher compatibility rate...

Aero Expo and Database Monitoring

Some years back (is my blog really that old...?) I had written about the AERO Expo, a global general aviation show in Friedrichshafen. Today, I received an invitation/marketing email reminding me about the upcoming expo. Looking over the list of exhibits and demonstrations, it is very similar to database systems or even information management. There is the core product (the aircraft/engine or database system) and then a good chunk is about the tooling and maintenance.

What - to me - stands out is the monitoring. It is critical in flight, to keep both your aircraft and your database system up and flying. I am not a pilot and cannot really comment on what is crucial, but for database systems performance monitoring is a very interesting topic. Did you know that InfoSphere Optim Performance Manager Extended Edition is capable of monitoring the application stack ("end to end monitoring")? To me as a frequent traveller this looks like making sure that both the aircraft is running well as well as understanding what is happening in terms of air traffic and on the ground, i.e., it provides the whole picture, not just a fragment.

Not sure whether I will visit the AERO this year (tickets anyone...?), I would look around to see if something similar to end-to-end monitoring is available.

BTW: For those in Switzerland or South Germany, take a look at this cool project for displaying air traffic.

Basics: Multi-language support in DB2 Information Center

Today, I want to point you to something very basic, something that many of us take for granted: Multi-language support in the DB2 Information Center. When you click on the link in the previous sentence, a new window should open and the DB2 Information Center with a page about its features should be displayed. Depending on your browser settings the page should display in German, Spanish, Italian, Traditional Chinese, etc. or English. On that page is a section that reads like this:

Basically, the DB2 Information Center displays the pages in YOUR language. This is nothing special and expected. By changing my browser settings and choosing a different language like shown here

and then reloading the frame inside the DB2 Information Center, I can now see the same section in a different language, e.g., German:

Now you may ask, why is he telling me? I often get asked about how things work in DB2 and where to find information. Just sending a link to a page in the DB2 Information Center and not worrying about the language is something I find very useful when working in an international context. Most prefer to read manuals in their native language. Moreover, using the integrated search works best when you know the search terms. But do I know the right terms when helping in a Spanish or French team? I can switch the browser, utilize the English search, then switch back to another language for the piece of information I found. And thereby learn another language just through reading about DB2 in Italian...

What is security and should I care?

I just returned from a longer trip. Without telling you where I travelled, I want to, I really have to talk about "security". I had to undergo checks in front of the hotel, when entering office buildings and malls, and several additional security checks at the airport. My suitcase and backpack were X-rayed here and there, quickly or with more focus, I was padded down, scanned, or just eyed up. But what for? So I started to test the checks.

I didn't hand over my cell phone or wallet before entering the scanner, I kept my backpack on while walking through them, I ignored the beeps. And I was successful - as expected.

Entering Information Management, database systems: Do you care about security, about protection of the data? Who has access to the system and who should have? Do you have checks in place and are they really working or are only show? Do you watch those in charge of protecting the data?

If you have some time, read a little bit about the DB2 security model or learn about the various tools in the IBM InfoSphere Guardium product portfolio.

BTW: I care about security, my personal security, that of others, and that of data.

P.S.: Highly trained professional in closed environment. Do not attempt at home...

Business trips and database migrations

As you might have guessed I have been travelling and I am going to travel again. Every business trip brings some risks and has associated costs, but each one also has its opportunities or specific goals. Many of my business trips are related to database migration projects, enabling customers or business partners to move from Oracle to DB2.

Depending on the trip destination there is more or less risk associated. There could be accidents, health issues, robbery or mugging, the risk of loosing your luggage, and of course of not getting enough sleep. To mitigate some of the risks, preparation is key, first of them knowing about the involved risks, what to watch out for. Often, you can benefit from the experience of others, e.g., reading a travel guide, reading hotel reviews, etc.

In terms of costs, usually there is a rough estimate based on data that can be quickly gathered and some travel experience. It helps to decide whether it is worth travelling. Would someone spend 5000 EUR for a 1000 EUR opportunity? What brings the trip in the long run?

Migration projects are similar to business trips. There are costs associated, there are risks, but also a benefit. Companies migrate from Oracle to DB2 because they want to save money, perform better, simplify their data centers and reduce the server and storage footprint, grow their businesses, enjoy competitive advantages, benefit from technology advances, or any combination of these. So they look into what it costs to get there and turn to IBM. Based on data that can be quickly gathered and migration experiences, a decision to move forward and to look into details can be made - similar to a business trip. Tools like MEET allow to analyse the level of compatibility and to quickly give an overview of problems. Experience with Oracle to DB2 migration projects, big and small, allows to attach costs to the issues found. A proof of concept allows to gain experience, similar to some business trips in a controlled environment to validate assumptions and to build up a portfolio of hotel and airline reviews, and to train the business travellers.

And finally comes the decision to embark, to start the journey. There are risks, but there are also best practices that can be followed and tour guides that can help if needed. As with business trips, there are tour reports of others: Coca-Cola about moving from Oracle to DB2 at the IOD conference in October, customers talking at the IDUG conferences about their migration projects, blog posts detailing why projects succeeded and lessons learned.

At some point everybody faces the question the first time: Do I want to travel? Especially for migration projects it is an unusual question that might only come up once and not everybody feels comfortable with. It might be similar to the first trip to a foreign country. But then there are resources to help and to assist with the decision and along the journey.

What is a smart phone?

Recently, my kids started pushing me to buy a new mobile phone for myself. Not that the kids really care about what I am buying, they want the old one as hand-me-down. They want to use the old mobile phone as gaming device. Anyway, as I am being pushed, I am looking into the market of smartphones. But what is a smart phone...?

Is it a phone that shuts down after enough work hours and tells me "family time"? Does it support "social integration", i.e., so that I talk with other people next to me and not just with folks on the phone (do you happen to enter rooms where everybody is talking, but only on the phone and not with each other)? Does it have an intelligent camera to enjoy a scenery live and not just as picture (there was a beautiful sunset over the Alps on a recent flight)? What happens when I press the "home" button? How well works the navigation assistance with a long shopping list in a crowded mall? And does it work without recharging for a whole week?

We'll see what phone I find. Any recommendations? My kids are pushing...

Data inconsistency causing 30 minute flight delay

On Friday evening I was traveling back home. The last leg from FRA to FDH is with a regional jet which always is parked in an outside position and requiring a bus transfer from the terminal. I was happy because the bus was leaving the terminal 20 minutes before the departure time, usually a sign of on-time arrival (that's what counts). However, it turned out differently...


Once we arrived at the airplane, all the passengers boarded and took their seats. We then waited for the usual "boarding complete" and welcome message which didn't come. Instead, the flight attendants counted the passengers, re-counted, compared their passenger list with the seated crowd, and then got busy. At first, they asked the passengers in business class to show their boarding passes. After some more discussions, they went around in economy class and asking for specific passengers. Suddenly, at least for us boarded passengers, two more passengers were brought to the plane and also another car from operations arrived.

With some more discussions between crew, operations, and some passengers, the puzzle eventually got solved: It seemed that the crew was handed an outdated list (which they discovered) as there were some last-minute changes. Some of the new passengers arrived on time for boarding while some others on the first list arrived late. Hence we had a mix-up in the passenger count and with some names. All this caused a delayed departure of 30 minutes (and 20 minutes for arrival).

Friday Fun: What people drink on the airplane...

Earlier this week when flying back I had some spare minutes and actually read the fineprint on the back of my boarding pass. The section "Dangerous goods in passenger baggage" notes that, among others, poisonous substances and radioactive materials are prohibited. However, at the bottom it is pointed out that, among others, this rule does not apply to alcoholic drinks. Well, that explains the behavior and facial expression of some fellow travelers...

Some flights, appliances, and "one size fits all"

As I wrote before, I was traveling within Europe. Similar to the Skymall brochure in North America, some airplanes feature the Worldshop brochure. I used some spare time and read about the new G3Ferrari Pizzamaker. This brought up memories of the Popcorn Maker, the Ice Cream Maker, the Donut Maker, the bread machine, and all the other advertised appliances. How many of them do you own?

One reason I don't own (all of) them is the storage space required when they are not used. Another reason is that they usually can only do what they have been designed for (sometimes not even that!) and it often requires reading additional instructions, getting familiar with the do's and don'ts. Sometimes, some of those "specialty appliances" are something en vogue for few months, then disappear again. I trust my (hightech) oven and my induction stove on helping me out when I need a pizza or some popcorn and they work fine for base regular food. They are my "one size fits all" solution to my culinary requirements.

The same goes for my database system. I trust DB2 to handle all kinds of data: Strings, numbers, dates, timestamps, and - yes - XML documents. DB2 can manage my data for OLTP applications and in BI scenarios. Sometimes, the crust on the pizza is not as crispy as that from a true Pizzamaker, but I can prepare a pizza side by side with a nice plum tart in my convection oven. Hmm, a taste of Fall...

Real life: May all your transactions commit!

Image via Wikipedia

About two weeks ago I had to buy train tickets for a trip to Paris. Because I had a special promotion code that could only be applied by a sales agent I went to the train station. There, a loooong journey into the inner workings of the sales and reservation system started. One hour later, I left with tickets, a long line of waiting customers, and a frustrated sales agent.

Recently I received a promotion letter (20 EUR off) by Deutsche Bahn, the state-owned German train operator. In the fineprint it stated that it could only applied when booking through a sales office (instead of online), was good till September, had minimum requirements on the ticket price and the type of train, and most importantly, once applied was non-refundable. Because I wouldn't use it for private travel, I thought about saving some bucks for IBM and planned to use it for a train-based business travel for which I needed to apply the IBM corporate client tariff. And I had so-called BahnCard, a frequent traveler card for 50% discount. Looking back, combining all this, it already called for disaster.

Once I was next in line, I mentioned the corporate client tariff, showed my identification for the corporate client tariff, the BahnCard, and the promotion letter. Few minutes later, I had my ticket, the promotion was applied and the credit card was swiped. Unfortunately, when I held the ticket in my hands, I couldn't see the corporate client tariff. No problem, the sales agent said, we'll look into it. After consulting with another sales agent, the "application support" was called in on how to apply the corporate client tariff. Next, the old ticket was canceled and a new one generated, all looking good. The idea was to apply the refund for the old to the new ticket, putting the difference back to the corporate credit card. However, because of the non-refundable promotion, the system went into a loop. It could not fully cancel the old ticket because parts of it were paid for by a non-refundable payment. The sales agent went into different menu options for payment, trying to charge a negative amount to the credit card, trying to refund the difference as cash, etc. All actions produced error messages because of incompatible offer conditions.

After several long minutes and an increasingly long line behind my (now sweaty back), the sales agent decided to call in help from the application support center again. The support engineer was able to remotely log into the transaction and see details, but was neither able to cancel the transaction nor to complete it. Together, they discussed options on how to convince the system to "get the job done". Finally, with my OK my credit card was charged again - paying the ticket a second time. Then a cash refund outside the system was initiated after performing and documenting a full cash inventory. Eventually, I left the sales office after about one hour, smiling to the "next in line" call.

When you design promotions, make sure the system is not only able to handle them, but also all the error cases. Those are best practices of engineering.

Traveling too often? Dealing with not so easy customers? Try this...

If you are traveling too often or, of course because of the economy, have to cut down your budget for roses and chocolate, here is something to try. It doesn't say whether you can change the message using a USB plug or whether it is useful to deal with customers that need that little extra attention. Anyway, try it and let me know...