Happy Holidays

With the last post this year, I wish you Happy Holidays, time for family and some rest. All the best for 2010, may all your transactions commit.

Henrik

Excuse me, what year do we have? - Some usability enhancements for XQuery in DB2

Imagine that in the XML world there is already 2010, but in your relational world it's still 2009.  This or the reverse could happen if you directly use fn:current-date() in XQuery and the CURRENT DATE special register in SQL. The reason is that XQuery by definition operates in and returns UTC-based time and date values, e.g., whereas the SQL context in DB2 refers to the local timezone.

To work around possible issues, user could call fn:adjust-date-to-timezone() to convert a data (or analogously a time or dateTime value) to a specific, e.g., the local timezone. Well, the world got simpler just recently. DB2 9.7 FP1 introduced new (DB2-specific) functions to retrieve values in the local timezone directly, named current-local-date, current-local-time, current-local-dateTime, and local-timezone.

Having different years in the SQL and XQuery context might be a rare event, but the usability improvements are very handy - not just close to year end.

XML Processing on z/OS

A new Redbook has just been published that discusses XML Processing on z/OS. The book starts out with XML Essentials, so that even beginners to the XML world can make use of this book. After the introduction the different options of XML processing, ranging from COBOL, PL/I over CICS, IMS, and DB2 pureXML to Rational Developer and Java-based solutions are discussed in an overview section. The rest of the book then deals in detail with the different options and also includes a chapter about performance and cost considerations.

Orwell, 1984, and Google

Forbes has an interesting article "When Google Runs Your Life". Basically, Google is on its way to impact (control? profile?) most of your private and corporate life. It has its reach into search, mapping and navigation, online and offline applications, into communication (both landlines and mobile). Through its browser and operating systems as well as DNS services it is also at the door to the Internet. To top that, by way of the AdSense and Analytics services there is a web of checkpoints plastering the Information Highway and more.

You talked with someone or left a message? You transcribed and analyzed it. You got emails? Analyzed and only the advertising got optimized. You surfed to some fancy places? Google already got the big picture of your personality.

In the article Eric Schmidt, Google's CEO, is cited as:

Privately, however, he has told friends to keep off a computer anything they want to keep private.

In several European countries there are already discussions about Google and compliance with privacy laws. It will be interesting to see how this moves forward.

Why? Motivation (based on fun)

Over the weekend I took a look at the Top Viral Video Ad Campaigns and noticed something by Volkswagen. It's not directly by them but by a project they sponsor which is called The Fun Theory. The idea is to introduce a fun factor to (at least for short term) change habits for the better. So far they have three videos and all are very interesting and funny.

By introducing something unusual into a common, daily setting they are creating curiosity and then fun. With that they are motivating people to change habits, maybe only once, maybe for longer.

Many people (I **not** included) come up with New Year's Resolution. Usually those resolutions are related to changing a habit for the good and people tend to fall back to their regular routine after a month or so. And in most cases they do so because one important ingredient is missing: Motivation (and fun).

Shock or Joy? Power Meters and the core of everything: Data

In many countries the end of year also is the time when power meters, or utility meters in general, are read out. When we lived in California the meters were manually read once a month and the monthly bill reflected the previous month's usage - with all its ups and downs. The feedback was more or less immediate which was good, but the amount to pay also significantly alternated between highs in Winter and Summer and lows in Spring and Fall.

In Germany, the meters are typically read once a year and the monthly amount is based on the averaged out estimate for the next year. You can plan ahead for the upcoming months in terms of payments, but for most people there is no direct feedback on their consumption. Our power meters will be read out over the next few days and I am happily expecting the annual statement sometime in January.

In some regions the Utilities have started deploying Smart Meters. In California, PG&E is changing to them, in Germany households are expected to move to them over time, too. I am still waiting for mine.

Why I am interested in it? First, direct feedback is valuable to improve or optimize consumption - monthly data is better than annual, daily or even immediate input better than monthly. Second, it is interesting to see what kind of data could be made available and in what format. My format of choice would be XML because of its flexibility and tooling support. Third, based on finer granularity, better pricing could be available (avoid the peak hours and save). Fourth, based on the data, you or a third-party company could analyze your consumption, compare with peer groups and look into finding ways to improve your monthly bill. Some companies already offer energy optimization services to enterprises and take a share of the savings.

Last, it would be fun to store XML-based energy data in DB2 pureXML and manage and analyze it myself.

What is at the core of all this is the data. Without its availability (in a usable format), there is no insight, no improvement (greener planet), no founding of new companies.

Friday Fun with Airlines

Yesterday, Adam's marketing thoughts were motivated by an old airline commercial and the feeling of coming home after a trip. That got me started last night because the feeling of "coming home" or "already feeling close to home" is what probably many travelers, including myself, share. I started to look for some funny videos that demonstrate my recent airline experiences. Here they are:

1) Flying has changed, it is safer today than it was, but the recent habit of charging for everything separately is annoying. Why not combine it?



2) The second video is about my experience as a tall person (198 cm / 6'6"). I have to crawl inside regional aircrafts. And in most others the overhead bins are a frequent hazard.



P.S.: Have a nice weekend to you all on DB2, Oracle, Informix, MySQL, MS SQL Server, PostgreSQL, Derby, and other database systems!

Tell me what you are (or could be) - Admin functions for inlining

I recently wrote about LOB Inlining, a new feature in DB2 9.7. It allows to store LOBs together with the actual row data and thereby save space and in most cases improve performance. XML Inlining has been around since DB2 9.5 and proved very valuable for many customers. A common usability issue, and we can argue whether small or big, was that it was hard for XML data to estimate whether it got inlined or not. This was caused by the fact that when you insert XML data it is in text form and then is parsed and converted into the internal, native format. Thereby it can shrink or grow, depending on the data. No simple tooling was available to estimate whether it would be inlined or whether it actually got inlined.

Well, the XML parsing and the resulting size changes are still present in DB2 9.7 (we still have native XML support), but tooling that can be used for both XML and LOB data has been added in the form of two admin functions:

• ADMIN_EST_INLINE_LENGTH works on both a XML or LOB column that is passed in as parameter. The return function returns either the estimated length of the column value for each of the column values or an indicator of whether the length would be too big or the estimate cannot be performed.
• ADMIN_IS_INLINED basically returns either a 1 or 0 (yes/no) for each column value of whether it has been inlined or not.

Both functions are not meant to be run on your production system with millions of rows per table, but for either test environments or for [performance] problem diagnostics (think of a nicely written WHERE clause attached to filter down the number of rows). The functions help DBAs to determine a good inline length and also to verify whether values are then actually inlined as planned.

To make usage of the usability function even more usable (nice sentence, right?) a new sample program has been added that demonstrates how the two admin functions can be applied to XML and LOB columns.

Holiday Preparations - Your action required!

I recently wrote about how companies make use of their address data during the last weeks of the year. Today, I thought to join the **fun** of ongoing holiday activities and start my own giveaway/sweepstakes. I will be crowning my "Reader of the Year" later this month. To be eligible, you don't have to submit silly forms, post holiday videos to YouTube, solve puzzles like "December is the ____ month of the year" or get your kids to paint your family.

All I am asking is that you leave some basic information, similar to other sweepstakes. Please comment on this post, with your name, your full address, your birthday, your annual income (full dollars/Euros/etc. is ok), your occupation, your detailed family status, and your highest degree of IT certification (no school information required!).

P.S.: And remember the fun part.

Call with IBM experts on Using JDBC with pureXML (updated)

Tomorrow, Friday 4th, is another meeting of the group calling themselves pureXML Devotees. Details can be found here. The call will start Friday on 1pm US Eastern time and will cover DB2 on z/OS as well as DB2 for Linux, UNIX, and Windows.

As usual, the meeting will be recorded and the slides and replay information should be available on the same website a couple days after the call. Enjoy!

Update: The replay information and slides are now available at the pureXML Devotees website.

Let's phone, shop, and whatever.... - Testing new limits

For most of the activities these days data will be produced, ranging from your life and household to administration, manufacturing or services. A lot of the data will end up in data warehouses, either directly or in some condensed, aggregated, distilled way. And while people were talking about 1 TB or 10 TB warehouses only few years ago, scaling up to 100s of Terabyte or even Petabytes (PB) is discussed often now.

One of the enhancements in DB2 9.7 is for addressing this trend. Up to version 9.5 distribution maps were limited to 4096 entries (4 kB), now  up to 32768 entries are possible. In a partitioned database the distribution key, i.e., the columns used to figure out on which database partition the entire row ends up, is important because it determines how evenly the data is split between the partitions. The more evenly balanced the distribution is, the better balanced typically the performance is.

To assign a database partition, the distribution key is hashed to an entry in the distribution map. The more entries in the maps, the smaller the skew. With the new increased distribution map in DB2 9.7, the skew remains small even for databases with a larger number of database partitions.

How do you test it? Increase your calling, shopping, driving, consuming. This will not only kick-start the economy, but also grow the enterprise warehouse and make sure new limits are tested (and introduced)...

The importance of correct and integrated data

You can deliver a product as fast and "good" as promised, but still loose a repeat customer. Here is a real life example.

Over the past days I ordered several books online. Usually, I use one specific company "A" (any guesses?) for two reasons. First, in Germany by law all German language books have to be sold for the list price, neither discounts nor coupons can be applied. "A" is offering free shipping on all books, regardless of their list price. Second, "A" is usually very quick in delivering the books and provides tracking information. I used this company as usual for some books.

Then, for an English book where every book seller can offer individual prices, I shopped around and found that I could save 2 EUR over at company "B". The product details showed that the book was in stock and would be sent the next day. I ordered the book in the evening and then started to wait for a confirmation email. And waited. The next morning I received 2 emails. The first was the acknowledgement of my order. The second was a status change email - now the book wasn't in stock anymore, but had a delivery time of 3 weeks. Ouch.
I checked the order status online and then the product page again. Both showed the 3 weeks for delivery. Why not cancel the order and shop somewhere else? I sent an email via contact form, tried to cancel online, and called the service hotline. On the phone they were friendly, confirmed the change to 3 weeks, wondered why it had changed over night and pointed out that it would be hard to cancel, but they would try anyway. The reason was they are working with a 3rd party fulfillment center. One day later, first the status said "trying to cancel", then it changed to "package sent", and in the afternoon the book arrived.

Company "B" delivered the product I ordered on time. However, their data isn't integrated and both customer and customer service are left to wonder about what is correct. Will I shop there again? Not this year... ;-)

Learn about MySQL and DB2

When you work with database systems for a longer time it is always a good idea to step back from time to time and look at "your world" from some levels up (isn't it the season for that anyway?). Now there is a book that helps you to get a good overview of not just DB2, but also MySQL. Any idea what book I am talking about?

After some years since the first edition, IBM just updated the MySQL to DB2 Conversion Guide (a so-called Redbook - no Oracle involved). What is good about the book is that they start with a high-level introduction into DB2 9.7 with its architecture and database objects and components, the available tools and utilities, and different ways of accessing the data (APIs, languages, etc.). Thereafter, they focus on MySQL and give a similar introduction. Only if you understand your source and the target platform a migration can be planned well. The rest and biggest part of the 450+ pages then deals with the necessary steps to successfully convert an existing MySQL-based application to one running on DB2.

It is something worth reading. And if MySQL is/was not your world, maybe you are interested in reading the Oracle to DB2 Conversion Guide.

Address databases (CRM), the holiday season, and load balancing

It's the time of the year when - it seems so - all companies of the world remember that they have stored "customer" address records somewhere or could acquire some cheaply. At least in my case, most of those companies didn't care for me (to me?) the rest of the year. Now they are sending friendly letters and emails, wishing the best for the season and yes, there are great deals I probably wasn't aware of.

Because from their point of view everybody seems to have plenty of time now during the Advent (nobody attending plenty of celebrations and ceremonies? nobody shopping for gifts? nobody preparing house and garden for the winter? nobody ...?), they suggest to switch all kinds of insurances, think about additional pension funds, or consider other "life-changing" events.

And then there are all those companies trying to sign me/us up for their now daily newsletters with special deals every day, with sweepstakes offering TVs and cash if I am willing to have my address data be distributed to another set of companies ("sponsors").

Finally, yesterday a letter arrived with a set of greeting cards attached which we should use for our holiday mails for friends and family. The front of the card was designed nicely ("how nice!"), the back was mostly advertising ("what a waste" -> trash).

If all those companies that feel the need to maintain my address data in a database all year and consider me a customer only now please would start treating me a customer throughout the year. I had time during the Summer break to look into insurances. I had money to buy the big ticket item after bonus payments in Spring. I was in the mood for pension/retirement planning when I saw the trees turn yellow/brown/red and then the leaves come down in Fall. So, pleeeeaaassseee, consider Load Balancing for us poor consumers, too, and start making use of your databases in different ways.