Wednesday, November 24, 2021

Rate-limit Kafka event generation with kcat and bash

Traffic for event streams
Recently, I worked with IBM Cloud Event Streams which is a message bus built with Apache Kafka. I was looking for a simple command-line tool to test my Event Streams instance and to stream access logs into it. That's when I ran into kcat (formerly known as kafkacat). It is a generic command line Kafka producer and consumer and easy to install - just use a Docker image. All worked well, I could even read a file of historic Apache access logs and, line by line, send them over. But I still faced the issue of controlling how much to send, how to throttle it. I solved it using a bash script.

Tuesday, November 23, 2021

Get educated: The IDUG Virtual 2021 EMEA Db2 Tech Conference is coming up

The virtual Db2 conference is coming up
Do you remember meeting in person, at a conference? Well, hang on to that memory because the International Db2 User Group (IDUG) conference for EMEA is going to be virtual again this year. The "IDUG Virtual 2021 EMEA Db2 Tech Conference" is scheduled for December 6-10. If you have not registered, I recommend doing it. Here are some highlights and links to explore further.

Thursday, November 18, 2021

On serverless data scraping, cloud object storage, MinIO and rclone

Building a data lake the serverless way
This fall, I started another side project. It involves mobility data and its analysis and visualization. As first step and still ongoing, I am building up a data lake (and maybe integrating it into a data fabric). For that, I regularly download data from various sources, then upload it to a Cloud Object Storage/S3 bucket. Once in cloud storage, I process it further, often involving tools like the MinIO client and rclone. Let me give you a quick overview and links to some more background reading...

Wednesday, November 3, 2021

Db2 SQL PL book as PDF and many useful doc links

Tell the world about Db2 documentation
Yesterday was one of those days when I, again, worked as information broker. I received a question, asked around myself, passed on information that I learned, and finally handed back an answer to the originator. With this blog post, I am closing the loop and share what I learned. A, I forgot, it is about the Db2 documentation.

Wednesday, October 13, 2021

IBM Cloud resource reclamations: Some more details and my best practices

Reclaim resources
This is a quick follow-up to my "To restore or delete" post in which I discussed IBM Cloud resource reclamations and shared some tricks. Today, I have some more details and also want to share with you my best practices on handling resources.

Monday, October 11, 2021

Db2 on Cloud credentials and how to connect from your app

Over the past 12 months, Db2 on Cloud migrated instances to a new deployment model. It seems to be based on the IBM Cloud Databases model and resulted in a different structure for the service credentials. Previously, Db2 on Cloud instances were accessible on the typical ports 50000 and 50001. The JSON-based credentials object included ready-to-use connection URIs. Now, after the change, the VCAP object has more parts, but is missing the easy-to-use URIs. In the following, I am showing you how to consume the new credential structure in your app and to successfully connect to Db2 on Cloud.

Wednesday, September 22, 2021

To restore or to delete: Working with IBM Cloud resource reclamation

Resource management
Over the past weeks, I provisioned many resources in my IBM Cloud account. Typical to cloud computing, I also deleted and re-provisioned service instances. As part of that work, I dealt with resource reclamation and sometimes restored services or made sure they were "hard deleted", immediately gone for good.

In this blog post, I am going to give a brief overview of resource reclamation in IBM Cloud and share some tips and tricks from my experience.

Friday, September 3, 2021

Serverless Twitter Bot using IBM Cloud

A Twitterbot at work
Some months ago, I discussed cron-like scheduling on IBM Cloud. One of my personal use cases is to send out tweets. In this post, I am going to look into details, into how I implemented a Twitter bot, deployed it to IBM Cloud Code Engine, and how it is managed.

Wednesday, July 28, 2021

Password expiration and vacation planning

Ready for vacation: Passwords
The months of July and August are typical vacation times. Many people work with (at least mental) packing list to prepare for some time off. On my list are passwords. I can relax knowing that no password will expire when I am away. The reason is that for some systems it is a big hassle to reset expired password or accounts. Thus, I prepare accounts for vacation.

Monday, July 26, 2021

How to connect from Python to Db2


One of the recurring questions I have seen is "how to connect from Python to Db2". I have blogged about Python and Db2 a couple times before. Recently, the question has been popping up again more frequently. From my view, the increased security with mandatory SSL/TLS encryption and the use of Python-based Jupyter notebooks for data science and AI / ML projects are the drivers behind that increase. Moreover, there is Db2 on-premises and the Db2 on Cloud / Db2 Warehouse on Cloud plus container-based offerings. Today, I am trying to sort this out and answer that question. Again.... :)

Tuesday, July 20, 2021

Cloud Security: BYOK vs. KYOK explained

Keep and bring your own key
When talking about cloud security and key management systems (KMS) for data encryption, we often hear the terms BYOK and KYOK. But what do they mean and what is the difference? Let me try to explain in this quick write-up. BYOK stands for "bring your own key" and refers to the ability to import an existing - your own - encryption root key into a (cloud-based) key management system.

Monday, July 12, 2021

Cloud-based HSM with PKCS #11 for Db2 Native Encryption

Manage your encryption keys
When using Db2 databases, there are a different ways to encrypt the stored data (data at rest). One of them is to use the built-in feature, Db2 native encryption. It supports three kinds of keystores, a local keystore file or centralized key management systems (KMS) based on KMIP or PKCS #11 protocols.

Today, I want to point you to a tutorial in the IBM Cloud documentation. It discusses using Hyper Protect Crypto Services PKCS #11 for Db2 native encryption.

Thursday, July 1, 2021

Db2 11.5.6 is available


You probably already have noticed that a new release of Db2 for Linux, UNIX, and Windows is available, Db2 11.5.6. You can download the Db2 Fix Pack via the usual support site. The related documentation highlights the following features:

  • Improved high availability with Advanced Log Space Management,
  • Graph modeling and analysis of Db2 data using IBM Db2 Graph,
  • Restrictions lifted on accessing column-organized tables
  • Technical preview update to Machine Learning Optimizer
  • New Click-to-Containerize utility

Aside from the highlights page, I usually go over the enhancements by category. Here are my personal highlights:

I am sure that we are going to learn all the details at the IDUG EMEA 2021 conference in Edinburgh, Scotland, in October. Mark your calendars.

Tuesday, June 15, 2021

Quickly deploy the serverless cloud mailer using Terraform

In the era of instant messaging we all still receive emails. They are used for status updates, security alerts or just for proposing really great offers. Recently, I blogged about how to have the IBM Cloud Security Advisor send out alerts using your SMTP-based email delivery service. Later, I made the solution core, a serverless action available as separate project "cloudmailer" on GitHub and blogged about it: A Serverless Function for Sending Emails on IBM Cloud. Continuing this side project, I now added Terraform support. Thus, using "terraform apply" you can now automatically deploy everything including the SMTP configuration. See the instructions in code repository for details.

Friday, May 28, 2021

IBM Cloud: Send out alert emails using SMTP

Last year, I blogged about how to use Slack or email notifications for security IBM Cloud security issues. Now I added another code sample, showing how to send out notification emails using any SMTP server. The code is written in Python and Node.js and deployed as IBM Cloud Functions-based webhook. The core part in charge of connecting to an SMTP email server and sending the email is isolated as dedicated action. Thus, it is possible to use it on its own, e.g., to send out emails unrelated to the IBM Cloud Security & Compliance Center.

Wednesday, May 19, 2021

My best practices: How to search Db2 documentation

I often answer technical product questions, for Db2 and IBM Cloud topics. To provide relevant links and to back up my "hunch" after reading a question, I typically search the relevant documentation. But what are efficient way to search in the Db2 documentation? What are good ways to find the relevant parts in the IBM Cloud documentation? Here are my best practices on searching documentation.

Thursday, May 13, 2021

Wireshark with Lua on RHEL / CentOS

Wireshark with Lua-based dissector

What do you do on a rainy public holiday with COVID19 restrictions in place? Finally get Wireshark to work with Lua support to have custom dissectors. Dissectors are useful to turn binary garbage into readable TCP or UDP packet content. Lua is a scripting language and a supported way of adding dissectors in Wireshark. Unfortunately, the install package for Red Hat Enterprise Linux does not include Lua support. Compiling Wireshark on my RHEL 8.3 does not simply work because it requires Lua version 5.2 for my scripts to work. And RHEL either has version 5.3 or 5.1 which both are incompatible (long story). So, let's get going.

Tuesday, April 6, 2021

Upcoming Db2 IDUG conferences


A quick post on the upcoming Db2 conferences, organized by the International Db2 User Group (IDUG)

  • The IDUG North America "2021 NA Db2 Tech Conference" is going to be a virtual event again. It is scheduled from June 7-16, 2021. The conference agenda is already available and registration is open.
  • The European IDUG conference, 2021 EMEA Db2 Tech Conference, is still slated to be an in-person event. I hope to meet you in Edinburgh, Scotland, this year in October. The conference team offers mentoring for new speakers. The submission deadline just passed, but if you are a customer with a good proposal, there might be .... 

Last, but not least, I recommend to take a look at the IDUG page with regional Db2 user groups. Some of them offer virtual events, too, and facilitating networking and exchange of information during the pandemic.

Tuesday, March 30, 2021

cron-like scheduling on IBM Cloud

cron-like scheduling

Some days ago I stumbled over my 2015 post "Bluemix: Simple cron-like service for my Python code". It is not just the name Bluemix which is dated. Since then, it has transformed into IBM Cloud Platform and has added serverless compute options like IBM Cloud Functions (OpenWhisk) and recently IBM Cloud Code Engine. Both of them support "eventing", event-based execution of code. And both support Python code as well as many other programming languages like Node.js, Go (Golang), PHP and more. So, what does it take to set up cron-like scheduling? Not much.

Thursday, March 11, 2021

Cloud tutorial on serveless web app and eventing

A follow-up from my last post on Python decorators:

Solution architecture
Some years back, on IBM Cloud, we introduced an IBM Cloud solution tutorial for GitHub traffic analytics based on Cloud Foundry and IBM Cloud Functions. A Cloud Functions action is triggered daily to collect traffic data. The action stores the data in a Db2 database. Users can then analyse the data in a Python Flask app served by Cloud Foundry.

Today, that same solution scenario and app are still available, but they are served by IBM Cloud Code Engine. Code Engine is a fully managed, serverless platform that runs your containerized workloads, including web apps, microservices, event-driven functions or batch jobs. The slightly renamed tutorial — "Serverless web app and eventing for data retrieval and analytics" — demonstrates how the existing app can be containerized and both served as web app and and used to process the daily data collection event.

Continue reading the article on the IBM Cloud blog.

Friday, March 5, 2021

Pseudo-decorators for my Python Flask app

Secured Python Flask app on Code Engine
Recently, I migrated an existing Python app from a Cloud Foundry to an IBM Cloud Code Engine deployment. The Flask app uses decorators for the routes and for OIDC-based authentication. For usability, the app should start up even without anything configured yet and the OIDC decorators being invalid. It required to some IMHO tricky coding.

Monday, March 1, 2021

JWT token authentication in Db2 runtimes like Python or Node.js

Python script connecting to Db2 with JWT
Some weeks ago I discussed how to configure JWT-based token authentication in Db2. I set up Db2 to accept JWS identity tokens and then connected to my test database using the command line. But how do you connect from a programming language like Python or Node.js? Here is what I needed.

Friday, February 19, 2021

Great chatbots in no time

Chatbots take over customer service
Last week, I was coach again at a chatbot hackathon. Chatbots have been around for a long time and I have blogged about tips & tricks for developing chatbots and resources many times. With Covid-19 transforming the world towards more digitalization and self-service, chatbot deployments grew significantly. In this post, I want to look back at the hackathon and share some wisdom for building great chatbots.

Tuesday, January 26, 2021

IBM Cloud CLI: Some tips and tricks - make it a home

IBM Cloud CLI options
In the past, I have blogged several times about how to use the command line. Today, I want to share some tips and tricks for the IBM Cloud CLI (command line interface). Many of you work with the ibmcloud command, but have you ever looked into its help option? Here is a quick look at them and how to put them to good use...

Monday, January 18, 2021

Follow-up to Db2 and JWT: What is JOSE...?

JWT: Encoded or decoded security claims
JWT: Encoded or decoded security claims
Last week, I wrote about Db2 support for JSON Web Tokens (JWT). Today, I have a small follow-up with some reading material on JWT and related topics like JOSE, JWS, JWK as well as OAuth and OpenID Connect.

Some JWT history and standards

The first draft for JSON Web Tokens, JWTs, is already 10 years old. It is from December 2010. The early draft states:

Tuesday, January 12, 2021

Db2 Security: Configure JSON Web Token (JWT) authentication

Db2 login using JWT access token
Db2 login utilizing a JWT
Since V11.5.4, Db2 allows to consume JWT (JSON Web Token). The business case is to support SSO (single sign-on). You can configure Db2 to accept access tokens issued by external Identity Provider (IDPs), including the IBM solutions or open source projects like gluu. V11.5.5 brought some JWT improvements, e.g., support for multiple labels. On a recent Friday, I set up my Db2 Docker container to accept JWTs, not those issued by an external identity service, but my own.