TimeXtender Job Scheduling with Qlik Application Automation

Trigger TimeXtender jobs with Qlik Application Automation

In this post we show you how to trigger a TimeXtender job using Qlik Application Automation. Even if you don’t (yet!) use TimeXtender this might be of interest to you, as we cover a very common scenario: triggering a remote process via an API, waiting for it to finish and checking the result. Along the way we’ll also share some Qlik Application Automation tips.

The challenge

We at Bitmetric love using TimeXtender, an agile, low-code platform for Data Warehouse Automation. It lets us quickly build a robust and scalable data foundation, with full documentation and auditability. Of course, what better place to present all that beautifully structured data than in Qlik Cloud?

In a typical scenario, we first want to ingest data into the Data Lake, then update the Data Warehouse. Finally, we present the data in Qlik Sense.

💡 If you want to know more about TimeXtender, we have a webinar coming up soon.

The challenge is that both environments have their own schedulers for jobs. As the Qlik scheduler doesn’t know when the TimeXtender job is done, we include a safety margin. That way, we’re relatively sure that all processing in TimeXtender is finished before we start loading data into Qlik Sense. While not the most efficient method, it does get the job done, most of the time…

A better approach: Qlik Application Automation

Qlik Application Automation is Qlik’s no-code automation builder. While frequent use cases are triggering downstream alerts and processes and automating administrative tasks within your Qlik tenant, with all its integrations and building blocks the possibilities for automation are limited only by your imagination. It’s quickly become a favorite feature here at Bitmetric.

In this example, we’ll use Qlik Application Automation call multiple TimeXtender APIs to do the following:

Initiate a TimeXtender job by making a call to the TimeXtender job execution AP

Wait until the job is done by monitoring the TimeXtender job status API

Determine the next steps based on the job’s success or failure, by examining the TimeXtender job logs through the API

Let’s see how each of these steps work in Application Automation, and learn some automation tips along the way!

Step 1: start the TimeXtender job

To access the TimeXtender API and start our job, we need two things: an API key and a Job Id. We store these in a variable so we can reuse them in the various API calls.

Next we call the Job Execution API. This starts the job, if it’s not running already, and returns a Trigger Id. We can use this Trigger Id to retrieve the logs of the job in step 3.

After the job is started, we go to the next step.

💡 Using labels to separate different steps in your automation is a good way to keep everything readable and organized.

Starting a TimeXtender job from Qlik Application Automation

Step 2: loop while the job is running

Implementing a while loop using Qlik Application Automation

TimeXtender provides a separate API endpoint to check the status of a job. In this step we call that API and check the result. If the job has finished, we go to the next step. If it hasn’t, we wait 30 seconds, loop back and then check again.

💡 Though there are loop blocks in Application Automation, these can only be used to process a list, not as a for-loop or while-loop. However, we can still build these loops using labels. In the flow on the left, as long as the job has not finished we jump back to the label at the beginning of the loop. Creating a de facto while-loop. Those who’ve been around coding a bit longer may recognize labels as Goto statements.

Step 3: check the result of the TimeXtender job

In the final step, we use the Trigger Id that we stored in step 1 to retrieve the logs for the job execution. Based on the result, successful or failed, we finish the job or raise an error.

💡 Of course, we can take many other interesting actions at the end of this automation. For example: start subsequent Qlik Sense reloads tasks or send a notification to Slack with the job results. Whatever fits your process!

Checking the result of the TimeXtender job in Qlik Application Automation.

Sharing Qlik Application Automations

Sharing an Application Automation template on Qlik Cloud

After creating an automation, you can save it as a reusable template. This allows you to reuse the same logic to run different TimeXtender jobs. You can also share this template with others within your Qlik tenant.

💡 Remembers to remove sensitive data from the automation before publishing it as a template or exporting the workspace. In this example you want to blank out the API key and Job Id.

You can also share the automation outside of your Qlik tenant. To do this, right click in the automation and select Download workspace. This downloads a JSON file with the definition of the automation, which you can upload to another Qlik tenant using Upload workspace.

Want to try this automation yourself? Get started by downloading it here.

Export an automation from Qlik Application Automation

More from the Bitmetric team

Take your Qlik skills to the next level!

Since 2013, the Masters Summit for Qlik is the premier advanced training for Qlik. Join us in Vienna and take your Qlik skills to the next level.

Join the team!

Enjoying these challenges? We have many more, and we’ll even pay you to solve them 😉 Check out our job opening.

API Automation Qlik TimeXtender

How can we help?

Barry has over 20 years experience as a Data & Analytics architect, developer, trainer and author. He will gladly help you with any questions you may have.

Call us

Mail us