Developing a custom data provider for TimeXtender

⚠️ Update: This article describes an older TimeXtender connector approach and may no longer reflect the current platform. The architectural problem and project experience remain relevant.

As part of our TimeXtender consultancy work, we were recently asked to migrate a custom data provider from a Qlik environment to TimeXtender. The data source is a custom-built SOAP API. Extracting the data for a single table involves calling multiple endpoints. It also involves obscure pagination logic and splitting string data into multiple rows and columns. While the supplier of the API still maintains it, it’s not actively supported or developed. We wouldn’t be getting any help or changes there. A nice challenge!

While TimeXtender supports data connections for SOAP, because of all the steps involved this could not be used. However, TimeXtender supports creating custom data connectors. This lets you develop a connector that is caters to your specific needs. It’s not restricted by the limitations of built-in connectors. We decided to take this route and learn some new things in the process.

But first, the result:

The task

Custom connectors in TimeXtender work as follows: using a reference DLL provided by TimeXtender, you develop a DLL that holds the logic for your connector. This DLL is then installed on the server where the TimeXtender client is used, after which you configure TimeXtender to add a custom data source. After that, there’s no difference between adding any other data source. The connector holds all the logic for any function or feature that is usually handled by TimeXtender, up and until the moment the data is loaded into TimeXtender. This means that creating configuration files, testing the connection to the data source, making a selection of the tables and fields you want, previewing the table, fetching the data and implementing any data selection rules created by users all needs to be handled in the connector.

The implementation

There is little official documentation regarding custom connectors and how to build them, but fortunately TimeXtender has been very helpful in supporting Bitmetric throughout the development. The connector has to be built in .Net 4.8, and is divided into several parts: the configuration properties, the data table, the data source and finally the provider that ties it all together. The data table and provider are kept to the default code as most of the logic is in the settings that you create to configure the connector and the data source where you actually connect to the source you want to fetch the data from and perform all the other logic mentioned earlier. While I won’t go into actual code, to give an impression of the required logic here is a list of the functions you’ll need to implement:

• Creating, loading and saving the connection settings
• Connecting to the data source
• Fetching a list of the tables and columns
• Fetching data from the tables selected in TimeXtender
• Formatting the data into an object that TimeXtender can ingest
• Performing data selection based on the rules created in TimeXtender per field
• Returning the data to TimeXtender

After developing the connector you end up with a custom DLL file. This file is installed on the TX server where you want to use the connector. PowerShell is used to add it to the GAC (Global Assembly Cache). This allows TimeXtender to find the DLL and support it as a custom data connector.

The result

The connector is added as a Custom Data Source in a Business Unit. From there, the workflow is the same as adding any other data source. You configure it, select the tables and columns you want, add any data selection rules, then Execute to load the data into TimeXtender. The animations below show the main functionality of the connector.

Adding and testing the new custom data provider for TimeXtender

Reading the custom data source and selecting tables and fields

Loading data with the custom data provider for TimeXtender

Viewing the loaded data

Conclusion

Thanks to TimeXtender supporting creating your own connectors we’ve been able to support a data source that we otherwise would’ve had to leave out of TimeXtender.

Of course, if you run into obscure or unsupported data sources, our TimeXtender consultants can help with integrations, custom connectors and implementation.

More from the Bitmetric team