The First GPT-Based AI Assistant For Data Catalogs Announced

illumex is happy to introduce the groundbreaking impact of our latest add-on: GPT-Based AI Assistant! 

This is the first ever GPT-based AI assistant for cataloging, data governance, data management, and collaboration.

At illumex, we’re no strangers to delivering groundbreaking solutions for data, analytics, and governance practice. From the very beginning of our public offering, we’ve been leveraging our proprietary Graph, Deep Learning, and Large Language Models to bring you AI-based auto-generated business terms and recommended metrics and analyses. 

Building upon this success, we are now adding a new and exciting supplement that takes data exploration to new heights – illumex GPT-Based AI Assistant!

We designed this new feature to cater to the challenges faced by data practitioners and stewards, who must regularly deal with painstaking documentation and time-consuming data analysis. By eliminating the manual task of documenting data and semantic entities, we empower you to focus on what truly matters: extracting meaningful insights and driving business success faster.

Business terms, metrics, queries, and data objects get automated documentation

It’s up to you to decide whether you use the illumex organic recommendation engine or the third-party provider for AI-assistant (we currently partner with Open-AI GPT) to suggest documentation for any data or semantic entity and translate SQL to natural language – from a snippet of an SQL code to a full-blown, hundreds-of-rows query.

Now it is even easier to collaborate around the auto-generated semantic entities through recommendations and insights and to certify business and data definitions with the AI Assistant’s help. To understand the full potential of the AI-based auto-generative ecosystem we provide, I’ll review each of the data and semantic entity types.

Auto-generated Business Terms-Now With The Suggested Description By Ai Assistant

Business Terms help you to come up with a common language across data producers and data consumers. They provide a platform where a single logical entity has a single business definition and a single data source to address all questions about the term and its attributes. “Lead” is a Business Term, for example. “Source” could be its attribute.

illumex automatically maps your entire Business Terms library by scanning your metadata and connecting data elements under a new semantic entity according to your unique organizational language. This language includes acronyms, industry lingo, and your organizational standard, along with your analytical query usage patterns.  

Additionally, since illumex continually monitors the metadata across your ever-evolving data stack, our system sends alerts on changes whenever there is an impact on your terms.  

Now, with the new illumex AI Assistant, the last mile of documenting and certifying terms with their attributes is also covered, supplementing the automated heavy lifting of generating those Business Terms in the first place.

With a well-defined and documented business terms library, you can eliminate tribal knowledge, easily maintain a collaborative Data / Analytics / Business dictionary, discover data in scale, and shorten the time to insight. 

Auto-generated Business Metric-Now With The Suggested Description By Ai Assistant

Defining primary metrics and KPIs is essential for any data-driven organization. These metrics help us track our progress toward our goals and make better decisions. Each metric has its own formula for calculation, and each formula has a data source with the relevant data.

For example, if we aim to increase sales, we might define a primary metric as “Total Sales.” The formula for total sales is “number of units sold * price per unit.” The data source for total sales is our sales database or CRM system.

There are two ways to create your metrics in illumex:

• Define a formula over the auto-generated business terms (the same way you would do it in Excel or Google Sheets), and add a free text name and description to it. That’s all! The metric gets an auto-generated SQL query with a single definition of its SELECT statement with the formula calculation, FROM statement with a certain table (or a result of joined tables), and WHERE clause with the applied logic and filters.
• Choose your metric from the list of auto-suggested metrics constructed by illumex based on your usage patterns and industry standards, with SQL query and formula over business terms included.

Regardless of how you maintain your metric store, illmex’s AI Assistant now suggests the name and description to streamline the process further.

For either of the mentioned approaches, illumex can also identify common partitions and filters frequently used with the metric. Scanning your usage patterns can also provide insights into the misuse and conflicts of certified metrics.

With a single source of truth for metric formulas, business definitions, query logic, and data origins, you can now implement governance to prevent cases where two departments in your company have two different answers to the same question. This type of augmented governance makes it easier to implement a data mesh architecture.

Collaborative analysis-now with the suggested description by ai assistant

An analysis is a saved query that includes business context, making it searchable and reusable. This means that you can find and reuse previously existing queries rather than start from scratch. The practice of reusing queries encourages collaboration and even builds user communities within your organization.

illumex allows you to find queries that were saved and certified by other users and recommends you save frequently used ones. This saves time and effort, eliminates tribal knowledge, simplifies documentation, and helps you get the most out of your data without duplicated costs.

When you save a query as an analysis, illumex automatically provides business context, like the data sources, usage patterns, associated business terms, and related analyses. And you can add more business context to it, like the purpose of the query and free text definitions. This context makes it easier for you to find and understand the query and can also help others use it.

The illumex AI Assitant has you covered with a concise suggested analysis description – even for the longest query you can imagine.

By saving and reusing queries and providing them with business context and a well-written description, you make your analytics process more intuitive, faster, and more accurate. This can help you to make better decisions and improve consistency and alignment across the business.

Queries, lineage, and transformations-now with the suggested description by ai assistant

Time is our most valuable resource. Understanding the source, logic, and outcome of every query in your business is crucial for managing analytics costs, accuracy, and consistency. Unfortunately, these efforts are not feasible to perform manually. The results could be outdated analytics, duplicate data pipelines, misaligned dashboards, and a lack of trust among decision-makers.

With illumex AI Assistant, we harness the power of natural language processing to translate every SQL query into descriptive, human-readable text automatically.

Every query in your data stack gets business context and becomes easily discoverable.

Our advanced column-based data lineage feature empowers you with comprehensive visibility into the journey of your data, enabling you to perform root cause analysis and impact analysis like never before.

With column-based data lineage, you can effortlessly trace the origins and transformations of specific data elements throughout your entire data ecosystem. By understanding the complete lineage and logic applied to a table, column, view, and even a BI component, you can easily identify the root causes of issues or discrepancies, saving you valuable time and effort in troubleshooting.

Additionally, the ability to perform impact analysis allows you to assess the potential consequences of changing a specific data element to other tables, views, reports, BI dashboards, and even a specific visual. By understanding how alterations to one table or column can ripple across your entire system, you can make informed decisions and mitigate any unintended effects, thus ensuring the integrity and reliability of your data and analytics.

But despite the fact that the lineage and transformations are easily trackable with illumex built-in lineage, some of the transformation queries include hundreds of lines of code and could be challenging to comprehend. illumex AI assistant translates the queries to text and distills the logic more concisely and clearly.

Unlock the power of column-based data lineage and gain deep insights that enable you to uncover the root causes of issues and make confident, well-informed decisions with minimal risk.

Tables and columns have their story told-now with the suggested description by AI assistant

illumex makes it exciting to document and review data entities! Instead of the standard practice to look for an owner of every data entity, illumex provides incredibly informative stats about tables and columns’ metadata, such as age, rows count, retention period, length, data type, and whether or not it can be nullable.

However, a picture is worth a thousand words: With illumex, you can jump to Lineage or to visual Data Exploration and see your data landscape in a beautiful graph, with connections between the different entities based on the team’s practices and usage. Imagine an interactive ERD diagram of thousands of tables and columns.

From there, you can dive into the Data Tab for more information about your most popular fields. The Data Tab Enrichment set of features shows you on which fields you applied the most JOINS or WHERE clauses or which aliases you used for a single field. This information can be used to better understand how your data is being used and to improve your data management processes.

You also get alerts on unused and duplicate entities and review the suspected duplications and overlaying fields. All of this helps to eliminate waste and concentrate on what is essential.

And now illumex is getting even better! With all the richness of supporting statistics, visual relational models, and recommendations, we’re adding a way to easily describe each data object with an auto-generated description provided by AI Assistant.

The first AI assistant for data catalog is now available

This is going to be game-changing for data teams everywhere. The AI Assistant will make it so much easier to understand your data and to make better decisions with it.

Are you still waiting for your Data Catalog to add Generative AI Assistant?

 Stop waiting. Start using.

 GPT-based AI assistant is a natural supplement to the illumex turnkey Semantic Layer.

✨No waiting Lists!

✨ No marketing mockups!

✨ Get instant access on June 5th!