“You transformed how our function has worked with the business for over 15 years in just 3 weeks”.
This was the opening statement by the VP Data & Analytics at a cyber security software and services company during the review of their onboarding process.
Moving from an old Enterprise Data Warehouse to a new cloud system was an excellent opportunity to formalize how the organization documented data and analytics. Allowing our solution to scan metadata files on both the old and the new landscape produced a complete business glossary solution with usage-based lineage and certification in less than a week.
The customer’s Analytics and Data teams got together to go over the information and confirm the auto-generated terminology and descriptions based on their domain expertise. These were the results after three weeks:
- For the first time, analytics got documented within a platform that was a middle layer between the data teams and the business analysts. All the descriptions, questions and related glossary were certified coherently.
The analytics team estimated this will save them 12 hours a week in self-service data discovery or waiting for a colleague’s reply. Previously, they needed to ask the data team any question they had through a Slack channel with a next-day response SLA.
Both analytics and data teams felt the tremendous impact on the internal communication within days, as previous frustrations around turnaround times and miscommunication around the way analytics should be developed got immediately eradicated.
This is how one of the analysts described the new process: “Instead of asking for a data set, we can now explain what we are presenting and our data engineers can check in illumex to see which tables and columns they need to load into Tableau”. - “We serve government and very large enterprises, each with different analytics needs. We needed to stop keeping that information in people’s heads.”
Our customer’s second objective was looking for ways to “remove the risk associated with tribal knowledge” (taken from an email sent by the customer).
Our customer provided dedicated analyst times to customize dashboards and views of their security profiles. These analysts would naturally keep this information in their mind and only document or transfer it when explicitly obligated.
The aforementioned posed the following risks:
– losing important information during internal knowledge transfer;
– growing customer frustration from mistakes and constant repetitions of the implementation process;
– and long hours of reverse engineering.
“Adding illumex to our onboarding process with customers means all their custom analytics are documented, and we use the platform’s certification to get the customer’s written approval. This way, we record each customer’s queries separately.” - “The automation of mapping new data seriously looks like magic!”
Every time a new customer is onboarded, their data needs to be added to the system and held separately from other customers, so this would require lengthy manual mapping.
Using our platform, our customer simply needs to create a read-only user, give it access to the new schema, and within hours the data is fully mapped, presenting a huge speed factor for the onboarding process.
Moving between Enterprise Data Warehouses is famously known as the 3rd most stressful thing in life after death and moving home (sad, but true).
Our customer described this process as “company breaking” because of both operational and cultural challenges created by this inevitable move.
We started to evaluate needs in late February, and the solution was fully live at the beginning of April. Since then, we have added Lineage to the system and are working towards loading more data directly from their BI application (Tableau).
Unlike many stories we hear, this one has a very happy ending. Both the Data teams and the business analytics teams collaborate better through transparency and have a seen material impact to efficiency. Their external customers also felt the benefit in efficiency and the increase in trust for the data and analytics their vendor provides.
Photo by Raphael Wild on Unsplash
