Home Big Data Metrics and KPIs for Information Groups – Atlan

Metrics and KPIs for Information Groups – Atlan

Metrics and KPIs for Information Groups – Atlan


Examples of Quantification from the Atlan Neighborhood

Survey outcomes from Vendr’s SaaS Tendencies Report (Q3 2023) verify what many knowledge groups have already skilled: It’s getting so much more durable to put money into new applied sciences and approaches. Purchases are down 37% year-over-year, and scrutiny over the fee and advantage of investments imply that greater than half the time knowledge groups spend contemplating the precise expertise is spent on negotiation, alone.

With growing scrutiny on the price of present expertise and groups, and better requirements for investing in new applied sciences and sources, knowledge groups are sometimes requested to speak the worth of their applications and proposed initiatives. The place technical advantages have been as soon as persuasive sufficient to shut gaps and inefficiencies in course of and expertise, knowledge groups should deal with extra planes of worth, and start to quantify the operational advantages, and enterprise worth, of their work.

Information leaders all through the Atlan Neighborhood have already begun this significant change, transferring past funding arguments that embody modernity, scalability, velocity, and consumer expertise. Within the under examples, you’ll see meticulous measurement of the progress of their applications, the operational advantages yielded by knowledge groups and customers, and the enterprise worth their knowledge helped to unlock.

Discovering, Measuring, and Speaking Worth

The widespread thread of all worth measurement is baseline evaluation. Whether or not knowledge groups are beginning an enrichment program, automating a key workflow, or delivering new capabilities or knowledge to their customers, taking a second to know the “earlier than state” makes it potential to measure your impression as your work progresses, then completes.

Within the Atlan Neighborhood, we see three ranges of worth and related Metrics and KPIs that knowledge groups can and must be measuring:

  1. Program Worth – Holding knowledge groups and stakeholders to account on the tempo, completeness, and high quality of their work.
  2. Operational Worth – Understanding the operational advantages and quantified efficiencies of recent applications, processes, and applied sciences.
  3. Enterprise Worth – Shifting past the “knowledge as a service” mannequin, and into one in all partnership with the enterprise, understanding the impression of information and analytics outputs on enterprise initiatives.

1: Program Worth

The primary degree of measurement for knowledge groups is that of their applications. From understanding the satisfaction of their finish customers to monitoring stakeholder contributions, there are quite a few “low-hanging fruit” metrics that must be recorded and communicated to stakeholders and leaders.

Outlined by progress in opposition to your staff’s objectives, and suggestions out of your finish customers, knowledge groups ought to pay cautious consideration to the relative enhancements they make by means of new applied sciences and processes, constantly converse with and survey knowledge customers, and set measurable objectives for his or her stakeholders to decide to.

Monitoring Stakeholder Contribution

Swapfiets, the world’s first bicycle subscription service, launched their Information Governance program by providing a self-service, trendy knowledge catalog on Atlan. Key to their success is an everyday collaborating and reporting cadence, with six cross-functional Information Governance Committees composed of information analysts, knowledge house owners, and knowledge stewards throughout their enterprise domains.

In that month-to-month assembly, we deal with three governance matters, that are possession, documentation, and high quality. For documentation and possession, Atlan is absolutely helpful. It’s additionally helpful for high quality to search out out the place within the knowledge lifecycle high quality points come up. Through the assembly, we write out motion factors and decide to duties like verifying extra phrases.”

Lisa Smits, Information & Analytics Staff Supervisor

Swapfiets now maintains 125 outlined metrics throughout these six enterprise domains, and with month-to-month committee conferences, has created a mechanism for accountability and attribution, and a platform for usually reporting their progress towards enrichment objectives.

Quantifying Expertise & Satisfaction

Utilizing experiential or NPS surveys, or a proxy metric like Slack messages, tickets, or escalations, are all probably efficient methods to quantify consumer satisfaction. Essential to speaking a transparent before-and-after is amassing a baseline measurement previous to releasing new tasks or experiences. 

For instance, measuring the variety of knowledge questions a Information Engineering staff receives previous to the discharge of a glossary, then three months into its adoption, paints a transparent image of adjusting consumer behaviors that may be simply communicated with stakeholders. 

Chargebee, a market-leading answer for recurring income administration, makes use of Atlan to create reusable knowledge merchandise, and to satisfy a rising inner urge for food for knowledge. Earlier than the introduction of self-service, Chargebee’s Information Engineering staff was accountable for responding to requests for knowledge. Additionally accountable for delivering knowledge to Chargebee’s prospects, their Information Engineers usually needed to de-prioritize inner requests.

This was an enormous problem. Inner knowledge requests have been pushed to the again of the queues as prospects have been at all times a precedence. This meant that we weren’t assembly SLAs and there was unhappiness amongst our stakeholders and plenty of escalation, which led to unhappiness inside the staff as nicely.”

Lloyd Lamington, Enterprise Options Supervisor, Chargebee

Previous to implementing Atlan, Lloyd and his staff fastidiously scoped their “earlier than state”, logging the variety of knowledge requests acquired per quarter, what number of of them have been repetitive or requested for uncooked knowledge, and the variety of escalations when SLAs have been missed because of de-prioritization.

With a baseline established, Chargebee’s knowledge staff have been in a position to quantify important enhancements after launching Atlan to help self-service, and that zero escalation requests have been acquired for the reason that rollout.

2: Operational Worth

Past monitoring consumer satisfaction and stakeholder contribution, important alternative exists for knowledge groups to save lots of value, and drive efficiencies in the best way that they, and their end-users, function.

Essential to correct measurement, and a compelling narrative to speak to management and stakeholders, is establishing a baseline measurement, or “earlier than state” to check enhancements in course of and expertise in opposition to. Instruments for establishing these baseline measurements might embody:

  1. Monetary Evaluation – Retaining a radical document of the price of consumer licenses, compute, and knowledge storage.
  2. Effort Scoping – Figuring out the accountable staff members, and the hassle, value, and time required of competing approaches or applied sciences.
  3. Information Staff Surveys – Figuring out how a lot time is spent on servicing requests, repetitive duties, and diagnosing breakages.
  4. Information Client Surveys – Figuring out how a lot time is spent on knowledge discovery, or how lengthy their requests take from submission to closure.
  5. Analyzing Programs – Calculating common challenge completion instances or the quantity of tickets in instruments like Jira, or measuring the common variety of Slack messages and the time their underlying points take to finish.

With a baseline established, any enhancements to groups, processes, and expertise can and must be measured in opposition to it, then communicated to stakeholders and management.

Measuring Value Financial savings

Acquainted to many knowledge groups are potential value saving alternatives, reminiscent of optimizing knowledge warehouse property or deprecating unused dashboards, eradicating unutilized knowledge pipelines, or optimizing costly queries.

Mistertemp, a pacesetter in recruitment and momentary staffing primarily based in France, adopted Atlan to enhance the navigability and usefulness of their newly applied trendy knowledge stack.

Previous to implementing Atlan, Mistertemp’s coverage of connecting with supply knowledge, then figuring out what was helpful after the very fact, was resulting in important technical debt, and diminished the navigability of their knowledge property.

All these fast choices created loads of property in Snowflake that mainly with out a enterprise use have been by no means actually touched or by no means actually documented or by no means actually linked to our BI device or some other device. So they simply stayed there being synchronized, costing us cash.”

David Milosevic, Head of Information and Analytics, Mistertemp

Utilizing automated lineage and recognition metrics, David and his staff decided that of their 30,000 knowledge property in Snowflake, simply one-third have been used within the previous 12 months. These property have been then deprecated, considerably bettering the navigability of their knowledge property, and lowering pointless value and energy to take care of them.

Operational Worth for Information Groups

Measuring Time-to-value for Technical Implementations

There are myriad choices for knowledge groups implementing new applied sciences and tasks, from hand-coding, to open-source instruments, to an array of SaaS options. However key to quantifying the worth of those efforts is cautious evaluation of the hassle concerned in every of those alternate options.

Within the case of changing or modernizing present applied sciences, knowledge groups ought to take care to stock the technical and experiential shortcomings of their legacy expertise, and punctiliously scope the hassle expended on their earlier strategy, drawing a transparent comparability between new and outdated.

Porto, a Brazil-based insurance coverage and banking chief with 13 million purchasers, sought to interchange a legacy knowledge catalog to maximise the ROI of their knowledge stack, and enhance knowledge literacy throughout their group.

We used to have a knowledge catalog. We had Informatica EDC. The motivation to search for a unique answer got here from some difficulties in implementing sure options like lineage, a enterprise glossary. It was form of irritating, and it got here to some extent the place we felt just like the expertise was form of blocking us from reaching the place we wished to go.”

Danrlei Alves, Senior Information Governance Analyst, Porto

Selecting Atlan to interchange their legacy catalog, Porto’s staff shortly built-in key knowledge sources, then used customized metadata fields as a migration goal for his or her present documentation, metadata, and relationships between tables, columns, schemas, and databases.

In simply six weeks, Porto’s knowledge staff launched their new catalog, with content material at parity with their legacy answer. However by drawing on expertise with their earlier answer, and preserving a selected stock of the enhancements they desired, Porto’s staff have been ready to attract a stark comparability between the 2 options, attaining in six weeks what as soon as took them greater than 2 years, speaking important enhancements in time-to-value.

Within the case of one-time tasks, knowledge groups should spend time estimating the hassle concerned in every proposed strategy, inclusive of the variety of staff members to allocate, time spent for every of those staff members throughout the length of the challenge, and the estimated completion date.

Tide, a UK-based digital financial institution with practically 500,000 small enterprise prospects, sought to enhance their compliance with GDPR’s Proper to Erasure. Making certain all private knowledge was deleted upon request first meant aligning on what constituted private knowledge, then figuring out the place that knowledge lived and the way it moved by means of their structure. With a complicated knowledge property, the guide effort concerned was in depth.

Folks must go into the databases and attempt to translate my checklist of non-public knowledge components. There have been 31 components to search out in our databases, and we’ve greater than 100 schemas, every with between 10 to twenty tables. So it will be loads of work to determine it. If we have been very diligent and did it for each schema, then it will in all probability be half a day for every schema. So half a day, 100 instances.”

Michal Szymanski, Information Governance Supervisor, Tide

Utilizing rules-based bulk automation in Atlan, the Tide staff was in a position to determine, tag, and safe this private knowledge in mere hours, a powerful feat in and of itself. However by first scoping the baseline guide effort wanted to finish the method, Tide’s knowledge staff have been in a position to report practically 50 days of effort saved.

Measuring Information Service Effort

For a lot of knowledge practitioners, a relentless back-and-forth answering easy questions from knowledge customers is a big drag on productiveness. Even if this drawback is straightforward to know, knowledge groups ought to take care to measure the impression these requests have on their productiveness. 

By surveying staff members to know the period of time they spend on these questions, and by together with a measurable metric like Slack messages acquired or tickets processed, knowledge groups can create a transparent baseline of effort, in opposition to which they’ll quantify productiveness positive factors.

Octane, a speciality lender providing financing for powersports automobiles, makes use of Atlan to doc their knowledge property, then make them out there by way of self-service. Previous to introducing Atlan, questions on Octane’s knowledge have been answered in a shared Slack channel, the place their Information Engineering staff was accountable for responding. 

Aware of the potential effectivity advantages of introducing self-service with Atlan, Octane’s Information Product Supervisor, Alex Bendix, analyzed the period of time their staff spent on these questions.

Every engineer was answering questions 10 to twenty% of their time. And then you definately multiply that over a sure variety of staff throughout the staff, that’s lots of of hours per 30 days that you just’re shedding when it comes to productiveness.”

Alex Bendix, Information Product Supervisor

By meticulously quantifying the period of time their staff spent answering questions on knowledge, and measuring the variety of Slack messages the staff was receiving after implementing Atlan, Alex and his staff have been in a position to report that they have been now saving 200 hours of Information Engineering effort per 30 days.

Measuring Efficiencies by Eliminating Repetitive Duties

Much like measuring the hassle expended answering questions, knowledge groups also needs to take care to measure the hassle they expend on recurring obligations, setting a baseline of effort to measure productiveness positive factors from options reminiscent of automation.

Past their documented success accelerating implementation time, Porto’s Information Governance staff has additionally quantified recurring financial savings from automating beforehand guide duties.

Danrlei and his staff began by figuring out repetitive work, together with assigning house owners, documenting property, and securing private knowledge, then created a framework to make sure that solely the info property that demanded their private consideration have been surfaced to the Information Governance staff.

If we think about every thing we’re doing now with Atlan in comparison with earlier than we had Atlan, we’re saving 40% in effectivity, when it comes to time and costly operational duties for every thing associated to governance. It is a 40% discount of 5 individuals’s time. We’re utilizing the time financial savings to deal with optimizing our processes and upleveling the kind of work we’re doing.”

Danrlei Alves, Senior Information Governance Analyst, Porto

With these automations operating in opposition to a knowledge property of over 1 million property, Danrlei and his staff have been shortly in a position to determine the diminished quantity of governance duties they have been now accountable for, then report this important enchancment.

Measuring Time-to-resolution

Lastly, productiveness positive factors from accelerating “hearth drill” duties like Root Trigger Evaluation, or troublesome investigatory duties like Impression Evaluation, additionally profit from a transparent accounting of effectivity positive factors. Fortunately, with these workstreams usually managed on instruments like Jira, measuring reductions in time-to-resolution might be so simple as analyzing effort and time spent on these duties earlier than and after enhancements.

Takealot, a South African eCommerce and Retail chief, makes use of Atlan to enhance technical understanding of their knowledge property, and drive enterprise self-service. Previous to Atlan, every time Takealot’s knowledge staff have been knowledgeable of a bug, they’d conduct conventional dash planning, figuring out the time wanted to analyze and resolve the problem.

By analyzing these decision workstreams, their staff decided they have been spending half of their time investigating the issue and figuring out the placement of a breakage, setting a baseline metric to enhance in opposition to. Then, by using automated column-level lineage on Atlan, the Takealot staff might extra shortly examine the basis trigger of information points, slicing their situation decision time in half.

As an alternative of trawling by means of all of the code, you may shortly comply with lineage backwards and examine it at each level to see what’s taking place. Earlier than, it might take per week or two weeks relying on how troublesome a bug was to handle, with 50% of that point being investigating what the issue was and the place it’s damaged earlier than really making use of the repair and getting it into manufacturing. I’d say we’ve in all probability halved that point.

Group BI Supervisor, Takealot

Operational Worth for Information Customers

Measuring Friction in Information Discovery

Whether or not points with navigability, strict entry controls, or restricted context, there are myriad causes knowledge customers have a troublesome time discovering and understanding the info out there to them. As a way to perceive the place to make enhancements, and to measure the good thing about these enhancements, knowledge groups ought to analysis the hassle their end-users expend on these actions.

Nasdaq, accountable for their eponymous inventory trade in New York Metropolis, and an array of expertise and monetary service merchandise, makes use of Atlan to evangelize their knowledge technique and enhance their governance observe.

Key to their implementation course of was time spent surveying Nasdaq’s knowledge customers, driving higher understanding of how the info staff might enhance their means of discovering and making use of knowledge.

Almost 100 customers responded to the survey, with 75% of respondents reporting spending time attempting to know knowledge. Respondents who spent six hours or extra hours pers week on knowledge, spent two of these hours attempting to know the context round what they already had entry to.

Take into consideration that! A 3rd of their time each week is spent simply attempting to know what’s there. Think about if we might deliver a product in that helps scale back that effort and actually allows them to get proper to the center of the issue — to drive knowledge merchandise from insights into the enterprise. And that’s what we’re attempting to get to.”

Michael Weiss, Senior Product Supervisor, Nasdaq

With this survey knowledge in-hand, and a mechanism for bettering the invention expertise for his or her knowledge customers, Michael and his staff have a “North Star” metric to measure their success, probably representing lots of of hundreds of {dollars} in financial savings in knowledge shopper effort, alone.

Measuring Time to Perception

For knowledge groups which have a service-driven relationship with their knowledge customers and prospects, baseline measurement of the time processes take to finish, like constructing dashboards or finishing queries, makes it easy to speak the impact of course of enhancements and new applied sciences.

Infillion, a Digital Media Platform serving the Fortune 50 and extra, makes use of Atlan to sew collectively a fancy knowledge property, and supply data-driven insights to their prospects

A vital a part of their Enterprise Intelligence staff’s obligations are supporting account groups, whose prospects count on frequent, correct reporting on advertising marketing campaign efficiency. Beforehand processed manually, these requests would take as many as three weeks to finish, with one week of ready for Information Engineering bandwidth, and one other two to supply a report.

Utilizing Atlan Scheduled Queries, Daniel’s staff now spends half-hour or much less responding to requests for knowledge, eliminating the necessity for scarce Information Engineering sources, and reporting a 3-week time financial savings for his or her enterprise companions and prospects on every request.

It’s allowed us to totally automate reporting with none engineering sources and develop into self-sufficient, and it’s positively eliminated a giant bottleneck in our course of. A 3-week ready interval is simply not scalable. So this positively lower that all the way down to a half an hour, perhaps much less, to arrange.

Daniel Chon, Director of Enterprise Intelligence, Infillion

Measuring Effort

Lastly, on a extra refined degree, an in depth stock of the duties knowledge customers undertake, mixed with an evaluation of the hassle expended and common personnel value for every of those duties, is a labor-intensive, however highly effective device for measuring the particular impression of a knowledge staff’s efforts.

Zip, a digital monetary service firm and buy-now-pay-later pioneer, makes use of Atlan to assist their knowledge customers shortly, reliably, and simply discover, perceive, and use the capabilities of their trendy knowledge stack.

Leroy Kahn, Zip’s Information Administration Lead, developed a mannequin to estimate the enterprise worth yielded from every exercise his colleagues would carry out on Atlan, and a course of to trace the quantity of those actions as a part of their enterprise case for a contemporary knowledge catalog.

Via consumer surveys, and the alternatives to enhance that they inventoried throughout duties involving discovering, sharing, understanding, and utilizing Zip’s knowledge, Leroy decided a median period of time saved for every exercise or occasion kind for Atlan customers. Based mostly on common wage, every minute of time saved was value $1 (AUD) in effectivity positive factors, which might then be multiplied by the point saved per occasion in Atlan.

By analyzing utilization metrics, Leroy and his staff have been in a position to account for every exercise carried out on Atlan throughout a 30-day interval, evaluating it in opposition to the time and greenback financial savings, projecting $390,000 AUD in effectivity positive factors.

3: Enterprise Worth

Whereas measuring the efficiency of information staff applications, and punctiliously recording the worth of operational enhancements is vital, there are a restricted variety of alternatives to make these enhancements. As a way to proceed speaking the worth of their work, knowledge groups should try to know what knowledge is used for, and to what impact. This necessitates transferring past a mannequin the place requests are serviced, and towards a mannequin of partnership with enterprise customers.

Whereas knowledge groups might need grown accustomed to constructing new capabilities or servicing requests which are then picked up by an analyst or data-savvy enterprise consumer, transferring past this paradigm means understanding the aim of those requests, and following up on their impression.

Information groups can start placing this into observe by recording a enterprise objective every time a request is made. From a easy stock of which groups or colleagues made these requests, to extra complicated data of tasks like lowering buyer churn or figuring out cross-sell or up-sell alternatives, knowledge groups can start evaluating their work in opposition to achievement of enterprise goals, and following up for particular anecdotes of what impression these insights made.

A quick-growing pet care firm is among the many finest examples of enterprise worth measurement within the Atlan Neighborhood. After migrating to a brand new telecommunications system, their name middle staff was underperforming, resulting in diminished buyer engagement, and lack of income.

Whereas knowledge was out there from their new system, its format was distinctive to telecommunications programs, dubbed a Name Element Document (CDR), making it troublesome for his or her Information Analysts to analyze the issue and suggest an answer.

I’ve labored in conditions like this earlier than, and if we didn’t have Atlan it will have taken six months. We’d be answering questions like ‘What do you imply by occasion code?’ or ‘What’s the distinction between extension quantity and regroup quantity?’.”

Information Governance Lead

Utilizing Atlan, a member of their staff with Telecommunications expertise labored to narrate Snowflake property with CDR knowledge, and translated, then documented what every of the info factors meant. Now in a position to perceive the context round this knowledge in a self-service method, their Information Analysts shortly carried out their evaluation in PowerBI, and recognized three core issues for under-performance.

Taking an Operational Worth lens to this achievement, their knowledge staff might report that they have been in a position to help this evaluation in 30 days with Atlan, relatively than 6 months. However by sustaining an in depth relationship with their counterparts within the enterprise, and by preserving knowledgeable about how knowledge evaluation drove a decision, their knowledge staff can now report that they supported a ten% improve in bookings by means of their name middle, a big improve in income, and a mission-critical achievement.

Whereas fastidious reporting on Program Worth and Operational Worth are essential, knowledge groups who construct robust relationships with their enterprise counterparts, and have the curiosity and persistence to know how their work is used downstream, can tie their work to an important issues their organizations are going through.

Key Takeaways

Throughout Program, Operational, and Enterprise worth, the most effective examples of quantification within the Atlan Neighborhood have numerous key commonalities.

  1. Curiosity about, and dedication to, the worth of their groups, applications, processes, and expertise.
  2. Constant assortment of quantitative and qualitative knowledge previous to funding or adjustments to processes or expertise.
  3. A mechanism for capturing adjustments in habits, sentiment, or effort, constantly measured in opposition to a baseline and communicated to stakeholders and leaders.

Picture by Patrick Perkins on Unsplash



Please enter your comment!
Please enter your name here