Jul 14, 2026
DataKitchen Webinar: Stop Clicking, Start Asking: The AI Playbook for Data Quality & Observability - Transcript
00:00:54

Christopher Bergh: Just Hello everyone. My name is Chris Berg and I'll be hosting our webinar today. So our webinar is about how to use AI to do data quality and observability. And so, um, before we start, I have a little bit of housekeeping. Um, slides and recording and transcription will be shared this week. Um, on the lower right hand of the Google Meet screen is a chat button. Uh, you should see this uh looks like a a comment section. If you have questions, just type them in there. I'll try to answer them either during the call or uh afterwards. And so, what's our what's our webinar about? So, we're going to talk about something called MCP, which is model context protocol, which is a way for tools like chat GPT or Claude to programmatically talk to other tools. And so, our open-source data quality tool, TestGen, has a we've built a very fullfeatured MCP server, which is the name for the port that connects to MCP. So, we're going to talk a little bit about what MCP is and why it matters.

00:04:47

Christopher Bergh: We're going to talk about tests in MCP. And then we're going to spend the majority of the time on kind of thinking in a broad way about how you interact with this new class of tools. And so, we'll talk a little bit about TestGen and its great UI and why would you want to use a UI if you could just plug it into your chat tool and dialogue with it. And then why would you want to dialogue with your with a tool that tests your data, profiles your data, automatically um helps you understand what's going on in your data estate? Why would you want to do that typing? What use cases would matter? And then a third case, well, you can then build something called an agent. You can build a small program that does something automatically for you. and do you want to do that? Under what conditions would that make sense? And we'll talk about a design pattern that we like that would make it sense and and give you some instructions on how to do it yourself.

00:05:52

Christopher Bergh: And so that's our that's our discussion today. um test gen's MCP and then these three modes modes of dealing with an external tool. Um, one is maybe just use the tools UI, maybe just use it to talk to it in a in a fat or third, maybe you can build something with it. So, uh, MCP stands for model context protocol. It's a it's a UI that allows you to talk or not a UI, an API that allows you to talk or a tool like an LLM to talk to your piece of software. And what it means really is that you can have more than one MCP connection to your tool. So you can have several. And so that actually enables sort of a a plugin or think of it as USBC for tools to plug into models. And so uh it's only been out for I don't know a year or so. Um it's still fairly new. It's still under revision. Um but one of the benefits here is that you can now um start asking uh any any number of tools how they are and how they should work and and what they can do.

00:07:04

Christopher Bergh: And so from a architecture po uh perspective, there's a host. Think of claw desktop or chat GPT or cursor or co c co-pilot and then there's a client and a server. And so what we've built in test gen is an MCP server. And the way the protocol works is it's got capabilities that are resources, prompts, tools, samplings, and kind of in practice tools do all the work. um they could query a database, run a job, send a message, they could uh talk to something. They're kind of a think of it as a a programmatic way to replace the user interface of your favorite tool. And if you actually go into tools like chat GPT or Claude, you can see there's a whole library of of MCPs or plugins that worked that work with it now. And so why does it matter? Well, one of the nice things about LLM is it can you can talk to it, you can dialogue with it, it can bring in information from the web and now it can bring information from other tools.

00:08:07

Christopher Bergh: So, a lot of your tools will have um different MCP interfaces. So, you may have a a a tool that talks an internal tool, you may have an external tool. And so our our focus on testgen is to make it so that it's open source. So you don't need enterprise pricing. And really we think MCP is becoming sort of the standard interface. And if you want to be able to talk to your data and understand its quality, understand problems with it, uh understand if there's any frequency or volume or schema issues, being able to plug it into an MCP server is great. And having it run behind your firewall uh and run on your laptop, the same one you use for your favorite tools is a good thing. And so So our MCP server um before I start I just want to talk a little bit about our test tool just for people who don't know. So we have an open- source fullfeatured data quality tool. What it does is it sits on top of all your data and learns it.

00:09:13

Christopher Bergh: It profiles it and then it makes some recommendations on how to clean it up. So it'll it'll find very common errors uh in data and then what it does is it has its own sort of AI algorithm that generates hundreds of custom data quality tests. We try to get you 80% there in terms of coverage on all the data quality tests you need across all your tables and then it can execute those tests in various ways on schedule or called from your orchestrator. Um it also does um machine learning anomaly detection. So it'll uh watch your data over time to see if the scheme is wrong or the volume changes above a threshold and then it does uh data quality scoring and the whole thing sort of runs uh in a package uh runs against your database. We don't copy your data. It comes in full feature with a UI and rules for one user and our enterprise version is very reasonable starting at $100 per user or $100 per database connection unlimited tables. Um and so the MCP uh toolkit comes completely fullfeatured as part of the open source version.

00:10:17

Christopher Bergh: Um and so what happens here is if you think about the process that people use to understand data, they need to discover data. They profile data. Our profiling engine makes some recommendations on how to clean data hygiene. You can create tests and then run them. You can monitor the tests and score the data. And that cycle uh is what we're trying to engender with MCP. So we have a very complete interface. Uh you can do all these things. You can connect to a database and profile and test and create tests and updates tests and score and operate. You can do almost everything you can do through the test gen UI through um the MCP. And so, um, if we look at it, there's sort of 96 different parts to what, uh, uh, MCP has. And some of them, I think, uh, what's really important is that your your tool be gated by who you are. So, there's a permission and authentication authorization model. And some of the tools are um, you know, um, or all the tools are sort of in context of who you are.

00:11:29

Christopher Bergh: So you can't sort of step out of your security bounds. And then it's just using all the same operations underneath. So uh if you add a test or modify a task, it shows up in the UI, shows up in the logs, just like uh just like operating through the UI or through the command line. And so here we've got a cheat sheet for you that sort of summarizes all the things that you can do, including discovering your data state, profiling your data, um triaging issues, generating and managing tasks, Running. test, investigating failures, monitoring tables, scoring quality, and just set up your schedules and run. And so, um, we also have, uh, in addition to an MCP UI, which tends to be more textoriented, we also have a REST API, which is in our which is a more structured API. if you want to be able to integrate it programmatically. So for instance, if you want to do CI and CD, you probably want to do export tests, import tests, you probably want to do them with the REST API.

00:12:29

Christopher Bergh: And so it works pretty easy to set it up. There's um you know, you just have uh uh um sign in with an OOTH token, connect it, and work. And um our setup guide covers it. So it's it's um there's no separate price. You should be able to do this and connect it up to your local uh chat GPT fairly quickly. And so that's it. So that's what Testgen is a tool that does. Data quality and data observability. Um it generates te it generates tests. Um it runs those tests. It uh scores your data quality and and it has a we think just a really great UI. Um and so why would you want to use anything else? You know, we put a lot of time in the UI. So for instance, we actually in our last release built a faceted browser for all the tests that we have to be able uh kind of like shopping for tests. If you go into uh whenever you have high cardality data like you're buying something on Amazon, there's just lots of different attributes of it and tests can have different attributes.

00:13:27

Christopher Bergh: So for instance, their test category, their test type, what they work on, algorithms, and so we built a faceted browser. Um, and so that's great, but uh you should also be able to do that in context with other tools. And that's why this chat or this LLM UI is is important. And you can uh use it for a bunch of different reasons. So we see MCP as kind of a second front door. It's not a replacement for the UI. And in fact, given that you have to wait a few seconds every time you answer a question with MCP, it feels a bit slower. And so I still like the UI for most things. Um, and so our UI isn't going anywhere, right? And I guess the point I want to say is when use the UI kind of when you want to be able to go in and get some things quickly, look at a quality dashboard, look at a test results, um be able to change a test, it's kind of the UI that you should live in and work in every day.

00:14:28

Christopher Bergh: But it sort of stops that it doesn't actually integrate and and doesn't allow you to think. And one of the ways that people have been using AI certainly in our company um is not only to create things like code or or content but also to to ideulate and talk and and sort of MCP helps you fill that gap. You can sort of talk to your data, understand what's working, get recommendations. And so because it's got the same security model, you don't have to worry um uh if you're going to break something by using this programmatic UI. And so um now I'm going to talk about two use cases um on if on how you want actually want to use MCP. And so the first use case is really AI dialogue. It's kind of chatting with your data and data quality. The second case I'm going to talk about is something called an agent which are programs that are using it. So this is probably the most common case um and I'll probably spend most of my time on it here.

00:15:26

Christopher Bergh: And so, um, when you hook up test gen to chat GPT, you start to be able to ask questions. You can say, test gen, you know, what tables do I have? How many rows are there? Um, you can start to help diagnose root causes. You could say, "Show me all the all the test errors." Help me help you prioritize those tests. You can bring in other sources of information. So, one way that we've used this is in our orchestrator tool, we hook up the recipes, which are sort of the process lineage, all the steps, and helps us prioritize where to do the fixes. And so, it can help you prioritize, diagnose, find root causes when something goes wrong. And it can also help you update source data quality. So for instance um if something looks strange in your data you have some fields that have errors on you can actually use the MCP and use your tool to update the source data to make the changes to fix the data and you can also go into to recommend tests and improve your test suites by adding new tests and you can do almost everything the UI can do.

00:16:40

Christopher Bergh: And so what it allows you to do is sort of very linearly kind of start an investigation and work through it screen by screen and see as it scrolls by. Um and so here's some examples, right? Like you can ask a question, what data do I have and how healthy it is? And it can tell you the score. It could tell you um the hygiene results, the test results. You could pick in on a project or pick in on a table and learn more and more about it. And then you can actually start thinking through like well what's dragging down the score? What is causing that score to be low? And so we can actually help you understand why the data score is there, what the hygiene and and the test results are. And then you can drill in and say, "Okay, here's something that looks show me the rows behind it." You can actually go in and look at the source data or chart them. And then um you could go in and and do very complicated things like look at a table, find the latest test failures, investigate the roost causes, recommend fixes, graph the failure counts in the last 30 days, then suggest any new tests we should add and add them to the suite.

00:17:48

Christopher Bergh: So you can make these very compound sentences. And what's great about that is you can just type that in, let it run. It takes several minutes for it to run. um not because test gen slow just the interaction with the model and then it'll come back and and and um uh give you a recommendation or make the changes and I'm going to show you some example about that and then you can actually have them recommend some new tests. Um, TestGen does a great job, we think, covering a large variety of tests, but um, it's always great to have a second set of eyes. And so, we have our own internal deterministic AI engine that, uh, generates data quality tests, but you can actually go in and add some more. And so, it's it's been interesting to see what, for instance, claude adds more tests that we don't have. And so it's a it's a great way of not only saving tokens for us to autogenerate tests, but it's a great way to have a second pair of eyes. And so we also have a bunch of what are called slash commands, which are just ways to take a series of prompts and put them together like health check or investigate failures or triage hygiene issues or table health or profiling overview.

00:19:01

Christopher Bergh: very simple uh commands that do a number of things together and these are great things for you to learn in the MCP API. And so um you know I think this idea of dialogue is is great because what what you can do is start addressing a bunch of different things. And so um I'm going to give a demo here. I'm not going to go use claude uh be because it just takes a few minutes uh for it to go and you guys have short attention spans uh Ben as well as I. So I'm going to show you the results. So the first thing up here at the top is sort of I I had a question look at all the hygiene issues and this is uh comes from the demo data that comes with our product when you install it. So you can just uh grab test gen installing locally. usually gets going with uh installations usually somewhere less than 10 minutes and then start doing this yourself hooking it up and so um you can look at all the hygiene issues and the data test failures and lists the first three I should investigate and so it goes in and pulls all the hygiene issues and here's the three issues in the data that should should have there's a duplicate customer ID um there's duplicate credit cards and this column is completely empty.

00:20:20

Christopher Bergh: So it it not only found issues in the hygiene tests and the data but it prioritized for those for you. Now going on to the sec second side well how would I fix the customer ID issue inde by customers? Well, since it knows the schema, um, it knows the database type, it can actually give you the actual schema or the actual SQL to to make the fix. And so, you don't really have to think, um, you just have to sort of copy and paste the SQL to do these fixes. And here's another example. Um, uh, a case of like what's the SQL to fix the hygiene issues with similar values matching on a standardized table. So, here we've got this hygiene issue. If you look at it, there's a bunch of different terms for ebike. And so that's pretty annoying, right? Um, and so how can you actually fix the data to do that? Well, what it can do is actually give you a recommendation on how to fix it. And then you can apply that fix um and actually uh change the record.

00:21:22

Christopher Bergh: So their ebike is now just grouped together under one term. And so one case is actually patching or fixing your data. Now, you may want to do this, you may not want to do this. Um, you may want to take that SQL and send it to someone else. Uh, it depends upon your business on patching the data. Um, and so, uh, as a when I've worked as a data engineer, I've patched plenty of data sets. Um, kept the SQL around to fix it in case it happens again. Um, in other cases, I've sent the SQL or sent the recommendation. We have a feature in test gen called an issue report where it summarizes all with the issues and you can send it to your the person who owns the data. So we're not recommending that in every case you find a data hygiene issue you should patch it but there are cases where you may want to do it yourself and and a tool like claude can help you do that. And so um now it can also do and recommend custom tests.

00:22:19

Christopher Bergh: So what testgen does is it generates a whole bunch of tests and so it can explore the test coverage across all the tests and find gaps and so there in the test data there's 214 tests and there's no cross table business rule test and so that came in it's automatically generated. So here's when the discount amount exceeds the product maximum allowed discount. And so it generates the SQL for you and then it can take that SQL and inject it into a custom test uh if you decide yes or no. So that's a a a great way to increase your test coverage, increase your test uh library by having um a you know your favorite AI tool look at your look at your hygiene results, look at your tests and actually create tests for you. So another case um about this case of recommending and adding new tests. So what I did is uh to do this data I I sort of created some I used chatbt to create some dirty data for me and then I downloaded some dirty data some data with data quality problems from the internet and so I actually just have typed this into or copied and pasted this into our MCP.

00:23:39

Christopher Bergh: So it it I had a bunch of things create a database connection. I had a bunch of connection information here and then it it created a table group um and then it profiled the data and then it gave me a summary. Then it created a test suite for that. Then it ran the test suite, summarized the results. Then it reviewed the test results and hygiene issues and gave me a list of recommended tests. And then I reviewed it and said, "Okay, go ahead and create the test." And add a note on the test itself saying you're it's created from claude. So what that looked like is uh I went into chat GPT. I just took that language and then I've stored pasted it in. It created a connection. It added the test and created the test suite. And so what that looked like is it ends up looking like in the product. And let me just um share a different tab. So it goes in into my into my test gen.

00:24:48

Christopher Bergh: I can see the test gen UI and it built this table group and it actually went in and added a whole bunch of custom tests. And so here I put a note in to this saying that this node is actually created from claude. And so it created this custom test. And you can see if I go in and edit the custom test, it actually just put in a small SQL query. And since Claude and Chat GPT are very good at knowing the SQL dialects and then it actually ran all those tests to make sure that they they worked. And so here I've got a bunch of new tests that they've recommended that they've added into my test suite. And so this is great. Like we've got uh lots of different sources, our way of using our our deterministic AI. Uh you've got other sources of information to create more and more test coverage. And so what we like about this is is being able to use your favorite tool, your favorite LLM to not only uh look at all your test results, look at your hygiene results, but but continuously improve the process.

00:25:59

Christopher Bergh: And so if I go back to kind of sharing my sharing my other screen you can kind of see the see the results again. So plug test gen in. It learns your data. It profiles your data and you can start talking to it. You can say you can even use it as a way to understand your data. How many tables do I have? How many columns do I have? Give me some examples in the columns. You can just uh because it uh profiles your data. You can help it understand what problems you have in your data. You can help it fix your data. You can help it recommend new tests. And so uh every time it come you get a new data set. You could copy it and profile it and run it automatically. And so that comes into this case of like when should you type and when should you just program? like I showed you a case before where I just sort of cut and pasted something from a document and I did the whole thing just by cutting and pasting and coming back at the end.

00:27:02

Christopher Bergh: Um, so another way to do it is is to say I want to create something called an agent which is a program that both talks to an LLM and talks to the test gen MCP at the same time to do something automatically. And so you know I think that's pretty interesting right so an agent is kind of a program that acts upon your behalf and the cost of programming now has become very low given claude code and um all the other tools that are out there. So typing on the left here, huming, asking a question, talking to the MCP server is great and uh we think that's probably the most case that people will use it in. But actually building an agent, running it on schedule, doing something automatically um is also a way to do is also a great way to do this. And so the question becomes, well, I've got this I've got this agent that's automatically running. I don't know if it's running right. Is it adding? Is it automatically fixing my data correctly? Um, how do I get some view on what it's doing?

00:28:11

Christopher Bergh: Um, and so that's one of the challenges with autonomous agents is sort of control and feedback and so in guard rails. And so what you don't want is kind of your agent rewriting your source tables at 2 a.m. And so that's one of the challenges with both the blackbox nature of using tools like cloud to write code. um and you want to be able to have some control over it. And so what I'm going to propose here is a pattern and I'm going to give you PRD to implement that pattern in cloud code on your own. Sort of two small programs. One finds the problem and proposes fixes and then one applies those fixes only after you've approved. And so um running it automatically it's it sounds great but like models hallucinate the model versions and behavior changes over time. Um and so this it's running autonomous agents. There's a lot of press about it. Um there's very few cases where I've seen it completely automated autonomously be successful. And so what we're proposing here is kind of putting a break in the middle using the automation but giving you a place to a gate to stop and do it.

00:29:33

Christopher Bergh: And so what we have here is kind of two programs. One is sort of a a program that runs that you schedule that sort of finds hygiene issues and proposes fixes. And then another program that looks at the status. In this case, we're using Jira as a place in the middle to say, "Okay, let's uh let's see if we actually want to fix it." And so it runs it only after you've approved it. And the way it works is instead of having it run and automatically work. What it runs and creates a ticket in Jira and then you disposition the ticket saying please fix. And then there's another agent that runs polling Jira to say, "Hey, what you what should you fix?" And so the control here is good because it gives you a chance to review everything that happens. And so what that looks like in um in Jira itself, it creates this ticket. And let me share uh my Jira screen here. And so here I've got an example of the ticket.

00:30:38

Christopher Bergh: And so I've got a list of uh my program put these tickets into Jira. Um and then what it did is it generated this ticket that actually does a whole bunch of possible things. So, let's see if I can make this uh a bit bigger. So, I don't want to edit it, but let me show you this on the left. It gives you a summary of what happens. And then it has a whole bunch of recommended fixes that it can go through. And it I it embeds the SQL on what you're going to do. And so this the way this works is it you go in and then disposition this saying I want to as please fix. you can change the definition needs reply do not fix fix complete and then another program picks it up and says okay and reads what's in the ticket and then actually makes the fixes and so this is a I think a a great pattern to put some control in the middle and if I go back to my uh webinar here and and show you my slides um uh what That means is you know having an unsupervised AI always run always make the changes it's very fast but it's has some risk to it takes new new effort to be able to go through and and type in perhaps manual coding or go through it manually it's it could be slow it has a lot of safety a lot of accuracy but it could take a lot of

00:32:07

Christopher Bergh: time so this human gated agent where um it's always making recommendations you're reviewing on them and then automatically fixing them as fast. We I like that pattern. And so what that looks like from an example standpoint and and uh is that uh and to give you a flowchart here, there's a a a Python program called test gen to Jira. What it does is it talks to the test it talks to Claude which talks to the test gen MCP server and says, "Hey, give me some hygiene issues." And then another prompt in says, "Well, how would you fix these hygiene issues?" And then it takes those and pushes them into a ticket into Jira. So it does some introspection. It learns the schema. It learns what um dialect it is. And then it writes those um that SQL into the ticket saying you should fix it. So nothing's been done here. Just a list of recommendations. And then there's another program that runs again from cron or from your scheduler that called Jira apply fixes.

00:33:09

Christopher Bergh: It sort of polls Jira saying is there something I should fix? Is there something I should fix? If the reviewer happens to go in at the first and says yes, you should fix it. Then it takes it uses claude and then actually makes the changes to the source uh uh database will um make the changes. And so this pattern is uh um something that uh we can we can do. And so the benefits is that you get automation but you also keep the veto and bad data kind of stops getting in before you look at it and kind of the MCP server gives the agent sort of eyes and hands in memory. Um but the policy part is put into a tool that you use. Now we happen to like use Jira. Um but what I've done is I've built a had cloud build a PRD that you can download and use yourself. So you can put it into your own tool and it will follow this pattern um to be able to use it and then you can update it and if you happen to use uh not use Jira but use Service Now or some other tools to do your workflow, it can work in the same way.

00:34:12

Christopher Bergh: And so you can download the PRD and load it into Collide code and create it yourself. Um or you can actually go in and and read about how we did it more in this blog. And so, um, kind of going, uh, going back here, Castton's got a great UI. You can do everything. We have this programmatic way of interacting called MCP. It opens up these two new channels. One is using an AI dialogue to learn more, to search root cause, to understand your data, to diagnose problems, to update data problems, and to create tests and improve d improve um test gen configuration and using it as an AI agent. You can do all those things automatically. You can build programs to do it, but we suggested a paradigm that allows you to put some uh control uh underneath it. So the conclusion is if you want fast direct answers just use the testg UI. If you want to diagnose root cause, update your source data quality and make changes to data, improve your test suites by adding new tests, well you can use the test gen MCP in an LLM.

00:35:23

Christopher Bergh: And then if you want to automatically repeatedly fix data, add tests or automate data quality, you can create an agent with control kind of following the pattern. And um one of the the whole sort of agent infrastructure is actually completely in flux right now. How you run agents, how you deploy them. Um there are 68 companies wanting to do that. Plus your LLMs are wanting to have agent infrastructures. And so um uh the framework at which you run your agent we're very obnox uh um we're purposely being agnostic on because there's what's going to happen I think in the next six months or year there's going to be agent infrastructures that people are going to deploy and want to run automatically and maybe it's going to be in your data bricks or maybe it's going to be in some other infrastructure but being able to create an agent that talks to your data be able to make changes with control I think this pattern is going to And so we're really excited about our new MCP. You can download and use it if you uh just saw uh even Rocky agrees.

00:36:28

Christopher Bergh: And it's really simple for you to start. Um it's all open source. So just uh download the the test gen installer, type test gen install. That's all you need. Um it runs we even have a simple version that doesn't require Docker. It just requires Python. um download our cheat sheet and then uh if you want to build an agent look look at our PRD and so um it's a great way to start using it fixing your data um having value and if you want to put something into production go ahead it's completely The Apache 20 open source and we have a very reasonable uh pricing model if you want to expand. And so that's it in terms of my webinar. It's a bit shorter today. Um h I don't see any questions in it. Um uh any questions or comments I can answer before we go. All right. Yeah, please go ahead. Go ahead.
Jyoti: Yeah.
Christopher Bergh: either type uh uh Yoti either type them in the chat or or go ahead and

00:37:27

Jyoti: Uh yeah. Yeah. Yeah.
Christopher Bergh: speak.
Jyoti: Um it's a little elaborate so I'll speak. There are two questions I have.
Christopher Bergh: I can't hear
Jyoti: Uh am I audible?
Christopher Bergh: you.
Jyoti: Am I audible? Hello.
Gil Benghiat: I yeah, I hear you. Um, Chris, do you hear Yoti?
Jyoti: Hello. Uh I am not
Gil Benghiat: Oh,
Christopher Bergh: Um uh he seems to be speaking but I can't hear them.
Gil Benghiat: I Chris,
Jyoti: touching.
Christopher Bergh: Um just to fill the gap again for everyone,
Gil Benghiat: can you hear
Christopher Bergh: we will share the slides in the recording and um all the material at the end of this uh
Gil Benghiat: me?
Jyoti: I think Chris is unable to receive our
Christopher Bergh: in a website that you'll receive. If you have any questions, I think the best way is is to type it in the uh type it in this into the lower right hand chat
Jyoti: audio.
Christopher Bergh: message.
Gil Benghiat: Sorry, Yoti. Yoti. Um, hi, this is Gil from Data Kitchen. If you want to um ask your question,

00:38:21

Jyoti: Yeah.
Gil Benghiat: I could give it a
Jyoti: Uh, sure.
Christopher Bergh: Yeah, I don't I don't hear him,
Jyoti: I'm not sure.
Gil Benghiat: shot.
Christopher Bergh: Gil, unfortunately.
Jyoti: Oh. Uh,
Christopher Bergh: Um,
Jyoti: am I audible now, Chris?
Christopher Bergh: let me Hey, uh, hey, Gil, can you I hear I hear it now.
Gil Benghiat: I can hear I can hear her. All
Jyoti: Okay.
Christopher Bergh: Sorry.
Jyoti: Okay. Okay. I'm so sorry.
Gil Benghiat: right.
Jyoti: Yeah. Um there are two questions I have Chris. Uh one uh at the very beginning there was a discover uh data phase that you uh had shared. Um just wanted to ask like in the traditional data quality process uh generally we focus upon the critical data elements by picking a few columns from the tables. Uh in this approach are we targeting all the columns irrespective which ones are uh critical not um uh and in order if we have to even identify the critical data elements uh which are a few columns is there an option to feed any kind of a business process document which then helps us identify all those um which are uh which may be related to some you know critical business proc uh critical business areas as like maybe analytical reports or like maybe financial columns which then become our critical data elements.

00:39:40

Jyoti: That's the first question. Um should I go for the second or do you want to take up the first
Christopher Bergh: Yeah. Yeah. Yeah.
Jyoti: one?
Christopher Bergh: Let me um can I just show you how we think about it? I think I I I think that makes a lot of sense, right? Don't don't boil the ocean. Do every data element. focus on just a few um the few critical ones. And so what TestGen does is actually um allow you to kind of go in and say what's a critical data element or not. And so um we have a case it identifies things that are PE PII. It identifies things that are critical data elements. And so for instance um here we've got credit card as a critical data element. And then you can edit this uh in the UI or through the API to say what are your set of critical data elements. So um and uh so both it identifies them automatically and allows you to
Jyoti: Okay.

00:40:38

Christopher Bergh: um to update them um and say which elements you think are critical data
Jyoti: Okay. All right. Uh, thanks a lot. This helps Chris.
Christopher Bergh: elements.
Jyoti: And my second question was related to the ticket that got raised in Jira. uh and it gets automatically raised through um uh through AI and it it identifies um it it chooses please fix uh option in Jira so that somebody is able to go ahead uh look into what records there are. So does it also attach what are the bad records which needs to be really fixed because the whole column may not be empty. Let's say uh it has picked up a column which has some empty fields and some of them are populated. So those empty fields are raised as a as an issue. And when we say please fix uh somebody needs to really know what to fix, which column to fix and and what is a bad data really. So does it attach that information as well or the records as well.
Christopher Bergh: Um, so so there's there's a couple parts. There's product features and then there's this um building an agent itself.

00:41:50

Christopher Bergh: And so the the agent code that I built doesn't attach any data to the ticket. It just embeds the SQL uh to it. And so this is a a custom feature. The code is something that you'd have to generate on your own and and run.
Jyoti: Okay.
Christopher Bergh: Um we do have the ability uh in the product for you to have something called an issue report where it will go in and if you find a problem. So for instance, if I go into um a profiling run and I look at an issue with um the data, I can then take this for instance this uh has non-standard blank values and I can generate an issue report. And what that does is gives you a chance to and I can show you here in a second uh what that looks like um by changing my tab here. It's a sharable issue report that has kind of a definition, the column, it has some SQL on how to locate it and it does does give you some information in it. But I I do believe that we do not share the PII data in the issue report.

00:42:57

Christopher Bergh: We try to make sure that that is uh that's hidden. And I think that's I I'll have to find out exactly if at your question if if we let sort of PII data go out when we we do the the data. Um we had a product feature on that a while ago. I just can't remember. Gil, do you remember uh on that if the issue about PII data and and issue reports?
Gil Benghiat: No, I think I think you got it right. I mean, that's um I don't think we we we uh hide PII data from display.
Jyoti: Mhm. Understood. Understood. Makes a lot of sense.
Gil Benghiat: Yeah.
Jyoti: Yeah. Thank you so much for that. That's it from me.
Gil Benghiat: Yeah.
Christopher Bergh: Okay, thank you.
Jyoti: Thank you.
Christopher Bergh: Sorry about not hearing your
Gil Benghiat: Questions here.
Christopher Bergh: question.
Jyoti: No, no, no problem, Chris. Thank you.
Christopher Bergh: Okay, now are there any other questions I can answer? All right. Well, thank you again for um attending. We really encourage you to give it a try. It's it's open source. We support you really well. If you run any it getting it hooked up to your um your claude should be simple, but then I've had some issues trying to get it hooked up to claude. So, we're are we're very good about giving support. So, uh try out MCP, try out Testgen. Uh we're really happy to help help you uh uh get through all the the installation parts. So, again, have a great rest of your day. will share the slides and we'll share um the recording here uh to you shortly.
Jyoti: Thank you. Thank you. Thanks everyone.
Christopher Bergh: Thank you.

Transcription ended after 00:44:51

This editable transcript was computer generated and might contain errors. People can also change the text after it was created.