---
title: "Solving the Agile Dilemma - Speed and Safety for AI-ML-Enabled Medical Devices"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/o8fck6g381"
content: auto-caption transcript, proper-noun corrected
---

# Solving the Agile Dilemma - Speed and Safety for AI-ML-Enabled Medical Devices

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/o8fck6g381)

---

Hi, everyone. Again, welcome. Thanks so much for joining us today. So the purpose of today's session is to explore how agile and CICD can bring speed and compliance together even for a device as complex as one that includes AI or machine learning. So very exciting stuff ahead.

We're gonna introduce ourselves in a bit. Just before we begin, a few housekeeping items. This webinar is being recorded and will be sent out with the slides following the conclusion. We'll also, of course, share, you know, other resources in the chat during the webinar. Please go ahead and, put questions in the q and a at any time.

Our colleagues are on the line, and we'll be surfacing your questions from the chat to us throughout the presentation. We'll, of course, do our best to answer as many questions as possible live. And we also have, an an incredible team of subject matter experts, staffing our chat today to answer any questions you can't get to, on the air. If you have to leave at any point, of course, that's okay. You'll see a survey that we'd really appreciate you completing.

It's gonna give us feedback on this webinar. And finally, there's a a link, that Joe posted, in the chat to register for our next webinar, so you can, you know, go ahead and do that at any point. And, yeah. So first off, you know, the reason we're here today, I want to introduce Ketryx. Ketryx is an AI native compliance platform purpose built for life sciences to deliver safer products faster.

Right? How do we do this? We interoperate your tooling across the various teams across your organization and apply your QMS roles across it all. And with that data, we can also automatically generate the compliance evidence and documentation that you need. Right?

And with our AI agents that, you know, Bailey will, demo later on in the presentation, we can automatically trace requirements, assess change impacts, validate test coverage, and flag compliance risks in real time. And for quality teams, this means less manual chasing of evidence and more confidence in audits mission readiness. For R and D teams, the benefit is more time spent coding in developer enabled tools in the environments they're used to, not in manual documentation. Okay. And now over to you, Bailey.

Tell us who you are. Yeah. Thanks, Maayan. So hello, everyone. Welcome to our webinar.

I'm Bailey Canter, our director of solutions here at Ketryx. A little bit about me, my background was originally in physics. I got my degree from UNC Chapel Hill, so go Tar Heels. And then I went on to be a systems engineer in the defense space working on a product line at Raytheon. After that, I worked as a back end software developer, and then that role is also the scrum master and release manager.

So I got to see both ends of writing requirements as a systems engineer, implementing them into the source code, and then trying to gather all of that, put it all together for that release readiness, and ultimately how challenging that can be to be agile in a regulated industry. And so now I'm very happy to be here at Ketryx to help teams solve solve similar challenges that I faced in my previous roles. And so our team here, our solutions team here is dedicated to doing just that. We work with teams in the med tech space who have complex systems. They're trying to deliver expert care to patients.

And, they run to similar challenges with trying to automate that evidence of compliance. And so our solutions team works intimately on the presale side to help clients, create solutions that use Ketryx and AI native platform to help automate those evidence of compliance and to embed that, compliance workflow into the way you work. So that's a little bit about me and, our solutions team here at Ketryx. Excited to chat more about agile today in the webinar. And now I'll pass it over to Mayan.

Yeah. Thanks, Bailey. I can attest that Bailey's team can help you with your problem, big or small. They're they will come up with the solution. So my name is Mayan Shvoh.

I'm the AI lead at Ketryx. My path took me through both academia and industry. I was, fortunate to to, to be able to do research at companies like IBM, Samsung, Philips, even in the health care domain. My PhD in AI focused on agents before they were cool, focusing beneficial agents. And I'm, I'm I'm very happy that I ended up in Ketryx, where some of those ideas from my research, landed in kinda, you know, real world use both internally and with our customers, in in the in the shape of our agents and our assistant, which are, you know, QMS informed, your QMS.

Right? Informed by your specific the customer's QMS, and and really have an understanding of your organizations and your regulatory needs. And, what's really cool about, you know, AI at Ketryx is that, we use it ourselves. Right? So we we we epitomize dogfooding, and we use it internally.

And I see how impactful our assistant and agents are, internally for our teams as well as, of course, for our customers. And because Ketryx is built at the quality of a medical device, we really you know, we're using that AI, building it again for internal use and external, really, and it helps us understand kind of customers' needs in this space, right, because Ketryx is such your unique piece of software. So I'm very excited for Bailey to be showing you those agents later on in the presentation. Very happy to be at Ketryx, and, yeah, let's talk some, agile AI and ML. So now that you know who we are, you know, here's what to expect from today.

Here's the agenda. We'll start with why agile development is is difficult in regulated environments as as many of you presumably know, then explore why AI and machine learning kinda raises the stakes and adds new challenges. And after that, we'll cover, you know, process changes. We'll cover process changes that make agile really work safely in in regulated development. And finally, we'll look at how combining agile with CICD enables continuous combined releases.

We'll also show Bailey will show you an amazing demo of Ketryx applying agile methodologies in tools like Jira to develop an AI enabled device. So I hope you're all very excited because I surely am. So before we dive in, we'd like to get a sense of kinda where everyone's coming from. So if you will, look at the poll area, in the in the in the chat there and, kind of try to answer the question, if you will. What development methodology does your engineering team use using kind of waterfall, agile, agile with CICD, some hybrid?

And, you know, Bailey, I'm sure you've seen many kinds of teams come to you with different kinda engineering methodologies. Right? Oh, yeah. Absolutely. I've seen I've seen it all, and I've worked in it all, from Kanban to, you know, traditional scrum to waterfall.

I've seen the the pros and cons of each, the benefits and the challenges of, trying to work in all of them. And, typically, I see, you know, a combination of them all. You know? It's really hard to be truly agile or truly waterfall. So I'm interested to dive into that and see what, the folks on the call, what methodologies they're using.

That's awesome. Yeah. Same here. Let's see. Our poll will be closing soon.

And, yeah, we see here in the results, which I think you can see in front of you, kind of the the plurality of votes goes to Agile with CICD, but then also, you know, this hybrid mix of agile and waterfall. So, you know, really, there's a good mix here, but a lot of you are using, agile or some some hybrid, which which come which includes it, in the mix. So let let us get yeah. The results are still and that's it. We're gonna put on the slides again.

There we go. And, yeah, over to you, Billie, to now take us into the world of agile. Yeah. Thank you, Maayan. Thanks everyone for your response to the poll.

Now that we have a sense of your current development practices, we will begin by, you know, laying some foundational groundwork on agile itself. And we'll start with a a brief introduction to those agile methodologies, referenced in the poll. So agile on the left is a flexible project management approach that breaks projects down into smaller iterative phases called sprints. Each sprint aims to deliver an increment of code that can be tested and improved and deployed independently, aiming for that continuous improvement, flexibility, and faster feedback loops, and continuous innovation. However, for med tech teams aiming for the same speed, the challenge arises because every single iteration must be meticulously tied back to requirements, risks, and test evidence to comply to stringent standards like IEC sixty two thousand three hundred and four, ISO one three four eight five, and ISO one four nine seven one.

So the problem arises there when, you know, you try to work agile while complying to those those standards. And what this leads to is teens working agile in one system while documenting waterfall in another. I think we saw that a little bit on the in the poll, a little bit of hybrid of both. So waterfall on the right is that linear sequential project management approach. It emphasizes establishing detailed requirements and then designing, developing, testing, and deploying in this waterfall or structured like phase with predictable timelines.

In this iteration or in this approach to project management, you're trying to align to manual check points which creates, you know, a a cascading effort of updating artifacts, artifacts. And so, you're not able to move iteratively or or in a cyclical nature kind of like agile is. And so what is really at the core of this dilemma? We see teams like you mentioned or like you saw on the poll that are doing a combination of agile and waterfall. But why is that the case?

Why do we struggle to be, you know, purely agile or or purely waterfall? And so let's dive into to that a little bit. On one hand, we have the promise of agile. That's speed, flexibility, and continuous improvement, which leads to fast releases and innovation. And this is what the r and d teams crave.

On the other hand, we have the reality of regulated medical device development. You have to have meticulous documentation and traceability and evidence for every change. This is what quality teams must ensure. So how do we maintain the speed and flexibility? How do we reconcile these two seemingly opposing forces?

I think or we think that this disconnect between agile development and, waterfall is really this disconnect between, these two with trying to generate that documentation and that evidence of compliance. So how does this line align with, the v model? So on the left, we have teams that are working in ALM tools like Polarian and Windchill, which are naturally built for, compliance to IEC six two three zero four. On the right, we have your, SDLC tools like Git and your CICD pipeline. But we have this gap in between that often leads to manual copying and pasting to try and get that information between your natural ALM tools that are great at complying to standards like sixty two thousand three hundred and four, maybe are more waterfall like, and the tools on the right which support agile.

And so this can mean that change management becomes very painful because every update requires rechecking requirements, risks across disconnected systems. And then you bring AI enabled devices into the mix, and the problem becomes even worse. You're in this endless cycle of trying to work in this area in the middle of copying and pasting information between these two frameworks to get that full cycle and understand, your full management of your product. And so in this sense, true agility can feel impossible. And so at Ketryx, we believe that automatically generating the evidence of compliance is how you fill that gap.

So I hope that's a good segue into, our next, topic, which is the comp complexity of introducing AI into products. Mayan? Yeah. Thanks so much, Bailey. Yeah.

And let's now you know, we've been talking about agile and regular development. And, you know, Bailey alluded to the fact that when you throw AI into the mix, everything gets even more complex than it already is. So let's zoom in on in on that on AI. Ai and machine learning are, you know, very heavily set in the current zeitgeist, and they're really changing how medical software is being built and updated. Right?

But, of course, not every organization is is at the same stage of adoption, So it's time for yet another poll, and to kinda get a sense of where the audience is. We just like to know, you know, how advanced is your organization in introducing AI or machine learning into your products? Are you kinda, you know, not exploring yet, early exploration, research phase? Maybe you're already piloting some some use of it. Maybe you have some limited use of it in production maybe for some of your products or maybe core to your product.

So let us now, turn over to the poll. And, Bailey, I'll put you on the spot again. I'm sure you get teams across the spectrum on on, you know, where they are in integrating AI into their into their products. Right? Yeah.

Absolutely. You know, AI has become a hot topic recently, and that's not news to anybody here. Especially when talking with enterprise clients, they're really excited to figure out how we can use AI in our current workflows and, how challenging that could be for a lot of reasons. How do you naturally integrate it with your tool stack as well as validate it and ensure that every change you're making with AI is tracked and can be trusted? Absolutely.

Yeah. It's a whole, you know, a lot of unchartered, kind of territory and waters there that that we're all managing. Right? And so it's good to have you know, I'm when I joined Ketryx, right, I, I was so thrilled to see all the subject matter experts that we have on hand, right, to that kinda offer a steady hand in these, again, uncharted waters. And so as we've been developing AI internally at Ketryx, as again, which is built to the to the level and the quality of a medical device, we've been applying the same methodologies that we use to develop Ketryx and that we offer our customers to develop our AI.

So, again, as I mentioned earlier, that gives you just this very important perspective into the needs of our customers that, you know, from this poll, many of them are integrating AI into, into their product. So that's very cool to see, and you you are in the in the right place for the seminar for this webinar. I'm sorry. So as this poll is closing, yeah, we see here it's very, very cool. We see we see a nice spread.

You know, we see in the poll results a bunch of people already piloting. Some of them, you know, some of them even have it core to their to their product. Some are in early phase. So it's really a nice spread here. And I think this this message of this, of this webinar, is kinda, you know, should resonate for for for everyone who is on any place in that spectrum.

And and, you know, of course, across the industry, we're seeing a rapid a rapid rise in AI and ML enabled devices. Right? And that kind of evident also by this by this small poll that we conducted, not just in volume, but but also in the kind of breadth of of applications. Right? We're also seeing kinda how you know, what regulatory authorities, what they consider to be medical devices.

We see that evolving over time. For example, you know, AI scribes for physicians being considered a medical device, certain regulatory domains, right, which it didn't used to be even even a year ago. AI is moving, you know, well beyond, you know, kinda radiology into surgery, cardiology, drug delivery, and even kind of manufacturing and supply chain. And, you know, for example, our customer, Heartflow, develops AI powered noninvasive cardiac imaging software, right, just as an example. And the chart on the right, really shows how kinda approvals have accelerated in the last decade regulatory approvals for AI enabled devices with especially sharp growth in the past few years likely due to the advent of, you know, generative AI, transformer based AI techniques like LLMs, etcetera.

And the key takeaway for us is that, you know, AI is no longer just just a niche. It's not no longer, you know, few and far between. It's become foundational across diagnosis, treatment, and operations, right, across many fields. And, you know, as as Bailey mentioned, right, agile is already challenging in regular environments, but AI enabled devices just raise the bar, you know, even higher in that complexity level. And unlike typical software, right, these machine learning models, they require frequent retraining and revalidation.

Right? You want to make sure, that your your, you know, AI enabled device, for example, is is, you know, aligned and appropriate for its real world uses. And agile tools just don't capture those events or the evidence that regulators expect. The datasets that that these models are trained on, they also change, you know, quite often. Right?

And even small shifts, even changes to the training methodology can really affect the safety and performance, right, which is really what we care about. And that can that can break the traceability back to requirements and risks, which is a big no no. On top of that, you know, AI devices release smaller updates more frequently. And instead of making teams more agile, this just amplifies documentation overhead. It just slows the teams down, right, which is the opposite of what we want.

And and AI, of course, also requires expanded post market monitoring and ongoing drift detection. And these are continuous compliance tasks that just don't fit neatly into the sprint cycles in agile that Dale talked about. And let's let's now dive a bit deeper into kind of model drift and data drift since it creates, you know, unique risks that make agile especially hard in regulated environments, and and the FDA has fortunately recently offered guidance on this. So, you know, and perhaps this is one of the strongest arguments for agile in in AI enabled devices. It comes from this recent research, from the FDA at SPIE Medical Imaging twenty twenty four, paper titled, and this is a mouthful, as usual for academic papers, quantifying input data drift in medical machine learning models by detecting change points in time series data.

This paper was presented by FDA researchers, and it showed that, you know, you can statistically detect drift in real world medical data streams very early. Right? And and this result from the FDA researchers, it sounds like really good news, and it absolutely is. But it also means, you know, that you as an as an organization, as a product, you'll be detecting this kind of statistical drift more often. Right?

And every such detection is a potential retraining and revalidation spend. Right? More documentation overhead and so on and so forth. So the challenge is in a waterfall process, that frequency just becomes untenable, unmanageable. Right?

Each detection triggers a massive revalidation cycle that can take even months. And so teams either, you know, delay acting on drift, which is risky, but performance quietly degrade, also not desirable. In an agile and CICD process, drift detection is exactly what you want. The pipeline absorbs the signal. Right?

It it it now you retrain, you revalidate, you regenerate your compliance artifacts, and you ship a safe update quickly, which is awesome. So what's the main takeaway here? Right? This paper makes the case clear. Early detection, you know, only delivers value if your development process can actually keep pace.

Without agile and continuous integration deployment, which, you know, Bailey will introduce, more in just a moment, early detection just creates bottleneck. Right? With agile, agile is the key to safe continuous AI and ML releases. And in the context of Ketryx, you know, how can Ketryx monitor data drift? Well, you know, as a best practice, teams should write automated tests, right, for detection of data drift.

They check for performance issues or or data drift for in their models. And they can use whatever testing system, they use, with Ketryx. And we integrate with a bunch of platforms including GitHub, TestRail, etcetera. And if any one of those tests fail, Ketryx will actually prevent a release from moving forward until the issue is resolved. Right?

And, of course, teams can use, you know, the Ketryx assistant and agents or AI for recommendations on how to write these drift detection tests. So that's really cool. And now back over to you, Bailey, to take us further, you know, deeper into agile and CICD. Awesome. Thank you, Maayan.

Really cool to hear your perspective as someone who works so closely with AI and to hear your take on it. So thank you for walking us through that. To take a step back, I think it's really important here to walk through continuous integration and continuous deployment, so CICD. And I know that, teams here are already using that and that's great, because CICD is really central, in in being in being agile. So continuous integration, we're we're working on making small code changes in a shared repo, that are automatically built and tested, helping you catch problems earlier.

And then continuous deployment, we're automatically running that CI pipeline so that you can push those changes out automatically, ensuring that new features and bug fixes can go live within minutes of being built and tested. And together, they form that CICD pipeline that is automating your build, test, and releasing. And in the agile world, this is exactly how you move quickly. Agile is really that framework, and CICD is how you implement that speed, the agility, the flexibility of agile. So why is c CICD so important especially for medical devices?

Firstly, it's about speed. It's being able to to get those changes out into the hands of users, whether that's bug fixes or new features, and that exactly is the promise of Agile. Then it's about quality. You wanna make sure that you catch bugs early, and that means they're, you know, often cheaper and easier to fix. And it's also about reducing risk because you're making small changes frequently, then if we have an issue, we know exactly what change likely resulted in that issue and can roll back changes quickly.

And then in a regulated development, CICD can also drive compliance. And that means with the right setup that every pipeline can generate that evidence of traceability and enforce your QMS. So that's where we see CICD really shine in helping solve the agile dilemma, giving you both speed and safety. To take this a little bit further, we we see the strength of agile. We see the strengths of CICD and where they bring really big advantages.

But, what teams find is when you go to actually implement this, especially in a regulated industry, you also bring about challenges when it comes to validation. And that's again because every single change requires meticulous documentation and evidence and revalidation for every change. And so while agile and CICD thrive on constant change in a regulated industry, you have a lot of checks that you need to go through. So fast and quality may not go together, with Teams current setups. And so there are some things that you can do, and then I'll propose my next slide that you can set up to really allow you to use agile and CICD so that documentation isn't so much of a burden and you can create the that evidence of compliance as you're working with every iteration.

So this is how we really think of implementing that and how to how to how to create that strategy. That's first with ensuring that you set up a a correct architecture. That's really taking a complex system and breaking it down into a systems and systems architecture where you can deploy at or code, deploy, and test each of those subsystems independently, but also at that top level systems, manage it at the systems level as well. Excuse me. The second is embedding testing alongside your development.

So that's utilizing automated tests in your CICD pipeline. And then thirdly, ensuring you have end to end traceability as you're working and set up at the end. And then Ketryx can really help you enforce those procedures and automatically generate that evidence compliance as you're working so that a lot of these artifacts aren't, you know, afterthoughts once you've finished coding, but are really being done as you're working. And that's really where we see the strength of agile and CICD combined. And so with that, I'm really excited to show you how we take these four steps, into the Ketryx platform.

Alright. I'm going to share my screen now as we transition to the demo environment. AKA the best part of the webinar. I know. Everyone's always like, alright.

Just get to the just get to the slides. Meeting potato. Absolutely. Alright. At this time, I'm sharing my screen.

Mayan, are you able to see my screen okay? I am. Yeah. Awesome. And feel free to use the chat to ask any questions.

We'll try our best to answer them live or have our team answer them in the chat as well. I'll also save five minutes at the end of this demo to, you know, save some time for questions at the end. So for those of you who haven't seen the Ketryx dashboard before, the Ketryx platform before, welcome. We're very excited to walk you through how you can implement agile, and regulated industries integrating directly with your CICD pipeline, and also adding the complexity of an AI tool as well. Here, we're looking at the Ketryx platform and we're managing multiple different projects.

We're utilizing that systems to systems architecture that I mentioned to manage this project that references a GenAI platform. Each of these projects that you see here contains all of your design controls, your design inputs, outputs, and it's connected to a preferred tool that you're using for whether for requirements or for source code or for test management. And like I mentioned, each of these projects can contain their own workflow, their own approvals so that you can release them independently and on different versions. We're going to work into releasing this case insights validated project, which is a top level project referencing this Gen AI platform. This Gen AI platform also has some of its requirements in Jira and we're connected to a Git repository as well.

And you can see we're utilizing that version control to allow these two subsystems to be on different versions, and reference it as well. Over here on the left is this complaint processing validate application. This is another product that's referencing that GenAI platform to do complaint processing here, utilizing that systems assistance architecture to maximize reuse. In this case, we'll only be looking at these two subsystems for the demo today. We'll first go into this case insights validate application.

We'll see some work that needs to be done in order to get this to release ready, and that will involve going to the Gen AI platform, creating a test case for an automated test that lives directly in the source code. We'll release the Gen AI platform, and then we'll go back to that top level project and release the full product. In this way, I hope to elucidate how easy it can be to connect to your CICD pipeline and automatically generate the evidence of compliance to quickly go ahead and release this full product. Alright. So let's dive in.

Again, we'll first take a look at this case insights project, and I'll jump straight to the traceability matrix. This case insights project has, I think this is seven columns, a seven column matrix here with user needs and system requirements managed in Jira at the top level. And it's referencing some requirements, some software items, some test cases, and you can see they're acting as risk controls here, which is what RC means. And we're referencing these from items that live in the subsystem to populate this full traceability matrix. And then again, we have test cases that are also at the verification level and validation level being pulled in.

Now this is a real time view of traceability. This isn't something that gets created as an afterthought or after your development is complete. We're building this traceability matrix and allowing you to easily understand where you may have gaps and traceability or where you need some work to be done in order to get this to a release ready state. We embed some design controls at the top here that allow you to easily understand where you may have those gaps. So you wanna make sure that in some cases, you have, you know, all your design inputs covered by design outputs.

And so checks like that can be created at the top here to ensure that you're working towards that release ready state. And this traceability matrix is highly configurable. We work with many teams with many complicated workflows that have different ways or different columns, different workflows, different item types, to build out that traceability. And so we can work or we work with teams to build out the traceability matrix that they're used to seeing or that is native to their workflow. Okay.

The other thing I wanted to show you was how we use our AI assistant agents and AI intelligence throughout our platform. And this assistant here is able to understand all the context in our system of systems architecture. So being able to see all the items that are in this top level project as well as the items that live in the subsystem. And so to give you a better understanding of what we're building here with you today in real time, I'm gonna ask the assistant, tell me about tell me about what the case insights validated application bits. Great.

So this will give us a little bit of context into what we're building here. In this case, we are actually using an AI the GenAI platform to to get some health insights about, patient data. And so I can expand this here to show you exactly what AI is doing to generate a response for you here. So I asked it just to tell me what we're building here and to get some more context around this case insights validated application. So the assistant responded telling us a little bit about it.

It's a personalized health insights and recommendation platform for diet, exercise, and sleep using AI technology. And we have some key features here that it's used to, again, gather this information not just from the system, but the subsystem as well. I think this is extremely helpful, especially in industries where you have a lot of subject matter experts or SMEs that you need to go to to get information. But with our assistant here embedded into your workflow, it understands that full broad picture of everything that's going on in your system so that you can allow the SME to work on dedicated value added work and really reference your assistant for, information like this. But our assistant doesn't only gather information.

You can also ask it to write a test case for ten, for example. I may need to provide it with the name. So not only do we connect to all of the different platforms to provide you with the most relevant accurate information about your system. But you can actually take action, write requirements, write test cases directly with this assistant here. It can even review your requirements and make suggestions to update them to make sure all of your requirements are written in a testable way.

And so here, we've actually analyzed this here, and you can see this with this highlighting that this is the requirement we're analyzing. And all I did was pass it the title, and it says, I can write a comprehensive test for you here. Looks like there's already one created. I think that's what it says here I'm summarizing. And so we can review this suggested item and it went ahead and wrote this test case for us with relevant test steps.

So this is just one way to, you know, work very quickly but in a controlled way, always requiring a human in the loop and keeping a robust audit trail. Great. I can talk about our AI capabilities for forever really. But instead, I'm actually going to switch into our Gen AI platform. Again, this is that subsystem that contains the code for this case insights full product connected to Jira and Git.

Jumping in here, we're gonna go look at the traceability matrix. And then you can see that the traceability matrix is missing. We have a test case that's missing for this transparency and audibility requirement. So let's go in and create that test case for this requirement. I've cheated a little bit here, and I've already written the test case just for the speed of this demo.

But I'm gonna check out to this specific branch that we're working on. This is the branch that Ketryx reads to know that I'm writing information, I'm writing test cases, so I can automatically report all of this evidence over to Ketryx and build that traceability. So here I can say at test, which is us me using my tagging language here, the Ketryx tagging language to build that traceability as you're working. This allows your developers to stay in their preferred developer tools and build traceability. You can think of this like conventional commits where you're tagging it to the Jira item and this is what I've done here.

I've actually just used the Jira ID to build that traceability. And so because Ketryx integrates so deeply with your CICD pipeline, we can go to the actions here and see that CICD build is being kicked off. This is, you know, running that build artifact. It's running those automated tests for you. And then what it's gonna do is it's going to actually report those test results over to Getrix.

And I can show you exactly how it does that. So right here again, we're showing this this test, mode here or the workflow that we've gone through. We set up the job, we've run dependencies, and we've reported that all over to Ketryx. And if I jump into the workflow file, I can show you exactly where we're doing that. And so right here, you can see that we're reporting tests directly to Ketryx with that success or with that failure.

And so jumping back over to Ketryx, I'm just going to do an automatic sync here for you. And because we integrate directly with your CICD pipeline, we can gather all those builds, all of those, commits over for you. And you can see that here, in the history. All of my commits are connected, to my repo, and I can track them here and also use this to generate, evidence of compliance for every commit and for every build that I've created. So going back to the traceability matrix, we just wrote a test case together.

We ran that automated test after I, you know, built traceability. And now you can see that this requirement, it has a test case and we can see the results of that automated test execution is passed. Now that we have all checks passing and we're green, we can jump to our release dashboard and see where we're at in this release process. Jumping to version one point two, this is the version we're working on together to release this insights platform. I'm gonna pause here at the release dashboard.

This is another very valuable check to determine where we're at in our release process. Do I have all of my design controls, dependencies, test executions run? And you can see all of the items that are in a controlled state. You can ensure all your design controls have passed as well as test executions. There's a lot of relevant information here that you can use to understand where you're at in that release readiness.

And then we can also go to all the documents which you can see is the next step that we need to complete in order for us to be release ready. So clicking over into the documents module, I can look at all of the documents that I need to complete or submit to the FDA, for example. And with just a click of button, I can go ahead and generate all of those documents. So what's happened is because I've made a change to an item that lives in the source code, any document that's referencing or relying on that underlying item, whether that item lives in Polarian as a requirement or Jira as a story or directly in the source code as a test. We ingest we connect to all of those systems, we ingest that information and automatically put it into a document for you.

So again, this documentation effort is a byproduct of the work you're doing. And so I generated all of that information that I need to get this into a release ready state. And now I can put my part eleven compliant signature around each document to make sure we're getting this ready for that release date. We embed these part eleven approval workflows directly at the item level or at the document level as well. And you can see as I'm quickly moving through these documents, how easy it is to track the changes that you're making for each document.

I just clicked this clock here to show me that we keep a robust audit trail of every time we generate a document, the change that we're making to the document, and who approved it, and when. And you can see that records created right here. Okay. Moving through to our last document that we need to complete to get this to a release ready state, We can now see, great. We've approved all of our documents.

We can we kept an audit trail of all of those changes. And now we can go back to our release and see where we're at in order to release this subsystem, GenAI platform. You can see now that all of our checks have passed, and we're ready to go ahead and release this subsystem. So I'm going to approve this again using my biometric signature to approve this release. And together, we've just released version one point two, and that confetti is always so rewarding.

Congratulations. Thank you, Liana. And I did it with the help of all of you guys here on this webinar today. So just to recap what we did at the subsystem, we were look we were, we navigated to the GenAI platform, which again is a subsystem connected to our Git repository. I kind of, glossed over the all items screen, but I think now is a great time to go back and and center around this all items screen.

I mentioned that we're connected to Git in this Gen AI platform as well as Jira. And this all item screen acts as the centralizing place to easily understand every item in my full life cycle. And I'm saying item because that's a term directly from IEC six two three zero four. We break your steps or your SDLC into an item level so that you can, work on each request or each requirement, each change request, each CAPA, approve them independently, and then use that controlled workflow to contribute to the automatically generated evidence like we just said on the release screen. And of course, like the traceability matrix, this is highly configurable.

We these are the default item types that Ketryx has out of the box, but we work with teams to create epics, stories, bugs, whatever you call the items in your system. One of the key advantages of the Ketryx platform is that it's highly configurable. This configurable workflow allows teams to really pick up the platform because we adapt to your current ways of working. We're not really a disruptive tool to make you change the way you're working. We're really an added measure on top of the way you're working to enhance your workflows and automatically generate that evidence of compliance.

So going back to this all items screen, we can see all the items that live across our system no matter where they actually live, whether their requirement acting as a risk control. We can see what state they're in. And another key feature that teams use this or another strong feature of the all item screen is that this really acts as a change control board. We can see every change that we've made across our systems with exact red lines to show us those changes. Ketryx, because it integrates with all of your systems, creates that robust audit trail for every single change.

And so having that is real evidence to an auditor that you're keeping track of every single change that you're making in your system. In this case, we've just updated this UUID because I reran the test and didn't make any changes to the test case. But you would see those red lines exactly here. And down below, this is us looking at the Ketryx version of this item. So this is actually a test case that lives directly in the source code and we're viewing it right here.

We're seeing the information that's implemented directly in the source code here as well the approvals and the relationships between records. So going back to this traceability matrix, we approved everything here. We released the GenAI platform. And now let's go back to that top level case insights validated application. I'm going to jump into this graph view here to show you a different structure of the architecture here.

We're again at that top level case insights platform. We're referencing the GenAI platform, and this is the insights platform we're using. And these are the different modules that are being implemented in that GenAI platform. And, of course, you can download this as a CSV or Excel, and you can use that feature along many modules in the in the Ketryx platform. You can, you know, just look at changed items here and export this as an Excel or CSV from all items screen as well as the traceability matrix here you could download as a CSV or Excel or Word as well.

Great. So now we're back at that top level case insights project, and we're ready to release it because we just went in and released that sub module, the Gen AI platform. So going into this top level release, we're working on version two point o. You can think of this as your product release even though we just released the Gena AI platform at version one point two. So again, this is that release dashboard here, but we've used some milestones at the top here to implement those maybe more waterfall like structures.

So so far, we've really seen how we can embed those agile practices using the Ketryx platform. We integrate directly with your CICD pipeline to gather those automated test results. We use the all items screen to very easily understand change management. All of this inherently supports agile. It's how teams are using Ketryx to speed up the way they're working.

But, of course, like we mentioned earlier, some things are naturally waterfall, and maybe that's requirements gathering. You wanna make sure all of your requirements have been gathered, approved, that you've had a meeting about it, you have meeting notes generated, all of that maybe before you start coding. And so CapEx has the flexibility to embed those more waterfall like activities using milestones for each release. So in this case, we have some milestones for requirements gathering, for defining that system architecture, and for releasing that GenAI platform right here, which you can see we just did together. So now all that's left is the system integration testing.

For the purpose of this demo, I actually enabled it so that you don't have to release this milestone in order to release this version. But this is the type of check and enforcement that you can apply. You can require that your teams, you know, release or approve those milestones before you can actually release your product. So now that we've released the GenAI platform and I was ready to go with all of the other changes at my top level platform, you can see that we don't that because we understand all of our changes and we made small changes in incremental approaches, we know exactly what's changed at both the component level or the sub module as well as at the top level. And so because we understand changes and we can integrate and use our systems to system architecture, We know not much else is changing and we have high confidence of that using our controlled items here, our design controls, and maybe we need to rerun some top level tests, which is indicated by this milestone.

But like I said, we'll leave that unattended to for now. So let's release this product together. Again, I'm going to use my biometric signature here to approve this top level product, and now we just released that product version. This is how you can have a a GenAI pro or product that relies on an AI algorithm that requires frequent updates that maybe requires training, or dataset updates. And you can easily go ahead and make those changes in the code, connect to or create a CICD pipeline to automatically run tests, build, report the that evidence over to Ketryx, by us being able to track all of your commits, all of your builds for you, and then we put that into the documentation for you.

So let's go take a look at one of those documents that we automatically generate. We'll open up this SRS here. Great. In this case, we've opened up an SCS, which is a Word document of our system design specification. This is again ingesting that information from the underlying platforms like the items that live in Jira, your test cases that live directly in Git.

And so we can see, how we automatically generate, these documents for you. I tried to expand it to make it a little easier to view, but it looks like that didn't help. So, again, this is these are items that live in those connected systems that, are being formatted directly into, you know, create your test report, your test plan, your risk matrix, your change plan. All of that can be automatically generated by Ketryx tracking, the changes that you're making in the underlying systems. Alright.

That is the bulk of how we use Keturic's help teams, you know, speed up development, you know, create, quicker releases, speed up that release time and integrate deeply with, agile and CICD in a regulated industries, especially, when you bring in the complexities of using AI platforms. Maayan, is there anything I missed or that you wanted to add to that? Yeah. I think that was really awesome. Thank you for that demo.

I think, you know, Ketryx really kinda, you know, showcases how to combine these deterministic, enforcing processes, right, that that are informed by your regulatory and QMS needs together with, you know again, we harness the power of generative AI in the form of our assistant and agents to really kinda, you know, help guide users, through that process. They're not alone in that, you know, scary daunting world of of regulation. And, of course, the user always have the control, so when the AI makes suggestions, etcetera, users choose when to approve or not. But combining those two, all the nice you know, these very powerful deterministic processes that Bailey showcased together with RAI, we have one, you know, heck of a system. So that's really awesome.

Yeah. Thanks for that, Maayan. I think we are. Yeah. I was just seeing if there's any questions in the chat that we can answer live.

Yeah. So I see a couple here. Can you see them? Yeah. Yeah.

I can see some. And, yes, of course, for those who had to drop, no worries. We will share the recording afterwards and, create this available to you to to refer to and share with teams. I think one question in the chat was that tier team prefers using user stories as product requirements, and there's no clear way to track what's being overwritten. This often leads to a chaotic and inconsistent requirements document, which becomes problematic when submitting to regulatory agencies.

Do you have any advice? Yes. I have lots of advice. This is actually a very common challenge that teams face is you are using user stories and maybe Jira as product requirements, and you wanna keep track of all of the changes that are being made. And so what Ketryx does because it integrates with your tools like Jira, where maybe your user stories are being written and maybe your requirements are written in Plarium, We integrate with those tools.

We have bidirectional sync with tools like Jira to keep track of all of those changes that are made. And then we can also, you know, write requirements for you based off of user stories. So something we didn't touch on was our agents. I showed you a little bit of the assistant sidebar and how teams use that, but we also have agents that run-in the background that the MyON is an expert on that can actually look at your user stories and write requirements for you or make updates to requirements based off of a user story. So it's a very flexible platform that allows teams to do exactly what this comments or this question suggested is continue to work the way you're working, and we can help you create that evidence that you need, and pulling all the relevant information that you need into that requirements document.

This was great. Thank you so much for a great demo and for being a wonderful cohost. Yeah. Absolutely. And, of course, if anybody's interested in doing a deeper dive and talking more about these these key principles that we touched on today.

Of course, you can set up time. My team would be happy to walk you through a platform and give a dedicated demo. So thank you everyone for attending. We really appreciate it.