---
title: "AI for Systems Teams - Where to Begin"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/i9szk54jo7"
content: auto-caption transcript, proper-noun corrected
---

# AI for Systems Teams - Where to Begin

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/i9szk54jo7)

---

Hi, everybody. My name is Erez. I'm the founder and CEO of Ketryx, where we help the world's largest and most innovative companies accelerate their product development. Today, I'm so happy to share this webinar about how AI for system teams works and where to begin. With me is my wonderful cohost, Bailey Cantor, who is our director of solutions.

Bailey. Hey, everyone. So excited about this topic today, which I'm really passionate about. Wonderful. Nice to have you here today, Bailey.

We're just gonna wait one more minute as people kind of get into the room. We have quite an audience today. This is a pretty exciting webinar because, Bailey, I remember when we were talking over lunch just a few weeks ago about the series of on sites we have with a few, of our largest partners. And we were sitting talking, like, we should kind of tell this story to other people and not just to the people we specifically work with, how AI is affecting these teams and what we're learning. And I know it's a big passion for you because you are a systems engineer at heart.

Yeah. Exactly. I. This couldn't have happened sooner. I mean, it would have been nicer if I had AI when I was working as a systems engineer in the defense space, but I'm really excited to see this this motion here to start thinking differently, equipping the systems engineers with the tools to move faster and be successful and solve some of the challenges that I found really frustrating in my earlier career.

And as I am talking with these Fortune five hundreds, you're exactly right. Everyone is asking that same question, and they're really excited to hear what we have to say and our approach for how we solve it. Yeah. And I think it's it's, like, kind of like the history of systems engineering where it is so complex to understand how a system work and how many different systems connect together and work. And then, like, understanding how AI is used within that context and how do you absorb the changes from one team to another team and coordinate all that is just rather complicated.

And I think that's what we're gonna break it down together and talk about how the best teams in the world are already doing it and already getting real world ROI, for that usage. Yeah. I'm so excited. So I'm just gonna wait one more second here and because we're still seeing people, come in, and then we'll get started. So what are we gonna talk about today?

Today, we're gonna talk about why I and your product and workflows break traditional systems models, how continuous compliance solves the validation bottleneck, and how to do it in practice, what continuous compliance looks like, and what proofs there are already from the field, including kind of milestones and best in class proof points. And then finally, we're gonna talk about how to apply AI native systems thinking to to your programs starting now and have a demo, a real world demo of how that looks and what people are doing today. With that, I just wanna introduce myself and Bailey. Eras Kaminski, like I mentioned, founder and CEO of Ketryx before this, studied computer science at MIT, worked at Wolfram Research on products like Mathematica, Wolfram Alpha, and the Wolfram Cloud helped engineers and mathematicians and physicists use the amazing power of computational mathematics. And then I was fortunate enough to join Amgen leading AI machine learning in helping understand what AI will be in this iteration of the life science industry.

Then we started Ketryx to to to build a community of people to help solve all challenges of how to use AI in safety critical product development. Awesome. And and I'm Bailey Cantor. I'm the director of our solutions team here at Ketryx. My background was originally in physics, and I started out as a systems engineer in the defense space where I saw a lot of the challenges that we'll talk about today.

And then I was in charge of our systems delivery at AMLO. So, yeah, I've got a lot of a lot of, tips, tricks, and, thoughts on systems engineering, SysML practices, and I'm really, really excited about how Ketryx solves those challenges and the approaches we take to, help the systems engineers. Yeah. And I remember when we first met, Bailey, you, like many of our early employees, you, like, found our website and were like, because you were searching for a tool for some of your work. And you're like, this is a pretty pretty cool thing.

Like, maybe I should look at what they're they're doing over there. And you went from kind of this r and d side to now working in solutions for for our partners. So it's always awesome to have a practitioner with us. Exactly. So what's Ketryx?

Just to remind everybody, Ketryx is an AI native compliance platform purpose built for regulated teams, safety critical teams, mission critical teams, and helps them deliver safer products faster. It's all about accelerating safe innovation. Ketryx allows the different teams in your organization to use the tools they're using, interconnect them through Ketryx, and then automatically enforce compliance and generate evidence of compliance through the tools you're already using. It's a tool built to accelerate this workflow, and we're very excited to share with you how this looks and all the work we've been doing over the last few years, leading up to Ketryx three point o, which is coming later on this year. We first wanna talk a little bit of a survey.

How does your team currently handle change impact assessments? We're gonna start the survey now. We'll share it in a second, and please fill it in. And in the meantime, I'll just say that we've seen over the last twelve and twenty four months an entire universe of of opportunities. We've seen companies still kind of do this on paper, with almost strings.

We've seen companies that do this in Excel. People who have built quite robust automated tooling, people who use AI for this, people use their existing kind of PLM, ALM, change management solution with AI, with automation, and and pretty complex solutions. We've met companies who have spent tens and hundreds of millions of dollars developing solutions here and try to understand, like, how can they still get better. So it's a very active field of development, and there's just so much going on. I'll just give a few more seconds to the survey cause I see folks are still filling it in, and then we'll share and we we'll see where where the crowd is today.

Yeah. And one thing I can add, Aras, is thinking about my time as a systems engineer. I worked on a product line for many different radar systems trying to do these change impact assessments, and I specifically remember a few core challenges. One of them being that the onboarding was so difficult. How do I learn how to do a change impact assessment?

How do I even understand all the tools that teams are doing their work in from the software teams in Jira, our own defect tracking tool, our own test management tool? You had to jump across all of those tools and teams to understand that, specific change. I remember going through rows and rows through doors to read every single line to understand what is implemented today and then have to go talk to a SME in person who wrote that requirement maybe twenty years ago and then trying to understand the impact of that change. Now this was such a manual process. I probably spent about, you know, eighty percent of my time just doing change impact assessments alone, which is why I'm so excited to talk to the teams that I talked to, you know, many Fortune five hundred companies who are trying to say or trying to find different ways to do this.

We don't have to do it manually anymore. Yeah. And I felt the same way. Like, when you get really into these procedures, you're like, gosh. Like, where is the system that is forcing me to do this and analyzing it for me?

And how can I use AI to do this faster? And I think that's what we really connected on. And I remember this long trip business trip we're on on a train talking about this for a few hours. Like, it, like, it has to be faster. Right?

Because this is where we're spending so much time, and it's so complicated. So let's just get the the survey and let's share it now, and have people kind of view results. And I wonder if you are where you thought you'd be. Okay. Moving on.

I wanna just kinda start this discussion by talking about what's going out there in the world. This is a graph we've been sharing recently about, a very unique, very cool class two product that is releasing very, very fast and developing very fast. So we can see that over this kind of false winter period of this year, they've rapidly increased the amount of commits they're doing. That's the purple line. Of course, we know coding models have gotten a lot better really in this kind of period, especially at the beginning of this year.

And they're just doing a lot more work and a lot more commits through AI. And at the same time, we're seeing something that to me is counterintuitive where they're releasing faster. I think most people would say that that can't happen. Like, if you're doing more code, you're releasing slower because there's a lot of work associated with that. And I guess that this entire webinar is really meant to explain how this is possible and what infrastructure this team needed to have in place to both accelerate the features they're delivering and accelerate the rate of their delivery, where they're landing in about five times a week a week developing this class two medical device.

So let's talk a lot a little bit about what's going on in different companies. Basically, there's two AIs that are coming into companies today. One AI is AI in the product. There's, features, whether that's LLM or more traditional machine learning, that are becoming much more enhanced than many of them for the first time possible with AI. And that amount of functionality like software and machine learning before it, is rapidly increasing the complexity of the products.

So it's not just that we're doing more. It's actually much more complicated to to make sure we're doing more correctly. And this is increasing kind of the gap between the approval time, the model actually getting updated, and then we need to develop different frameworks to help make sure that we are learning from new data even post approval. And that's why the FDA has plans like the predetermined change control plan and guidances around AI kind of life cycle management. And I really recommend reading this guidance that came out last year that I think explains a lot of of the agency's approach, and and I believe that a lot of the world will take back from this approach because the FDA is really leading the front.

At the same time, we also have AI emerging in our workflows. So AI in our nonproduct software, AI in our infrastructure helping us change faster, also requiring us to change faster because AI is also now a product in our workflows and is being updated often. We have just much more options, many more tools, Copilot suggestions, AI generated code, AI assisted spec generation from requirements, from the code. And there's more and more time now taking between the decision we're making and documentation because the world is accelerating. And I feel like everybody is feeling this, that just products are becoming more complex, products are moving faster, and the way we're developing them is moving much faster.

Earlier this year, the FDA even issued its first warning letter about how to use AI safely in workflows, and that's part of what we're gonna talk about today, in this webinar about how to do that and make sure that your usage of AI, is safe, compliant, but it also allows you to leverage it to the the kind of best outcome to accelerate your product delivery. And and what we're seeing right now, Bailey, I think we've talked about a lot, is this validation bottleneck. Right? Yeah. Exactly.

This is really the the core of where we're sit seeing teams sit. It's this validation bottleneck. It starts with this, you know, one x, you know, one x in, one x out, which, if I'm being honest, I think that's an assumption we're making. When I talk with teams and have implemented many teams using AI use cases, it's usually more like one x, three x. But what we're seeing is that you equip your software developers with the AI tools like Copilot, and they're able to develop software faster.

But what you still run up against is this bottleneck where you're developing ten times faster, but you haven't equipped with the rest of your your teams, your processes. You haven't aligned the architecture with your tools to be able to handle the compliance overhead. Those are things like approvals, traceability, document generation. Those are really important things that you must do in order to ensure high quality product. Yeah.

And I think that, like, solving the acceleration of the left side without solving the right side won't actually help us deliver products faster. And I hear this day in and day out where people almost ask me, like, I don't get the hype behind generative AI because it hasn't yet given me any ROI. Like, I'm not actually accelerating. What's happening, though, is I'm having a lot of work, and I'm getting a lot of complaints from my quality and validation assistant engineering team that there's so much work being generated, and is it the right type of work? Yeah.

And, Erez, you actually said something I want to highlight is you're exactly right. You can't just focus focus on one side of this. When talking with many of the Fortune five hundreds that I've gone to the past few weeks going on-site, they're looking for serious efficiency gains. They want to develop software so much faster. We're talking going from quarterly to weekly, biweekly releases.

And the way they're implementing AI just they're not seeing those returns. And it comes back to this bottle bottleneck. You have to look at all sides of it. You've got to fix the people, the processes, and the tools and equip all of those three key things with AI. And I think that's exactly what we'll see on these next few slides.

Yeah. And this is one of the quotes we've heard about it that, you know, development used to outpace compliance by one to three, and now it feels like it's one to thirty. Like, I need to spend thirty x more time in compliance because so much work is being done, and there's just so much effort. It's so much easier because of the the many, many, many horizontal tools to accelerate, r and d and codevelopment. Very hard to accelerate quality and compliance and all these assurance kind of design assurance, quality assurance aspect of the work.

Yeah. And so we get a lot of questions. How do you actually do it? So, I first wanna say that it's hard. It's not like there's this, kind of magic, magic moment or magic, tool that if you just connect, it solves everything.

It's a lot of different things, and that's the whole challenge. And at its core, I think it's about two different approaches to AI. One is this generative approach, which is using generative AI to generate content, to review content, to suggest ideas, to synthesize ideas. And the all other is this symbolic approach, that helps you ensure processes are delivered correctly, outputs are reliable, and and the documents are structured correctly as well as the structure of your traceability is maintained. People call this, neurosymbolic computing, and Ketryx is is one of the first kind of enterprise applications bringing that to the field.

So we can think of these generative agents that do little tasks. Right? They go and they write code or they go and they write a requirement or they generate requirements from code or they generate risks. They do these little tasks that people used to do. They're mapping kind of, you know, one task to another input to output.

On the other hand, we have these symbolic process agents that help make sure that, the quality system is executed correctly, whether that's from a process verification that we've done step one, three, four, and the right people have approved or from the aspect of, traceability that even if you generated the traceability across many different people and many different teams, still consistently a distributed set of people are reviewing them left to right, for example, making sure that use cases are approved before design inputs, are approved before design outputs, which are approved before tests, and making sure that all the different things we need to do happen in place. The real acceleration happens when you're able to harness both of them, both the symbolic approach and the generative approach in this neurosymbolic approach that is the right thing for people that work on critical products. And what we really wanna talk about and show today is three different high level workflows and strategies that people are putting today in practice to integrate AI, into safety critical and mission critical workflows in a way that is safe, that assures quality, and that generates the compliance evidence we need to prove both of those things. We're gonna talk about each of them for just a second here with some examples, and then, Bailey is gonna show us how this actually works in practice and kind of take us through these workflows. So the first one is that we need to build our architecture for change.

When we started Ketryx, we we talked a lot to folks about this need to design both the software, the hardware, the design controls, the risk management, and the way we build our team of delivery around, the need for change, which is gonna happen more and more often. You need to have reusable components within a system of systems. The second part is you need to have symbolic assurance of different quality workflows and integrate that into the your tools and how you work. One example is that if there is a lot of change, you need a way to deterministically check your traceability structure. And for some high risk products, make sure that, the v model is going correctly left to right step by step in threads of traceability and not just in columns of traceability.

The third thing oh, yeah, Bailey. I was just gonna say, I think that's a really important one because when talking with different teams, you know, executives going on-site who are running into these challenges of, k. This sounds great. We all agree what the challenges are, but how do you do this in practice? It always comes to these three things.

And maybe they've got one of them down. Maybe it's deploying an agent that looks at your tasks and creates a, you know, an output. But getting to all three is where they're starting to see those efficiency gains. Sorry to jump in there, Rez. Just really excited about that.

Perfect. And I and I I love how you're saying it, and I think that relates to this third part of, like, then you need a way to actually automate your change impact analysis and use AI for that, but still have the system ready for it from an architect perspective and the symbolic quality assurance controls in place for the AI not to create any needless risk. Right? Because it's very easy to introduce risk using AI systems, And it's very easy also to, like, not change and not try to move faster, but consumer, patients, the public needs us to move these systems and design them faster and at the same time find ways to make sure that what we're doing is safe. So this is kind of how we're gonna combine them all together and layer them.

So the first aspect of design architecture for change, I you know, we're gonna show what it means, but it's basically all about having a system of systems. So you have kind of maybe two different product systems that serve patients and providers or, you know, or consumers and and businesses, many different aspects or different types of consumers, maybe different mission partners if you're in the DOD, DOW space. And then then they have different systems they're using in different services or subsystems those systems are using in different artifacts like open source libraries or in house libraries or soup software of unknown provenance that are then consumed by these systems. Now if every time, we create a brand new system like they used to do in software, in the early two thousands and before, it creates a huge amount of work, and it actually produces less safe systems because there are less review and less anomaly and bug tracking and less, oversight happening, from the teams. So this is not just about moving faster.

It's also about moving safer. And the goal is to be able to have a service or a subsystem that you could just take out, modify, and plug back in, make sure everything works, and then generate the evidence so you can verify everything works. And this is not just an idea. This works in practice and has helped companies like Heartflo's and others drastically accelerate. So Heartflo in this case, there was a lot of manual documentation going on, a hundred thousand items working, and and, you know, all kinds of challenges in traceability as many companies have.

And with Ketryx, they are able to reduce the complexity of their IT systems from a hundred thousand to ten thousand items, serve quarter of a million patients, and it took ten weeks to implement this. This is kind of the beginning of the demo we're gonna show in a second and just huge value in this because less complexity, faster delivery, faster execution. Yeah. And, Erez, sorry. Yeah.

One thing I wanted to add here as well is I love this case study because I intimately saw this challenge as a systems engineer. Like I said, I worked across different product lines trying to manage different configurations or variants of a complex radar system. And so this diagram explains the architecture that I had to walk through manually for a change in fact assessment, and the scale of that is really challenging to do. And so Ketryx, you know, fits fits together, an architecture that supports change, which is extremely powerful. Yeah.

Couldn't agree more. And I think, like, in ninety percent of the times, this is what the first thing people wanna talk about on a call is, like, how do you actually do this? Exactly. The second, workflow and strategy we wanna talk about is compliance enforcement. In this case, it's a Fortune five hundred diagnostic company, and they're able to build compliance enforcement into their engineering workflows and cut their cycle time by eighty five percent.

It's a it's a massive increase, in velocity. And one example of this is making sure that design verification happens online and happens in a distributed way and not necessarily in in in meetings. But also the system keeps checking that people have done these things correctly, whether that's process milestones, project milestones, system milestones, or even just, like, day to day milestones between employees. So there are many different tools in this case that they were using. There was an issue with the documentation was coming in very late, causing a lot of rework.

And then there was late stage quality problems that that both came in the market, but also more controlled from the kind of quality assurance work happening late. Like, traceability, like in many companies, were done kind of late in the process instead of early in the process as an early gate. And then they used to find mistakes and spend a lot of time reworking them. And after they implemented Ketryx, they're able to reduce their documentation by seventy percent, moved from seven weeks to seven days for release cycles, and reduced their tooling costs by over fifty percent just because they consolidate things. The last strategy I wanna talk about, which is a wonderful diagnostic company is called Cytovel, is how they're able to do change impact, in minutes and not days with live traceability, with assurance, and with a view of how things work.

So in this case, they get a new change order. Ketryx helps them understand if there's new or existing requirements that need to happen, if those requirements based on a change impact assessment could impact specifications for hardware and software and then tests, and then how to make sure which of them have been touched actually and changed, and how to easily generate evidence of that with human in the loop. It's a really nice kind of culmination of a lot of different technologies. And what's cool is actually the implementation here was very, very simple because they didn't modify their whole system to make this work. The tools they were using, like Ketryx, modified around them, which is one of the powers of AI.

And for them, change impact analysis took week took weeks to map, a lot of SME coordination, a lot of overhead of busy work stealing time from other things they could do, like product development. And the result of this was seventy percent faster change impact analysis. Their initial impact analysis went from almost, over two weeks to minutes, and zero SME model bottlenecks, but everybody still got to participate in the review of the actual results and the verification that the changes were correct and in the search to make sure that nothing was missed. So I think that a lot of them would say that it was safer even because of they could do it earlier and then talk about it and not wait until they had a draft. So they're spending all the time they used to spend drafting things actually doing high quality systems engineering work.

Bailey, I know that when you first joined us, you're like, this is the future. And I remember this conversation where I asked you, what do you think we should be doing? And you said change impact analysis. And we dove into it together working through this with me, you, and Gabriel, and many other team members. So I'm excited for you to show us how this all looks coming together.

And let's move into the presentation after one last note, which is what's the actual goal of all of this together? The goal is to allow people to achieve continuous compliance with systems thinking. So if you could do all these things together, what you'll get is all the data sources you're using funneled, into specific documents around your technical work, you know, your SRS, your, testing report, your risk management report, designed for different jurisdictions, different regulatory regimes, different countries. And then you could use that to compile the actual submission that you will send to different regions. And we find companies that have medical products that are shipped to many different regions or other products, some of them medical, some of them automotive, some of them kind of in different fields that need similar evidence, but just rearranged within this kind of evidence system of system.

And so I'm not saying you need to do all these things to get here because this is one path, but you can do some of these things and solve some of this problem. But, eventually, this is what teams really want and how they they do decrease their overall work. With that, I'll take just a second, and let's move into our demonstration. So, Bailey, let's get started. I'm so excited to show you what this looks like in implementation.

So we'll start by showing you how you can design architecture that's built for change using our systems to systems architecture, how you build symbolic quality assurance into your workflows, and then we'll walk through an AI driven change impact analysis. So for those of you who are seeing Katrics for the first time, welcome. We're looking here at our systems to systems architecture where we have helped teams like we saw in the earlier slides, like HeartFlow really achieve, quicker product releases, and that's by making sure that your system is designed in a way that's built for change. We're doing that here by creating this regular rhythm notification top level system, which is where we'll generate our DHF, and breaking into two subsystems, the core services subsystem as well as the optical heart sensor subsystem. This is our hardware subsystem which is connected to Jira where our team is doing most of their work.

And in this case, the core services subsystem is connected to its own Jira project as well as Git. This allows your teams working in different components or subsystems to work in their own preferred tools. They can have their own processes. They can even be released and versioned independently, like you can see here with the different versioning of the components so that you can then pull in the different versions of components into your top level product release, which may be on two point o in this case. From this view, we're also able to see the release cadence, our release progress, so how we're trending to our product level release.

We can see which subsystems may be slowing us down more often. Like, in this case, we 've got a lot of work to do from a design control perspective in order for us to be at a hundred percent, whereas our core services subsystem seems to be moving a lot quicker. And from this, we can more accurately report on our product level releases. So jumping into the regular rhythm notification subsystems, this is our top level system. We're gonna jump straight to this graph view.

We'll look at this graph view, which will help us visualize what I was just saying, and contextualize how we think about architectures. So it's probably very common with some of the diagrams you see today that represent systems to systems. But in this case, we're looking at that core services subsystem, the different modules that flow up into it, as well as the optical heart sensor subsystem and its features and modules. Now this is a very simplified example to show you, you know, how we support this architecture, but, really, we work with teams who have many different complex architectures. So we're talking hundreds of modules that could be on this graph view here that we're helping them versions that you can ensure that your architecture is built for speed, agility, and change.

So when we think about change, there's really no better way to view it than our traceability module. Navigating to the traceability module, here's where you start to see all of the different components come in together to form your end to end traceability view. So this is that real time trace view that's pulling in information from connected systems. Remember this project, this team is working on Jira. So we're writing items in Jira.

We're seeing them contribute in real time to this traceability view, and we're able to, very easily understand the impact of that view. So here, change impact becomes very easy because we're able to pull in the systems, get a whole level view of all of our components, and how they contribute to our top level product. Here, we're able to click this bead icon to show upstream and downstream impacted odd items. You can see here we have different item types, like software items implemented in Jira that are being pulled from our core services subsystem that implement this top level IRN, a regular with a notification system, as well as down to test cases. So we work with many teams who have a myriad of different tools.

In this case, it's Jira and Git, but we have worked with teams who use JAMA, Polarion, X-ray, TestRail, all the different tools. Ketryx can integrate with those tools and provide you with this real time end to end traceability. So once you have an architecture that's built for change, you have your systems and systems set up, you then want to build some quality assurance directly into your workflows. One way to see that is with these controls at the top of the traceability view. We can click on the system requirements sixty percent, and it'll filter our trace view to show us where we have gaps in traceability.

What you can also see is where you have product validation test cases missing. So these are those deterministic checks that help you understand not only where you have gaps, but what you're missing in your quality system. So where do you not have coverage? Where do you not have approvals? Where do you have failing or missing test cases?

All of these checks are brought to you in real time so that you can have a higher confidence in the quality product that you're building instead of identifying gaps really late in your VMV cycle. So now that we see an example of how you can build these, you know, symbolic quality assurance workflows directly into traceability, into your approvals of your items, we'll take a look at an assistant workflow that shows you how you can do an AI driven change impact analysis. Clicking this assistant here, we're gonna drop in a CAD drawing. This CAD drawing is going to require us to make a change to our system. So in this case, I'm going to ask the assistant, which has knowledge of all of our connected systems, so the items in Jira, and understands this traceability graph, which is now your knowledge graph that the assistant can traverse across like nodes in a graph to understand the relationships in our very complicated system and then also make recommendations.

So it's going to look at the items that are currently traced, so that's what the traceability view is showing you, But it's going to do more than that. It's gonna look across the systems, so the components, it's gonna look across the tools, and it's also gonna tell you what's missing. So where do you have relationship that doesn't exist or probably should exist? So it can show you what requirements are already, you know, written that could be traced to a test case that exists today. Or another use case that I often work with teams on is do looking for redundant or conflicting requirements.

You know? As a systems engineer in my previous role, I spent a lot of time searching through doors to find existing requirements or where is it redundant. And without AI, was really challenging and often resulted to man many, many days doing manual looking through tables to identify that. With this assistant here, you can codify an agent to go execute that workflow for you so that you spend more time doing things like, a change impact analysis with Ketryx helping you make it a lot easier. So let's take a look at what the assistant found.

It's ran an impact assessment across the systems and subsystems across the tools and analyze the CAD drawing that I gave it. So here's an example of what that CAD drawing looks like for the regular rhythm notification wearable where we can see that we're specifying that maybe the tolerance should change or maybe the size or dimensions of this wearable. And now we're gonna take a look at what the assistant found. So what is the impact on the existing items in my system? So here you can see it's look looking at the impact on the heart sensor subsystem.

It's looking at the different item types, why it's impacted, so providing a rationale, and the impact on the top level product itself. So you can go through each of these and understand very easily what are the impacted items, the requirements, what test cases do I need to re execute. So regression testing in VNB becomes a lot easier when you have a powerful knowledge graph like this to assist you. And the critical gaps. So these are items that should exist but don't.

So it's not only gonna look at what already exists, but it's gonna make suggestions on what should change as well. Let's take, an example of the assistant giving us a suggestion, and let's take action on the suggestion it gave us. So in this case, what we're doing is we're ensuring that the human stays in the loop. So it's gonna walk us through the entire process of a change impact analysis, but it's only gonna make a change once I, as a human, have reviewed and accepted it. So ensuring that we are making sure that you have part eleven approvals.

So making sure that we also have an audit log of every single change that is made with the assistant and with the human that's taking action. And let's take a look at what this looks like together. So we can review the suggested item to that IRN requirement that lives in Jira, and with it, you can see clearly red lines and green lines of the change. So in this case, it looks like we've added some additional text here to support the changes that were being invoked by that CAD drawing. So we'll save the changes directly to this item, which will then push those changes directly to our connected system.

We just made a change to this item here, I r n dash one. That's the intended use for this regular rhythm notification device. And let's jump into Jira and see that change. So in this case, we have the intended use, the device construction, which was pushed from the assistant directly to the connected system. And then we'll want to, you know, reopen this item, review those changes together, and then we can move it through its approval workflow.

So once we've changed it to a resolved state, we can then approve as owner this item, which is then, the way to move these items through a part eleven workflow. So keeping the audit trail, making sure that you can have your biometric signature if you choose, attach each of these items, and then allow it to move through this approval group. In this case, I'm a super user, so my approval worked for all of these approval groups, but you can also separate approval owner. Going back to Ketryx now, we'll see a key detail here. This item that was previously approved now shows us this banner here, missing approval.

We can click into the item and see what's changed. Clicking into the item in Jira, we can see that we have an important notification here that the design input has been updated since the last time this item was approved. We have a reverification flag, which is a deterministic check that allows us to understand changes easily. In this case, we'll want to view the change so we can view the change together, which will direct us to those exact red lines and green lines showing us what's changed so we can decide if we need to now make a change to any downstream items. Maybe we need to rewrite a test case, for example, add a test step.

In this case, I'm gonna say that, you know what, this didn't have an impact on this downstream item. So we can dismiss it, and here it's gonna prompt you to provide a specific rationale. So not impact on requirement. So we can provide a rationale and dismiss this change here. Then we can continue with business as usual implementing this change.

So with that, we saw the way to use the assistant to execute a change impact assessment from a CAD drawing. Of course, this can also be from a plain text description or summary of what the change is. We also saw that when we invoked that change, it pushed that update keeping a human in the loop directly to a connected system, and then we saw the impact on the downstream items that maybe would be impacted by that upstream change. And finally, let's take a look at how this pulls together to get you to an end to end release of your product. So we'll go to two point o here, which is the version that we're working on for our regular rhythm notification product, And we'll see here that we have a few metrics here that help us understand release readiness.

So how are we trending to our product level release? You know, where or what impacted requirements are here, what risk controls exist, items awaiting my approval, everything that's designed to help you understand the compliance activities that are a byproduct of the work that's being done in your connected systems. And we also have milestones that are enforcing the subsystem releases at the top so that you can't release your top level product unless your subsystems have also been released. And tying it all together, we'll go to our documents. So with just a click of a button, we can pull that metadata from those connected systems.

So that change to that Jira item that we just made together is now being pulled and ingested into our documentation here. And using our templated language or templating language, we can generate an SRS document that you're generating for your teams today. Here's an example of what that would look like. This is our SRS here that has a full table of contents, where you can also pull in any of the metadata that exists in those connected systems, like the description, the relationships, and so on. Okay?

So just to recap, we started this off with understanding systems to systems, so building the architecture for change. We then took a look at how we can build those quality assurance into these workflows with the computational controls, with that reverification flag, and how we can use the assistant to help us, you know, on a workflow that's really near and dear to my heart change in fact because that was a really long and tedious process that took a lot of time in my previous roles, and I'm really excited that our systems engineers are now equipped to mood move with speed and quality in their day to day work. So with that, I'm gonna pass it back to you, Rez, to close out. Thank you, everyone. Bailey, thank you for a wonderful presentation.

That was just excellent. I know it was a big dream of yours to bring all these things together, and it was your inspiration to make this webinar. So thank you for that. And I wanted to ask you for the audience, given that you are a systems engineer, you know, now and forever, and you work in this at that kind of the highest stakes environment, knowing what you know now about how AI is being used in companies, how AI, you know, is being used at Ketryx and what we're building towards, how do you see that developing over time, this role, and where do you see it going? Yeah.

Well, I certainly hope it changes drastically from the way that I was doing it originally. It's really going to be about the people who are the innovators, who want to do things differently, who see the opportunity of AI. And like we said in the webinar, map the architecture, the tools, the processes into the way that they're doing it to make it more efficient. So, hopefully, it's extremely different than the way it is now. Yeah.

And I think patients are looking for that change. And and to me, when I go and talk about this, I feel this change of, like, the systems engineer going from being part of the process to being someone who architects the process, architects the system, and is really governing how things get done and making it all make sense with an army of agents and and kind of symbolic verification tools underneath them. I think it's one of the most exciting times to be a systems engineer, and I know that you're helping people achieve that. So we can stop for a second. We can take a look at questions.

And I just wanted to thank everybody for their time joining us today. Thank you, Bailey. Thank you to the audience. Thank you for everybody who made this possible. And, really, the best of luck, helping change how we develop the most important systems in the world.

Thank you so much. Alright. Great. Thank you everybody for joining us for the demonstration here. We do have a few questions that have come in in the chat throughout this presentation.

First is how do you handle traceability when requirements live in different tools? Say, Jira for software and Polarion for hardware. And this is an excellent question. The first is understanding that each of your different connections live in Ketryx as a whole. So for organization wide, some teams may work primarily in Jira, some primarily in Polarion, some a mix of both.

When we set up connections for your global Ketryx instance, we have the ability to set up a number of different connections overall. And then per project, we'll determine what space within each of those tools we're connecting to. For example, I might have in my core services within my system of systems, I'm developing my software in Jira separately from I might have my system requirements in Polarion. When we are in a given project, these connections are based upon what is available for what has been set up at the organization level. Our next question is, how do I find requirements that have no verification coverage?

Super common problem and also super easy for us to see here in Ketryx. We're gonna take a look at that same traceability matrix that we saw in Bailey's demo earlier. And what we're gonna use is these control filters up here at the top. So right away, in my trace matrix, I can see I have verification tests for about fifty percent of my subsystem requirements. There are two different ways for me to narrow the screen down.

Because I'm looking specifically at subsystem requirements as my root of my query, I can select the column, and this will break out so that each row is unique to a subsystem requirement. From here, we can also select the subsystem verification control filter, and this will narrow it down with both my subsystem requirements as the focus and only the rows where I am missing a subsystem verification test. At this point, you have the option to either write net new tests or use the AI assistant to suggest a potential test case for any of your subsystem requirements that are missing a test. This segues us nicely into a third question we've received throughout this demo, which is what stops the AI from changing a requirement or a test without anyone noticing? And the most important thing to understand here is in accordance with the requirements in six two three zero four, AI is not gonna make any changes in your system without a human being in the loop as those changes are made.

So when we're using this assistant, if I prompt this to suggest a change for this user interface and notifications requirement here, it's going to give me an example of what change it's going to propose, and I will, as the human in the loop, authorize that change or reject it. So this requirement currently says, we will generate a notification within thirty seconds of detecting an irregular rhythm. If I wanna prompt, propose a change to this requirement to reduce the notification window to ten seconds. The system is going to read this requirement. It's gonna reference any documentation or quality standards in my system, and it's going to have a button down here at the bottom that says review suggested edit.

Once this is done generating, if I open this review suggested edit modal, this gives me an option where just as if I were creating a net new item, the ability to review the content that's going to be included. This includes new versus original values and any changes in the text, including red lines, in this case, changing a three to a one, and any information about changes to requirement types or taxonomic information. I have the option to save these changes here. And once I select that, because this record has now updated, by changing this, my status goes from closed, which is both finished and approved, to an item in an unresolved state. Currently, the work is still completed, but we see I now have to reapprove because a change has been made to this item.

This here segues nicely into question number four. How do I know what's actually changed since our last release? In code forward organizations, change is happening every second of every day. So the first and most fundamental way to understand change in between releases is here on our release dashboard. Like Bailey mentioned earlier, this is a great overview of each of the different types of items and the different, status of those items, but we have a breakdown specifically per item type.

How many items are new in this version and how many items have changed? We also have a breakdown of how many of each of these items are in a controlled state versus uncontrolled. So this is a great one stop shop for you to jump in and say, for any requirements in my system that have changed since my last version, show me these in my all items screen. When we've narrowed our all items down to items that have changed, we'll see this diff column here has this yellow chevron icon. This indicates change as well as that release dashboard.

If I were to clear the filters here, I can see some of the items would have a checkbox. Some would have an x if they're excluded. If I select the diff icon here, this will open red lines on an item by item basis. This is comparing one record to another. So since my last record in my last release, this item has been reopened and the state is no longer approved.

If we wanna view more history on any given item that has this chevron in the diff column, we can see that using the history on the item. So if I open this details panel, I can see a complete record of all changes for this item and the ability to filter down to only controlled states of the item. So if multiple changes are happening to an item in between approvals, you can filter down to only the most recent approved version of that change. Looks like we don't have any other questions coming in in the chat. Thank you all very much for your time today.