---
title: "Empowering the SDLC Process with Tools to Bridge R&D and Quality"
type: webinar-transcript
publisher: Ketryx
source: "https://fast.wistia.net/embed/iframe/t1t7e1qc3e"
content: auto-caption transcript, proper-noun corrected
---

# Empowering the SDLC Process with Tools to Bridge R&D and Quality

*Ketryx webinar — transcript of the recorded session.*

[▶ Watch the recording](https://fast.wistia.net/embed/iframe/t1t7e1qc3e)

---

Welcome, everyone. It's really exciting to see the the size of the audience that we have today to talk about this really important topic, which is which is bridging the gap between r and d and quality. To to get started, Andrew and I are gonna go through introductions. We're gonna introduce ourselves and our respective organizations, and we'll talk a little bit about what brought this collaboration together. We're then gonna move on to talking about the the challenges and the tensions that are involved in building, medical device software, particularly the tension between the the two organizations, the the builders and the quality people, the the developers and the QA.

We're then going to discuss the the consequences that can result from those tensions, like the the really negative outcomes that can occur, and then we're gonna move on and spend the bulk of our time in this section four about best practices from the perspective of both process and tooling that can allow you to alleviate these tensions to the extent that you can, to allow the process to to be executed in the highest quality and the fastest way, and then we'll we'll take questions at the end. So with that, Andrew, would you mind giving a quick introduction to you and Rook? Yeah. Absolutely. Thanks, Jake.

Hello, everyone. I'm Andrew Wu. I'm very excited to be here. I'm the VP medical software at Rook Quality System and general manager of our APEC office. My background in training is about medical engineering and computer science.

Served as, CTO, engineering manager throughout my tenure before joining Rook full time in twenty twenty. I got fifteen years experiences in that device space, running cross functional team, bringing product from to the market from product inception all the way to commercialization. Right? Next slide, please, Jake. Yep.

Thank you. Rec Quality System, I'll briefly introduce our firm, is a quality regulatory consulting firm dedicated to helping company ranging from start ups to Fortune five hundred companies to develop and maintain quality management systems. You know, we consider it is imperative to provide and establish a quality and regulatory driven strategic road map alongside with our clients while they're advancing their technology development and business operation. What we have found a lot of time is the quality and regulatory road map can serve as a catalyst to optimize your business operation, not just on the engineering front, but also on a clinical validation front and multiple fronts. Right?

It's also important to mention that our team consists of experienced r and d and quality engineers who, in fact, have same experiences that I I do, participated in that device development process in the past, and they can be very hands on and develop technical documentation alongside with you while giving you advice on the best practices and help you focus on the product development. Right. That's all about me and the firm. Thank you, Jake. No no problem, Andrew.

So my name is Jake Stowe. I am the VP of client operations here at Ketryx. I, I went to graduate school at MIT, and that's where I, I worked at Amazon as a process engineer while I was doing that as a, working within one of the fulfillment centers with some of the robotic applications. After I graduated from MIT, I moved on to Amgen. And I was at Amgen for five years, and I worked primarily in roles in quality and in manufacturing for pharmaceutical operations.

I was actually a major deviation investigator in the largest biopharmaceutical manufacturing plant in the world, for a while. And then my last role at Amgen, I worked in a team that specialized in building validated applications from of machine learning and mechanistic models in Amgen's process development function, and that's where I kinda got my introduction into the space. And then I I left to come Ketryx, and my my main role at Ketryx is to make our clients as successful as possible within our tools. And what Ketryx is is a platform for assisting and accelerating developing validated applications. And we take, I think, a a very different tact from the rest of the market on this problem, Whereas instead of trying to build a solution that solves every problem within the space, which is extremely numerous and hard, our approach to it is we connect to systems that, that engineers already use to automatically extract the information that you need from those systems to be able to furnish evidence as a part of an audit or as a part of a submission.

And I will I'll talk a little bit in more detail as we move through the the presentation today. So let's move on to talking about the the core the core challenges intentions that we were help we're hoping to solve and we're helping to solve. And let's baseline on the fact that building software for medical devices is just hard, and we think it's hard for three reasons. The first reason is that the the process of building a medical device is just fundamentally different than building, like a website or a game with software. In general, if you're building a validated application, you're building something that can touch people's lives, that can impact their health, and you're you're solving a problem in the real world.

And that's just a harder scientific and engineering problem than, you know, building an ecommerce website, for example. The the second problem that you encounter in this space is that the tooling market as it is leaves medical device developers and QA in in a valley. Where on one side of the valley, you have all of the applications that are in wide use within, the tech industry, such as Jira and GitHub, which on their own are not fit for purpose for developing validated applications. They themselves are not validated for doing this type of work. And on the other, you have a bunch of applications that were primarily developed for hardware that primarily support a much slower release cycle than you would expect for developing a modern cloud application.

And then final reason is that if you are you're running a validated software development release life cycle, you are running a compliance process in parallel with that that is going to add exponential complexity to the process. The the interaction between both the development process and the compliance process is not like a linear increase in complexity. It's an exponential increase. And kind of the fundamental thing here is the interaction between r and d and quality and quality and regulatory. And there's various lenses through which you can see this very productive tension.

One is kind of the fundamental responsibilities of these two groups within the organization. The r and d group is responsible for building the product, and the quality and regulatory folks are responsible for ensuring that the product that is released is released according to the procedures and is has had all of the risks fundamentally mitigated. And this tension can be thought about through various lenses. So one of them is like the the responsibilities. Another is the way that these two groups tend to think about process.

So, developers are, like, fundamentally think about software development as trying to go as fast as possible and trying to iterate it iterate as quickly as possible. They wanna be able to release fast. They wanna be able to put their, their software into the field so that they can see how users respond to it and adapt to it. That's the kind of the fundamental thinking that they are taught. Whereas the quality of regulatory personas that work within the process are are very concerned about ensuring that the process is being executed rigorously against the standard operating procedures that are within the organization.

And it tends to, not always, but it tends to focus more on kind of a waterfall way of thinking about things. Because within that waterfall, you can understand these are the things that need to be completed at this phase. These are the things that need to be completed at this phase, and these are the things that need to be completed in this phase. And the the the tension or the friction between these two ways of thinking about the process shows up in a lot of organizations. I'm sure you see it in your own organizations.

And then the final lens that you can see this through is through the lens of risk. And I think everyone who works on a medical device is fundamentally interested in ensuring that the products that they release are are safe and effective. But there's this other tension as well in ensuring that the the product actually goes out. Like, no product is ever perfect. And at some point, you need to be able to ship so that the the life saving or the life changing technology that are being developed gets into the hands of the people who can use it.

And on if if you if you think about which organization emphasizes what risk, developers tend to emphasize wanting to deliver such that people can use the technology that they're building, and quality is trying to emphasize the the safety aspects of it. And what I think the most important thing to say here is that both are necessary. In the in the event that you think too hard about, trying to make the product perfect, nothing is ever perfect and it won't go out. But if you don't focus on trying to make the product perfect at all, you end up having really negative consequences downstream within the market. And so we'll talk, you know, briefly about the consequences here, and then, you know, Andrew and I will get into kind of the meat of this presentation, which is the best practices of how you can you can ease these tensions and help both both sides win in this process, which is the way to build the best products.

I think the the core failure chain that Andrew and I are focused on here is when these two groups, because of their different perspectives and needs within within this process, become misaligned fundamentally in terms of communication, in terms of priorities, in terms of procedures. That's kind of the first stop on this chain of events. What that often leads to is this process compliance ambiguity, and what that means is that certain activities may be occurring out of step with the the controls that are set within the procedures, and that can lead to an inadequate audit trip. And and what that means is that the the the the process is no longer diagnosable. If there is a if there's data that is being stored in one system that is operating in one phase of the SDLC and there's data being stored in another system that is out of phase with it, then this misalignment has grown to the the design controls themselves, and it's no longer clear which system should be governing in terms of what is currently the source of truth.

And that leads to the device not being in a validated state. That's actually the definition of the device not being in a validated state when the the the artifacts themselves are not in alignment and the design controls don't adequately describe what's being built. And the downstream impact of that is potentially device safety, efficacy, and security issues, which are the core things that we're all trying to avoid. And I think, Andrew, you have this example that you want to talk about that you see very frequently in your practice. Yeah.

Absolutely. So let's just dive into a scenario, right, and and make this, a very concrete case that we can try to diagnose together. Right? So this like Jake was saying, to ensure that the device is safe, efficacious, and secured is the eventual goal before releasing the software to the market. Right?

But how do we ensure that is the case? Right? And when we're thinking about generating the evidence for regulatory authority like the FDA to really give a go decision on this device is go good to go to market. Obviously, it depends on the auto trails. Right?

You gotta generate all the adequate documentation that clearly and accurately depict what happened, right, during the development process. K? Another question comes down to how would the quality team work with the development team to generate those documentation. Right? Extracting artifacts, software development artifacts from the development tool this a necessary step.

You know, for instance, we list out a few. Requirement definition, design aspect, design detail, architecture detail design, it will be stored in tool like Jira. Right? The code repository, all the version history would be stored in, you know, Git or, you know, party a tool like GitLab. Right?

The test cases might be test cases, the evidence would be stored in a test measure tool like X-ray. So when the quality team is out of sync with where the development is really at and to have a very extensive knowledge on how and when to extract those artifact from the development tool. This can be a very troublesome scenario that the quality team would be in. Right? So let's just take a quick example.

Right? When you're trying to gather the requirement definitions, gonna go to Jira. But what if it's structured in a way that the quality team actually cannot comprehend? Or the quality think the quality team consider that, hey. The way that these requirement are being constructed or the archetype of these requirement is not really fulfilling what the FDA needs to see.

There will be a lot of gun downstream consequences. Right? And what if the the version control records, including the code base that built from the ground up by the software dev team or a off the shelf software, that the team incorporated as part of the system. What if their version record, it's actually mixed up? How do they extract those information?

Right? And how do we know it's actually, in fact, depicting what exactly happened during the death process. Right? So I I always love to say that Jake mentioned to me multiple times during our conversation. You could be running state of the art quality process, right, but totally unable to produce the evidence of that compliance.

Right? I think this really stemmed from the knowledge gap, right, across the two teams, like Jake was saying. How and when to support the other party to fulfill their responsibilities. Right? Delivering a solution fast is what the dev team is seeking for, but the compliance mindset with adequate audit trail is what eventually need to manifest itself and get to the commercialization stage.

Right? So fulfilling those two mindset, it's not impossible, but it definitely takes quite a lot of practices and optimization to get to that stage. And this is where we wanna dive into the best practices. Yep. Is it okay if I move on from the slides, Andrew?

Thank you. Awesome. So let's move into this one. Alright. So that's leverage, the gold standard, for developing software.

There's a guidance called IEC sixty two zero four. This is, again, the gold standard that FDA and a lot of regulatory body relies on to judge the software theft process, whether it's in compliance or not based on the framework. Right? So the green boxes as annotated by, in fact, the the provisions that calls out in this particular guidance. Right?

Start with development planning, five point one to requirement definition, five point two, all the way up all the way down. Architecture design, detailed design, and lower level testing, higher level testing, eventually release. Right? And on the right hand side, this is a depiction of agile practices, described as multiple layer of abstractions. Right?

So we're talking about building a software. We're gonna build it by pieces. Right? Eventually, we'll interview all the event together and get to the shippable software state, verify it, and eventually come to the release. Right?

And after all the releases, are done by several iterations, stuff like that, and then you conclude the project to a certain state where you can secure regulatory clearance, or, you know, maintain your software as a validated state based on what the regulator would perceive. Right? So let's just dive into what do we mean by multiple layer of obstructions. Right? So sits right in the middle.

It's a story layer. Right? Story layer, by definition, it's a small piece of functionality. You know, for instance, user sign in. Right?

This depict a expected behavior that the software would react when the user is interacting with the software in certain way. Right? They put in the credentials. They press submit, and there will be a request being sent to the back end to verify that, hey. This user actually gives valid credentials.

So that would constitute a story. Increment, typically, this is what we called by the end of every sprint, you will have a set of functionality being built. You know, for instance, user management functionality. Aside from user sign up, you might have to have to build forget password. You might have to build, sign up, stuff like that, right, to constitute a a group of user story.

And by definitions, founded development team, it's called epic in, you know, in some instances. Right? When we go up a level, releases. Right? So this is where there's a shippable software being, available to the end user to do user acceptance settings.

Right? So just take, you know, this example. If this project is about building a cyanide automated lesion segmentation tool that streamlines the ischemic stroke, detection. Right? What what would be the release?

Right? The release would be the user management plus a few functionality, that, allow the physician to triage the ischemic stroke condition in certain way, but it wouldn't be as complete as the final release that pushes through the FDA review, process. Right? So release at the end of the day, release is the final gating item to ensure that this functionality is built right and robust, and it actually allow the user to do user acceptance testing. Right?

The project would always be ongoing because you have to maintain the software throughout its surface life throughout its life cycle. Right? So project will always be ongoing. The release would stop when you call out a version and ship it to the the user. Right?

So let's just dive into each of the layer, and illustrate the best practices. And I would Jake, next slide, please. Thank you. Yep. K.

So I I love the consequence slide that we built. And so I'm gonna tie it back up to how these practice best practices can really remediate and prevent the consequences from happening. Right? And start from the story layer because this is, you know, Sorry. I'll start with the project layer first because that actually serve as a a pretty clear foundation to ensure that the mutual understanding across the two teams are correctly aligned.

K? At the project level, right, we work with a lot of our client to get the r and d team and quality team aligned on the process and objectives, right, through multiple ways. In a nutshell, this is where we provide a solid methodology to the team to avoid the team align misalignment issue and train them to understand the responsibility of each department and intend to keep them accountable to voice the process compliant ambiguity. And how do we do that exactly? Right?

So the first layer that we have to look through the compliance perspective. Right? The quality team should always try to develop the work instruction and the SOP and really engage r and d team to review and share their honest opinion on how feasible the processes are. This is, you know, this is a testament of how the quality team would be convey these practices and embed those practices into the r and d process. Because keep in mind, how would the r and d team move forward with the development is actually not rely on the definition on work instruction at SOP.

It's actually rely on the criteria called definition of done. Right? So there is a set of criteria the development team would rely on to move the project forward. But what if the definition definition of done needs to be harmonized with the work construction? And this is kind of the essence of this talk is how do the team understand each other, right, throughout the process.

Wouldn't it be necessary to inject some of the necessary quality compliance practices into the definition of done? Right? So the r and d team definitely need to acknowledge that engagement from the quality team is necessary, right, sometime to fulfill the definition of done during development process. You know, for for instance, right, asking a quality team to conduct the design input review, focusing on requirement analysis so that they can suggest revision to requirements, to test cases based on their assessment of the requirement architecture, and how these requirement might influence overall VMB strategy, which the dev team is gonna have a difficult time comprehend. What do we mean by adequate from the FDA point of view?

How do we approach a VNB strategy in a very efficient way and convince the FDA that, hey. We have done this in a stepwise rigorous manner. Right? So it's all about mitigating a risk. Right?

And that comes down to another process that, you know, the dev team is very foreign with is risk management process that focus on product oriented risk. Right? And like Jay was say Jayco was saying earlier, the risk that the dev team consider is the speed of development. It's not about the product safety, efficacy, or security concerns. Right?

So this is where the quality team needs to come in to bring their understanding of the risk management standard best practices so so that it can be an integral part of the software dev activity. And you might be asking why that is, right? And I think it's very evident, right? Based on the risk valuation and the risk scenarios that's being captured, there is a very high possibility where you need to tweak your design. You need to introduce certain additional requirement definition that's built within the software to convince the FDA that, hey.

We are bringing down the risk level down to the acceptable level. Right? This can, you know, come from the initial front end design, architecture design, or even the selection of soup or off the shelf software. Right? So a lot of this, in fact, needs to be covered during the project layer.

Right? So the methodology needs to be laid out in a in a very well manner. So let's dive into the story. So before we before we move on from this, Andrew, we have a a question for Rakesh. And your question is, what are some of the differences in best practices across different risk levels of software, I e class a, class c per IEC six two three zero four?

I'm can I can I take this question at the beginning and then hand it up? Yeah. Yeah. So, Rakesh, it's a great question. I think six two three zero four is a wonderful standard for a lot of reasons, but one of the one of the reasons is is that it's very explicit about what requirements are you need to meet or what clauses you need to meet for the different risk levels.

So I think the the recommendations that we are providing today are more about how you take those clauses and you translate them into a set of procedures. So when, Andrew says something like the the r and d team needs to be involved in the definitions of the work instructions in the SOP even though the quality team is responsible for outputting that document. A part of that consideration should be, what are the clauses of six two three zero four that we need to meet? Right? I think there's a there needs to be an assessment at the beginning as a part of the risk management process and developing what classification of risk your your device is gonna fall into, that then the output of that process is how does the SDLC itself meet the clauses that are required by six two three zero four.

And so I think the to answer your question, at least from my perspective, is that the the advice that we're giving is general to all of the classes of risk, which needs to be modified based upon the the risk assessment itself for the device. Andrew, do you have anything you wanna add there? Yeah. Absolutely. So if I were to, you know, give a very precise answer on this, I would say the best practices would be generalizable.

Like, what we will talk about today, definitely a group with Jake, is generalizable across different risk level based on the software safety classification. Right? But what's more important is the quality and personnel have a clear understanding on the emphasis of the activity that he or she needs to focus on for class a versus class b, first class c. Right? Because if you have approached regulatory agencies like FDA in the past, and you will understand why six zero two zero four is structured that way based on the classification is, in fact, because depends on the use scenario, depends on the safety profile of the product.

There would be certain activity that you need to demonstrate much more concrete evidence on. Right? So leveraging the best practices to produce more information on certain area the FDA consider risky for your device. So that's the focus of why, you know, this IC three zero four standard calls out by risky, safety classification is because the emphasis that the quality team needs to put on is in fact different across different classification. Awesome.

You should we move on to the story layer now, Andrew? Yeah. Absolutely. Alright. So at the story layer is all magic happens here.

You know, both team have to pay very close attention to how the story is actually being written, implemented, tested because this layer impact almost all IEC two thousand three hundred and four processes as you can see from, you know, this green highlighted box, and other than a release layer, everything actually has been touched upon in the story layer. Right? So at this level, we intend to put together adequate documentation or, in other word, audit trails for the auditor exam. This this is where these audit trail actually would be generated. Right?

And if I can jump, you know, a bit forward, right, if you think about story layer, increment layer, release layer, right, you gotta care more about the integral part or the integration activity, right, across those layer. But the essential information that you are, in fact, producing the audit trail, the building block is in fact in a story. Right? So I just wanna emphasize that's that that concept is actually very, very important. How the story all come together, how to integrate it together, how to ensure ensure that there there is no conflict or regression, scope is being determined properly when you are putting everything together.

That's the challenge when you move on to the next from a story to increment, incremental release. K? But almost all almost all the activity that I gotta demonstrate best practices is in fact in the sort. Right? For instance, we like to ensure that the requirement are not just not not just informed by the end users ask on the functionality.

It would also be informed by what we, you know, already talked about, risk evaluation results. You know, typically, there will be additional requirement be added, right, to address the safety, security, or even interoperability concern. Right? When you deploy some software to the eventual, environment, you gotta ensure that the information is being transmitted right. Gotta ensure that all the, the communication or the connection, to other system outside of your system boundary are being done right.

Right? So that actually would be part of a validation that the FDA needs to see as well. Right? So that's how that's why you need to pay very close attention to requirement definition state when you're in fact writing the story. It's also very important to ensure that the test cases are built with good test coverages, right, at different test levels, right, from a lower level testing, unit level, integrate integration level, all the way to system level.

Right? And ensure that you have sufficient amount of different type of data to explore different task path. And these are, in fact, the evidence that the regular would like to see in the eventual submission because you can, in fact, capture the edge cases, right, through these best practices. Right? You know, user can go array off label to do something.

Right? And it might be by accident most of the time, or it might be intentional when there is an attacker. Right? So you gotta make sure that, you know, you consider all the scenario that how this software would be behaving. So, in fact, acceptance criteria defined by every verification task very important.

Right? So last but but not least, that's just gonna make sure that the dependencies with the with the rest with respect to each other, with with respect to each story is being called out properly, like we just said. When you are compiling a few stories all together, when it's being completed, when you're integrating all these building blocks together, how do you make sure that the scope of regression can be determined very precisely if you don't do this traceability on dependencies at this level? Right? It it's gonna be very difficult for you to trace it back in the next layer and say, okay.

This is how we determine regression. Right. Okay. Now I wanna pass it on to Jake and talk about how the Ketryx offer can help, translate all these back to practices through this through the functionalities that the Ketryx platform offers. Sure.

And I think the the main goal here is just to provide an example of one of the ways that these problems can be solved. Because I think the the the consequences going back to the consequence chain that we talked about earlier, and the one of the main things that happens at the beginning is that teams get misaligned. And a lot of the best practices that we have in our in our slides are about avoiding the this misalignment. And so I I just wanna quickly show you one way, of solving this and what we mean when we talk about, allowing systems the systems of work to serve as the source systems and creating audit trails directly from the the work itself. So I opened a a standard demo Ketryx, organization here, and these are just some demo projects that I have available.

And these projects map almost exactly onto the oops. Excuse me. Almost exactly onto the concept of a project in t I. It's t r forty five. Right, Andrew?

That's where this I forgot to mention. It is the Yeah. R forty five. Yeah. Thank you.

And I'm I'm gonna start by talking about how we help instantiate this level of abstraction into everything that you do, as a part of building a six two three zero four compliant product. So if I flip back here and I click into this monolithic project demo, you're gonna see the first screen that I come to is called the all items screen. And what items they refer to are actually a concept directly out of six two three zero four. These are configuration items. And the like, I think the the practical way to talk about them is that they consist of all of the different design controls that you would manage as a part of the six two three zero four compliant process.

And if we take, for example, requirements and we look at that, we see that we have several requirements that are included as a part of this version two of the product that we're building here. And if I just scroll down to one of these and I click into one of them, you're going to see pretty much everything that you would expect from a controlled record. You're going to see the unique identifying information. You're going to see the description of the requirement itself, the context, the rationale. You're also going to see any approvals that have been applied here, relations to other records, and release documents that contain this record.

The key thing that you're not going to see, because it's not present, is the ability to edit this requirement within the system because the work is not taking place in Ketryx. Ketryx is just providing a layer of transparency that quality and regulatory professionals need for performing their jobs. The work is taking place in Jira. And I won't I won't go into a ton of detail about about this, but I think for for the audience here, the main thing that I wanna emphasize is that we do three things to Jira, to make it fit for purpose for doing this type of work, and we do this to all of the source systems that we connect to. The first thing is that every every item has the concept of its local traceability.

So as you're building out a story, it's very easy for you to see how this requirement inter interacts with the test cases that it connects to, the specifications it connects to, and where it sits within the requirement tree. So it's easy to define the boundaries around the the branches and the traceability matrix that you are trying to affect with a story. The second thing that we do is we provide you with the ability to approve at the granular level of the item itself. So I think many organizations that we interact with, they perform approvals at the level of the entire set of design controls on the documents themselves. But that makes it hard for the process to become modular because all of the design controls have to change all at one time.

By allowing the approval that take place at the level of the item itself, it allows different aspects of the system to be worked on independently. And so if I just I'll quickly demo this approval mechanism really quickly, and then we'll move on to what I think is the most important aspect of what we do with these objects. So I've reopened this ticket. I'm just gonna come in here and remove the rationale from this. And I'm gonna hit save.

And now I want to approve this ticket. So I'm gonna take this, and I'm going to place it into a resolved state. And this is gonna do two things. One thing it's gonna do is it's gonna make this ticket immutable, which is exactly what you wanna do if you're going to go through an approval process with one of these objects. You don't want people to be able to come in and change this.

And the second thing that we're going to do is we are going to I need to approve a training document. So I'll talk about that as well. In general, with the system, this button would turn blue and I would be able to approve it, but, unfortunately, in this demo, there is actually I need to go through a training process to go through this approval. So this document is empty. This is just for demo purposes.

But if I come through here and I acknowledge this document and I apply my signature saying that I have read it, that will now ungate my ability. Here. Let me just oh, I can actually yeah. View. Show here.

And I'll just go back here. I now can refresh this screen. And because I've performed that training, that approvals button will now allow me to sign. And I'm a superuser in this, in this instance, so my signature will count for all of these, but, I'm sure you can see that you can define different groups for doing the approval at this very granular level. And so now I've I've approved the record.

And, again, I think getting to the the real core of what we're talking about, which is how to maintain alignment between between the groups. If I click on this requirement and I come back to the screen that we started at, if I scroll down, you'll see that there's a complete version history of all of the changes that were made to that ticket, which is exactly the data that you need as a QA RA professional for performing your duties, and it just automatically happens. You can see the changes that I I put in the system. It's already here. So if you see that that rationale field has been removed.

And if I wanna see the difference between this ticket and the previous approved version, I just have to hit the diff here. So, yeah, here we go. I'm gonna take this question. So Ruchi has a question, which is how is this different from Jira tracking and Bitbucket? I think, Ruchi, your question came through before I I I went through all of that.

As far as I'm aware, I I look at a lot of software in this space, and I am not aware of any system that does the syncing in the way that we do, which is basically real time with Jira. I know that there are some tools out there that do some of the connections, but this very live, up to the minute tracking and, also as well being able to have the Alcoa plus plus audit trail on the changes, is not something that I've seen in any other tools. If you have further questions, please post them in the chat. If I've ended up answering them, as I went through the demo, I hope I hope that helped. So I think the the core thing to come back to is being able to create a single source of truth out of the systems that are the systems that are being used by the developers, and it gives total and complete transparency into what's happening within the and real time transparency into what's happening within the development process.

And the final thing that I'll touch on is just that this can be done for multiple systems. So if I go back into this project, you'll see that there are Jira items here, but this also can take place in Git based repos as well. So I'm just gonna pass this back to you, Andrew, and we can move forward. Great. Yeah.

I think I think there's an important point I would like to make. At the story level, this is where you can try to mitigate the consequence, where we call out inadequate audit trail. Right? A lot of time, you gotta generate those, even at the story layer. And now we move on to the increment layer.

Right? So this is where you have to start getting into integration workflow of compiling multiple completed tickets, task, stories altogether because this is where potentially a software, a software built will be established. And potentially, this is where a release can happen. Right? It won't happen every time.

But, once, you know, a a few epics has been, you know, completed, then a release could happen. Right? So at this stage, you have to demonstrate good traceability practices. I think this Jake has been mentioning quite a lot throughout the the demo. Right?

Across requirement definition, require, across verification items, and even the risk item. Right? Remember that risk item traceability is, in fact, a essential piece that the FDA would judge how adequate the requirement and verification items can substantiate your claims. Right? So you can certainly, you know, consider, you know, looking at how robust the change management practices is to kind of judge how good of a traceability practice that you are complying to.

Right? You know, for instance, if they're based on the feedback at the end of increment, at the end of a sprint, and, the user, the end user suggested during the sprint demo that, hey. There's a new functionality I would like to add in because that helps me, you know, with certain scenarios. Right? And you go back to the backlog, and you drag it up.

But now you have to decide on, okay, what are the impact, right, of the existing, completed stories that this new backlog item might potentially have? Do I need to reopen those story, or those item and triage the priority of that? Should I tackle this in the next sprint, or should I tackle that, in another sprint? And what are the technical debt that we're thinking about right now? You know what I mean?

What are the potential design changes that we're thinking from the technical point of view. Right? So change management process is, in fact, a very crucial compliant practice to ensure that the traceability practices are done right. Right? So at at each increment, we have to re revisit the topic that we brought up earlier about risk management.

Right? Architectural decision dependent on multiple things. Right? Off the shelf software selection, SVOM inventory maintenance, interoperability requirement, cybersecurity requirements, or even function boundaries. Right?

Some of the device function that you might be building that's not considered as device function while some of them are. So how do you draw the boundary? Right? And at each increment, you gotta revisit those factors, right, because that would potentially in introduce a new change management, change, request or path that the dev team need to take on and the quality team needs to be fully aware. To consolidate all the test evidence configuration record at the end of each, increment.

It's very, very important. So that's where you can leverage a CICD pipeline, and be sure that there's a sprint review being carried out at the end of sprint to ensure that everything that we built is behaving as expected in a nutshell. In other word, the device this increment describing the device functionality is, in fact, kept in the validated state. So quite a lot of quality related work item would be addressed here at the integral level. Right?

So I'll pass it back to Jake, to talk about a few examples. Thanks, Andrew. So returning to, like, the topic of how to maintain like, this alignment and kind of the entire thing that we're driving towards, and specifically with relation to the activities that Andrew just mentioned, specifically traceability and risks. I think one of the core things that we we work towards with everything trying to make all of the information real time is having a live traceability matrix, something that allows you to see the state of the tracing for all of the objects in your project at any time. And so if I'm I'm looking at this project, I can see here that I have these objects in relation to each other, and I have visual indicators that indicate to me whether the traceability links were broken and what else needs to be done.

I'm actually going to go in here and break one of the links so that we can see what this looks like, when things do not look perfect. So let me reopen this. I am going to remove the fulfill requirements here, and then I will just hit refresh here really quickly. And then I will come back to the demo, And we now see, hold on, that we now have design inputs that are no longer covered by design outputs. So if someone had gone into this specification item and removed the tracing, Ketryx automatically detects that the project no longer has the required tracing to be in a validated state.

It also is going to tell you that you do not have all of the items controlled. And what's what's great about these checks is that they help you to automatically identify those places. So if I click on that, I can see very quickly where the gaps in my traceability exist. And this is being updated in real time all the time. So there's never a period of time when you don't really know the state of validation for the the product that's in development.

And I also am able to look to see which items are not controlled. And, of course, we have all of these nice visual indicators here that will flag for you, which objects themselves need your attention. So I will. And then, of course, let's let's also talk a bit about risk management. Here again, risks are full class citizens within Ketryx.

They they are they participate in the linking system, the in the traceability system, And you can see that we have, an interface here that complies precisely with one four nine seven one. So you'll you can specify hazards, sequences, events, hazardous situations, the harms that are coming out of that risk. You can do an initial risk evaluation. You can apply these risk control measures, and you can look at and then you can describe them, and then you can do a residual risk evaluation. And if you need to add an additional benefit risk analysis, that's also possible here.

We give each organization the capability of configuring specific risk models for different hazard types if you want, and we've even built this little widget that allows you to visually see the assessment based on the one four nine seven one framework. But I think the most important thing to emphasize here and and bringing it back to, again, staying in alignment, knowing what's important is the risk control measures and how that shows up in the system. So if I go back to the all items screen, for example, and I wanna look at all of the items that are risk controls at a part of this project, that's as easy as clicking a button. I can see all of the requirements and the specification items that are serving as risk controls. I can even see which tests are testing those.

And I think the most important thing is if I wanna see the risk controls that are new or changed as a part of moving from version one to version two. That's also as easy as a click of a button. And then finally, I'll just touch on, you know, the testing activities as Andrew was touching on as well at the increment level. I am also able to see those tests which are testing risk controls within the system so that you can ensure that those items are included as a part of your test plan when you go to perform an incremental release. Wonderful.

So I'm gonna pass this back to you, Andrew. And do you want me to advance to the release level? Yes, please, Jake. K. Right.

Lastly, let's talk about release. Right? So the release layer absolutely can be considered the last checkpoint to ensure all the device safety, efficacy, security endpoints are being addressed properly. Right? So a good analogy of release versus increment, I I can think of if we were to talk about from the hardware point of view and more concrete ideas.

Right? So increment can be considered as subassemblies. Right? So you build all these components, but, obviously, you have to put it into the inventory and track it. Right?

You gotta make sure that all the self assembly are good for use where you're finally putting the finished good altogether before you release a lot of finished good, which will be the release itself. Right? So release records definitely needs to be kept in place, you know, following the release procedure very, very properly, precisely. Right? Formal design review, go no go decision, needs to be made across the team, how to orchestrate the release through these best practices that we've mentioned just now.

In fact, all needs to be manifest in the final design review process. Right? So to be a little bit more concrete, right, configuration item must be thoroughly documented, and the record must be kept up to date, especially when there will be updates given the post appointment testing results. Right? There might be some configuration item that would ring into some issue and cause, you know, certain remediation that needs to happen from the engineering point of view, and the configuration records have to be updated, right, even post release.

Right? And worst case scenario worst case scenario is you have to revert back to your prior version. Right? So that's how this configuration record and version control practices are super, super important even at the release stage. Right?

So utilizing CICD pipeline on automated test, like we just, talked about, ensures that all at the integral level, and and determining a regression scope is properly, done and tracked. Right? So at the end of the day, the quality team and even regulatory team, right, they gotta make sure that auto release record at each release really matches the expectation from the regulatory submission. You know, if you look at a five ten k, everybody calls out a release version. Right?

Because that's the baseline that you're asking the FDA for review to give you that clearance. Right? So I'm gonna pass it back to Jake for that last portion of the case study. Yeah. You know, we have we have two questions that I'm gonna address really quickly about Ketryx, Andrew, before we move on.

So, Al, you asked, can you choose whether you only have p one and p two or use three levels of probability, p one, p two, and p three? The current functionality in Ketryx is based on one four nine seven one, and so there's only those two levels of p one and p two. But we already have, on our product roadmap expanding to make it much more flexible so that you can use multiple levels of probabilities. So that will be a capability that's in the product soon. It's It's just not what's currently in the validated state of Ketryx.

And then Rakesh asked is, you know, the for the risk management platform for Ketryx, for things like cybersecurity and human factors, the FDA has recently been asking for risk management processes beyond one four nine seven one, and can the hazard evaluation be configured to other standards or processes? And the answer to that is yes. So we allow you to define different risk models based upon, the to the hazard type itself, and we often do that configuration for our customers. So if you have a different, if you have a different risk analysis model that you wanna use for cybersecurity, that can just be done at the grain of the hazard type itself. And then so I'm gonna mark those, and then just moving on.

So moving up to the level of the release, we also have global controls and global transparency here. So if I click on the releases in this project, and I can see that there are two two versions of this specific project here. And you can see that version one has already been approved and version two has been the version that we've been working in. When I click into this version, the first thing that's going to pop up is a dashboard that gives you a very high level summary of where your progress is towards bringing your product into a validated state. This is gonna show you the how many design controls have actually been, put into under control and reviewed, and whether the test executions that have been performed are controlled and whether they've been passed.

You also can click on each of these to filter. So if you wanted to see all the young the uncontrolled items, you can just click here, and you're gonna see the software specification that I opened up, recently. And on the right hand side, you're going to see this release checklist. This is the highest level of control within a project itself. And what this does is it defines a sequence of steps that need to take place in order for you to be able to release this.

And Ketryx can actually take this even farther. In the event that you want to integrate the product directly into your CICD pipeline, we can actually fail or not I've not failed builds, but failed deployment to a production server in the event that all of these items have not been met. That's kind of the highest level of control that our our customers sometimes require. And it's it's very easy to configure this list to add items to it and to place different controls into it if you have certain things that you want to achieve. One of the ways to do that is with our milestone system.

I won't get into that here because I wanna be like, I wanna make sure that we're staying on time, but that is an aspect of the product of making this very granular. And the final thing that we do is we allow you to generate release documents directly from the system of work. So if, for example, if I come down to the system requirement specification here and I hit this generate button, the system is now extracting all of the most current data from the requirements and placing them into a document. I'm just gonna hit download, Open this. And let me I'm gonna have to just open that, and I will pull this onto the screen.

So this is a document that was just generated, and you can see, like, all the requirements broken down by their types and that the data has been pulled from the source systems and placed directly into this document. And, of course, you know, not all of our customers use this as an EQMS and for document control, but, some of them do, and you can actually do the approvals for the documents directly within Ketryx. So if I wanted to approve that system requirement specification, I could click here. This is perfectly part eleven compliance signature system here, and I can confirm that. I can approve the document, and that can also gate performing the final release here.

You can see the I guess the the documents have been removed from this, but there's usually a checkbox here in the default that says all of the documents been have been approved. So I am going to really quickly let me move this back into this, and here we go. Just got a little a little mood there. So I will move forward into the final slide. Alright.

And so I think the the summary that we wanna give to you all here is that being able to bridge the gap between quality and r and d is a product of both selecting the right tools and having the right culture. From the tooling perspective, I you wanna ensure that you're building a single source of truth, that documents are a side effect of the work being done and not an independent process, and you wanna be able to have engineering controls embedded in the tools themselves. You wanna ensure that things can be done within the compliance sequences of steps. Right. In a nutshell, from the cultural point of view, right, first priority, neutral understanding.

Right? What are the each others ask, and what are the priority of those? And how do we make sure that each of the team, in fact, understand where the other team is at and their status? Right? Without that layer of communication, you you wouldn't be able to depict the exact thing that happened during dev by producing documentation.

Right? And once again, we I have been chatting with Jake quite a lot about this. Right? Documentation should not be the main way of communication. Right?

Documentation should be a byproduct of these practices that the team agreed upon and try to commit to other team's goal. And by following that practice, you will be able to build the documentation very robustly and eventually let the FDA reviewer judge whether this is adequate to commercialize as a final netting device product on the market. So I know I know we're at time. Akin, I see that you have a question, and we will follow-up via email after this. But I wanna thank everyone for coming to the webinar.

Really appreciate your attention. I wanna reiterate that both Rook US and Ketryx are here to help. So if you have, a desire to, you know, see more about our tool or seek Andrew's services, we're here. We we would love to talk to you. And as you sign off, you're going to get that survey.

Please tell us how we did. We are very interested in your feedback. Thank you, everyone. Really appreciate the time. Thank you.