Description:RELATED ART

[0001] Embodiments of the present specification relate generally to a functional safety validation system, and more particularly to a system that tests and validates if one or more application containers to be deployed in a software defined vehicle satisfy functional safety requirements.
[0002] Software defined vehicles have transformed the automotive industry that traditionally relies on hardware to a software-centric electronic device on wheels. The software defined vehicle offers a variety of benefits over a traditional vehicle. For example, the software defined vehicle allows original equipment manufacturers (OEMs) to offer new features, improve vehicle performance, and enhance vehicle safety through software updates without vehicle owners really needing to visit service centers to avail these benefits.
[0003] Generally, the software defined vehicle includes application containers that execute associated operations including safety critical functionalities in the vehicle. Examples of such application containers include an advanced emergency braking container, an adaptive cruise control container, and a lane departure warning container. These application containers residing in the software defined vehicle provide comfort and convenience to the driver and passengers of the vehicle and also ensure the safe navigation of the vehicle, safety of the driver and passengers, and safety of the people and objects in the surroundings of the vehicle. Hence, the functional safety aspects of these application containers need to be thoroughly tested and validated before deploying the application containers in real-world vehicles.
[0004] Functional safety validation of the application containers typically involves simulating unexpected faults that could potentially occur in real-world scenarios, and injecting such faults intentionally into the application containers to assess how the application containers respond to those faults. By simulating and injecting the faults, a test engineer evaluates robustness and ability of the application containers to detect, diagnose, and recover from the unexpected faults that may potentially occur in real-world scenarios.
[0005] Conventional test systems performing such functional safety validation of the application containers in a laboratory environment typically use complex and costly external hardware devices for simulating and injecting the faults into the application containers. Further, such conventional test systems are capable of performing the functional safety validation only when actual hardware in which the application containers are to be deployed in real-world scenarios is available, thereby relegating safety testing to later development stages.
[0006] Further, in the software defined vehicle, a failure of one application container should not lead to a cascade of failure of other application containers, resulting in serious vehicle safety concerns. International organization for standardization (ISO) 26262 defines freedom from interference as the absence of cascading failures between two or more elements that could lead to the violation of a safety requirement. Conventional systems perform a freedom from interference test in a laboratory environment, and identify if the failure of one application container leads to a cascade of failures by injecting data and simulating stress environments, performing performance testing, and monitoring end to end data, which are all time-consuming processes and require expensive third-party software tools.
[0007] Further, the software defined vehicle conventionally includes a plurality of different automotive safety integrity level (ASIL) rated application containers co-existing and deployed in the same high-performance computer. These different ASIL rated application containers need to be subjected to a mixed criticality test, which entails validating application containers at an individual level as well as validating if the application containers satisfy overall functional safety requirements when they function together as a unit. Typically, the conventional systems perform the mixed criticality test using external software tools and hardware devices, which are expensive. Moreover, the associated testing methodology is tedious and requires a significant amount of time.
[0008] Further, the conventional systems transmit data packets, related to the application containers successfully subjected to the functional safety validation, from a cloud platform to real-world vehicles via an over-the-air (OTA) communications link. The OTA communications link uses specific communication protocols that need to be validated to identify if the transmitted data packets include any defects or errors.
[0009] Accordingly, there remains a need for an improved functional safety validation system that tests and validates if the application containers satisfy functional safety requirements without requiring any additional external hardware devices and simulation tools, and further effectively tests and validates transmission of secure data packets to real-world vehicles.

BRIEF DESCRIPTION

[0010] It is an objective of the present disclosure to provide a method for performing functional safety testing. The method includes generating a bytecode that mimics a test condition associated with an application container to be tested using a bytecode generating system. The application container is stored in a test server. Further, the method includes verifying if an injection of the generated bytecode into a kernel memory space of an operating system hosting the application container introduces one or more errors in behavior of the operating system. Furthermore, the method includes injecting the generated bytecode into the kernel memory space of the operating system only upon verifying that the injection of the generated bytecode does not introduce the one or more errors in the behavior of the operating system. Moreover, the method includes comparing an output generated by the application container post injection of the generated bytecode into the kernel memory space of the operating system with an expected output. In addition, the method includes successfully validating compliance of the application container with one or more functional safety standards when the output generated by the application container matches with the expected output. The method further includes automatically transmitting the application container that is successfully validated from the test server to a target system for deploying the application container in the target system.
[0011] Successfully validating compliance of one or more application containers includes generating the bytecode representative of null data and one or more flag values used to mimic a failure of a first application container selected from the one or more application containers. Further, the method includes verifying if the injection of the bytecode representative of the null data and the flag values into the kernel memory space of the operating system hosting a second application container that is different from the first application container introduces the one or more errors in the behavior of the operating system using a bytecode verifying system. Furthermore, the method includes injecting the bytecode representative of the null data and the flag values into the kernel memory space of the operating system only upon verifying that the injection of the bytecode into the kernel memory space of the operating system does not introduce the one or more errors in the behavior of the operating system. Moreover, the method includes determining if the second application container correctly identifies the failure of the first application container and generates an alert message indicating the failure of the first application container post injection of the bytecode representative of the null data and the flag values into the kernel memory space of the operating system. In addition, the method includes identifying that the failure of the first application container does not cause the second application container to fail when the second application container has correctly identified the failure of the first application container and generated the alert message.
[0012] Successfully validating compliance of one or more application containers includes identifying a first memory address space in the kernel memory space of the operating system where a first application container runs on the operating system. Further, the method includes identifying a second memory address space in the kernel memory space of the operating system where a second application container runs on the operating system. Furthermore, the method includes generating the bytecode that mimics a sensor output that is related to the first application container and is unrelated to the second application container, and injecting the bytecode that mimics the sensor output into the second memory address space. Moreover, the method includes determining if the second application container generates an error message and restarts post injection of the bytecode into the second memory address space. In addition, the method includes identifying that the second application container complies with the one or more functional safety standards when the second application container generates the error message and restarts post injection of the bytecode that mimics the sensor output that is unrelated to the second application container into the second memory address space.
[0013] Successfully validating compliance of one or more application containers includes generating the bytecode that mimics a failure of a hardware device that is interlinked to a subset of application containers selected from the one or more application containers. Further, the method includes injecting the bytecode into the kernel memory space of an operating system hosted by the hardware device for mimicking the failure of the hardware device. Furthermore, the method includes identifying a corresponding timestamp associated with the failure of the hardware device and associated failure of each of the subset of application containers interlinked to the hardware device. Moreover, the method includes verifying if each of the subset of application containers correctly identifies that a reason for the associated failure is the failure of the hardware device based on the corresponding timestamp. In addition, the method includes verifying if each of the subset of application containers generates an associated error message and shuts down an associated operation post injection of the bytecode that mimics the failure of the hardware device into the kernel memory space of the operating system. The method further includes identifying that the subset of application containers comply with the one or more functional safety standards when each of the subset of application containers generates the associated error message and shuts down the associated operation post injection of the bytecode that mimics the failure of the hardware device into the kernel memory space of the operating system.
[0014] Successfully validating compliance of one or more application containers includes generating a first bytecode, a second bytecode, and a third bytecode that mimic presence of a first application container, a second application container, and a third application container selected from the one or more application containers within a virtual target system. Further, the method includes injecting the first bytecode, the second bytecode, and the third bytecode into the kernel memory space of a first operating system, a second operating system, and a third operating system for mimicking presence of the first application container, the second application container, and the third application container within the virtual target system. Furthermore, the method includes generating and injecting a fourth bytecode that mimics a sensor input into the kernel memory space of the second operating system. Injection of the fourth bytecode causes the second application container to interfere with one or more functionalities of the third application container. Moreover, the method includes determining if the first application container interferes with the one or more functionalities of the third application container while the second application container is interfering with the one or more functionalities of the third application container. In addition, the method includes successfully validating operation of the first application container, the second application container, and the third application container as a unit when the first application container does not interfere with the one or more functionalities of the third application container while the second application container is interfering with the one or more functionalities of the third application container.
[0015] Automatically transmitting one or more application containers that are successfully validated from the test server to the target system includes generating and injecting a first bytecode representative of one or more network data packets and one or more checksum codes into the kernel memory space of an operating system hosting a vehicle agent to mimic availability of the target system for communication with the vehicle agent. Further, the method includes generating and injecting a second bytecode representative of one or more modified network data packets and one or more corresponding checksum codes into the kernel memory space of the operating system that mimics transmission of a defective data packet from the target system to the vehicle agent. The defective data packet corresponds to one of a compromised data packet, a data packet affected by a malware, a data packet including an error message, and a data packet including one or more messages in an undesired format. Furthermore, the method includes verifying if the vehicle agent correctly identifies one or more defects in the defective data packet represented by the second bytecode injected into the kernel memory space of the operating system and generates an alert message including the one or more identified defects post injection of the second bytecode into the kernel memory space of the operating system. Moreover, the method includes successfully validating a first level functionality of the vehicle agent when the vehicle agent generates the alert message including information corresponding to the one or more identified defects post injection of the second bytecode into the kernel memory space of the operating system.
[0016] Automatically transmitting the one or more application containers that are successfully validated from the test server to the target system includes generating a third bytecode that mimics a defective application container. The defective application container corresponds to one of a compromised application container and an unsecured application container affected by a malware. Further, the method includes injecting the third bytecode that mimics the defective application container into the kernel memory space of the operating system for mimicking transmission of the defective application container from an over-the-air server to the vehicle agent. Furthermore, the method includes determining if the vehicle agent correctly identifies one or more defects in the defective application container post injection of the third bytecode into the kernel memory space of the operating system. Moreover, the method includes successfully validating a second level functionality of the vehicle agent when the vehicle agent correctly identifies the one or more defects in the defective application container post injection of the third bytecode into the kernel memory space of the operating system. Automatically transmitting the one or more application containers that are successfully validated from the test server to the target system includes generating a fourth bytecode that mimics an unsecured data packet in a first file format. The first file format is different from a standard second file format in which a secure data packet is transmitted from a first application programming interface (API) port system of the target system to a second API port system of the test server. Further, the method includes injecting the fourth bytecode into the first API port system of the target system, and transmitting the fourth bytecode that mimics the unsecured data packet from the first API port system to the second API port system through the vehicle agent. Furthermore, the method includes verifying if the second API port system correctly identifies that the unsecured data packet received from the first API port system is in the first file format that is different from the standard second file format. Moreover, the method includes successfully validating one or more functionalities of the second API port system of the test server when the second API port system correctly identifies that the unsecured data packet received from the first API port system is in the first file format.
[0017] Automatically transmitting the one or more application containers that are successfully validated from the test server to the target system includes injecting the third bytecode that mimics the defective application container into the kernel memory space of an operating system associated with a sandbox residing in the target system for mimicking downloading of the defective application container from the test server by the sandbox. Further, the method includes determining if the sandbox correctly identifies the defective application container represented by the third bytecode injected into the kernel memory space of the operating system of the sandbox and prevents a deployment of the defective application container in the target system. Furthermore, the method includes identifying that the sandbox executes one or more associated functionalities correctly when the sandbox correctly identifies the defective application container and prevents the deployment of the defective application container in the target system. Successfully validating compliance of one or more application containers includes generating and injecting a first bytecode and a second bytecode into the kernel memory space of a first operating system and a second operating system selected from the one or more operating systems for mimicking presence of a first application container and a second application container within a virtual target system. Further, the method includes generating and injecting a third bytecode that mimics a sensor input into the kernel memory space of the first operating system, and verifying if the first application container provides a microservice to the second application container post injection of the third bytecode into the kernel memory space of the first operating system. The microservice includes transmission of the sensor input from the first application container to the second application container.
[0018] Furthermore, the method includes identifying if the second application container receives the sensor input transmitted from the first application container within a particular time period. Moreover, the method includes successfully verifying the microservice provided by the first application container to the second application container when the second application container receives the sensor input from the first application container within the particular time period. Successfully validating compliance of one or more application containers includes generating and injecting a bytecode into the kernel memory space of each of a first operating system and a second operating system for identifying utilization of one or more of a memory resource and a computing resource of a virtual target system by each of a first application container and a second application container. The first operating system hosts the first application container having a first safety critical value and the second operating system hosts the second application container having a second safety critical value that is lesser than the first safety critical value. Further, the method includes identifying if utilization of one or more of the memory resource and the computing resource of the virtual target system exceeds a designated memory resource threshold and a designated computing resource threshold, respectively using the virtual target system.
[0019] Furthermore, the method includes verifying if the virtual target system performs one or more of shutting down or limiting an operation of the second application container having the second safety critical value when one or more of the memory resource and the computing resource used by the virtual target system exceeds the designated memory resource threshold and the designated computing resource threshold, respectively. Moreover, the method includes successfully verifying one or more functionalities of the target system when the virtual target system shuts down or limits the operation of the second application container. Successfully validating compliance of one or more application containers includes generating and injecting a first bytecode that mimics a sensor input into the kernel memory space of a first operating system hosting a first application container. The first application container generates a first application output upon injecting the first bytecode that mimics the sensor input into the kernel memory space of the first operating system. Further, the method includes transmitting the first application output generated by the first application container to a second application container via a communications link. Furthermore, the method includes generating and injecting a second bytecode into the kernel memory space of a second operating system hosting a second application container to identify if the second application container has generated a second application output within a designated time period based on the first application output. Moreover, the method includes verifying if a virtual target system shuts down an operation of the second application container and increases an associated computing resource previously allocated to the first application container when the second application container fails to generate the second application output within the designated time period. In addition, the method includes successfully validating one or more functionalities of the target system when the virtual target system shuts down the operation of the second application container and increases the associated computing resource previously allocated to the first application container.
[0020] It is another objective of the present disclosure to provide a system for performing functional safety testing. The system includes a test server, a bytecode generating system, a bytecode verifying system, a bytecode injecting system, and a test validation subsystem. The test server includes a virtual target system. The virtual target system corresponds to a virtual model of a target system and includes one or more operating systems hosting one or more application containers to be tested. The bytecode generating system that resides in the virtual target system and is configured to generate one or more bytecodes that mimic one or more test conditions associated with the one or more application containers to be tested. The bytecode verifying system is communicatively coupled to the bytecode generating system and is configured to verify if an injection of the one or more generated bytecodes into one or more kernel memory spaces of the one or more operating systems introduces one or more errors in behavior of the one or more operating systems.
[0021] The bytecode injecting system that is communicatively coupled to the bytecode verifying system and is configured to inject the one or more generated bytecodes into the one or more kernel memory spaces of the one or more operating systems only when the bytecode verifying system verifies that the injection of the one or more generated bytecodes does not introduce the one or more errors in the behavior of the one or more operating systems. The test validation subsystem that is communicatively coupled to the bytecode injecting system and the target system. The test validation subsystem is configured to compare one or more outputs generated by the one or more application containers post injection of the one or more generated bytecodes into the one or more kernel memory spaces of the one or more operating systems with one or more expected outputs. Further, the test validation subsystem is configured to successfully validate compliance of the one or more application containers with one or more functional safety standards when the one or more outputs generated by the one or more application containers match with the one or more expected outputs. The test server automatically transmits the one or more application containers that are successfully validated to the target system for deploying the one or more application containers in the target system. The target system includes one of a physical main electronic control unit, a software defined vehicle, a rail system, an avionic system, a drone, a robotic vehicle used in one or more of an industrial application, a retail application, a healthcare application, a security application, and a surveillance application, an aquatic vehicle, an internet of things device, a gaming system, a consumer electronic device, a hardware-in-the-loop test bench, a software-in-the-loop test bench, a model-in-the-loop test bench, and one or more servers hosting virtual electronic control units.

BRIEF DESCRIPTION OF DRAWINGS

[0022] These and other features, aspects, and advantages of the claimed subject matter will become better understood when the following detailed description is read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
[0023] FIG. 1 illustrates a block diagram depicting an exemplary functional safety validation (FSV) system used to execute a functional safety testing of application containers to be deployed in a target system, in accordance with aspects of the present disclosure;
[0024] FIG. 2 illustrates a block diagram depicting various components associated with the FSV system of FIG. 1 used to execute the functional safety testing of the application containers, in accordance with aspects of the present disclosure;
[0025] FIG. 3 illustrates a flow diagram depicting an exemplary method for injecting a fault into an application container and identifying an output generated by the application container in response to the injected fault using the FSV system of FIG. 1, in accordance with aspects of the present disclosure;
[0026] FIG. 4 illustrates a flow diagram depicting an exemplary method for performing a freedom from interference test using the FSV system of FIG. 1, in accordance with aspects of the present disclosure;
[0027] FIG. 5 illustrates a flow diagram depicting an exemplary method for executing a mixed criticality test using the FSV system of FIG. 1, in accordance with aspects of the present disclosure; and
[0028] FIGS. 6A-B illustrate a flow diagram depicting an exemplary method for testing functionalities of a vehicle agent that ensures a secure communication between the target system and a cloud sever using the FSV system of FIG. 1, in accordance with aspects of the present disclosure.

DETAILED DESCRIPTION

[0029] The following description presents an exemplary functional safety validation system. Particularly, embodiments described herein disclose the functional safety validation system that validates if application containers satisfy functional safety requirements defined by, for example ISO 26262, which is an international functional safety standard for electric and electronic systems in road vehicles.
[0030] As noted previously, software defined vehicles generally include different ASIL rated application containers, which perform various safety critical functionalities for ensuring the safety of the vehicle, passengers, and the people and objects in surroundings of the vehicles. Therefore, functional safety aspects of such application containers have to be thoroughly validated before deploying the application containers in real-world software defined vehicles.
[0031] Typically, conventional functional safety validation (FSV) systems perform functional safety validation of the application containers using external hardware devices and simulations tools, which are generally expensive. Further, conventional methods used for the functional safety validation are tedious and are time-consuming as they require simulation of various safety critical scenarios and execution of those scenarios individually with respect to the application containers.
[0032] For example, conventional methods of performing freedom from interference and mixed criticality test involve testing the application containers against risk producing elements, and executing one or more of functionality testing, performance testing, communication testing, hardware-in-loop testing, stress testing, and/or resource allocation analysis, which are all time-consuming processes and require different expensive software tools and hardware devices. Examples of such software tools include ControlDesk, AutomationDesk, MATLAB, and Vector CANoe. Examples of such hardware devices include SCALEXIO, CANcaseXL, and a high-end graphics enabled personal computer.
[0033] In order to address the aforementioned issues, embodiments of the present disclosure provide a FSV system that does not require any additional external hardware devices and simulations tools for validating functional safety aspects of the application containers. Specifically, the FSV system performs the functional safety validation by generating a bytecode that mimics an unexpected fault that may potentially occur in real-world scenarios. The FSV system then injects the generated bytecode into a kernel memory space of an operating system to evaluate how an application container responds to that particular fault. In particular, the FSV system performs the functional safety validation by injecting the bytecode into the kernel memory space of the operating system, which simulates real-world faults without need for any other external hardware devices and simulation tools, thus providing a cost-effective solution.
[0034] It may be noted that different embodiments of the present FSV system may be used in different application areas. For example, the FSV system may be used to test and validate functional safety aspects of application containers to be deployed in software defined vehicles, rail systems, microcontrollers, avionic systems, electronic systems, Internet of Things devices, gaming systems, and consumer electronics such as smartphones and smart televisions. Further, the FSV system may be used to test and validate functional safety aspects of application containers to be deployed in an aquatic vehicle, a drone, and a robotic vehicle used in one or more of an industrial application, a retail application, a healthcare application, a security application, and a surveillance application. However, for clarity, an embodiment of the present FSV system is described herein in greater detail with reference to testing and validation of functional safety aspects of automotive applications.
[0035] FIG. 1 illustrates a block diagram depicting an exemplary FSV system (100) used to execute a functional safety testing of application containers (102A-N) to be deployed in a target system (104). An example of the target system (104) includes a physical main electronic control unit (ECU) (104) that typically resides in a software defined vehicle (106). Specifically, the FSV system (100) is used to execute the functional safety testing and validate if the application containers (102A-N) to be deployed in the physical main ECU (104) of the software defined vehicle (106) satisfy functional safety requirements defined by an international standard. An example of the international standard includes ISO 26262.
[0036] With increasing complexities in the automotive industry due to software and mechatronic based solution implementations, there are increasing risks of random failures of vehicle components such as systematic failures, hardware failures, and software failures. Such random failures occurring in vehicles in real-world scenarios pose a huge risk to human lives and objects in the vehicle surroundings. ISO 26262 defines a series of standards including guidelines to mitigate such risks associated with the random failures of the vehicle components. In addition, ISO 26262 requires the functional safety testing of the application containers (102A-N), which entails simulating or recreating real-world random failures or faults in a laboratory environment and further verifying if the application containers (102A-N) are capable of handling such random failures or faults without compromising the vehicle safety. The application containers (102A-N), successfully passing the functional safety testing, are generally capable of handling random failures that may occur in real-world scenarios efficiently, and thereby mitigate associated risks.
[0037] In one embodiment, the FSV system (100) described in the present disclosure provides a framework to perform the functional safety testing of the application containers (102A-N) in a laboratory environment in an efficient manner without needing any additional external hardware devices and simulation tools. Specifically, the FSV system (100) performs the functional safety testing by simply accessing one or more kernels of one or more operating systems, as described in detail with reference to subsequent paragraphs.
[0038] For executing the functional safety testing of the application containers (102A-N), the FSV system (100) includes a virtual main electronic control unit (ECU) (108) deployed in a test server (107). In one embodiment, the test server (107) corresponds to a cloud server or a local server. In certain embodiments, the virtual main ECU (108) corresponds to a virtual model of the physical main ECU (104) or a high-performance computer that usually resides in the software defined vehicle (106). An architecture of the virtual main ECU (108) closely mimics an architecture of the physical main ECU (104). Further, the virtual main ECU (108) includes a set of components that are present in the physical main ECU (104) except for the fact that those components are virtualized or simulated components. Specifically, the virtual main ECU (108) includes a virtual system-on-chip (110), and a hypervisor (112) including a plurality of operating systems (114A-N) that run on top of the hypervisor (112). Further, the virtual main ECU (108) includes a virtual safety island (116) that hosts a real-time operating system (RTOS) (118) of the virtual main ECU (108).
[0039] In certain embodiments, the application containers (102A-N) residing within the virtual main ECU (108) are generated, for example, using a Docker platform, an OpenWrt platform, a Kubernetes platform, or a Podman platform. These application containers (102A-N) run on top of the RTOS (118) and/or the operating systems (114A-N). These application containers (102A-N) correspond to different types of ASIL rated applications rated as one or more of ASIL A, ASIL, B, ASIL C, and ASIL D. In one embodiment, all of these application containers (102A-N) may run on a single operating system. For example, all the application containers (102A-N) may run on the RTOS (118). Alternatively, each of the application containers (102A-N) may run on a dedicated operating system. For example, the application container (102A) may correspond to an advanced emergency braking container (102A) and may run on a first operating system (114A). The application container (102B) may correspond to a lane keep assist (LKA) application container (102B) and may run on a second operating system (114B) that is different from the first operating system (114A). The application container (102N) may correspond to a camera application container (102N) and may run on the RTOS (118).
[0040] When executing the functional safety testing of a single application container (102A), for example, an application container (102A), the FSV system (100) accesses a shared memory space within a kernel memory space of an operating system hosting the application container (102A). Further, the FSV system (100) injects one or more bytecodes into the shared memory space for executing the functional safety testing of the application container (102A). It may be noted that injecting the bytecodes into the shared memory space may modify only the functionalities of the application container (102A) and does not affect the functionalities of other application containers (102B-N) that may be running on the same operating system and the same kernel memory space.
[0041] When executing the functional safety testing of a plurality of the application containers (102A-N) simultaneously, the FSV system (100) directly injects bytecodes into kernel memory spaces of operating systems instead of injecting the bytecodes into the shared memory space within the kernel memory spaces. For instance, the FSV system (100) requires to simultaneously test the application containers (102A, 102B, and 102N) running on the first, second, and real-time operating systems (114A, 114B, and 118), respectively. In the previously noted example, the FSV system (100) directly injects a first bytecode related to testing of the application container (102A) into a kernel memory space of the first operating system (114A) instead of injecting the first bytecode into the shared memory space within the kernel memory space. Similarly, the FSV system (100) directly injects second and third bytecodes related to testing of the application containers (102B and 102N) into corresponding kernel memory spaces of the second and real-time operating systems (114B and 118), respectively, instead of injecting the second and third bytecodes into the shared memory spaces within the corresponding kernel memory spaces.
[0042] In certain embodiments, each of the operating systems (114A-N) and the RTOS (118) includes a corresponding user memory space (122) and a corresponding kernel memory space (124). In one embodiment, the user memory space (122) includes a bytecode generating system (202) and a log file analyzing system (214), as depicted in FIG. 2. FIG. 2 illustrates a block diagram depicting one or more exemplary components of the FSV system (100). In certain embodiments, the bytecode generating system (202) is communicatively coupled to a test automation system (126) (shown in FIG. 1) via a communications link (128). Examples of the communications link (128) include a controller area network, a local interconnect network, a FlexRay, a Wi-Fi, network, an Ethernet, a cellular data network, and an over-the-air communications link. In certain embodiments, the communications link (128) may operate using one or more of a transmission control protocol (TCP), a scalable service-oriented middleware over IP (SOME/IP), TCP/IP, one or more application programming interface (API) calls, and any other suitable protocols.
[0043] In one embodiment, the test automation system (126) automatically generates one or more test cases for executing the functional safety testing of the application containers (102A-N). An example of the test automation system (126) includes a Robot Framework. In certain embodiments, the test cases generated by the test automation system (126) include details of test conditions that the bytecode generating system (202) need to mimic or simulate in order to test the functional safety aspects of the application containers (102A-N). The test automation system (126) also generates and stores outputs expected from the application containers (102A-N) when these test cases are executed and successfully validated by the test automation system (126).
[0044] In certain embodiments, the bytecode generating system (202) generates bytecodes that mimic the test conditions based on the details of the test conditions included in the test cases. In one embodiment, the test conditions correspond to one or more of faults and operational conditions associated with the application containers (102A-N) need to be tested prior to deployment of the application containers (102A-N) in the software defined vehicle (106). Specifically, the test conditions correspond to faults that the application containers (102A-N) may encounter in real-world scenarios. The test conditions may also correspond to real-world test scenarios in which functionalities of the application containers (102A-N) need to be tested. The test conditions may also include mimicking physical presence of the application containers (102A-N) within the virtual main ECU (108) or the physical main ECU (104), sensor outputs, failure of one or more application containers (102A-N), failure of one or more hardware devices, and/or a physical presence of the software defined vehicle (106). The test conditions may also include mimicking generation of defective application containers, and/or transmission of defective or unsecured data packets between the test server (107) and the physical main ECU (104). The test conditions may also include identification of a memory resource and a computing resource of the virtual main ECU (108) used by the one or more application containers (102A-N) during functional safety testing of these application containers (102A-N).
[0045] In one embodiment, the bytecode generating system (202) then provides the generated bytecodes as inputs to a bytecode verifying system (204) in the kernel memory space (124). Subsequently, the bytecode verifying system (204) identifies if the injection of the generated bytecodes into the kernel memory space (124) of the operating systems (114A-N and 118) causes errors or causes the FSV system (100) to crash, as described in detail subsequently with reference to FIG. 3. When the bytecode verifying system (204) identifies that the injection of the generated bytecode into the kernel memory space (124) may introduce errors or crash the FSV system (100), the bytecode verifying system (204) generates a bytecode error message that is displayed to a user such as a test engineer via a user interface (130) associated with a display unit (132) of the FSV system (100).
[0046] Alternatively, when the bytecode verifying system (204) identifies that the injection of the generated bytecode into the kernel memory space (124) may not introduce any errors or crash the FSV system (100), a bytecode injecting system (206) injects the bytecodes into the kernel memory space (124) of the operating systems (114A-N and 118). Subsequently, a test validation subsystem (208) in the kernel memory space (124) captures outputs generated by the application containers (102A-N) upon injecting the bytecodes into the kernel memory space (124) of the operating systems (114A-N and 118).
[0047] Further, the test validation subsystem (208) compares and identifies if the outputs generated by the application containers (102A-N) match with corresponding expected outputs previously generated and stored by the test automation system (126). In one embodiment, the test validation subsystem (208) identifies that the application containers (102A-N) satisfy functional safety requirements defined in the international standard when the outputs of the application containers (102A-N) match with the expected outputs. Otherwise, the test validation subsystem (208) identifies that the application containers (102A-N) do not satisfy the functional safety requirements when the outputs of the application containers (102A-N) fail to match with the expected outputs. Further, the test validation subsystem (208) generates and displays one or more failure messages via the user interface (130) that indicate to the user that the application containers (102A-N) fail to satisfy the functional safety requirements to enable the user to make suitable changes and fix issues associated with the application containers (102A-N).
[0048] In certain embodiments, in addition to the test validation subsystem (208), the kernel memory space (124) also includes a log file generating system (210) that stores test case results indicating successful and/or unsuccessful execution of the test cases in an associated log database (212). Further, the log file generating system (210) provides the test case results stored in the associated log database (212) as inputs to a log file analyzing system (214) for enabling the user to access the test case results.
[0049] In certain embodiments, the application containers (102A-N) that successfully pass the functional safety testing are transmitted from an over-the-air (OTA) server (134) to an OTA client (136) in the physical main ECU (104) via a vehicle agent (138). The OTA client (136) then deploys the tested application containers (102A-N) in the physical main ECU (104) of the software defined vehicle (106). In certain embodiments, the vehicle agent (138) corresponds to a gateway pod that acts as an interface between the test server (107) and the physical main ECU (104). The vehicle agent (138) verifies if any messages exchanged between the test server (107) and the physical main ECU (104) are secure and are not compromised. The FSV system (100) may be used to test if the vehicle agent (138) correctly performs associated functionalities of identifying unsecured messages exchanged between the test server (107) and the physical main ECU (104).
[0050] For example, the FSV system (100) may be used to generate one or more bytecode that mimics defective data packets, and subsequently inject the bytecode in an operating system (140) of the vehicle agent (138), which mimics transmission of unsecured or compromised data packets from the test server (107) to the physical main ECU (104) via the vehicle agent (138). Subsequently, the FSV system (100) identifies if the vehicle agent (138) correctly recognizes the data packets as unsecured or compromised, and further identifies if the vehicle agent (138) generates an alert message, as described in detail subsequently with reference to FIGS. 6A-B. When the vehicle agent (138) generates the alert message correctly, the FSV system (100) successfully tests the functionalities of the vehicle agent (138). Otherwise, the FSV system (100) identifies the vehicle agent (138) to be not working properly and displays the alert message in the user interface (130) to alert the user about improper functioning of the vehicle agent (138).
[0051] In certain embodiments, the FSV system (100) may also be used to execute the freedom from interference test and identify if the failure of one application container (102A) leads to a cascade of failures of other application containers (102B-N), as described in detail subsequently with reference to FIG. 4. The FSV system (100) may also be used to execute the mixed criticality test, which entails validating the functionalities of the application containers (102A-N) at an individual level as well as validating if the application containers (102A-N) satisfy overall functional safety requirements when the application containers (102A-N) function together as a unit, as described in detail subsequently with reference to FIG. 5.
[0052] In one embodiment, the FSV system (100) and associated systems including the software defined vehicle (106), virtual main ECU (108), virtual system-on-chip (110), hypervisor (112), virtual safety island (116), test automation system (126), OTA client (136), vehicle agent (138), bytecode generating system (202), bytecode verifying system (204), bytecode injecting system (206), test validation subsystem (208), log file generating system (210), and log file analyzing system (214), for example, may include one or more of general-purpose processors and specialized processors to execute the functional safety testing of the application containers (102A-N). In certain embodiments, the FSV system (100), software defined vehicle (106), virtual main ECU (108), virtual system-on-chip (110), hypervisor (112), virtual safety island (116), test automation system (126), OTA client (136), vehicle agent (138), bytecode generating system (202), bytecode verifying system (204), bytecode injecting system (206), test validation subsystem (208), log file generating system (210), and log file analyzing system (214) may include one or more of graphical processing units, microprocessors, programming logic arrays, field programming gate arrays, integrated circuits, system on chips, and/or other suitable computing devices. Additionally, certain operations of the FSV system (100), software defined vehicle (106), virtual main ECU (108), virtual system-on-chip (110), hypervisor (112), virtual safety island (116), test automation system (126), OTA client (136), vehicle agent (138), bytecode generating system (202), bytecode verifying system (204), bytecode injecting system (206), test validation subsystem (208), log file generating system (210), and log file analyzing system (214) may be implemented by suitable code on a processor-based system, such as a general-purpose or a special-purpose computer.
[0053] A flowchart (300) depicting an exemplary method by which the FSV system (100) injects a fault that the application container (102A) may encounter in a real-world scenario and identifies an ability of the application container (102A) to handle such a fault is described subsequently with reference to FIG. 3. It is to be understood that the FSV system (100) is similarly capable of injecting faults and verifying the fault handling abilities of other application containers (102B-N). The order in which the exemplary method is described is not intended to be construed as a limitation, and any number of the described blocks may be combined in any order to implement the exemplary method disclosed herein, or an equivalent alternative method. Additionally, certain blocks may be deleted from the exemplary method or augmented by additional blocks with added functionality without departing from the claimed scope of the subject matter described herein.
[0054] At step (302), the bytecode generating system (202) receives a test case from the test automation system (126) for executing the functional safety testing of the application container (102A). For example, the application container (102A) corresponds to an advanced emergency braking (AEB) container (102A). In one embodiment, the received test case is indicative of a fault that the bytecode generating system (202) needs to generate in order to verify the ability of the AEB container (102A) to handle the fault when it occurs in a real-world scenario. For instance, the received test case may be indicative of a faulty image with missing pixel data in order to verify the ability of the AEB container (102A) to handle a scenario of sudden failure of a camera in a real-world driving scenario. In certain embodiments, the test case also includes a response that is expected from the AEB container (102A) upon subjecting the AEB container (102A) to the fault.
[0055] At step (304), the bytecode generating system (202) generates a bytecode that mimics the fault based on an instruction and the test case received from the test automation system (126). For instance, with reference to the previously noted example, the bytecode generating system (202) generates a bytecode in a binary format that mimics the faulty image with missing pixel data based on an instruction and the test case received from the test automation system (126). An example of the instruction includes “clang -O2 -target bpf -c ebpf_program.c -o ebpf_program.o”. In the previously noted exemplary instruction, “ebpf_program.c” corresponds to software codes that generate the faulty image with missing pixel data, and “ebpf_program.o” corresponds to a portion of the instruction that generates the bytecode corresponding to the software codes.
[0056] At step (306), the bytecode verifying system (204) verifies if an injection of the bytecode into the kernel memory space (124) of the operating system (114A) hosting the AEB container (102A) introduces an error in a behavior of the operating system (114A). For example, the bytecode verifying system (204) identifies if the injection of the bytecode causes the virtual main ECU (108) to utilize an associated computing resource greater than a first threshold and/or an associated memory resource greater than a second threshold using register value tracking, direct packet access, and/or pruning techniques. When the injection of the bytecode causes the virtual main ECU (108) to utilize the associated computing and/or memory resources greater than the respective first and second thresholds, the bytecode verifying system (204) identifies that the injection of the bytecode is likely to introduce an error in the behavior of the operating system (114A). Subsequently, the bytecode verifying system (204) generates a bytecode error message, and displays the generated message to the user via the user interface (130).
[0057] Alternatively, when the injection of the bytecode causes the virtual main ECU (108) to utilize both the associated computing and memory resources lesser than the respective first and second thresholds, the bytecode verifying system (204) identifies that the injection of the bytecode does not introduce any errors in the behavior of the operating system (114A). The bytecode verifying system (204) then generates a bytecode verified message. Further, the bytecode verifying system (204) transmits the bytecode verified message to the bytecode injecting system (206) via the communications link (128) for enabling the bytecode injecting system (206) to inject the bytecode into the kernel memory space (124) of the operating system (114A).
[0058] At step (308), the bytecode injecting system (206) injects the bytecode mimicking the fault into the kernel memory space (124) of the operating system (114A) when the bytecode verifying system (204) successfully verifies that the injection of the bytecode does not introduce any errors in the behavior of the operating system (114A). In one embodiment, the bytecode injecting system (206) injects the bytecode into the kernel memory space (124) via an ethernet port (142) in the physical main ECU (104). Specifically, the bytecode injecting system (206) injects the bytecode into the ethernet port (142) of the physical main ECU (104). Subsequently, the test validation subsystem (208) verifies if the kernel memory space (124) of the operating system (114A) receives the bytecode injected into the ethernet port (142) of the physical main ECU (104). When the kernel memory space (124) of the operating system (114A) correctly receives the bytecode, the test validation subsystem (208) identifies that the ethernet port (142) is working properly. Otherwise, the test validation subsystem (208) identifies that the ethernet port (142) is working improperly when the kernel memory space (124) fails to receive the bytecode injected into the ethernet port (142). Alternatively, the bytecode injecting system (206) may directly inject the bytecode into the kernel memory space (124) of the operating system (114A) instead of injecting the bytecode via the ethernet port (142) to mimic the fault that may potentially occur during a real-world operation of the software defined vehicle (106).
[0059] At step (310), the test validation subsystem (208) captures an output generated by the application container (102A) upon injecting the bytecode into the kernel memory space (124) of the operating system (114A). For instance, with reference to the previously noted example, the application container (102A) may correctly identify that the faulty image represented by the injected bytecode lacks certain pixels upon injecting the bytecode into the kernel memory space (124) of the operating system (114A). Further, the application container (102A) may generate an output including an error message that indicates lack of certain pixels in the faulty image represented by the injected bytecode. At step (312), the test validation subsystem (208) compares and identifies if the output generated by the application container (102A) matches with the expected output that is previously generated and stored in the test automation system (126).
[0060] At step (314), the test validation subsystem (208) successfully executes the functional safety testing of the application container (102A) when the output generated by the application container (102A) matches with the expected output. For instance, with reference to the previously noted example, the test validation subsystem (208) identifies that the application container (102A) satisfies the functional safety requirements defined in the international standard such as in ISO 26262 when the output including the error message generated by the application container (102A) matches with the expected output. Otherwise, the test validation subsystem (208) generates and displays a failure message to the user via the user interface (130) when the output generated by the application container (102A) fails to match with the expected output. The failure message, thus displayed via the user interface (130), indicates to the user that the application container (102A) fails to satisfy the functional safety requirements defined in the international standard.
[0061] Conventional FSV systems use external hardware devices and expensive simulations tools for injecting faults into an application container and for performing the functional safety testing of the application container. In contrast to the conventional FSV systems, the present FSV system (100) performs the functional safety testing by simply generating a bytecode that mimics an unexpected fault that may potentially occur in real-world scenarios, and by injecting the generated bytecode into the kernel memory space (124) of the operating system (114A). Thus, the present FSV system (100) effectively performs the functional safety testing without need for any external hardware devices and expensive simulation tools, which makes the FSV system (100) an optimal and economical in-lab system for executing the functional safety testing of the application containers (102A-N).
[0062] In certain embodiments, the FSV system (100) also allows testing of the application containers (102A-N) against virtual scenarios without need for any external simulation systems that simulate those virtual scenarios. For example, the FSV system (100) may be used to test and verify functionalities of the AEB container (102A) against virtual scenarios. To that end, the bytecode generating system (202) generates a bytecode representative of data values indicative of a road friction, and vehicle dynamics of an ego vehicle and an obstacle. In addition, the generated bytecode mimics a real-world scenario of the obstacle entering into a path of the ego vehicle such that a distance between the obstacle and the ego vehicle is greater than a designated distance threshold, for example, of 12 meters. In the previously noted example, the bytecode verifying system (204) verifies if the injection of the bytecode representative of data values into the kernel memory space (124) causes an error in the behavior of the operating system (114A). When the bytecode verifying system (204) successfully verifies that the injection of the bytecode does not cause any error in the behavior of the operating system (114A), the bytecode injecting system (206) injects the bytecode into the kernel memory space (124) of the operating system (114A). The injection of the bytecode into the kernel memory space (124) mimics the real-world scenario of the obstacle entering into the path of the ego vehicle when the distance between the obstacle and the ego vehicle is greater than the designated distance threshold without using any external simulation systems.
[0063] Upon injecting the bytecode, the test validation subsystem (208) determines if the AEB container (102A) correctly identifies presence of the obstacle in front of the ego vehicle and generates an obstacle warning message. When the test validation subsystem (208) determines that the AEB container (102A) has correctly identified the presence of the obstacle and generated the obstacle warning message, the test validation subsystem (208) identifies the AEB container (102A) to be functioning properly.
[0064] In another example, the bytecode generating system (202) generates another bytecode representative of data values that mimic a real-world scenario of a sudden entry of the obstacle into the path of the ego vehicle such that a time-to-collision between the obstacle and the ego vehicle is lesser than a designated time threshold. In this example, the bytecode injecting system (206) injects the bytecode into the kernel memory space (124) of the operating system (114A). Subsequently, the test validation subsystem (208) verifies if the AEB container (102A) generates an emergency brake activation message upon injecting the bytecode into the kernel memory space (124). When the AEB container (102A) generates the emergency brake activation message as expected, the test validation subsystem (208) identifies the AEB container (102A) to be functioning properly.
[0065] In certain embodiments, the FSV system (100) may also be used to perform the freedom from interference test, as noted previously. In real-world conditions, a failure of an application container (102A) deployed in the physical main ECU (104) should not lead to a cascade of failures of other application containers (102B-N) interlinked to the application container (102A). Otherwise, the safety of the software defined vehicle (106) will be highly compromised due to the cascade of failures of the other application containers (102B-N). In one embodiment, the FSV system (100) may be used to verify if a failure of one application container (102B) leads to a failure of another application container (102A) in a laboratory environment before deploying such application containers (102A-B) in a real-world vehicle, as described subsequently in detail with reference to FIG. 4.
[0066] FIG. 4 illustrates a flow diagram depicting an exemplary method (400) for performing the freedom from interference test without using any external hardware devices and expensive simulations tools. At step (402), the bytecode generating system (202) receives a test case from the test automation system (126) for executing the freedom from interference test. For example, the bytecode generating system (202) receives a test case for testing if a failure of the camera application container (102N) leads to a failure of the AEB container (102A). The test case received by the bytecode generating system (202) may indicate to the bytecode generating system (202) that the bytecode generating system (202) needs to mimic a crash of camera application container (102N) to verify if the crash of the camera application container (102N) causes the AEB container (102A) to fail.
[0067] At step (404), the bytecode generating system (202) generates a bytecode that mimics a failure of the camera application container (102N) based on the test case received from the test automation system (126). For example, with reference to the previously noted example, the bytecode generating system (202) generates a bytecode representative of one or more null data values and one or more flag values, which mimic the crash or the failure of a camera. At step (406), the bytecode verifying system (204) verifies if an injection of the bytecode representative of the null data values and flag values into the kernel memory space (124) of the operating system (114A) introduces an error in the behavior of the operating system (114A), as described previously with reference to FIG. 3. At step (408), the bytecode injecting system (206) injects the bytecode representative of the null data values and flag values into the kernel memory space (124) of the operating system (114A) when the bytecode verifying system (204) successfully verifies that the injection of the bytecode does not introduce any error in the behavior of the operating system (114A).
[0068] The bytecode representative of the null data values and flag values, thus injected into the kernel memory space (124), causes the AEB container (102A) to identify that the camera application container (102A) has failed and is transmitting data packets that have no image data. At step (410), the test validation subsystem (208) determines if the AEB container (102A) has correctly identified the failure of the camera application container (102N) upon injecting the bytecode into the kernel memory space (124). Further, at step (412), the test validation subsystem (208) determines if the AEB container (102A) generates and displays an alert message that indicates the failure of the camera application container (102N) via the user interface (130). At step (414), the test validation subsystem (208) identifies the AEB container (102A) to be working properly and that the failure of the camera application container (102N) has not affected functionalities of the AEB container (102A) when the AEB container (102A) generates and displays the alert message via the user interface (130) as expected. Otherwise, the test validation subsystem (208) identifies that the failure of the camera application container (102N) has hindered the functionalities of the AEB container (102A) when the AEB container (102A) fails to generate the alert message as expected.
[0069] Another example of the freedom from interference test includes testing if the failure or a malfunctioning of the camera application container (102N) hinders the functionalities of the AEB container (102A). When the camera application container (102N) malfunctions and transmits images at a frame rate that is greater than a specific frame rate, a memory associated with the AEB container (102A) may crash. In such a scenario, the AEB container (102A) is expected to generate a warning message to indicate to the camera application container (102N) that the camera application container (102N) is transmitting the images at a frame rate greater than the specific frame rate.
[0070] In one embodiment, the FSV system (100) may be used to verify if malfunctioning of the camera application container (102N) causes the AEB container (102A) to fail using the FSV system (100). To that end, the bytecode generating system (202) generates a plurality of bytecodes that mimic images captured by a camera. Subsequently, the bytecode injecting system (206) injects each of the plurality of bytecodes sequentially into the kernel memory space (124) of the operating system (114) at designated intervals such that the AEB container (102A) identifies that the AEB container (102) receives the images at a rate that is greater than the specific frame rate. The test validation subsystem (208) then determines if the AEB container (102A) generates a warning message upon injecting the bytecodes sequentially at the designated intervals into the kernel memory space (124) of the operating system (114A). When the AEB container (102A) generates the warning message as expected, the test validation subsystem (208) identifies that the malfunctioning of the camera application container (102N) does not cause the AEB container (102A) to fail. Otherwise, the test validation subsystem (208) identifies that the malfunctioning of the camera application container (102N) causes the AEB container (102A) to fail when the AEB container (102A) fails to generate the warning message as expected.
[0071] In certain embodiments, the physical main ECU (104) in the software defined vehicle (106) includes a single operating system, for example, the operating system (114A) that runs more than one application container such as the AEB container (102A) and the LKA container (102B). In such scenarios, possibilities exist for a memory of the AEB container (102A) to interfere with a memory of the LKA container (102B). For example, a radio detection and ranging (RADAR) sensor in the software defined vehicle (106) generally captures distance data associated with objects in the surroundings of the software defined vehicle (106), and provides associated outputs as inputs to the AEB container (102A) for enabling the AEB container (102A) to take appropriate action. However, when the RADAR sensor wrongly provides the associated outputs as inputs to the LKA container (102B) instead of providing the outputs to the AEB container (102A), the LKA container (102B) may get corrupted. In such a scenario, a fault handler of the LKA container (102B) is expected to generate an error message and restart the LKA container (102B).
[0072] In one embodiment, the FSV system (100) may be used to verify if the interferences between the memories of the AEB container (102A) and the LKA container (102B) are effectively handled by the fault handler of the LKA container (102B) in a laboratory environment. To that end, the bytecode generating system (202) identifies a first memory address space where the AEB container (102A) runs on the operating system (114A). Further, the bytecode generating system (202) identifies a second memory address space where the LKA container (102B) runs on the operating system (114A). In one embodiment, the bytecode generating system (202) identifies the first and second memory address spaces by accessing a memory usage file such as a “/proc/[PID]/maps” file associated with the virtual main ECU (108).
[0073] Subsequently, the bytecode generating system (202) generates a bytecode that mimics an output that is unrelated to the LKA container (102B). For example, the bytecode generating system (202) generates a bytecode that mimics an output of the RADAR sensor including distances to various objects in the surroundings of a vehicle, which is relevant to the AEB container (102A) but is unrelated to the LKA container (102B). Subsequently, the bytecode injecting system (206) injects the bytecode mimicking the output of the RADAR sensor into the second memory address space where the LKA container (102B) resides. Upon injecting the bytecode, the test validation subsystem (208) determines if the LKA container (102B) generates an error message. Further, the test validation subsystem (208) determines if the fault handler of the LKA container (102B) restarts the LKA container (102B) as expected upon injecting the bytecode. When the LKA container (102B) generates the error message and is restarted by the fault handler as expected, the test validation subsystem (208) identifies the LKA container (102B) to be working properly. Otherwise, the test validation subsystem (208) identifies the LKA container (102B) to be working improperly when the LKA container (102B) fails to generate the error message or is not restarted by the fault handler as expected.
[0074] In certain embodiments, the software defined vehicle (106) includes a plurality of application containers (102A-N) that are interlinked to each other and exchange information between them. When all such interlinked application containers (102A-N) fail together simultaneously or sequentially, there is likely a single root cause that causes all the interlinked application containers (102A-N) to fail. When such a root cause causes all the interlinked application containers (102A-N) to fail, the application containers (102A-N) are expected to execute a predefined defense strategy. For example, the applications containers (102A-N) are expected to generate one or more error messages and further to shut down their associated operations when a common root cause causes all the interlinked application containers (102A-N) to fail for ensuring the safety of the software defined vehicle (106).
[0075] In one embodiment, the FSV system (100) may be used to verify if the application containers (102A-N) correctly execute the predefined defense strategy as expected or not when a common root cause that causes a plurality of application containers (102A-N) to fail. To that end, the FSV system (100) instructs the bytecode generating system (202) to generate a bytecode and transmit the bytecode to the bytecode injecting system (206). The bytecode injecting system (206) then injects the bytecode into a kernel of an operating system that runs on a hardware device such as a graphical processing unit (144) in the virtual main ECU (108). In certain embodiments, the injection of the bytecode into the kernel of the operating system mimics a failure of the graphical processing unit (144). The test validation subsystem (208) then retrieves a particular timestamp at which the graphical processing unit (144) has failed from the log database (212). It may be noted that the failure of the graphical processing unit (144) also causes the application containers (102A-N), for example, corresponding to the AEB, LKA, and camera application containers (102A-N) to fail. The test validation subsystem (208) then retrieves a corresponding timestamp at which each of the AEB container (102A), the LKA container (102B), and the camera application container (102N) has failed from the log database (212).
[0076] From the retrieved timestamps, the test validation subsystem (208) identifies that the graphical processing unit (144) failed first followed immediately by failure of AEB, LKA, and camera application containers (102A-N). Accordingly, in this example, the test validation subsystem (208) identifies that the failure of the graphical processing unit (144) is the root cause for the failures of the AEB, LKA, and camera application containers (102A-N). Further, the test validation subsystem (208) verifies if the AEB, LKA, and camera application containers (102A-N) generate one or more error messages and further shut down their associated operations as expected post injecting the bytecode mimicking the failure of the graphical processing unit (144) into the kernel of the operating system. When the AEB, LKA, and camera application containers (102A-N) generate the one or more error messages and further shut down their associated operations as expected, the test validation subsystem (208) identifies that these application containers (102A-N) execute their defense strategies properly. Otherwise, the test validation subsystem (208) identifies that the AEB, LKA, and camera application containers (102A-N) do not execute their defense strategies properly when these application containers (102A-N) fail to generate the one or more error messages and shut down their associated operations as expected.
[0077] In certain embodiments, the physical main ECU (104) of the software defined vehicle (106) includes various ASIL rated application containers (102A-N), all of which coexist in the same physical main ECU (104). When these application containers (102A-N) are presented together as a unit, each of the application containers (102A-N) should execute their associated functionalities without any fault. The FSV system (100) of the present disclosure can be used to perform the mixed criticality test that entails testing whether each of the application containers (102A-N) execute their associated functionalities properly when the application containers (102A-N) operate together as a unit, as described in detail subsequently with reference to FIG. 5.
[0078] FIG. 5 illustrates a flow diagram depicting an exemplary method (500) for executing the mixed criticality test using the FSV system (100). At step (502), the bytecode generating system (202) generates a plurality of bytecodes that are used to mimic presence of the application containers (102A-N) within the virtual main ECU (108). For example, the bytecode generating system (202) generates first, second, and third bytecodes that are used to mimic presence of first, second, and third application containers (102A, 102C, and 102D) within the virtual main ECU (108). An example of the first application container (102A) includes the AEB container (102A). An example of the second application container (102C) includes an adaptive cruise control (ACC) container (102C). An example of the third application container (102D) includes a vehicle dynamics container (102D) used to control an accelerator, steering wheel, and/or a brake of a vehicle.
[0079] In one embodiment, each of the first, second, and third bytecodes generated by the bytecode generating system (202) is representative of associated data packets including flag values used to mimic presence of the AEB, ACC, and vehicle dynamics containers (102A, 102C, and 102D) within the virtual main ECU (108). At step (504), the bytecode injecting system (206) injects the generated first, second, and third bytecodes into the kernel memory space (124) of the operating systems (114A, 114C, and 114D) for mimicking the presence of the AEB, ACC, and vehicle dynamics containers (102A, 102C, and 102D) within the virtual main ECU (108). At step (506), the bytecode generating system (202) receives a test case from the test automation system (126) for performing the mixed criticality test of the AEB, ACC, and vehicle dynamics containers (102A, 102C, and 102D).
[0080] In one embodiment, the test case received from the test automation system (126) indicates to the bytecode generating system (202) that the bytecode generating system (202) needs to generate a fourth bytecode that mimics a virtual scenario of an ego vehicle following another vehicle at a particular safe distance. At step (508), the bytecode generating system (202) generates the fourth bytecode mimicking the virtual scenario of the ego vehicle following another vehicle at the particular safe distance based on the test case received from the test automation system (126). At step (510), the bytecode injecting system (206) injects the fourth bytecode into the kernel memory space (124) of the operating system (114C) hosting the ACC container (102C). The injection of the fourth bytecode into the kernel memory space (124) causes the ACC container (102C) to identify that the ACC container (102C) needs to activate an associated ACC mode and maintain the particular safe distance between the ego vehicle and another vehicle. Accordingly, the ACC container (102C) interferes with functionalities of the vehicle dynamics container (102D) and provides instructions to the vehicle dynamics container (102D) to control the accelerator, steering wheel, and/or brake of the ego vehicle such that the particular safe distance is maintained between the ego vehicle and another vehicle. At step (512), the test validation subsystem (208) determines if the AEB container (102A) interferes with the functionalities of the vehicle dynamics container (102D) while the ACC container (102C) is already interfering with the functionalities of the vehicle dynamics container (102D).
[0081] At step (514), the test validation subsystem (208) identifies that the AEB, ACC, and vehicle dynamics containers (102A, 102C, and 102D) are all operating as a unit appropriately when the AEB container (102A) is not interfering with the functionalities of the vehicle dynamics container (102D) while the ACC container (102C) is already interfering with the functionalities of the vehicle dynamics container (102D). Otherwise, the test validation subsystem (208) identifies that the AEB, ACC, and vehicle dynamics containers (102A, 102C, and 102D) are not operating as a cohesive unit appropriately when the AEB container (102A) is interfering with the functionalities of the vehicle dynamics container (102D) while the ACC container (102C) is already interfering with the functionalities of the vehicle dynamics container (102D).
[0082] In certain embodiments, the OTA server (134) transmits the application containers (102A-N) whose associated functionalities are successfully tested to the OTA client (136) via the vehicle agent (138) for enabling the OTA client (136) to deploy the application containers (102A-N) in the physical main ECU (104). In real-world scenarios, the vehicle agent (138) verifies if the application containers (102A-N) received as data packets from the OTA server (134) are secure and are not compromised before deploying the application containers (102A-N) in the physical main ECU (104). In certain embodiments, the FSV system (100) of the present disclosure can be used to verify if the vehicle agent (138) correctly identifies the data packets received from the OTA server (134) as secure or unsecured, as described subsequently in detail with reference to FIGS. 6A-B.
[0083] FIGS. 6A-B are a flow diagram illustrating an exemplary method (600) for testing functionalities of the vehicle agent (138) that ensures secure communication between the software defined vehicle (106) and the test server (107) using the FSV system (100). At step (602), the bytecode generating system (202) generates a first bytecode representative of network packets including data and checksum codes used to mimic a presence of the software defined vehicle (106). At step (604), the bytecode injecting system (206) injects the first bytecode in the kernel memory space (124) of the operating system (140) hosting the vehicle agent (138) for mimicking availability of the software defined vehicle (106) to communicate with the vehicle agent (138).
[0084] Subsequently, at step (606), the bytecode generating system (202) generates a second bytecode representative of network packets including modified data and checksum codes that mimic a defective data packet. In one embodiment, the defective data packet may correspond to one of a hacked or compromised data packet, an unsecured data packet affected by a malware such as a virus, a trojan, and a spyware, a data packet including an error message, and a data packet including one or more messages in an undesired format. At step (608), the bytecode injecting system (206) injects the second bytecode in the kernel memory space (124) of the operating system (140) for mimicking a real-world scenario of transmission of the defective data packet from the software defined vehicle (106) to the vehicle agent (138). At step (610), the test validation subsystem (208) verifies if the vehicle agent (138) correctly identifies one or more defects in the defective data packet represented by the second bytecode injected into the kernel memory space (124). In one embodiment, the vehicle agent (138) uses one or more techniques such as a security checksum and a security handshake for identifying the one or more defects in the defective data packet.
[0085] Further, at step (612), the test validation subsystem (208) further verifies if the vehicle agent (138) generates an alert message including the one or more identified defects and transmits the alert message to the virtual main ECU (108). At step (614), the test validation subsystem (208) successfully validates a first level functionality of the vehicle agent (138) when the vehicle agent (138) correctly transmits the alert message including the one or more identified defects to the virtual main ECU (108). In one embodiment, the first level functionality of the vehicle agent (138) corresponds to accurately identifying the one or more defects in the defective data packet transmitted from the physical main ECU (104) to the test server (107) via the vehicle agent (138).
[0086] At step (616), the bytecode generating system (202) generates a third bytecode that mimics a data packet to be injected into a communication port line (146) of the OTA server (134). In certain embodiments, the OTA server (134) includes the communication port line (146) to which a plurality of other communication port lines associated with the application containers (102A-N) are communicatively coupled. An example of the communication port lines that exist in the OTA server (134) and the application containers (102A-N) includes an ethernet port line. At step (618), the bytecode injecting system (206) injects the third bytecode into the communication port line (146) of the OTA server (134). At step (620), the test validation subsystem (208) verifies if the data packet represented by the third bytecode injected into the communication port line (146) of the OTA server (134) correctly reaches the one or more operating systems (114A-N) hosting the application containers (102A-N). At step (622), the test validation subsystem (208) identifies the OTA server (134) to be working properly when the data packet represented by the injected third bytecode correctly reaches the one or more operating systems (114A-N) hosting the application containers (102A-N).
[0087] At step (624), the bytecode generating system (202) generates a fourth bytecode that mimics a defective application container. In one embodiment, the defective application container corresponds to a hacked or compromised application container, or an unsecured application container affected by a malware such as a virus, a trojan, and a spyware. At step (626), the bytecode injecting system (206) injects the fourth bytecode into the kernel memory space (124) of the operating system (140) hosting the vehicle agent (138) for mimicking a real-world scenario of transmitting the defective application container from the OTA server (134) to the vehicle agent (138). At step (628), the test validation subsystem (208) determines if the vehicle agent (138) correctly identifies one or more defects in the defective application container upon injecting the fourth bytecode into the kernel memory space (124). At step (630), the test validation subsystem (208) successfully validates a second level functionality of the vehicle agent (138) when the vehicle agent (138) correctly identifies the one or more defects in the defective application container. In one embodiment, the second level functionality of the vehicle agent (138) corresponds to accurately identifying the one or more defects in the defective application container transmitted from the OTA server (134) to the physical main ECU (104) via the vehicle agent (138).
[0088] In certain embodiments, the software defined vehicle (106) exchanges vehicle and/or driver data with the test server (107) in real-world scenarios. The test server (107) uses such data received from the software defined vehicle (106) for providing value added services. For example, the software defined vehicle (106) periodically transmits a data packet including telemetry and/or vehicle diagnostic data to the test server (107) such that the test server (107) provides a value-added service by predicting an upcoming failure of one or more vehicle components and by alerting a user. Generally, the software defined vehicle (106) includes an associated application programming interface (API) port through which the data packet is transmitted to an API port in the test server (107) via the vehicle agent (138) for receiving value added services. While transmitting the data packet from the API port system of the software defined vehicle (106) to the API port system of the test server (107), one or more hackers may hack and modify the data packet for introducing security risks to the test server (107) and the software defined vehicles (106) connected to the test server (107). In real-world scenarios, the API port system of the test server (107) is expected to identify such an unsecured data packets transmitted to the API port system and generate an alert message.
[0089] In one embodiment, the FSV system (100) may be used to verify if the API port system of the test server (107) correctly identifies an unsecured data packet received from the API port system of the software defined vehicle (106) in a laboratory environment. To that end, the bytecode generating system (202) generates a bytecode that mimics an unsecured data packet in a first format different from a second format in which data is usually transmitted from a first API port system (148) in the software defined vehicle (106) to a second API port system (150) in the test server (107). The bytecode injecting system (206) then injects the generated bytecode into the first API port system (148). The first API port system (148) then transmits the bytecode mimicking the unsecured data packet to the second API port system (150) through the vehicle agent (138). Upon transmitting the bytecode mimicking the unsecured data packet to the second API port system (150), the test validation subsystem (208) verifies if the second API port system (150) correctly identifies that the data packet is unsecured and is in the first format different from the second format. Further, the test validation subsystem (208) verifies if the second API port system (150) generates and displays an alert message on the user interface (130) of the display unit (132). When the second API port system (150) correctly identifies that the data packet is unsecured and is not in an appropriate format and further generates the alert message as expected, the test validation subsystem (208) successfully verifies the functionalities of the second API port system (150). Otherwise, the test validation subsystem (208) identifies the second API port system (150) to be functioning improperly.
[0090] In certain embodiments, the software defined vehicle (106) generally includes a sandbox (152) that provides a secure onboard application execution environment. The sandbox (152) typically downloads one or more application containers (102A-N) from an application store in the test server (107). Further, the sandbox (152) identifies if such downloaded application containers (102A-N) are secure, and deploys the downloaded application containers (102A-N) in the software defined vehicle (106) only upon successful verification. The FSV system (100) of the present disclosure may be used to verify these functionalities of the sandbox (152) in a laboratory environment.
[0091] For example, the bytecode generating system (202) generates a bytecode that mimics a defective application container for testing functionalities of the sandbox (152). Examples of the defective application container includes an application container that includes a descriptive tag of another application container and/or the application container in a format different from a designated format. The bytecode injecting system (206) then injects the bytecode mimicking the defective application container into the kernel memory space (124) of an operating system associated with the sandbox (152) to mimic downloading of an application container from the application store.
[0092] Further, the test validation subsystem (208) determines if the sandbox (152) correctly identifies the defective application container represented by the injected bytecode into the kernel memory space (124) as the defective application container. Further, the test validation subsystem (208) determines if the sandbox (152) prevents deployment of the defective application container in the software defined vehicle (106). When the sandbox (152) correctly identifies that the application container is defective and further prevents associated deployment in the software defined vehicle (106) as expected, the test validation subsystem (208) identifies the sandbox (152) to be working properly. Otherwise, the test validation subsystem (208) identifies the sandbox (152) to be working improperly.
[0093] In certain embodiments, an architecture of the software defined vehicle (106) generally corresponds to a service-oriented architecture in which the application containers (102A-N) all act as microservices. One particular application container (102N) in the software defined vehicle (106) provides a microservice to another application container (102A) in the software defined vehicle (106). For example, the camera application container (102N) provides a microservice to the AEB container (102A) by transmitting associated camera images to the AEB container (102N) via a standard message-based communication protocol for enabling the AEB container (102A) to perform associated functionalities. Further, when the camera application container (102N) transmits images to the AEB container (102A), the AEB container (102A) needs to receive each of such images from the camera application container (102N) within a particular time period via the standard message-based communication protocol, which otherwise affects the safety of the software defined vehicle (106). The FSV system (100) can be used to verify such communications and microservice related messages exchanged between different application containers (102A-N) of the software defined vehicle (106) in a laboratory environment.
[0094] To that end, the bytecode injecting system (206) injects first and second bytecodes, for example, into the respective kernel memory spaces (124) of the operating systems (118 and 114A) for mimicking the presence of the camera application container (102N) and the AEB container (102A) within the virtual main ECU (108). Further, the bytecode injecting system (206) injects a third bytecode that mimics an image captured by a camera into the kernel memory space (124) of the operating system (118) hosting the camera application container (102N). The test validation subsystem (208) then verifies if the camera application container (102N) transmits the image injected as the third bytecode to the AEB container (102A) via the standard message-based communication protocol. Further, the test validation subsystem (208) verifies if the AEB container (102A) receives the image within the particular time period from the camera application container (102N). When the AEB container (102A) has successfully received the image from the camera application container (102N) within the particular time period as expected, the test validation subsystem (208) successfully verifies a microservice provided to the AEB container (102A) by the camera application container (102N). Otherwise, the test validation subsystem (208) identifies failure of the microservice between the camera application and AEB containers (102N and 102A) when the AEB container (102A) has not successfully received the image from the camera application container (102N) within the particular time period.
[0095] In real-world scenarios, the software defined vehicle (106) includes a plurality of application containers (102A-N) that run on the same physical main ECU (104). Features of these application containers (102A-N) will be continuously updated for introducing any new features and/or for modifying existing features. Additionally, application containers that are newly developed will also be deployed in the same physical main ECU (104). As a result, memory available to each of the application containers (102A-N) will be limited as the number of application containers (102A-N) deployed in the physical main ECU (104) continuously increases. Limited availability of memory generally affects working of each of the application containers (102A-N) and may even lead to a memory interference or a crash of the memory. In order to avoid these issues, the physical main ECU (104) in the software defined vehicle (106) continuously monitors usage of associated resources such as usage of memory and central processing unit (CPU) resources. When the memory and/or CPU resources usage is about to exceed a memory threshold and a CPU threshold, respectively, the physical main ECU (104) reduces a number of the application containers (102A-N) that are functioning actively and/or limits operations of individual containers to ensure those application containers (102A-N) function properly without any issues. In certain embodiments, the FSV system (100) can be used to verify these functionalities of the physical main ECU (104) in a laboratory environment.
[0096] To that end, the bytecode generating system (202) generates a bytecode that is used to fetch the memory and CPU resources usage. The bytecode injecting system (206) then injects the bytecode into the kernel memory space (124) of corresponding operating systems (114A-N and 118) for fetching and identifying the memory and CPU resources usage of each of the application containers (102A-N). For example, the bytecode injecting system (206) injects the bytecode into the kernel memory space (124) of the operating systems (114D-E) for fetching and identifying the memory and CPU resources usage of the container applications (102D-E) corresponding to the vehicle dynamics controller container (102D) and a warning message displaying container (102E), respectively. In one embodiment, the vehicle dynamics controller container (102D) has an associated first safety critical value. The warning message displaying container (102E) has an associated second safety critical value that is lesser than the first safety critical value. The test server (107) may store the first and second safety critical values associated with the vehicle dynamics controller and warning message displaying containers (102D-E) in an associated database (now shown in FIGS. 1-6B). If any of the application containers (102D-E) use the memory and CPU resources greater than the memory and/or CPU thresholds, the test validation subsystem (208) identifies such application containers (102D-E) based on their associated memory and CPU usage values.
[0097] Further, in the previously noted example, when the memory and/or CPU resources used exceeds a designated memory threshold and/or a designated CPU threshold, respectively, the test validation subsystem (208) determines if the virtual main ECU (108) that mimics the physical main ECU (104) performs an expected action. For example, the test validation subsystem (208) determines if the virtual main ECU (108) shut downs or limits an operation of the warning message displaying container (102E) having the second safety critical value when the memory and/or CPU resources used exceeds the designated memory and/or CPU thresholds, respectively. When the test validation subsystem (208) determines that the virtual main ECU (108) has correctly shut down or limited the operation of the warning message displaying container (102E) as expected, the test validation subsystem (208) identifies that the virtual main ECU (108) representative of the physical main ECU (104) is working properly. Otherwise, the test validation subsystem (208) identifies that the virtual main ECU (108) representative of the physical main ECU (104) is working improperly.
[0098] Similarly, each of the application containers (102A-N) running on the software defined vehicle (106) includes an associated processing time threshold. Vehicle safety is highly compromised when an application container fails to process data within the associated processing time threshold when the failed application container is one of multiple application containers (102A-N) that work together to achieve a particular functionality. For example, the AEB container (102A), camera application container (102N), and warning message displaying container (102E) operate together to achieve an emergency braking functionality. Generally, the camera application container (102N) provides images of the surroundings as inputs to the AEB container (102A). Subsequently, the AEB container (102A) determines a time-to-collision with an obstacle and further transmits the determined time-to-collision information to the warning message displaying container (102E) within a first designated time period. Subsequently, the warning message displaying container (102E) generates and displays an obstacle warning message to a driver within a second designated time period. However, when the warning message displaying container (102E) takes time that is more than the second designated time period to process the time-to-collision information received from the AEB container (102A) and to display the warning message, the physical main ECU (104) needs to shut down the warning message displaying container (102E). Further, the physical main ECU (104) needs to allocate an increased amount of CPU resources to the AEB container (102A) to activate an emergency braking rapidly to avoid collision between the software defined vehicle (106) and the obstacle. These functionalities of the physical main ECU (104) noted previously can be verified in a laboratory environment using the FSV system (100).
[0099] Specifically, the bytecode injecting system (206) injects bytecodes into the kernel memory space (124) of the operating system (114A) that mimics images of the vehicle surroundings provided as inputs to the AEB container (102A). Subsequently, the AEB container (102A) may determine the time-to-collision with the obstacle from the input images, and provide the determined time-to-collision information as input to the warning message displaying container (102E). The bytecode injecting system (206) then injects another bytecode into the kernel memory space (124) of the operating system (114E) associated with the warning message displaying container (102E) to fetch and identify if the warning message displaying container (102E) has generated a warning message within the second designated time period. When the warning message displaying container (102E) has not generated the warning message within the second designated time period, the test validation subsystem (208) transmits a delay-in-processing time message to the virtual main ECU (108). Upon receiving the delay-in-processing time message, the test validation subsystem (208) determines if the virtual main ECU (108) shuts down the operation of the warning message displaying container (102E) and allocates an increased amount of CPU resource to the AEB container (102A). When the virtual main ECU (108) shuts down the operation of the warning message displaying container (102E) and allocates the increased amount of CPU resource to the AEB container (102A) as expected, the test validation subsystem (208) identifies that the virtual main ECU (108) representative of the physical main ECU (104) is working properly. Otherwise, the test validation subsystem (208) identifies that the physical main ECU (104) is working improperly.
[00100] In certain embodiments, when the physical main ECU (104) of the software defined vehicle (106) downloads an application container (102A) or executes the application container (102A) from the test server (107), there are various files such as cache and temporary files that are created by default. The FSV system (100) provides an option to the user to indicate if the physical main ECU (104) needs to generate both cache and temporary files or needs to create only cache files during execution of the application container (102A). When the user selects the option that indicates that only the cache files are to be created, the FSV system (100) does not allow the physical main ECU (104) to unnecessarily generate and store the temporary files, which saves a significant amount of memory space associated with the physical main ECU (104).
[00101] In certain embodiments, the FSV system (100) can also be used to identify a root cause of a communication delay occurring between one container and another container. For example, the FSV system (100) assigns flags at one or more diagnostic points within software codes of an application container (102A) where faults are likely to occur. For instance, the FSV system (100) assigns a flag at codes corresponding to an ethernet port of the AEB container (102A) where the AEB container (102A) receives one or more images as inputs from the camera application container (102N). When the FSV system (100) identifies that the ethernet port of the AEB container (102A) receives the images at a frame rate that is lesser than a designated frame rate, the FSV system (100) raises the flag assigned at the ethernet port of the AEB container (102A). Further, the FSV system (100) raises the flag upon identifying the root cause for a delay in receiving the images from the camera application container (102N) as a slow frame per second rate of the camera application container (102N). Unlike the FSV system (100), accessing the kernel memory space (124) and raising flags are not possible in conventional FSV systems. However, accessing the kernel memory space (124) and raising flags are possible with the FSV system (100), which helps in identifying a time interval between each communication occurring between the application containers (102A-N) and a root cause of the delay in the communication.
[00102] While the FSV system (100) is described with reference to validation of whether the application containers (102A-N) to be deployed in the software defined vehicle (106) adhere to safety standard defined in ISO 26262, the FSV system (100) can also be used to validate the application containers (102A-N) to be deployed in other target systems. For example, the FSV system (100) may be used to validate if the application containers (102A-N) to be deployed in an avionics system adhere to safety standard defined in DO-178 or DO-254. In another example, the FSV system may be used to validate if the application containers (102A-N) to be deployed in a railway system adhere to railway safety standard defined in EN 50129. The FSV system (100) may also be used to validate various test bench setups such as hardware-in-the-loop test bench setup, a software-in-the-loop test bench setup, a model-in-the-loop test bench setup, and one or more servers hosting virtual ECUs. While validating these test bench setups, the FSV system (100) may be used to identify memory and CPU resources utilized by the application containers (102A-N) tested using these test bench setups. Further, the FSV system (100) may be used to identify details of communications occurring between the application containers (102A-N).
[00103] Conventional FSV systems perform functional safety validation of the application containers (102A-N) using external hardware devices and simulations tools, which are generally expensive. For example, executing the functional safety testing of the application containers (102A-N) using a conventional hardware-in-loop test setup and associated software tools costs approximately 3,50,00,000/- rupees, which varies depending upon a number of the application containers (102A-N) to be tested and scope of the testing. In contrast, when executing the functional safety testing for the same number of application containers (102A-N) and for the same scope of testing using the present FSV system (100), the test will cost at least 2.7 times lesser than the conventional testing costs as the present FSV system (100) does not require any additional external hardware devices and software tools for the functional safety testing. The FSV system (100) simply performs the functional safety testing by injecting a bytecode into the kernel memory space (124) of the operating system (114A-N), which simulate real-world faults without need for any external hardware devices and simulation tools. Hence, the FSV system (100) of the present disclosure provides a cost-effective in-lab solution for executing the functional safety testing of the application containers (102A-N).
[00104] Although specific features of various embodiments of the present systems and methods may be shown in and/or described with respect to some drawings and not in others, this is for convenience only. It is to be understood that the described features, structures, and/or characteristics may be combined and/or used interchangeably in any suitable manner in the various embodiments shown in the different figures.
[00105] While only certain features of the present systems and methods have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes.

LIST OF NUMERAL REFERENCES:

100 FSV system
102A-N Application containers
104 Physical main ECU
106 Software defined vehicle
107 Test server
108 Virtual main ECU
110 System on chip
112 Hypervisor
114A-N, 118, 140 Operating systems
116 Safety island
122 User memory space
124 Kernel memory space
126 Test automation system
128 Communications link
130 User interface
132 Display unit
134 OTA server
136 OTA client
138 Vehicle agent
142 Ethernet port
144 Graphical processing unit
146 Communication port line
148, 150 API port systems
152 Sandbox
202 Bytecode generating system
204 Bytecode verifying system
206 Bytecode injecting system
208 Test Validation subsystem
210 Log file generating system
212 Log database
214 Log file analyzing system
300-314 Steps associated with a method for injecting a fault into an application container
400-414 Steps associated with a method for performing a freedom from interference test
500-514 Steps associated with a method for performing a mixed criticality test
600-630 Steps associated with a method for testing functionalities of a vehicle agent , Claims:We claim:

1. A method for performing functional safety testing, comprising:
generating a bytecode that mimics a test condition associated with an application container (102A) to be tested using a bytecode generating system (202), wherein the application container (102A) is stored in a test server (107);
verifying if an injection of the generated bytecode into a kernel memory space (124) of an operating system (114A) hosting the application container (102A) introduces one or more errors in behavior of the operating system (114A);
injecting the generated bytecode into the kernel memory space (124) of the operating system (114A) only upon verifying that the injection of the generated bytecode does not introduce the one or more errors in the behavior of the operating system (114A);
comparing an output generated by the application container (102A) post injection of the generated bytecode into the kernel memory space (124) of the operating system (114A) with an expected output;
successfully validating compliance of the application container (102A) with one or more functional safety standards when the output generated by the application container (102A) matches with the expected output; and
automatically transmitting the application container (102A) that is successfully validated from the test server (107) to a target system (104) for deploying the application container (102A) in the target system (104).

2. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating the bytecode representative of null data and one or more flag values used to mimic a failure of a first application container (102N) selected from the one or more application containers (102A-N);
verifying if the injection of the bytecode representative of the null data and the flag values into the kernel memory space (124) of the operating system (114A) hosting a second application container (102A) that is different from the first application container (102N) introduces the one or more errors in the behavior of the operating system (114A) using a bytecode verifying system (204);
injecting the bytecode representative of the null data and the flag values into the kernel memory space (124) of the operating system (114A) only upon verifying that the injection of the bytecode into the kernel memory space (124) of the operating system (114A) does not introduce the one or more errors in the behavior of the operating system (114A);
determining if the second application container (102A) correctly identifies the failure of the first application container (102N) and generates an alert message indicating the failure of the first application container (102N) post injection of the bytecode representative of the null data and the flag values into the kernel memory space (124) of the operating system (114A); and
identifying that the failure of the first application container (102N) does not cause the second application container (102A) to fail when the second application container (102A) has correctly identified the failure of the first application container (102N) and generated the alert message.

3. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
identifying a first memory address space in the kernel memory space (124) of the operating system (114A) where a first application container (102A) runs on the operating system (114A);
identifying a second memory address space in the kernel memory space (124) of the operating system (114A) where a second application container (102B) runs on the operating system (114A);
generating the bytecode that mimics a sensor output that is related to the first application container (102A) and is unrelated to the second application container (102B);
injecting the bytecode that mimics the sensor output into the second memory address space;
determining if the second application container (102B) generates an error message and restarts post injection of the bytecode into the second memory address space; and
identifying that the second application container (102B) complies with the one or more functional safety standards when the second application container (102B) generates the error message and restarts post injection of the bytecode that mimics the sensor output that is unrelated to the second application container (102B) into the second memory address space.

4. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating the bytecode that mimics a failure of a hardware device (144) that is interlinked to a subset of application containers (102A-C) selected from the one or more application containers (102A-N);
injecting the bytecode into the kernel memory space (124) of an operating system hosted by the hardware device (144) for mimicking the failure of the hardware device (144);
identifying a corresponding timestamp associated with the failure of the hardware device (144) and associated failure of each of the subset of application containers (102A-C) interlinked to the hardware device (144);
verifying if each of the subset of application containers (102A-C) correctly identifies that a reason for the associated failure is the failure of the hardware device (144) based on the corresponding timestamp;
verifying if each of the subset of application containers (102A-C) generates an associated error message and shuts down an associated operation post injection of the bytecode that mimics the failure of the hardware device (144) into the kernel memory space (124) of the operating system; and
identifying that the subset of application containers (102A-C) comply with the one or more functional safety standards when each of the subset of application containers (102A-C) generates the associated error message and shuts down the associated operation post injection of the bytecode that mimics the failure of the hardware device (144) into the kernel memory space (124) of the operating system.

5. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating a first bytecode, a second bytecode, and a third bytecode that mimic presence of a first application container (102A), a second application container (102C), and a third application container (102D) selected from the one or more application containers (102A-N) within a virtual target system (108);
injecting the first bytecode, the second bytecode, and the third bytecode into the kernel memory space (124) of a first operating system (114A), a second operating system (114C), and a third operating system (114D) for mimicking presence of the first application container (102A), the second application container (102C), and the third application container (102D) within the virtual target system (108);
generating and injecting a fourth bytecode that mimics a sensor input into the kernel memory space (124) of the second operating system (114C), wherein injection of the fourth bytecode causes the second application container (102C) to interfere with one or more functionalities of the third application container (102D);
determining if the first application container (102A) interferes with the one or more functionalities of the third application container (102D) while the second application container (102C) is interfering with the one or more functionalities of the third application container (102D); and
successfully validating operation of the first application container (102A), the second application container (102C), and the third application container (102D) as a unit when the first application container (102A) does not interfere with the one or more functionalities of the third application container (102D) while the second application container (102C) is interfering with the one or more functionalities of the third application container (102D).

6. The method as claimed in claim 1, wherein automatically transmitting one or more application containers (102A-N) that are successfully validated from the test server (107) to the target system (104) comprises:
generating and injecting a first bytecode representative of one or more network data packets and one or more checksum codes into the kernel memory space (124) of an operating system (140) hosting a vehicle agent (138) to mimic availability of the target system (104) for communication with the vehicle agent (138);
generating and injecting a second bytecode representative of one or more modified network data packets and one or more corresponding checksum codes into the kernel memory space (124) of the operating system (140) that mimics transmission of a defective data packet from the target system (104) to the vehicle agent (138), wherein the defective data packet corresponds to one of a compromised data packet, a data packet affected by a malware, a data packet comprising an error message, and a data packet comprising one or more messages in an undesired format;
verifying if the vehicle agent (138) correctly identifies one or more defects in the defective data packet represented by the second bytecode injected into the kernel memory space (124) of the operating system (140) and generates an alert message comprising the one or more identified defects post injection of the second bytecode into the kernel memory space (124) of the operating system (140); and
successfully validating a first level functionality of the vehicle agent (138) when the vehicle agent (138) generates the alert message comprising information corresponding to the one or more identified defects post injection of the second bytecode into the kernel memory space (124) of the operating system (140).

7. The method as claimed in claim 6, wherein automatically transmitting the one or more application containers (102A-N) that are successfully validated from the test server (107) to the target system (104) comprises:
generating a third bytecode that mimics a defective application container (102A), wherein the defective application container (102A) corresponds to one of a compromised application container and an unsecured application container affected by a malware;
injecting the third bytecode that mimics the defective application container (102A) into the kernel memory space (124) of the operating system (140) for mimicking transmission of the defective application container (102A) from an over-the-air server (134) to the vehicle agent (138);
determining if the vehicle agent (138) correctly identifies one or more defects in the defective application container (102A) post injection of the third bytecode into the kernel memory space (124) of the operating system (140); and
successfully validating a second level functionality of the vehicle agent (138) when the vehicle agent (138) correctly identifies the one or more defects in the defective application container (102A) post injection of the third bytecode into the kernel memory space (124) of the operating system (140).

8. The method as claimed in claim 7, wherein automatically transmitting the one or more application containers (102A-N) that are successfully validated from the test server (107) to the target system (104) comprises:
generating a fourth bytecode that mimics an unsecured data packet in a first file format, wherein the first file format is different from a standard second file format in which a secure data packet is transmitted from a first application programming interface (API) port system (148) of the target system (104) to a second API port system (150) of the test server (107);
injecting the fourth bytecode into the first API port system (148) of the target system (104);
transmitting the fourth bytecode that mimics the unsecured data packet from the first API port system (148) to the second API port system (150) through the vehicle agent (138);
verifying if the second API port system (150) correctly identifies that the unsecured data packet received from the first API port system (148) is in the first file format that is different from the standard second file format; and
successfully validating one or more functionalities of the second API port system (150) of the test server (107) when the second API port system (150) correctly identifies that the unsecured data packet received from the first API port system (148) is in the first file format.

9. The method as claimed in claim 8, wherein automatically transmitting the one or more application containers (102A-N) that are successfully validated from the test server (107) to the target system (104) comprises:
injecting the third bytecode that mimics the defective application container (102A) into the kernel memory space (124) of an operating system associated with a sandbox (152) residing in the target system (104) for mimicking downloading of the defective application container (102A) from the test server (107) by the sandbox (152);
determining if the sandbox (152) correctly identifies the defective application container (102A) represented by the third bytecode injected into the kernel memory space (124) of the operating system of the sandbox (152) and prevents a deployment of the defective application container (102A) in the target system (104); and
identifying that the sandbox (152) executes one or more associated functionalities correctly when the sandbox (152) correctly identifies the defective application container (102A) and prevents the deployment of the defective application container (102A) in the target system (104).

10. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating and injecting a first bytecode and a second bytecode into the kernel memory space (124) of a first operating system (114A) and a second operating system (114B) selected from the one or more operating systems (114A-N) for mimicking presence of a first application container (102A) and a second application container (102B) within a virtual target system (108);
generating and injecting a third bytecode that mimics a sensor input into the kernel memory space (124) of the first operating system (114A);
verifying if the first application container (102A) provides a microservice to the second application container (102B) post injection of the third bytecode into the kernel memory space (124) of the first operating system (114A), wherein the microservice comprises transmission of the sensor input from the first application container (102A) to the second application container (102B);
identifying if the second application container (102B) receives the sensor input transmitted from the first application container (102A) within a particular time period; and
successfully verifying the microservice provided by the first application container (102A) to the second application container (102B) when the second application container (102B) receives the sensor input from the first application container (102A) within the particular time period.

11. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating and injecting a bytecode into the kernel memory space (124) of each of a first operating system (114A) and a second operating system (114B) for identifying utilization of one or more of a memory resource and a computing resource of a virtual target system (108) by each of a first application container (102A) and a second application container (102B), wherein the first operating system (114A) hosts the first application container (102A) having a first safety critical value and the second operating system (114B) hosts the second application container (102B) having a second safety critical value that is lesser than the first safety critical value;
identifying if utilization of one or more of the memory resource and the computing resource of the virtual target system (108) exceeds a designated memory resource threshold and a designated computing resource threshold, respectively using the virtual target system (108);
verifying if the virtual target system (108) performs one or more of shutting down or limiting an operation of the second application container (102B) having the second safety critical value when one or more of the memory resource and the computing resource used by the virtual target system (108) exceeds the designated memory resource threshold and the designated computing resource threshold, respectively; and
successfully verifying one or more functionalities of the target system (104) when the virtual target system (108) shuts down or limits the operation of the second application container (102B).

12. The method as claimed in claim 1, wherein successfully validating compliance of one or more application containers (102A-N) comprises:
generating and injecting a first bytecode that mimics a sensor input into the kernel memory space (124) of a first operating system (114A) hosting a first application container (102A), wherein the first application container (102A) generates a first application output upon injecting the first bytecode that mimics the sensor input into the kernel memory space (124) of the first operating system (114A);
transmitting the first application output generated by the first application container (102A) to a second application container (102B) via a communications link (128);
generating and injecting a second bytecode into the kernel memory space (124) of a second operating system (114B) hosting a second application container (102B) to identify if the second application container (102B) has generated a second application output within a designated time period based on the first application output;
verifying if a virtual target system (108) shuts down an operation of the second application container (102B) and increases an associated computing resource previously allocated to the first application container (102A) when the second application container (102B) fails to generate the second application output within the designated time period; and
successfully validating one or more functionalities of the target system (104) when the virtual target system (108) shuts down the operation of the second application container (102B) and increases the associated computing resource previously allocated to the first application container (102A).

13. A system for performing functional safety testing, comprising:
a test server (107) that comprises a virtual target system (108), wherein the virtual target system (108) corresponds to a virtual model of a target system (104) and comprises one or more operating systems (114A-N) hosting one or more application containers (102A-N) to be tested;
a bytecode generating system (202) that resides in the virtual target system (108) and is configured to generate one or more bytecodes that mimic one or more test conditions associated with the one or more application containers (102A-N) to be tested;
a bytecode verifying system (204) that is communicatively coupled to the bytecode generating system (202) and is configured to verify if an injection of the one or more generated bytecodes into one or more kernel memory spaces (124) of the one or more operating systems (114A-N) introduces one or more errors in behavior of the one or more operating systems (114A-N);
a bytecode injecting system (206) that is communicatively coupled to the bytecode verifying system (204) and is configured to inject the one or more generated bytecodes into the one or more kernel memory spaces (124) of the one or more operating systems (114A-N) only when the bytecode verifying system (204) verifies that the injection of the one or more generated bytecodes does not introduce the one or more errors in the behavior of the one or more operating systems (114A-N);
a test validation subsystem (208) that is communicatively coupled to the bytecode injecting system (206) and the target system (104), wherein the test validation subsystem (208) is configured to:
compare one or more outputs generated by the one or more application containers (102A-N) post injection of the one or more generated bytecodes into the one or more kernel memory spaces (124) of the one or more operating systems (114A-N) with one or more expected outputs; and
successfully validate compliance of the one or more application containers (102A-N) with one or more functional safety standards when the one or more outputs generated by the one or more application containers (102A-N) match with the one or more expected outputs, wherein the test server (107) automatically transmits the one or more application containers (102A-N) that are successfully validated to the target system (104) for deploying the one or more application containers (102A-N) in the target system (104).

14. The system as claimed in claim 13, wherein the target system (104) comprises one of a physical main electronic control unit (104), a software defined vehicle, a rail system, an avionic system, a drone, a robotic vehicle used in one or more of an industrial application, a retail application, a healthcare application, a security application, and a surveillance application, an aquatic vehicle, an internet of things device, a gaming system, a consumer electronic device, a hardware-in-the-loop test bench, a software-in-the-loop test bench, a model-in-the-loop test bench, and one or more servers hosting virtual electronic control units.