WO2017075623A1

WO2017075623A1 - Method and system for boot time optimization of embedded multiprocessor systems

Info

Publication number: WO2017075623A1
Application number: PCT/US2016/059797
Authority: WO
Inventors: Yogesh Vikram MARATHE; Kedar Satish CHITNIS; Rishabh GARG
Original assignee: Texas Instruments Incorporated; Texas Instruments Japan Limited
Priority date: 2015-10-30
Filing date: 2016-10-31
Publication date: 2017-05-04

Abstract

In described examples, an embedded multiprocessor system (100) includes: a multiprocessor system on a chip (SOC) (108); a memory (114) coupled to the multiprocessor SOC (108), the memory (114) storing application software partitioned into an initial boot stage and at least one additional boot stage; and a secondary boot loader configured to boot load the initial boot stage on at least one processor of the multiprocessor SOC (108). The initial boot stage begins executing, and flow of data from the initial boot stage to the at least one additional boot stage is disabled. The application software is configured to boot load a second boot stage of the at least one additional boot stage on at least one other processor of the multiprocessor SOC (108) and to enable flow of data between the initial boot stage and the second boot stage.

Description

METHOD AND SYSTEM FOR BOOT TIME OPTIMIZATION OF

EMBEDDED MULTIPROCESSOR SYSTEMS

[0001] This relates generally to embedded multiprocessor systems, and more particularly to boot time optimization for applications executing on such systems.

BACKGROUND

[0002] A class of safety systems, referred to as advanced driver assistance systems (ADAS), has been introduced into automobiles to reduce human operation error. Such systems may provide various functionality, such as rear-view facing cameras, electronic stability control, and vision-based pedestrian detection systems. The boot time of an automotive safety system may be crucial for safety and important to the overall user experience.

[0003] For example, an automotive safety system, such as a rear view camera system (RVCS) may be provided to show the rear view of the vehicle when the reverse gear of the vehicle is engaged. The RVCS may simply display the rear view or may include more intelligence to detect obstacles and provide visual and/or audible warnings to the driver. In either instance, the boot time of the RVCS is critical for safety and, if overly long, affects the overall user experience.

SUMMARY

[0004] In described examples for boot time optimization in embedded multiprocessor systems, an embedded multiprocessor system includes: a multiprocessor system on a chip (SOC); a memory coupled to the multiprocessor SOC, the memory storing application software partitioned into an initial boot stage and at least one additional boot stage; and a secondary boot loader configured to boot load the initial boot stage on at least one processor of the multiprocessor SOC. The initial boot stage begins executing, and flow of data from the initial boot stage to the at least one additional boot stage is disabled. The application software is configured to boot load a second boot stage of the at least one additional boot stage on at least one other processor of the multiprocessor SOC and to enable flow of data between the initial boot stage and the second boot stage.

[0005] In at least one example, a method for booting an embedded multiprocessor system includes boot loading an initial boot stage of application software of the embedded multiprocessor system on at least one processor of a multiprocessor system on a chip (SOC) in the embedded multiprocessor system, wherein the initial boot stage begins executing and flow of data from the initial boot stage to a subsequent boot stage of the application software is disabled, boot loading, under control of the application software, the subsequent boot stage of the application software on at least one other processor of the multiprocessor SOC, and enabling flow of data between the early boot stage and the subsequent boot stage by the application software.

[0006] In at least one example, an automotive safety system includes: a multiprocessor system on a chip (SOC); a camera coupled to the multiprocessor SOC; a memory coupled to the multiprocessor SOC, the memory storing application software partitioned into an early boot stage and a late boot stage, wherein the early boot stage is configured to capture video frames from the camera and to display the video frames on a display device, and the late boot stage is configured to detect objects in the video frames; and a secondary boot loader configured to boot load the early boot stage on a master processor of the multiprocessor SOC. The early boot stage begins executing, and flow of data from the early boot stage to the late boot stage is disabled. The application software is configured to boot load the late boot stage on at least one other processor of the multiprocessor SOC and to enable flow of data between the early boot stage and the late boot stage.

BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a block diagram of an example rear view camera system (RVCS).

[0008] FIG. 2 illustrates the simplified data flow of an example conventional RVCS application.

[0009] FIG. 3 is an example of a Vision Software Development Kit (SDK) software stack implemented on an example system on a chip (SOC).

[0010] FIG. 4 is illustrates the simplified data flow of FIG. 2 divided into an early boot stage and a late boot stage.

[0011] FIG. 5 is a sequence diagram illustrating an optimized boot process for the RVCS of FIG. 1.

[0012] FIG. 6 is a flow diagram of a method for booting an automotive safety system. DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0013] Like elements in the various figures are denoted by like reference numerals for consistency.

[0014] Embodiments of the disclosure provide for boot time optimization for embedded multiprocessor systems such as advanced driver assistance systems (ADAS) that enable rapid boot loading of high priority portions of application software and delayed booting of lower priority portions. For example, in a rear view camera system (RVCS), boot priority may be given to display of the first frame of the rear view video while booting of other parts of the RVCS application such as object detection is given lower priority. In some embodiments, the application software manages the boot order and boot loading of the lower priority portions of the application.

[0015] Embodiments are explained herein in reference to an example RVCS. Example embodiments are possible for other automotive safety systems and other embedded multiprocessor systems. Generally, an automotive RVCS is a real time embedded system implemented as a hardware unit with external analog and digital interfaces. FIG. 1 is a block diagram of an example RVCS 100. The RVCS 100 includes a multi-processor system-on-a-chip (SOC) 108, memory devices including read only memory 112, flash memory 114, and random access memory (RAM) 116, a microcontroller (MCU) 106, and a power management integrated circuit (PMIC) 110. The RVCS is coupled to a camera sensor 102 configured to capture a video sequence of the rear field of view of a vehicle. The RVCS 100 is also coupled to a display 104 configured to display the video sequence along with any overlays and warnings that may be generated as a result of processing the video sequence in the RVCS 100.

[0016] The ROM 112 may be used to store the power on reset (POR) boot code. The flash memory 114 may be used to store secondary boot code (i.e., as a secondary boot loader) and application code. The MCU 106 serves as the communication interface between the SOC 108 and the vehicular network. Accordingly, the MCU 106 hosts the network protocol stack, which is usually a controller area network (CAN) stack. The multiprocessor SOC 108 hosts the RVCS application.

[0017] Conventionally, the RVCS of FIG. 1 is booted as follows. Upon POR, one of the processors of the multiprocessor SOC 108 is boot loaded by code stored in the ROM 1 12. After execution of the initial built in self tests (BIST), control is passed to a secondary boot loader read from the flash memory 114 into RAM 116. The secondary boot loader prepares the SOC 108 for executing the RVCS application, performing functions such as initializing essential phased lock loops and clocks, enabling peripherals, loading the multiprocessor application images into RAM 116, and booting the other processors in the SOC 108. After the other processors are booted, the secondary boot loader passes control to the processor which will execute the RVCS application. All processors of the multiprocessor SOC 108 are booted before the RVCS application begins execution. Thus, the display of the video stream from the camera sensor 102 cannot begin until all the processors are booted. The display delay incurred due to this boot time may be annoying to the driver of the vehicle and may also impact safety.

[0018] FIG. 2 illustrates the simplified data flow of an example conventional RVCS application. In this example, the simplified data flow of the RVCS application is illustrated using the Vision Software Development Kit (SDK) available from Texas Instruments Incorporated (TI). A brief overview of the Vision SDK is provided herein. Additional information may be found in K. Chitnis, et al., "TI Vision SDK, Optimized Vision Libraries for ADAS Systems," Texas Instruments, April, 2014, which is incorporated by reference herein.

[0019] The TI Vision SDK is a multiprocessor software development platform for TI's family of ADAS SOCs. The software framework allows users to create different ADAS application data flows involving video capture, video pre-processing, video analytics algorithms and video display. The Vision SDK is based on a framework referred to as the "Links and Chains" framework and the user API to this framework is called "Link API."

[0020] Generally, links are software modules implementing a processing step in a video processing data flow. Each link executes as a separate thread and includes a message box that allows other links to communicate with that link. Each link also implements an interface which allows other links to directly exchange video frames and/or bits streams with the link. The unit of exchange between links is a system buffer. Multiple types of system buffers exist, such as video frame, video bit stream, and meta data types. A system buffer includes at least a buffer type field, a channel number identifying a video/processing channel in the system, a time stamp, a sequence number, a payload size, and a payload pointer.

[0021] The Link API enables creation, control, and connection of links. The control code is written on a single processor. Internally, the Link API uses inter-processor communication (IPC) to control links on different processors. Each link has a link identifier that indicates which processor executes the link. A connection of links is referred to as a chain. The chain represents the data flow of the application. After a chain is initiated, video frames "flow" through the chain without need for user application intervention. Table 1 includes brief descriptions of some of the links provided by the Vision SDK.

Table 1

[0022] FIG. 3 is an example of the Vision SDK software stack implemented on a configuration of the TDA3x SOC 300, one of the ADAS SOCs available from Texas Instruments. The SOC 300 of this example includes two image processor units (IPU), two digital signal processor (DSP) from the TMS320C66x family of signal processors available from Texas Instruments, and an embedded vision engine (EVE), also referred to as a vision processor or vision hardware accelerator. For example, additional information regarding the architecture of the TDA3x SOC is located in "TDA3x SoC Processors for Advanced Driver Assist Systems (ADAS) Technical Brief," Texas Instruments Incorporated, October 2014, which is incorporated by reference herein. Each of the software stacks 302, 304, 306, 308 illustrates the software executing on the respective processors of the SOC 300. Table 2 contains a brief description of the software layers.

Table 2

[0023] Referring again to FIG. 2, the depicted simplified data flow assumes the RVCS 100 of FIG. 1, in which the SOC 100 is the SOC 300 of FIG. 3. In this data flow, the Capture link 200, the Dup link 202, the IPC OUT link 204 the IPC IN link 212, the Merge link 214, the Sync link 216, the Alg link 218, and the Display link 220 execute on IPU1 of the SOC 300. The IPC IN link 206, the Alg link 208, and the IPC OUT link 210 may execute on the C66x or the EVE of the SOC 300. The Capture link 200 captures frames from the camera sensor 102. The Dup link 202 provides duplicate frame information to both the merge link 214 and the IPC OUT link 204. The IPC OUT link 204 in conjunction with the IPC IN link 206 communicate the frame information from IPU1 to the Alg link 208 executing on the other processor. In this example, the Alg link 208 implements an algorithm plugin for object detection. For example, object detection may include traffic sign recognition, lane detection and/or pedestrian detection.

[0024] The IPC OUT link 210 in conjunction with the IPC IN link 212 communicate the output of the Alg link 208 to the Merge link 214 on IPUl . For example, the output of the Alg link 208 may be a list of detected objects, including the coordinates of each object and the type of each object. The Merge link 214 merges the output of the Dup link 202 and the output of the Alg link 208 for consumption by the Alg link 218. In this instance, the Alg link 218 consumes both video frames from the Dup link 202 and algorithm output from the Alg link 208.

[0025] The Sync link 216 time synchronizes buffers from different channels. In this instance, the Sync link 216 matches the sources time stamps of video frames to metadata output by the Alg link 208 and forwards composite frames to the Alg link 218. A composite frame includes a video frame and the Alg link 208 output for that frame in a single buffer. In this example, the Alg link 218 implements a drawing algorithm. More specifically, the Alg link 218 receives composite frames and draws boundaries around detected objects in a video frame to visually indicate the detected objects. The Display link 220 displays the output of the Alg link 218 on the display 104.

[0026] As mentioned hereinabove, conventionally, all processors of the RVCS 100 are booted before the display of the video stream can begin. Further, as shown in the simplified data flow of FIG. 2, the initial frames are processed through all the links of the data flow before anything is shown on the display. In some embodiments, the data flow of the RVCS application is divided into two boot stages, referred to herein as an early boot stage and a late boot stage. The early boot stage includes the application data flow needed for capture and display of the video stream from the camera sensor 102 while the late boot stage includes the remaining data flow of the RVCS application. The early boot stage can booted and begin execution before the late boot stage is initiated.

[0027] FIG. 4 illustrates the simplified data flow of FIG. 2 divided into an early boot stage 400 and a late boot stage 402. The early boot stage 400 includes the Capture link 200, the Dup link 202, and the Display link 220. The remainder of the links is assigned to the late boot stage 402. To facilitate the division of the data flow into boot stages, Gate links 406, 408, 410 are added to the early boot stage 400 to control the flow of data between links in the early boot stage 400 and links in the late boot stage 402. A Gate link operates as a "gate" to control of the flow of data from a previous link to the next link. For example, Gate link 406 controls the data flow from the Dup link 202 to the IPC OUT link 204. A Gate link has two states: an on state in which data is allowed to flow through the link, and an off state in which data is not allowed to flow through the link. Further, when a Gate link is instantiated, the link is in the off state by default. The RVCS application can issue a command to change the state of a Gate link to the on state.

[0028] Also, a Merge link 404 is added to the early boot stage 400 to merge the output of the Alg link 218 and the Dup link 202 when the Gate link 410 is in the on state. When the Gate link 410 is in the off state, the link 410 forwards the channel received from the Dup link 202 to the Display link 220. When the Gate link 410 is instantiated in the early boot stage 400, the application provides information regarding any input links, including any links that will not be instantiated until the application boot loads the late boot stage 402, to the Gate link 410. During the boot loading of the early boot stage 400, the Merge link 404 receives all the information needed to start receiving data from the Gate link 410 at any time the Gate link 410 is changed to the on state by the application.

[0029] In addition to splitting the data flow of the RVCS application into an early boot stage and a late boot stage, the RVCS application includes functionality to boot load the late boot stage. Accordingly, after the early boot stage of the application is executing, the application initiates the boot loading of the late boot stage. In some embodiments, the secondary boot loader (SBL) is modified to boot only those processors needed by the early boot stage of the application and to include interfaces, i.e., application program interfaces (APIs), to allow the application to boot other processors and enable peripherals as needed for the late boot stage. More specifically, the SBL may be split into three layers: an SBL library layer, an SBL Utility layer, and an SBL application.

[0030] The SBL library layer contains APIs that may be used by an application for booting processors and for parsing and loading images. This library layer may be used by both the application and the boot loader application. For example, the SBL utility layer contains APIs for communicating between boot media and the master processor (i.e., the processor executing the application, APIs for data transfer, and APIs for interacting with various peripherals). This layer is used only by the boot loader application. The SBL application contains the secondary boot loader application that loads and boots the application image, e.g., the early boot stage of the application. An example of such an SBL is described in R. Garg and S. R., "TDA3xx Secondary Bootloader (SBL)", Texas Instruments, January, 2016, pp. 1-30, which is incorporated by reference herein.

[0031] FIG. 5 is a sequence diagram illustrating the boot process of the RVCS 100 using the modified SBL. Upon POR, the processor that will execute the RVCS application is boot loaded by the ROM boot loader. After execution of the initial built in self tests (BIST), control is passed to the SBL application read from the flash memory 114 into RAM 116. The SBL application boots the processors needed for the initial boot stage of the RVCS application, e.g., IPU1 of SOC 300 (the master processor) and the processor hosting the CAN stack. The SBL application also enables any peripherals needed for the initially boot stage, loads the image for the early boot stage of the RVCS application into RAM 116, and initiates execution of the early boot stage code. Upon execution of the early boot stage code, display of video frames begins. At this point, all of the Gate links (FIG. 4) are in the off state.

[0032] The RVCS application then boots loads and synchronizes the other processors required for the late boot stage using the APIs of the SBL library. Finally, the RVCS application starts execution of the late boot stage. As part of starting the late boot stage, the RVCS application sends commands to each of the Gate links (FIG. 4) to change the state of these links to on, allowing data to flow between the early boot stage links and the late boot stage links.

[0033] The RVCS examples above illustrate optimized booting in which an automotive safety application is divided into two boot stages. In some embodiments, the data flow of an automotive safety application is divided into more than two boot stages such that the initial boot stage has the highest boot priority, i.e., will be boot loaded first, and the remaining boot stages have lower boot priorities. For each boot stage, the processors needed for executing the functionality of the application assigned to the boot stage are booted, if not already booted during an earlier boot stage. Also, booting of the remaining stages is managed by the safely application.

[0034] FIG. 6 is a flow diagram of a method for booting an automotive safety system in which the application software is partitioned into multiple boot stages. The application software is partitioned such that functionality of the application that needs to come on line as soon as possible, e.g., capturing and displaying video frames, is included in the boot stage that will be boot loaded first. The remaining functionality of the application software is partitioned into boot stages that are boot loaded under control of the application software.

[0035] The flow of data between boot stages is controlled by software gate modules, such as the above described Gate link, which enable and disable data flow through the modules according to commands from the application software. When a boot stage is boot loaded, any gate modules in the boot stage may be disabled by default. The application may then enable gate modules as needed to allow the flow of data between boot stages.

[0036] As shown in FIG. 6, initially the initial boot stage of the safety system application is boot loaded 600 onto one or more processors of a multiprocessor SOC. Any peripherals needed for execution of the initial boot stage are also enabled. The initial boot stage begins execution. At this point, data flow between the initial boot stage and any subsequent boot stages is disabled. Accordingly, any gate modules in the initial boot stage are configured to not allow the flow of data through the modules.

[0037] The safety system application then boot loads 602 another boot stage of the application on one or more other processors of the multiprocessor SOC. Any peripherals needed for execution of the initial boot stage that are not already enabled are also enabled. Accordingly, the boot loading of the next boot stage is controlled by the safety system application. The safety system application may manage the boot loading of the next boot stage using an API of a secondary boot loader, such as the secondary boot loader described hereinabove. The next boot stage begins execution. At this point, data flow between the newly booted boot stage and the initial boot stage and between the newly booted boot stage and any boot stages yet to be boot loaded is disabled. Accordingly, the gate modules in the newly booted boot stage are configured to not allow the flow of data through the modules.

[0038] The safety system application then enables 604 the flow of data between the initial boot stage and the newly booted boot stage. Accordingly, the safety system application sends a command to any gate modules controlling the flow of data between the two boot stages that enables the flow of data through those modules. Any gate modules in the two boot stages that control the data flow to boot stages not yet boot loaded are not enabled. The safety system application 604 then repeats the boot loading of another boot stage and enabling of data flow for any remaining boot stages 606. For any boot stage boot loaded after the second boot stage, the flow of data is enabled as needed between that boot stage and any of the previously boot loaded boot stages.

Other Embodiments

[0039] For example, embodiments have been described herein in which the automotive safety system is a RVCS. Example embodiments are possible for other automotive safety systems, such as back over prevention, parking assistance, front collision warning, and automatic braking. For example, for other camera based safety systems, the early boot stage may be similar to that of the RVCS example in order to start displaying the video stream(s) quickly and the late boot stage may include the remainder of the safety system application software.

[0040] In another example, embodiments have been described herein in relation to embedded multiprocessor systems used for automotive safety. Example embodiments are possible for other embedded multiprocessor systems, such as automotive infotainment, heads up displays, and consumer electronics (e.g., wearable sports cameras).

[0041] In another example, embodiments have been described herein in relation to a TDA3x multiprocessor SOC and the Vision SDK. Example embodiments are possible for other multiprocessor SOCs and/or other software development frameworks for multiprocessor software development.

[0042] Although method steps may be presented and described herein in a sequential fashion, one or more of the steps shown in the figures and/or described herein may be performed concurrently, may be combined, and/or may be performed in a different order than the order shown in the figures and/or described herein.

[0043] Components in multiprocessor systems may be referred to by different names and/or may be combined in ways not shown herein. Also, the term "couple" and derivatives thereof are intended to mean an indirect, direct, optical, and/or wireless electrical connection. For example, if a first device couples to a second device, that connection may be through a direct electrical connection, through an indirect electrical connection via other devices and connections, through an optical electrical connection, and/or through a wireless electrical connection.

[0044] Modifications are possible in the described embodiments, and other embodiments are possible, within the scope of the claims.

Claims

CLAIMS What is claimed is:

1. An embedded multiprocessor system comprising:

a multiprocessor system on a chip (SOC);

a memory coupled to the multiprocessor SOC, the memory storing application software partitioned into an initial boot stage and at least one additional boot stage; and

a secondary boot loader configured to boot load the initial boot stage on at least one processor of the multiprocessor SOC, wherein the initial boot stage begins executing, and flow of data from the initial boot stage to the at least one additional boot stage is disabled;

wherein the application software is configured to boot load a second boot stage of the at least one additional boot stage on at least one other processor of the multiprocessor SOC and to enable flow of data between the initial boot stage and the second boot stage.

2. The embedded multiprocessor system of claim 1, wherein flow of data between boot stages of the application software is controlled by one or more gate modules, each gate module configured to enable or disable flow of data through the module under command of the application software.

3. The embedded multiprocessor system of claim 1, wherein the application software is configured to use a program interface to the secondary boot loader to boot load the second boot stage.

4. The embedded multiprocessor system of claim 1, wherein the embedded multiprocessor system is coupled to a camera to receive video frames, and the initial boot stage is configured to capture video frames from the camera and display the video frames on a display device.

5. The embedded multiprocessor system of claim 4, wherein the embedded multiprocessor system is a rear view camera system.

6. The embedded multiprocessor system of claim 5, wherein the secondary boot loader is also configured to boot load a processor hosting a controller area network (CAN) stack.

7. The embedded multiprocessor system of claim 1, wherein the at least one processor comprises an image processor, and the at least one other processor comprises at least one of a digital signal processor and a vision processor.

8. The embedded multiprocessor system of claim 1, wherein the embedded multiprocessor system is an automotive safety system.

9. A method for booting an embedded multiprocessor system, the method comprising:

boot loading an initial boot stage of application software of the embedded multiprocessor system on at least one processor of a multiprocessor system on a chip (SOC) in the embedded multiprocessor system, wherein the initial boot stage begins executing, and flow of data from the initial boot stage to a subsequent boot stage of the application software is disabled;

boot loading, under control of the application software, the subsequent boot stage of the application software on at least one other processor of the multiprocessor SOC; and

enabling flow of data between the early boot stage and the subsequent boot stage by the application software.

10. The method of claim 9, wherein flow of data between boot stages of the application software is controlled by one or more gate modules, each gate module configured to enable or disable flow of data through the module under command of the application software.

11. The method of claim 9, wherein the boot loading an initial stage is performed by a secondary boot loader, and the application software uses a program interface to the secondary boot loader to boot load the subsequent boot stage.

12. The method of claim 9, wherein the initial boot stage includes functionality to capture and display video frames on a display device.

13. The method of claim 9, wherein the at least one processor comprises an image processor, and the at least one other processor comprises at least one of a digital signal processor and a vision processor.

14. The method of claim 9, wherein the embedded multiprocessor system is a rear view camera system.

15. The method of claim 14, further comprising boot loading a processor hosting a controller area network (CAN) stack.

16. An automotive safety system comprising:

a multiprocessor system on a chip (SOC);

a camera coupled to the multiprocessor SOC;

a memory coupled to the multiprocessor SOC, the memory storing application software partitioned into an early boot stage and a late boot stage, wherein the early boot stage is configured to capture video frames from the camera and to display the video frames on a display device, and the late boot stage is configured to detect objects in the video frames; and a secondary boot loader configured to boot load the early boot stage on a master processor of the multiprocessor SOC, wherein the early boot stage begins executing, and flow of data from the early boot stage to the late boot stage is disabled;

wherein the application software is configured to boot load the late boot stage on at least one other processor of the multiprocessor SOC and to enable flow of data between the early boot stage and the late boot stage.

17. The automotive safety system of claim 16, wherein flow of data between the early boot stage and the late boot stage is controlled by one or more gate modules, each gate module configured to enable or disable flow of data through the module under command of the application software.

18. The automotive safety system of claim 16, wherein the application software is configured to use a program interface to the secondary boot loader to boot load the late boot stage.

19. The automotive safety system of claim 16, wherein the master processor is an image processor, and the at least one other processor comprises at least one of a digital signal processor and a vision processor.

20. The automotive safety system of claim 16, wherein the automotive safety system is a rear view camera system.