TWI809763B - Safety system for a mobile vehicle and control method thereof - Google Patents

Abstract

A safety system for a mobile vehicle includes a camera device and a central control device. The camera device is used to obtain a first image. When the central control device determines that the mobile vehicle is in a straight state, the central control device determines whether a target object is in the first range of a side of the mobile vehicle according to the first image. When the central control device determines that the mobile vehicle is in a turning state, the central control device determines whether the target object is in the first range of the side of the mobile vehicle according to the first image, wherein the second range is larger than the first range.

Description

Safety system and control method of mobile vehicle

The invention relates to a safety system of a mobile vehicle, in particular to a safety system and a control method for dynamically adjusting the monitoring range.

When the driver is driving the vehicle, his field of vision will have a blind spot due to the structure of the vehicle body, such as being unable to see vehicles or pedestrians on the side or rear of the vehicle. In order to improve safety, the driver needs to install a safety system to monitor the blind area of the field of view.

The object of the present invention is to propose a security system and its control method for dynamically adjusting the monitoring range.

According to the present invention, a security system for a mobile vehicle includes a photographing device, a host and a central control device. The photographing device is used to obtain a first image. The host is used for outputting information of a mobile vehicle. The central control device receives the first image and the mobile vehicle information. When the central control device judges that the mobile vehicle is in a straight-ahead state according to the mobile vehicle information, the central control device determines whether there is a target object within a first range on the side of the mobile vehicle according to the first image. When the central control device judges that the mobile vehicle is in a turning state according to the information of the mobile vehicle, the central control device determines whether there is the target object in a second range on the side of the mobile vehicle according to the first image , the second range is greater than the first range. The central control device uses artificial intelligence to identify the target object.

According to the present invention, a method for controlling a safety system of a mobile vehicle includes the following steps: a. Obtaining a first image; b. When the mobile vehicle is in a straight-going state, judging the position of the mobile vehicle according to the first image Whether there is a target object in a first range on the side; and c. When the mobile vehicle is in a turning state, judge whether there is the target in a second range on the side of the mobile vehicle according to the first image object. Wherein the second range is larger than the first range, and steps b and c are to use artificial intelligence to identify the target object.

The safety system of the present invention can dynamically adjust the monitoring range. When the mobile vehicle is in the straight-going state, the safety system selects a smaller first range (first area) for monitoring, so as to achieve faster and more real-time monitoring. When the mobile vehicle is in a turning state, the safety system selects a larger second range for monitoring, which helps to improve driving safety.

FIG. 1 shows a mobile vehicle applying the security system of the present invention, and FIG. 2 shows the monitoring range of the security system in FIG. 1 . The mobility vehicle 10 in FIG. 1 can be, but not limited to, a bus, a truck, a truck, a pallet truck or a car. The mobile vehicle 10 has a plurality of wheels 11, 12, 13 and 14 for moving the mobile vehicle 10. The mobile vehicle 10 is also provided with a security system 20, and the security system 20 is used to monitor whether there is a target on the side of the mobile vehicle 10. object. The target object can be but not limited to pedestrians or other mobile vehicles. The security system 20 includes camera devices 21 and 22 , a central control device 23 , a host 24 , a display device 25 and a speaker 26 . The photographing device 21 is disposed on the left side of the mobile vehicle 10 for acquiring a first image P1 on the left side of the mobile vehicle 10 . The photographing device 22 is disposed on the right side of the mobile vehicle 10 for obtaining a first image P2 on the right side of the mobile vehicle 10 . The range covered by the first image P1 includes a first area 301 and a second area 302 . The range covered by the first image P2 includes a first area 311 and a second area 312 . The first area 301 is located between the mobile vehicle 10 and the second area 302 , and the first area 311 is located between the mobile vehicle 10 and the second area 312 . In the case that the mobile vehicle 10 is a vehicle, such as a bus, the photographic device 21 can be installed on, but not limited to, the shell of the left rear-view mirror, the left side of the vehicle body or the left side of the roof, and the photographic device 22 can be installed on but Not limited to the housing of the right rear view mirror, the right side of the bodywork or the right side of the roof. In one embodiment, the photographing devices 21 and 22 can be but not limited to general or wide-angle cameras. When the photographing devices 21 and 22 are 180° wide-angle cameras, the photographing device 21 can monitor the left, left rear and left front areas. The camera device 22 can monitor the right side, right rear and right front areas. In an embodiment, the width B1 of the first regions 301 and 311 may be but not limited to the width of a vehicle, the width of a lane or 3.5m.

The host 24 in FIG. 1 is used for providing mobile vehicle information VI. In one embodiment, the host computer 24 may be a driving computer (Electronic Control Unit; ECU) that controls the mobile vehicle 10 . The mobile vehicle information VI includes but not limited to at least one of moving speed, steering wheel rotation angle, steering wheel rotation direction, turn signal, accelerator signal and brake signal.

The central control device 23 is connected to the camera devices 21 , 22 and the host computer 24 . The central control device 23 obtains the mobile vehicle information VI from the host computer 24 to determine whether the mobile vehicle 10 is in a straight-going state or a turning state. Since the monitoring methods on the left side and the right side are the same, for the purpose of brevity, only the environment on the left side of the mobile vehicle 10 will be described below. When the central control device 23 judges that the mobile vehicle 10 is in a straight-going state according to the mobile vehicle information VI, the central control device 23 judges whether there is a target object in the first range according to the first image P1 from the photographing device 21, wherein the first One range is the first area 301 . When the central control device 23 judges that the mobile vehicle 10 is in a turning state according to the mobile vehicle information VI, the central control device 23 judges whether there is a target object in the second range larger than the first range according to the first image P1, wherein the second range The second range is the first area 301 plus the second area 302 . In one embodiment, the central control device 23 uses artificial intelligence to identify the target object. In one embodiment, the artificial intelligence model used to identify the target object may be but not limited to MobileNet-SSD.

In the embodiment of FIG. 1 , the central control device 23 includes an image processing unit 231 , a control unit 232 and a communication interface 233 . The image processing unit 231 is connected to the photographing devices 21 and 22 for receiving a plurality of first images P1 and P2 provided by the photographing devices 21 and 22 . The image processing unit 231 identifies target objects in the first images P1 and P2 in an artificial intelligence manner to generate object information CDs. The object information CD represents the recognition result of the target object. In one embodiment, the object information CD includes object type and coordinate information. The coordinate information represents the position or range of the identified target object in the first images P1 and P2.

For example, as shown in FIG. 3 , the image processing unit 231 includes an image processing chip 2311 and an artificial intelligence (AI) chip 2312 . The artificial intelligence chip 2312 is connected to the image processing chip 2311 and the control unit 232 . The image processing chip 2311 is connected to the camera devices 21 , 22 and the control unit 232 . The image processing chip 2311 receives the first images P1 and P2, and performs image processing on the first images P1 and P2 respectively to generate the second images P1' and P2'. The image processing can be, for example, removing noise, downsizing, correcting distortion, or extracting a part of the first image P1 and a part of the first image P2. The artificial intelligence chip 2312 uses artificial intelligence to perform object recognition on the second images P1' and P2' to generate an object information CD. The artificial intelligence chip 2312 can, but is not limited to, utilize convolutional neural networks to implement artificial intelligence.

The communication interface 233 is connected to the host 24 , the display device 25 and the speaker 26 for receiving the mobile vehicle information VI from the host 24 and sending the data to the display device 25 and the speaker 26 . The control unit 232 is connected to the image processing unit 231 and the communication interface 233 . The control unit 232 receives the object information CD and the mobile vehicle information VI from the communication interface 233 to control the operation of the image processing unit 231 . In the embodiment shown in FIG. 1 , the communication interface 233 is further connected to the image processing unit 231 . The image output by the image processing unit 231 is transmitted to the display device 25 via the communication interface 233 .

Fig. 4 shows the first embodiment of the control method of the security system of the present invention. For the purpose of brevity, the following only describes the monitoring of the left side environment of the mobile vehicle 10 . Referring to FIG. 1 to FIG. 4 , in step S10 , the image processing unit 231 receives the first image P1 captured by the photographing device 21 . The range covered by the first image P1 includes a first area 301 and a second area 311 . The photographing operation of the photographing device 21 is continuously performed, so it will continuously output a plurality of first images P1 to the image processing unit 231 . In step S11, the central control device 23 judges whether the mobile vehicle 10 is in a straight-ahead state according to the mobile vehicle information VI. When the central control device 23 determines that the mobile vehicle 10 is in the straight-going state, the security system 20 will proceed to step S13. When the central control device 23 determines that the mobile vehicle 10 is not in the straight-going state, the security system 20 will proceed to step S12. In step S12, the central control device 23 determines whether the mobile vehicle 10 is in a turning state according to the mobile vehicle information VI. When the central control device 23 determines that the mobile vehicle 10 is in a turning state, the safety system 20 will proceed to step S14. When the central control device 23 determines that the mobile vehicle 10 is not in a turning state, the safety system 20 will return to step S10.

In step S13 , the central control device 23 judges whether there is a target object within the first range (the first area 301 ) according to the first image P1 . In one embodiment, the central control device 23 uses artificial intelligence to perform object recognition on the first image P1. When the target object is identified, the central control device 23 obtains the coordinates of the target object. When the central control device 23 judges that the coordinates of the target object are located in the first preset area of the first image P1, the central control device 23 judges that the target object exists in the first range, wherein the first preset area of the first image P1 corresponds to in the first range. When the central control device 23 judges that there is no target object within the first range, it returns to step S10. When the central control device 23 determines that there is a target object within the first range, the security system 20 proceeds to step S15 to remind the user.

In step S14, the central control device 23 determines whether there is a target object within the second range (the first area 301 plus the second area 302). For example, the central control device 23 uses artificial intelligence to perform object recognition on the first image P1. When the target object is identified, the central control device 23 obtains the coordinates of the target object. When the central control device 23 judges that the coordinates of the target object are located in the second preset area of the first image P1, the central control device 23 judges that the target object exists in the second range, wherein the second preset area of the first image P1 corresponds to in the second range. When the central control device 23 judges that there is no target object within the second range, it returns to step S10. When the central control device 23 determines that there is a target object within the second range, the security system 20 proceeds to step S15 to remind the user.

FIG. 5 illustrates an embodiment of calculating the coordinates of the target object in steps S13 and S14. 5 shows the first image P1 obtained by the photographing device 21. The central control device 23 determines the bounding box 27 (Bounding Box; BB) of the target object from the first image P1, and calculates the coordinates of the target object according to the bounding box 27. For example, the central control device 23 may, but not limited to, acquire the coordinates (x+w/2, y+h) of the center point 271 at the bottom of the bounding box 27 as the coordinates of the target object. When the central control device 23 judges that the coordinates (x+w/2, y+h) of the target object are located in the first preset area of the first image P1 (as shown in the hatched area in Figure 5), the central control device 23 judges the first Target object within range.

In an embodiment of step S15, the central control device 23 generates a warning signal and transmits it to the speaker 26 via the communication interface 233. The warning signal is an audio signal, and the speaker 26 emits a warning sound according to the first warning signal to remind user.

In another embodiment of step S15, the central control device 23 performs post-processing on the first image P1, and transmits the post-processed image to the display device 25, so that the post-processed image can be displayed on the display device 25. image to remind the user. Specifically, when it is recognized that the target object is in the first range or the second range, the control unit 232 instructs the image processing chip 2311 to perform post-processing on the first image P1. The image processing chip 2311 transmits the post-processed image to the display device 25 through the communication interface 233 . The post-processing includes but is not limited to distortion correction, marking or superimposing text in images, marking or superimposing images in images, brightness adjustment, color change, image distortion, (partial) image enlargement, (partial) image reduction, image Rotation, image segmentation and image stitching. The display device 25 may be, but not limited to, an electronic rearview mirror.

FIG. 6 shows a second embodiment of the control method of the security system of the present invention. For the purpose of brevity, the following only describes the monitoring of the left side environment of the mobile vehicle 10 . The operations of steps S10, S11, S12, S13 and S15 in FIG. 6 are the same as those in FIG. 4 . In step S14 of FIG. 6, when the control unit 232 of the central control device 23 judges that there is a target object in the second range, proceed to step S16, and when the control unit 232 judges that there is no target object in the first area 301, return to step S10. In step S16, the control unit 232 determines whether there is a non-road area within the second range, and if the non-road area is not included in the second range, proceed to step S15, and if the non-road area is included in the second range, proceed to step S17. The non-road area refers to other areas other than roads, including but not limited to sidewalks. In one embodiment, the control unit 232 can use the artificial intelligence chip 2312 to identify the non-road area with artificial intelligence.

In step S17, the control unit 232 of the central control device 23 determines whether the target object is in a non-road area. Specifically, when the coordinates of the target object acquired by the control unit 232 according to step S14 are located in the area corresponding to the non-road area in the first image P1, the control unit 232 determines that the target object is in the non-road area and proceeds to step S18. If the control unit 232 determines that the target object is not in the non-road area, go to step S15.

In step S18 , the control unit 232 tracks the traveling direction of the target object according to the subsequent plurality of first images P1 . For example, the control unit 232 can determine its traveling direction and judge whether the traveling direction is towards the mobile vehicle 10 according to the continuous multiple coordinates of the target object. When the control unit 232 determines that the target object in the second area 302 is not moving towards the mobile vehicle 10 , return to step S10 . When the control unit 232 determines that the traveling direction of the target object is towards the mobile vehicle 10, it further determines whether the target object is about to leave the off-road area, if yes, proceed to step S15, and if not, return to step S10. In one embodiment, when the distance between the target object and the boundary of the off-road area close to the mobile vehicle is less than or equal to a preset value, the control unit 232 determines that the target object is about to leave the off-road area. In an embodiment, the manner of reminding the user in step S15 may also be different according to step S13 , step S14 or step S18 .

In one embodiment, the target objects can be set as individuals such as pedestrians and two-wheeled vehicles (such as bicycles and locomotives), and the collision between such individuals and mobile vehicles is likely to endanger the safety of life. For the driver of a large vehicle (such as a bus or a truck), the number of warnings can be reduced to avoid distracting the driver's attention because the warning is only directed at individuals who are more likely to endanger their lives. Tired from too many alerts.

In other embodiments, it is also possible to adjust the order of certain steps in FIG. 4 and FIG. 6 . When can understand control method of the present invention from above description, comprise the following steps: Step a: obtaining a first image; Step b: When the mobile vehicle is in a straight-going state, determine whether there is a target object within a first range on the side of the mobile vehicle according to the first image; and Step c: When the mobile vehicle is in a turning state, determine whether there is the target object within a second range on the side of the mobile vehicle according to the first image. Wherein the second range is larger than the first range, and the steps b and c are to use artificial intelligence to identify the target object.

It can be known from the above description that when the mobile vehicle 10 is in the straight-going state, the safety system 20 monitors a smaller first range to achieve faster and more real-time monitoring. When the mobile vehicle 10 is turning, the safety system 20 monitors the larger second range. In other words, the present invention automatically expands the monitoring range when the driving vehicle 10 is turning, which helps to improve safety, especially for large vehicles, such as buses and trucks.

The above description is only an embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as above with the embodiment, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field, Within the scope of not departing from the technical solution of the present invention, when the technical content disclosed above can be used to make some changes or be modified into equivalent embodiments with equivalent changes, but all the content that does not depart from the technical solution of the present invention, according to the technical essence of the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments still fall within the scope of the technical solutions of the present invention.

10: Action Vehicle 11: Wheels 12: Wheels 13: Wheel 14: Wheel 20: Security system 21: Photographic installation 22: Photographic device 23: Central control device 231: Image processing unit 232: control unit 233: Communication interface 24: Host 25: Display device 26: horn 27: Bounding box 271: center point 301: the first area 302: second area 311: the first area 312: second area

Figure 1 shows a mobile vehicle to which the security system of the present invention is applied. Figure 2 shows the monitoring range of the security system in Figure 1. FIG. 3 shows an embodiment of the image processing unit in FIG. 1 . Fig. 4 shows the first embodiment of the control method of the security system of the present invention. FIG. 5 shows an embodiment of obtaining the coordinates of a target object. FIG. 6 shows a second embodiment of the control method of the security system of the present invention.

10: Action Vehicle

11: Wheels

12: Wheels

13: Wheel

14: Wheel

20: Security system

21: Photographic installation

22: Photographic device

23: Central control device

231: Image processing unit

232: control unit

233: Communication interface

24: Host

25: Display device

26: horn

Claims (32)

A security system for a mobile vehicle, comprising: a camera device used to obtain a first image; a host computer used to output information about a mobile vehicle; and a central control device connected to the camera device and the host computer to receive the The first image and the mobile vehicle information; wherein, when the central control device judges that the mobile vehicle is in a straight-going state according to the mobile vehicle information, the central control device judges the side of the mobile vehicle based on the first image Whether there is a target object within a first range; Wherein, when the central control device judges that the mobile vehicle is in a turning state according to the mobile vehicle information, the central control device judges the mobile vehicle according to the first image Whether there is the target object in a second range on the side, the second range is larger than the first range; wherein, the central control device uses artificial intelligence to identify the target object. The safety system for a mobile vehicle as claimed in claim 1, wherein when the central control device determines that there is the target object within the first range or the second range, the central control device generates a warning signal. The safety system of the mobile vehicle as described in claim 1, wherein when the central control device judges that there is the target object within the second range and the target object is located in a non-road area, the central control device according to the subsequent multiple sheets of the The first image tracks the traveling direction of the target object, and generates a warning signal when the target object moves toward the mobile vehicle and the distance from the boundary of the off-road area close to the mobile vehicle is less than or equal to a preset value . The safety system of the mobile vehicle as described in claim 1, wherein when the central control device judges that the mobile vehicle is in a straight-going state, it uses the artificial intelligence to perform a Object identification is used to obtain the coordinates of the target object, and when the central control device determines that the coordinates are located in a first preset area of the first image, the central control device determines that the target object exists in the first range. The safety system of the mobile vehicle as described in claim 4, wherein the central control device uses the artificial intelligence to perform object recognition on the first image to obtain the coordinates of the target object when judging that the mobile vehicle is in a turning state, And when the central control device determines that the coordinates are located in a second preset area of the first image, the central control device determines that the target object exists within the second range. The safety system for a mobile vehicle as claimed in claim 4 or 5, wherein the central control device determines the coordinates of the target object according to the bounding box of the target object. The safety system of the mobile vehicle as claimed in claim 6, wherein the central control device includes the coordinates of the center point of the bottom of the bounding box as the coordinates of the target object. The safety system of the mobile vehicle as described in claim 1, wherein the central control device includes: an image processing unit connected to the camera device, and the image processing unit uses the artificial intelligence to identify the target object in the first image to generate An object information; a communication interface, connected to the host; and a control unit, connected to the image processing unit and the communication interface, the control unit receives the object information, and receives the mobile vehicle information sent by the host through the communication interface ; Wherein, when the control unit judges that the mobile vehicle is in a straight-going state according to the mobile vehicle information, the control unit judges whether there is the target object in the first range according to the object information; wherein, when the control unit judges according to the Mobile vehicle information judges that the mobile vehicle is turning state, the control unit judges whether there is the target object in the second range according to the object information. The safety system for a mobile vehicle as claimed in claim 8, wherein when the control unit determines that there is the target object within the first range or the second range, the control unit generates a warning signal. The safety system of the mobile vehicle as described in claim 8, wherein when the control unit judges that the target object is within the second range and the target object is located in a non-road area, the control unit according to the subsequent multiple pieces of information on the object tracking the direction of travel of the target object, and when the target object moves towards the mobile vehicle and the distance from the boundary of the off-road area close to the mobile vehicle is less than or equal to a preset value, the control unit generates a warning signal . The safety system of the mobile vehicle as described in Claim 9 or 10 further includes a speaker connected to the central control device, and the speaker plays a warning sound according to the warning signal. The safety system of the mobile vehicle as described in claim 8, wherein the image processing unit includes: an image processing chip, receives the first image, and performs image processing on the first image to generate a second image; and an artificial A smart chip is connected to the image processing chip and the control unit, and the artificial intelligence chip uses the artificial intelligence to perform object recognition on the received second image to generate the object information. The security system for mobile vehicles as claimed in claim 12, wherein the artificial intelligence chip includes a convolutional neural network for realizing the artificial intelligence. The safety system of the mobile vehicle as described in Claim 8, wherein the control unit controls the image processing unit to perform post-processing on the first image according to the object information, and the post-processing includes distortion correction, labeling or superimposing text, Annotate or overlay images, adjust brightness, Color change, image distortion, image enlargement, image reduction, image rotation, image segmentation or image stitching. The safety system of the mobile vehicle as described in claim 14 further includes a display device connected to the central control device through the communication interface to display the post-processed image. The safety system for a mobile vehicle as claimed in claim 15, wherein the display device includes an electronic rearview mirror. The safety system for a mobile vehicle as described in claim 1, wherein the mobile vehicle information includes at least one of moving speed, steering wheel rotation angle, steering wheel rotation direction, turn signal, accelerator signal, and brake signal. The safety system for a mobile vehicle as claimed in claim 1, wherein the target object includes at least one of pedestrians, bicycles and motorcycles. A method for controlling a security system of a mobile vehicle, comprising the following steps: a. Obtaining a first image through a photographic device; b. When the mobile vehicle is in a straight-going state, a central control device judges the Whether there is a target object in a first range on the side of the mobile vehicle; and c. When the mobile vehicle is in a turning state, the central control device judges a target object on the side of the mobile vehicle according to the first image Whether there is the target object within the second range; wherein, the second range is larger than the first range; wherein, the steps b and c are to use artificial intelligence to identify the target object. The control method of the safety system of the mobile vehicle as described in claim 19 further comprises that the central control device generates a warning signal when the central control device judges that the target object is within the first range or the second range. The method for controlling the security system of a mobile vehicle as described in claim 19 further includes: when it is determined that the target object is within the second range and the target object is located in a non-road area, The central control device tracks the traveling direction of the target object according to the subsequent multiple first images; and when it is judged that the traveling direction is toward the mobile vehicle and the distance from the boundary of the non-road area close to the mobile vehicle is less than or When equal to a preset value, the central control device generates a warning signal. The control method of the safety system of the mobile vehicle as described in claim 20 or 21 further includes a speaker playing a warning sound according to the warning signal. The control method of the safety system of the mobile vehicle as described in claim 19, wherein the step b includes: using the artificial intelligence to perform object recognition on the first image to obtain the coordinates of the target object; and when the coordinates are located at the second When a predetermined area of an image is selected, the central control device determines that the target object exists in the first area. The control method of the security system of the mobile vehicle as described in claim 19, wherein the step c includes: using artificial intelligence to perform object recognition on the first image to obtain the coordinates of the target object; and when the coordinates are located at the first When a predetermined area of the image is selected, the central control device determines that the target object exists within the second area. The control method of the security system of the mobile vehicle according to claim 23 or 24, wherein the step of obtaining the coordinates of the target object includes the central control device determining the coordinates of the target object according to the bounding box of the target object. The control method of the safety system of the mobile vehicle as described in claim 25, further comprising that the central control device uses the coordinates of the center point at the bottom of the bounding box as the position of the target object mark. The control method of the safety system of the mobile vehicle as described in claim 19 further includes: using the artificial intelligence to identify the target object in the first image to generate object information; when judging that the mobile vehicle is in a straight-going state, The central control device judges whether there is the target object in the first range according to the object information; and when judging that the mobile vehicle is in a turning state, the central control device judges whether there is the target object in the second range according to the object information . The control method of the security system of a mobile vehicle as described in claim 27, wherein the step of generating an object information includes: performing image processing on the first image by an image processing chip to generate a second image; and using the manual The intelligence performs object recognition on the second image to generate the object information. The control method of the safety system of the mobile vehicle as described in claim 19, further comprising that the central control device judges that the mobile vehicle is in the straight-going state or the turning state according to a piece of mobile vehicle information, wherein the mobile vehicle information includes At least one of moving speed, steering wheel rotation angle, steering wheel rotation direction, direction light signal, accelerator signal and brake signal. The control method of the safety system of the mobile vehicle as described in claim 19, further comprising the central control device performing post-processing on the first image in response to the result of identifying the target object, wherein the post-processing includes distortion correction, labeling Or superimpose text, label or superimpose images, brightness adjustment, color change, image distortion, image enlargement, image reduction, image rotation, or image segmentation and image stitching. The control method of the security system of the mobile vehicle as described in Claim 30 further includes displaying the post-processed image through a display device. The control method of the safety system of the mobile vehicle as claimed in claim 19, wherein the target object includes at least one of pedestrians, bicycles and motorcycles.

Images