JP2015133089A - Road sign recognition apparatus and road sign recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a road sign recognition apparatus capable of appropriately communicating a road sign to a passenger.SOLUTION: A road sign recognition apparatus 100 comprises: an image recognition unit 14 detecting a road sign by executing an image recognition process on a picked-up image picked up by imaging a front side of a vehicle by an imaging unit 12 provided in the vehicle; a determination unit 32 determining the degree of importance of the road sign; and a display unit 44 capable of displaying at least two display contents side by side on a screen. If at least two road signs are detected, the first road sign out of the at least two road signs has a first degree of importance higher than a second degree of importance, and the second road sign out of the at least two road signs has the second degree of importance, then the first road sign is always displayed in a predetermined region fixed to a screen as a first display content out of at least two display contents and the second road sign is displayed in the other region of the screen as a second display content out of the at least two display contents at necessary timing.

Description

  The present invention relates to a device for recognizing a road sign (a road sign recognition device) and a method for recognizing a road sign (a road sign recognition method). In particular, the road sign is caused by a captured image.

  For example, Patent Document 1 discloses a vehicular display device, and the vehicular display device can display display contents on a head-up display, an in-meter display, and a center display according to the order of importance.

  Next, for example, Patent Document 2 discloses a navigation device, and the navigation device can display a plurality of road signs.

JP 2008-168799 A US Patent Application Publication No. 2012/0223845

  However, conventional devices or devices such as the vehicle display device disclosed in Patent Document 1 and the navigation device disclosed in Patent Document 2 are difficult for a passenger such as a vehicle driver to recognize a plurality of display contents or road signs. In other words, the inventors have recognized that conventional devices or devices cannot adequately display the road signs that are most needed for the driver.

  One object of the present invention is to provide a road sign recognition device capable of appropriately transmitting road signs to passengers. Other objects of the present invention will become apparent to those skilled in the art by referring to the aspects and best embodiments exemplified below and the accompanying drawings.

  In the following, in order to easily understand the outline of the present invention, embodiments according to the present invention will be exemplified.

In the first aspect, the road sign judgment device comprises:
An image recognition unit that detects a road sign by performing image recognition processing on a captured image in which the front of the vehicle is captured by an imaging unit provided in the vehicle;
A determination unit for determining the importance of the road sign;
A display unit capable of displaying at least two display contents side by side on the screen;
With
At least two road signs are detected, a first road sign of the at least two road signs has a first importance higher than a second importance, and the at least two road signs When the second road sign has the second importance, the road sign determination device displays the first road sign on the screen as the first display content of the at least two display contents. While always displaying in the predetermined area | region fixed, the said 2nd road sign is displayed on the other area | region of the said screen as a 2nd display content of the said at least 2 display content at a required timing.

  In the first aspect, a road sign having a high importance (first road sign) is always displayed in a predetermined area fixed on the screen, while another road sign (a second sign having no high importance). Is not displayed in a predetermined area fixed on the screen, but is displayed in another area of the screen. Therefore, a passenger such as a vehicle driver can easily recognize a road sign (first road sign) having high importance. Moreover, the passenger | crew can recognize the other road sign (2nd road sign) displayed along with the 1st road sign as needed. As described above, the road sign recognition device can appropriately transmit the road sign (including the first and second road signs) to a passenger such as a driver.

  If the position at which a road sign having high importance is displayed is changed, it is difficult for the occupant to predict where the road sign is displayed on the screen.

In a second aspect dependent on the first aspect,
When the next first road sign is detected, the current first road sign is replaced with the next first road sign, and the next first road sign is displayed in the predetermined area. May be,
Each of the current first road sign and the next first road sign may be a speed sign or a stop sign.

  In the second aspect, when the next road sign (the next first road sign) having high importance is detected, the next road sign (the next first road sign) is displayed in a predetermined area. The Here, each of the current road area (current first road sign) and the next road sign (next first road sign) having high importance is a speed sign or a stop sign. Therefore, when the current speed sign (current first road sign) and the next speed sign (next first road sign) indicate, for example, 50 [km / h] and 70 [km / h], respectively. In the predetermined area fixed to the screen, 70 [km / h] is displayed instead of 50 [km / h]. In this way, the road sign recognition device continues to display the latest speed sign in a predetermined area, so that, for example, the driver can easily recognize the speed sign. This allows the driver to drive the vehicle so that the vehicle speed is applied to the speed indicator.

  When a stop sign is displayed in a predetermined area fixed on the screen, the driver can drive the vehicle so that the vehicle temporarily stops. In other words, since the stop sign substantially indicates, for example, 0 [km / h], the driver can drive the vehicle so that the speed of the vehicle decreases to 0 [km / h]. .

In a third aspect subordinate to the first or second aspect,
The second road sign may include a speed sign resulting from weather.

  In the third aspect, the speed sign (second road sign) resulting from the weather is displayed in another area of the screen. That is, the screen displays a high-significance road sign (first road sign) such as a speed sign or a stop sign, for example, and the speed caused by the weather as another sign (second road sign). A sign can be displayed. When a speed sign due to the weather is detected, the driver can recognize the speed sign due to the weather displayed in another area of the screen. Accordingly, the driver can drive the vehicle more safely while considering the weather when driving the vehicle.

In a fourth aspect depending on any one of the first to third aspects,
The second road sign may include a prohibition sign that prohibits a predetermined state of the vehicle and a caution sign that alerts a passenger of the vehicle, and the second road sign in the other area The display may continue for a predetermined period of time,
When at least three road signs are detected and a third road sign of the at least three road signs has a third importance lower than the first importance; and The third road sign may be sequentially displayed in the other area.

  In the fourth aspect, when a prohibition sign that prohibits a predetermined state of the vehicle (for example, an entry prohibition sign, a parking prohibition sign, an overtaking prohibition sign) is detected, the driver is displayed in another area of the screen. A prohibition sign (second road sign) can be recognized. Further, when a warning sign (for example, an animal jumping attention sign or a child jumping attention sign) that calls attention to a vehicle occupant is detected, the driver can display the warning sign (second sign) displayed in another area of the screen. Road sign).

  In particular, the driver may drive the vehicle so that the vehicle does not enter a predetermined state, for example, so that the vehicle does not enter the road corresponding to the entry prohibition. The display of the prohibition sign (second road sign) such as the entry prohibition sign may be continued for a predetermined period so that the sign) can be recognized. Similarly, the display of the caution sign (second road sign) only needs to be continued for a predetermined period so that the driver can recognize the caution sign (second road sign).

  It should be noted that prohibition signs (second road signs) such as parking prohibition signs and overtaking prohibition signs are generally effective for long distances, so that the predetermined period can be set longer. In other words, the predetermined period may be a fixed period, but can be set for each other road sign (second road sign not having high importance).

  In addition, when there are a plurality of other road signs (second and third road signs that do not have high importance) to be detected, the road sign recognition device recognizes other road signs (having high importance). Second and third road signs) can be sequentially displayed in other areas of the screen. As a result, other road signs (second and third road signs not having high importance) are not displayed at the same time, and the driver can make other road signs (second and second signs not having high importance). It is easier to recognize the third road sign).

In the fifth aspect, the road sign recognition method comprises:
Detecting at least two road signs by performing image recognition processing on each of at least two captured images in which the front of the vehicle is imaged by an imaging unit provided in the vehicle;
When the first road sign of the at least two road signs is a speed sign or a stop sign, the first road sign is always displayed in a predetermined area fixed on a screen; and the at least two When a second road sign of the two road signs is different from the first road sign, the second road sign is displayed along with the first road sign at a necessary timing in another area of the screen. about,
Is provided.

  In the fifth aspect, the speed sign or the stop sign (first road sign) is always displayed in a predetermined area fixed on the screen, while the other road sign (second different from the first road sign) Is not displayed in a predetermined area fixed on the screen, but is displayed in another area of the screen. Therefore, an occupant such as a vehicle driver can easily recognize the speed sign or the stop sign (first road sign). Moreover, the passenger | crew can recognize the other road sign (2nd road sign) displayed along with the 1st road sign as needed. As described above, the road sign recognition method can appropriately transmit road signs (including the first and second road signs) to passengers such as drivers.

  In particular, when the speed indicator is displayed in a predetermined area, for example, the driver can easily recognize the speed indicator. This allows the driver to drive the vehicle so that the vehicle speed is applied to the speed indicator. In addition, when a stop sign is displayed in a predetermined area fixed on the screen, the driver can drive the vehicle so that the vehicle temporarily stops. Therefore, the driver can drive the vehicle more safely.

  Those skilled in the art will readily understand that the illustrated embodiments according to the present invention can be further modified without departing from the spirit of the present invention.

FIG. 1A shows a configuration example of a road sign recognition apparatus according to the present invention, and FIG. 1B shows a display example of a display unit. 2A, 2B, 2C, and 2D are explanatory diagrams of a series of screens (first operation example) that change when the vehicle travels in the first section. Indicates. 3 (A), 3 (B), 3 (C), and 3 (D) show another series of screens that change when the vehicle travels in the first section (second operation example). FIG. 4A and 4B are explanatory diagrams of a series of screens (third operation example) that change when the vehicle travels in the second section. The flowchart showing operation | movement of the road sign recognition apparatus of FIG. 1 (A) is shown. 6 (A), 6 (B), 6 (C), 6 (D), 6 (E), and 6 (F) can be recognized by the road sign recognition apparatus of FIG. 1 (A). Multiple road signs.

  The best mode described below is used to easily understand the present invention. Accordingly, those skilled in the art should note that the present invention is not unduly limited by the embodiments described below.

  FIG. 1A shows a configuration example of a road sign recognition apparatus according to the present invention, and FIG. 1B shows a display example of a display unit. As shown in FIG. 1A, the road sign recognition device 100 includes an image recognition unit 14 and a determination unit 32. Moreover, the road sign recognition apparatus 100 in FIG. 1A can further include an imaging unit 12 and a display unit 44. Of course, the road sign recognition apparatus 100 may not include some of these components, and the road sign recognition apparatus 100 may further include other components not shown. In addition, the road sign recognition apparatus 100 in FIG. 1A can input information or data on the in-vehicle network 50, or can output information or data on the in-vehicle network 50.

  The imaging unit 12 in FIG. 1A is provided in a vehicle. Here, the imaging unit 12 is arranged, for example, indoors of the vehicle, and can generate a captured image that captures the front (traveling direction) of the vehicle. Of course, the imaging unit 12 may be provided in front of an indoor mirror (rear view mirror), and the imaging unit 12 may be a shared imaging unit used for a drive recorder, for example. In addition, the imaging unit 12 may be disposed outside the vehicle, for example, outdoors, and the imaging unit may be provided, for example, at the front of the front grill.

  In this way, the imaging unit 12 configured with a camera, for example, is arranged at an arbitrary position of the vehicle so that the imaging unit 12 can image the front (traveling direction) of the vehicle. The imaging unit 12 preferably includes an imaging sensor (imaging device) such as a CCD or CMOS, and generates a digital captured image. The imaging unit 12 includes an imaging lens that forms an image of a subject (for example, a road sign located in front of the vehicle) on the imaging surface of the imaging sensor. The position of the imaging lens is automatically set by, for example, an autofocus motor. It is preferably driven or adjusted. In addition, the imaging unit 12 may continue to output a real-time captured image to the image recognition unit 14, for example, or may output the captured image to the image recognition unit 14 regularly or irregularly at a predetermined timing. May be.

  The image recognition unit 14 in FIG. 1A inputs, for example, a digital captured image and executes image recognition processing on the captured image. The image recognition unit 14 can detect whether a predetermined pattern such as a character or a graphic exists in the captured image and detect a road sign. The image recognition unit 14 in FIG. 1A detects, for example, a road regulation sign among the road signs, and the road regulation sign prohibits the vehicle from a specific traffic method or causes the vehicle to pass according to the specific method. Is designated as a vehicle. The road regulation sign includes, for example, a speed sign that is a regulation speed, and the regulation speed can for example prohibit the vehicle from traveling beyond the displayed speed (maximum speed). As described above, the image recognition unit 14 can detect a road regulation sign such as a regulation speed from the captured image and output the road regulation sign to the determination unit 32.

  The determination unit 32 in FIG. 1A determines the importance of, for example, the regulation speed (first road sign) detected by the image recognition unit 14. In other words, for example, when a regulated speed is detected, the importance of the regulated speed is high, preferably the highest, and the regulated speed (first road sign) is always displayed in a predetermined area fixed on the screen. The As shown in FIG. 1B, the display unit 44 displays, for example, a regulated speed of 40 [km / h] as display content on the right half of the screen. For example, it is preferable that the display content indicating the regulated speed of 40 [km / h] is the same (similar shape) or similar characters, figures, and the like as the road signs actually installed on the road (see FIG. 1 (B)), the driver can more easily recognize the regulation speed. Of course, the display content may be only a character representing a number (for example, “40”), or may be a character representing a number and an explanatory note (for example, “40 (maximum speed)”).

  By the way, the regulation speed may change depending on the weather. That is, for example, when it is raining, the regulation speed (maximum speed) may be set small. Accordingly, when the regulated speed (second road sign) caused by the weather is detected by the image recognition unit 14, the display unit 44 displays the regulated speed caused by the weather as display content on the left half of the screen, for example. Can do. Thus, for example, when it is not raining, the driver can recognize the current regulation speed in a predetermined area (for example, the right half) fixed on the screen, while for example, when it is actually raining, The current regulation speed can be recognized in other areas of the screen (for example, the left half). Accordingly, the driver can drive the vehicle more safely while considering the weather when driving the vehicle.

  A road sign (second road sign) having an importance level lower than that of a regulated speed (first road sign) is not only a speed sign that is a regulated speed caused by weather, for example, but a predetermined state of the vehicle. It is also preferable to include a prohibition sign (for example, an entry prohibition sign, a parking prohibition sign, an overtaking prohibition sign) to be prohibited. As shown in FIG. 1B, the display unit 44 can display, for example, prohibition of overtaking as display contents on the left half of the screen. The display content indicating the overtaking prohibition is preferably the same (similar shape) or similar character, figure, or the like as the road sign actually installed on the road. The display content (road sign) displayed on the screen is preferably adapted to the country where the vehicle exists or the country where the vehicle is manufactured or imported.

  Further, a road sign (second road sign) having an importance level lower than that of the regulated speed (first road sign) is a warning sign (for example, an animal jumping warning sign, a child jumping out) that alerts the vehicle occupant. It is preferable to further include a warning sign). The driver can drive the vehicle more safely while recognizing a warning sign (second road sign) displayed in another area (for example, the left half) of the screen.

  The display unit 44 in FIG. 1A includes, for example, a display, and the display unit 44 can have a screen. Further, the display (display unit 44) may be a common display for displaying a meter arranged on the instrument panel, for example. That is, an area which is a part of the display screen for displaying the meter may be configured as a screen as shown in FIG. Alternatively, the display (display unit 44) may be a dedicated display. In addition, the display part 44 may be comprised with the projector of a head-up display, for example, and the display part 44 does not need to have the screen which is a windshield (windshield), for example.

  The screen of the display unit 44 has at least two areas, and one of the at least two areas is a predetermined area (first road sign) fixed to the screen. The other of the at least two areas is also preferably a predetermined area (second road sign) fixed on the screen, so that the driver can further recognize the second road sign. Easy to do. For example, as shown in FIG. 1B, the screen of the display unit 44 may further include an area for displaying the current time, the current temperature, the travel distance (for example, a trip meter), and the like.

  Moreover, the road sign recognition apparatus 100 in FIG. 1A may constitute, for example, a navigation apparatus. In other words, the processing unit of the navigation device may have the function of the determination unit 32, for example. Alternatively, the processing unit of the navigation device connected to the imaging unit 12 may have the function of the image recognition unit 14 and the function of the determination unit 32, for example. In addition, the road sign recognition apparatus 100 may constitute a drive recorder, for example. In other words, the imaging unit 12 may be a drive recorder camera.

  2A, 2B, 2C, and 2D are explanatory diagrams of a series of screens (first operation example) that change when the vehicle travels in the first section. Indicates. In FIG. 2A, when the vehicle 10 travels on the first section of the road, the vehicle 10 is positioned in front of the road sign m1. That is, before the vehicle 10 passes the next road sign m1 (first road sign), the vehicle 10 passes a current road sign (first road sign) (not shown). For example, a regulated speed of 50 [km / h] is displayed as display content on the right side (see FIG. 2A).

  Next, in FIG. 2B, the vehicle 10 passes the road sign m1. At this time, the display unit 44 displays, for example, a regulated speed of 70 [km / h] as display content on the right side of the screen (see FIG. 2B). When the current speed sign (current first road sign) and the next speed sign (next first road sign) indicate, for example, 50 [km / h] and 70 [km / h], respectively, Instead of 50 [km / h], 70 [km / h] is displayed in the predetermined area fixed to (see FIG. 2B). In this way, the display unit 44 can always display a road sign (first road sign) having high importance in a predetermined area fixed on the screen. Further, the road sign recognition apparatus 100 can continue to display the speed sign (first road sign) that is the latest regulated speed in a predetermined area. Therefore, an occupant such as a driver of the vehicle 10 can easily recognize a road sign (first road sign) having high importance.

  Next, in FIG. 2C, the vehicle 10 passes the road sign m2 (second road sign not having high importance). At this time, the display unit 44 displays, for example, overtaking prohibition as display contents on the left side of the screen (see FIG. 2C). The other road sign (second road sign) is not displayed in a predetermined area fixed on the screen, but is displayed in another area of the screen. In other words, overtaking prohibition is not displayed in place of 70 [km / h] in a predetermined area fixed on the screen. Therefore, an occupant such as a driver of the vehicle 10 can easily recognize a road sign (first road sign) that is always displayed in a predetermined area fixed on the screen and has high importance. Moreover, the passenger | crew can recognize another road sign (2nd road sign) as needed. As described above, the road sign recognition apparatus 100 can appropriately transmit the road signs (first and second road signs) to a passenger such as a driver.

  In addition, the display unit 44 can display the first and second road signs side by side on the screen (see FIG. 2C). In other words, the second road sign is displayed at a necessary timing next to the first road sign that is always displayed. Therefore, the road sign recognition apparatus 100 can more appropriately transmit the road sign (including the first and second road signs) to a passenger such as a driver. In FIG. 2C, the display unit 44 displays the second road sign (passing prohibition) on the left side of the first road sign (regulated speed of 70 [km / h]). The unit 44 may display the second road sign on the right side, upper side, lower side, etc. of the first road sign.

  Next, in FIG. 2D, the vehicle 10 passes the road sign m3 (second road sign not having high importance). Here, the display unit 44 displays, for example, a regulated speed of 50 [km / h] as display content when it rains on the left side of the screen (see FIG. 2D). The other road sign (second road sign) is not displayed in a predetermined area fixed on the screen, but is displayed in another area of the screen. In other words, in the other area of the screen, a regulated speed of 50 [km / h] (the latest second road sign) is displayed when it rains instead of prohibiting overtaking. Thus, the display unit 44 can display, for example, two road signs (first and second road signs) at the maximum.

  3 (A), 3 (B), 3 (C), and 3 (D) show another series of screens that change when the vehicle 10 travels in the first section (second operation example). FIG. Similar to FIGS. 2A and 2B, the display unit 44 in FIGS. 3A and 3B has, for example, 50 [km / h] and 70 [km / h] on the right side of the screen, respectively. h] is displayed as the display content. Similarly to FIG. 2C, the display unit 44 of FIG. 3C displays, for example, overtaking prohibition as display contents on the left side of the screen.

  However, in FIG. 3C, the longest displayable period T1 of the road sign m2 (second road sign not having high importance) is set by, for example, the determination unit 32 of FIG. The longest displayable period T1 is, for example, 5 [min]. Since the road sign m2 (second road sign), which is a prohibition sign indicating prohibition of overtaking, is generally effective for a long distance, the longest displayable period T1 (predetermined period) can be set long. If there is no road sign m3 (third road sign not having high importance), for example, the display unit 44 may display, for example, 5 [min] from the time when the vehicle 10 passes the road sign m2. Until the time elapses, the display of overtaking prohibition (second road sign) can be continued.

  Of course, the display unit 44 displays the road sign m2 before the vehicle 10 passes the road sign m2 (for example, when the vehicle 10 passes 20-30 [m] in front of the road sign m2, for example, FIG. ), The display of the overtaking prohibition (second road sign) may be started. In particular, when the second road sign is a stop sign, an entry prohibition sign, or the like, the driver of the vehicle 10 can easily recognize the display of the second road sign that starts early or in advance. Or after the vehicle 10 passes the road sign m2 of FIG. 3C, for example (for example, when the road sign m2 is no longer detected by the image recognition unit 14 of FIG. 1A, for example). Display of prohibition of overtaking (second road sign) may be started. Moreover, the determination part 32 does not need to set the longest displayable period T1, and the display part 44 is, for example, until the vehicle 10 passes the road sign m3 (third road sign not having high importance). The display of overtaking prohibition (second road sign) may be continued only for a while.

  Next, in FIG. 3D, the vehicle 10 passes the road sign m3 (third road sign not having high importance). The display unit 44 can display, for example, three road signs (first, second, and third road signs). Accordingly, the display unit 44 alternately displays, for example, 50 [km / h] speed limit (third road sign) and overtaking prohibition (second road sign) on the left side of the screen when it rains. Specifically, when it rains, the display of the regulated speed (third road sign) of 50 [km / h] is continued for the continuation period t1 (predetermined period), and then overtaking is prohibited (second road) The display of the sign is continued for the duration t2 (predetermined period). After that, when it rains, the display of the regulated speed (third road sign) of 50 [km / h] is again continued for the continuation period t3 (predetermined period).

  For example, when two road signs m2 and m3 (second and third road signs) are alternately displayed on the left side of the screen, each of the durations t1, t2 and t3 (predetermined periods) is the same period. There may be. In addition, the display content (for example, 50 [km / h] regulation speed when it rains) initially displayed on the left side of the screen may be emphasized. As a result, the driver of the vehicle 10 can more easily recognize the display content (for example, the third road sign) initially displayed on the left side of the screen. Specifically, the duration t1 may be set longer than the duration t3. Or the display content (for example, 3rd road sign) corresponding to continuation period t1 may be temporarily displayed large.

  In FIG. 3D, the longest displayable period T2 of the road sign m3 (third road sign not having high importance) is set by, for example, the determination unit 32 in FIG. The longest displayable period T2 corresponds to, for example, the time when the vehicle 10 travels 220 [m]. As an example, the determination unit 32 can calculate a time required for the vehicle 10 to travel 220 [m] at a predetermined speed. Alternatively, the determination unit 32 inputs travel distance information from, for example, a meter ECU (not shown) via the in-vehicle network 50, for example, and when the vehicle 10 actually travels by 220 [m], the determination unit 32 determines the end point of the longest displayable period T2. It may be set. Of course, each of the longest displayable periods T2 and T1 may be the same period. Further, the determination unit 32 may not set the longest displayable period T2.

  In FIG. 3D, until the time corresponding to the end point of the longest displayable period T2, the display unit 44, for example, regulates 50 [km / h] when it rains (third road sign) and, for example, overtaking prohibited (Second road sign) can be alternately displayed on the left side of the screen. Thereafter, the display unit 44 can end the display of the regulated speed (third road sign) of 50 [km / h], for example, when it rains. Further, until the time corresponding to the end point of the longest displayable period T1, the display unit 44 can display, for example, overtaking prohibition (second road sign) on the left side of the screen, and then the regulation speed (second road) The display of the sign) can be terminated.

  FIG. 4A and FIG. 4B are explanatory diagrams of a series of screens (third operation example) that change when the vehicle 10 travels in the second section. Here, for example, the second section in FIG. 4 (A) follows the first section in FIG. 3 (D), for example. However, in FIGS. 4A and 4B, the longest displayable periods T1 and T2 are not set. Alternatively, in FIGS. 4A and 4B, the vehicle 10 or the determination unit 32 determines both the time corresponding to the end point of the longest displayable period T1 and the time corresponding to the end point of the longest displayable period T2. Waiting for the arrival of

  4A, the display unit 44 can display, for example, three road signs (first, second, and third road signs). For example, two road signs m2, m3 are displayed on the left side of the screen. (Second and third road signs) are displayed alternately. Each of the continuous periods t1, t2, t3, and t4 (predetermined periods) in FIGS. 3D and 4A may be the same period. For example, the display content in which two road signs m2 and m3 (second and third road signs) are alternately displayed first may be emphasized. Specifically, for example, the duration t1 may be set longer than the duration t3, and the duration t2 may be set longer than the duration t4.

  Next, in FIG. 4B, the vehicle 10 passes the road sign m4 (fourth road sign not having high importance). The display unit 44 can display, for example, four road signs (first, second, third, and fourth road signs). Therefore, on the left side of the screen, the display unit 44 displays, for example, a regulated speed of 30 [km / h] (fourth road sign) when snow falls, and a regulated speed of 50 [km / h] (third) when it rains. Road sign) and overtaking prohibition (second road sign) are sequentially displayed. Specifically, the display of the regulated speed (fourth road sign) of 30 [km / h] when snowing continues for a continuation period t5 (predetermined period). After that, when it rains, the display of the regulated speed (third road sign) of 50 [km / h] and the prohibition of overtaking (second road sign) are the continuation period t3 (predetermined period) and continuation, respectively. It continues for a period t4 (predetermined period). After that, when snow falls, the display of the regulated speed (fourth road sign) of 30 [km / h] is continued again for the continuation period t6 (predetermined period).

  Each of the continuous periods t3, t4, t5, and t6 (predetermined periods) in FIG. 4B may be the same period. Note that, for example, when the snow falls, the display content on which the regulated speed (fourth road sign) of 30 [km / h] is initially displayed may be emphasized. Specifically, for example, the duration t5 may be set longer than the duration t6.

  FIG. 5 is a flowchart showing the operation of the road sign recognition apparatus 100 in FIG. The image recognition unit 14 in FIG. 1A performs image recognition processing on the captured image captured by the imaging unit 12 to detect a road sign, while the determination unit 32 determines that the detected road sign is, for example, a restriction. It is determined whether or not the speed (maximum speed), that is, a speed indicator (step ST01 in FIG. 5). When the detected road sign is a speed sign, the determination unit 32 sets the importance of the road sign (speed sign) high. Here, the number of importance levels is, for example, two (high or low), and the importance level of a road sign (speed sign) is, for example, high or fixed to “1”, for example.

  When the detected road sign is not a speed sign, the determination unit 32 determines whether or not the detected road sign is a stop sign (step ST01 in FIG. 5). When the detected road sign is a stop sign, the determination unit 32 sets the importance of the road sign (pause sign) high. Here, the importance of the road sign (temporary stop sign) is also high, for example, or fixed to “1”, for example. Since the stop sign substantially indicates, for example, 0 [km / h], the road sign having a high importance (first road sign) includes not only the speed sign but also the stop sign.

  Step ST01 in FIG. 5 is repeated until a road sign having a high importance (first road sign) is first detected. In other words, for example, after the engine of the vehicle 10 or the road sign recognition device 100 is started, the display unit 44 is fixed to the screen until a road sign (first road sign) having high importance is detected. Nothing is displayed in the predetermined area. On the other hand, when a road sign (first road sign) having high importance is detected in step ST01 of FIG. 5, the display unit 44 displays a speed sign (for example, FIG. A)))) or a stop sign is displayed (step ST02 in FIG. 5).

  Next, the determination unit 32 determines whether or not the next road sign is detected (step ST03 in FIG. 5). In other words, after the captured image captured by the imaging unit 13 does not include the road sign detected in step ST01, the determination unit 32 determines whether a new road sign is detected (step ST03 in FIG. 5). ). After the next road sign or a new road sign is detected, the determination unit 32 determines whether or not the detected road sign is a speed sign or a stop sign (step ST04 in FIG. 5).

  When a road sign (first road sign) having high importance is detected in step ST04 in FIG. 5, the display unit 44 displays a speed sign (for example, FIG. 3B) in a predetermined area fixed on the screen. Refer to))) or a stop sign (step ST05 of FIG. 5). Next, the determination unit 32 can determine whether or not to stop the display on the display unit 44, in other words, for example, whether or not the ignition switch of the vehicle 10 is turned off (step ST06 in FIG. 5).

  When the display on the display unit 44 is not stopped, the road sign recognition apparatus 100 executes steps ST03 and 04 again. In step ST04 of FIG. 5, the new road sign is not only a road sign having a high importance (first road sign) but also a road sign not having a high importance (for example, second to fourth road signs). Contains. In FIG. 5, the road sign recognition apparatus 100 can display, for example, four road signs (first to fourth road signs), but the road sign recognition apparatus 100 can display, for example, steps ST07, ST09, ST10. , ST11 may be omitted, for example, only two road signs (first to second road signs) may be displayed. Alternatively, the road sign recognition apparatus 100 may add a plurality of steps, for example, between step ST09 and step ST11 without limiting the number of road signs displayed on the screen, for example.

  In step ST04 in FIG. 5, a road sign (first road sign) having high importance is not detected, and road signs (for example, second to fourth road signs) having no high importance are detected. Sometimes, the determination unit 32 can determine the number of detected road signs (for example, second to fourth road signs). For example, when the number of detected road signs (for example, the second to fourth road signs) indicates 1, only the second road sign is detected, and the third road sign is not detected. Accordingly, the display unit 44 can display the second road sign (for example, see FIG. 3C)) in another area of the screen (steps ST07 and ST08 in FIG. 5).

  Next, when the number of detected road signs (for example, the second to fourth road signs) indicates 2, for example, only the second and third road signs are detected, and the fourth road sign is not detected. . Therefore, the display unit 44 can sequentially display the second and third road signs (for example, see FIG. 3D) in other areas of the screen (steps ST09 and ST10 in FIG. 5). Next, when the number of detected road signs (for example, second to fourth road signs) indicates, for example, 3, the second, third, and fourth road signs are detected. Accordingly, the display unit 44 can sequentially display the second, third, and fourth road signs (for example, see FIG. 4B) in other areas of the screen (step ST11 in FIG. 5).

  6A, 6B, 6C, 6D, 6E, and 6F are recognized by the road sign recognition apparatus 100 of FIG. 1A. Shows several possible road signs. As shown in FIG. 6A, the image recognition unit 14 in FIG. 1A has a speed sign (first road sign) indicating a regulated speed (maximum speed) of 40 [km / h], for example, A prohibition sign (second road sign) indicating prohibition of overtaking can be recognized. Thus, for example, the first road sign and the second road sign may be detected substantially simultaneously.

  Further, as shown in FIG. 6B, the image recognition unit 14 in FIG. 1A performs, for example, a temporary stop sign (first road sign) indicating “stop” and a prohibition sign (for example, parking prohibition). Second road sign) can be recognized. In addition, the image recognition unit 14 may also detect, for example, an auxiliary sign among the road signs, and recognize a set of road signs indicating the end of a regulated speed (maximum speed) of, for example, 40 [km / h]. it can. In other words, the image recognition unit 14 more appropriately recognizes a temporary stop sign (first road sign) indicating, for example, “stop” in consideration of, for example, road regulation signs and auxiliary signs among the road signs. Can do.

  Further, road signs that do not have high importance (for example, second to fourth road signs) are, for example, prohibition signs (FIG. 6C) that indicate vehicle entry prohibition signs, for example, warning signs that indicate that animals may jump out. There is a warning sign or an animal jumping out warning sign (FIG. 6 (D)), for example, an attention sign indicating a pedestrian crossing or a child jumping out warning sign (FIG. 6 (E)), such as a school, kindergarten, nursery, etc. A warning sign indicating a warning sign or a child jumping out warning sign (FIG. 6F) can be included.

  The present invention is not limited to the above-described exemplary embodiments, and those skilled in the art will be able to easily modify the above-described exemplary embodiments to the extent included in the claims. .

  DESCRIPTION OF SYMBOLS 10 ... Vehicle, 12 ... Imaging part, 14 ... Image recognition part, 32 ... Judgment part, 44 ... Display part, 50 ... In-vehicle network, 100 ... Road sign recognition apparatus .

Claims (5)

An image recognition unit that detects a road sign by performing image recognition processing on a captured image in which the front of the vehicle is captured by an imaging unit provided in the vehicle;
A determination unit for determining the importance of the road sign;
A display unit capable of displaying at least two display contents side by side on the screen;
With
At least two road signs are detected, a first road sign of the at least two road signs has a first importance higher than a second importance, and the at least two road signs A predetermined area in which the first road sign is fixed to the screen as the first display content of the at least two display contents when the second road sign has the second importance. And the second road sign is displayed as a second display content of the at least two display contents at a required timing in another area of the screen. apparatus. When the next first road sign is detected, the current first road sign is replaced with the next first road sign, and the next first road sign is displayed in the predetermined area. And
2. The road sign determination apparatus according to claim 1, wherein each of the current first road sign and the next first road sign is a speed sign or a stop sign.   The road sign judgment device according to claim 1 or 2, wherein the second road sign includes a speed sign resulting from weather. The second road sign includes a prohibition sign that prohibits a predetermined state of the vehicle and a caution sign that alerts an occupant of the vehicle, and the display of the second road sign in the other area is Continue for a certain period of time,
When at least three road signs are detected and a third road sign of the at least three road signs has a third importance lower than the first importance; and The road sign judgment device according to any one of claims 1 to 3, wherein the third road sign is sequentially displayed in the other area. Detecting at least two road signs by performing image recognition processing on each of at least two captured images in which the front of the vehicle is imaged by an imaging unit provided in the vehicle;
When the first road sign of the at least two road signs is a speed sign or a stop sign, the first road sign is always displayed in a predetermined area fixed on a screen; and the at least two When a second road sign of the two road signs is different from the first road sign, the second road sign is displayed along with the first road sign at a necessary timing in another area of the screen. about,
A road sign recognition method comprising:

Images