JP7184705B2 - ARROW DIRECTION DETERMINATION DEVICE AND ARROW DIRECTION DETERMINATION METHOD - Google Patents

Description

The present disclosure relates to an arrow direction identification device.

BACKGROUND ART Conventionally, there has been a demand to accurately identify road signs for purposes such as automatic driving control of vehicles. Such signs include, for example, one-way signs and signs that display arrows, such as signs indicating predetermined restricted traffic sections. Japanese Patent Application Laid-Open No. 2002-200000 discloses that an object classifier is switched according to map information to identify what kind of sign the object is.

JP 2018-97738 A

In a sign displaying an arrow, the direction of the arrow has an important meaning. In the technology described in Patent Document 1, the type of sign including an arrow, in other words, the meaning indicated by the direction of the arrow, such as the fact that it is a one-way street or the start or end of a restricted section, is included in the map information in advance. there is Therefore, the direction of the arrow can be identified based on such map information. However, the technique described in Patent Document 1 uses map information to which a huge amount of data is added, so the processing power, power consumption, and cost required for the apparatus for identifying the direction of the arrow are enormous. For example, when such a device is mounted on a vehicle and used, the technology of Patent Document 1 may not be applicable due to limited processing resources. Therefore, there is a demand for a technique that can easily and accurately identify the direction of an arrow in a sign that displays an arrow.

The present disclosure can be implemented as the following forms.
[Mode 1] An arrow direction specifying device (100) for specifying the direction of an arrow on a sign displaying an arrow, wherein the direction of the arrow displayed on the sign is captured in an image obtained by an imaging device that captures an image of the surrounding environment. a marker specifying unit (12) specifying an arrow; and pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size that includes the specified arrow. counting the number of pixels having a pixel value associated with the color representing the arrow; A histogram generation unit (13) for generating a vertical histogram indicating the total value, and a direction identification unit (14) for identifying the direction of the arrow using the vertical histogram, wherein the direction identification unit is , the change in the horizontal direction in the vertical direction histogram is approximated by a linear straight line, and if the slope of the linear straight line is positive, the direction of the arrow is specified as rightward; An arrow direction identifying device that identifies the direction of an arrow as leftward .
[Mode 2] An arrow direction specifying device (100) for specifying the direction of an arrow on a sign displaying an arrow, wherein the direction of the arrow displayed on the sign is captured in an image obtained by an imaging device that captures an image of the surrounding environment. a marker specifying unit (12) specifying an arrow; and pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size that includes the specified arrow. counting the number of pixels having a pixel value associated with the color representing the arrow; A histogram generation unit (13) for generating a vertical histogram indicating the total value, and a direction identification unit (14) for identifying the direction of the arrow using the vertical histogram, wherein the direction identification unit is , in the vertical histogram, at least the left peak, the right peak, and the average value of the total values of the horizontal central part in the vertical histogram, and the ratio between the average value and the left peak An arrow direction identification device that identifies the direction of the arrow based on a first ratio and a second ratio that is the ratio of the average value to the right peak.
[Mode 3] An arrow direction specifying device (100) for specifying the direction of the arrow in a sign displaying an arrow, wherein the direction indicated on the sign is captured in an image obtained by an imaging device that captures an image of the surrounding environment. a marker specifying unit (12) specifying an arrow; and pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size that includes the specified arrow. counting the number of pixels having a pixel value associated with the color representing the arrow; A histogram generation unit (13) that generates a vertical histogram indicating the total value, and a direction identification unit (14) that identifies the direction of the arrow using the vertical histogram, wherein the indicator is the main An auxiliary mark attached to a mark, wherein the mark identification unit recognizes the main mark in the captured image, and uses the position of the recognized main mark as a reference to identify the auxiliary mark that is estimated to exist. An arrow direction identification device that cuts out a marker estimation area from the captured image and generates the vertical direction histogram for the cut out auxiliary marker estimation area.
[Mode 4] An arrow direction specifying device (100) for specifying the direction of the arrow in a sign displaying an arrow, wherein the direction indicated on the sign is captured in an image obtained by an imaging device that captures an image of the surrounding environment. a marker specifying unit (12) specifying an arrow; and pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size that includes the specified arrow. counting the number of pixels having a pixel value associated with the color representing the arrow; A histogram generation unit (13) for generating a vertical histogram indicating the total value, and a direction identification unit (14) for identifying the direction of the arrow using the vertical histogram, wherein the histogram generation unit is , before the vertical histogram is generated, pixels arranged in the horizontal direction for each pixel position along the vertical direction in the arrow-containing region and having a pixel value associated with the color representing the arrow; counting numbers, and generating a horizontal histogram showing the total horizontal number of counted pixels per pixel location over the vertical length of the arrow-containing region; An arrow direction identification device for generating the vertical histogram, with a predetermined range in the vertical direction including a vertical position indicating a maximum value in the histogram as a processing target.

According to one aspect of the present disclosure, there is provided an arrow direction identification device (100) for identifying the direction of an arrow in a sign displaying an arrow. This arrow direction identification device includes a sign identification unit (12) that identifies the arrow displayed on the sign, and a predetermined arrow that includes the identified arrow in the captured image obtained by an imaging device that captures an image of the surrounding environment. counting the number of pixels arranged in the vertical direction for each pixel position along the horizontal direction within the arrow-containing area of the size determined and having a pixel value associated with the color representing the arrow; a histogram generating unit (13) for generating a vertical histogram indicating the total number of the counted pixels for each pixel position in the vertical direction over the entire horizontal length of the containing region; a direction specifying unit (14) that specifies the direction of the arrow using a histogram.

According to the arrow direction specifying device (100) of this form, pixels arranged in the vertical direction for each pixel position along the horizontal direction within the arrow containing region of a predetermined size including the specified arrow. count the number of pixels having a pixel value associated with the color representing the arrow, and sum the number of pixels counted for each pixel location over the entire horizontal length of the arrow-containing region. is generated and the direction of the arrow is specified using the vertical histogram, the direction of the arrow can be specified without using map information to which a large amount of data is added. Therefore, the processing power, power consumption, and cost required for the apparatus for identifying the direction of the arrow are not enormous, and the direction of the arrow on the sign displaying the arrow can be easily and accurately identified.

The present disclosure can also be implemented in various forms other than the arrow direction identification device. For example, it can be implemented in the form of an arrow direction identification method, a control method of an arrow direction identification device, a computer program for realizing functions and methods of the device, a storage medium storing such a computer program, or the like.

1 is a block diagram showing a functional configuration of an arrow direction specifying device as one embodiment of the present disclosure; FIG. 4 is a flowchart showing the procedure of arrow determination processing; FIG. 3 is an explanatory diagram showing an example of a maximum speed limit sign accompanied by an auxiliary sign containing an arrow; FIG. 4 is an explanatory diagram showing an example of a candidate region cut out from the image shown in FIG. 3; 9 is a flow chart showing a procedure of clipping processing of an estimated auxiliary sign area. FIG. 11 is an explanatory diagram showing an image obtained by resizing the clipped estimated auxiliary marker region; 4 is a flowchart showing a procedure of histogram creation processing; FIG. 4 is an explanatory diagram showing an example of a horizontal direction histogram; FIG. 4 is an explanatory diagram showing an example of a vertical histogram; It is explanatory drawing which showed the feature-value in the vertical direction histogram. 4 is a flow chart showing the procedure of direction identification processing in the first embodiment. 9 is a flow chart showing the procedure of direction identification processing in the second embodiment. FIG. 11 is a flow chart showing a procedure of direction identification processing in the third embodiment; FIG. FIG. 4 is an explanatory diagram showing an example of a main marker including an arrow;

A. First embodiment:
A1. Device configuration:
An arrow direction identification device 100 shown in FIG. 1 is a device that identifies the direction of an arrow indicated on a sign. In this embodiment, the sign showing the direction of the arrow corresponds to the auxiliary sign used in conjunction with the main sign. As such an auxiliary sign, for example, an auxiliary sign indicating a section (traffic restricted section) where a predetermined speed limit is set is applicable to a main sign indicating that a predetermined speed limit is set. Specifically, in Japan, a supplementary sign with an arrow pointing right indicates the start of a traffic-restricted section, a supplementary sign with an arrow pointing in both directions indicates that it is in a section with restricted traffic, and a supplementary sign with an arrow pointing left indicates the end of the restricted traffic section.

The arrow direction identification device 100 is configured as an ECU (Electronic Control Unit) including a CPU (Central Processing Unit) 10 , a memory 20 , and an external interface (external I/F) 15 . The CPU 10 and memory 20 are communicably connected to each other via an internal bus. In this embodiment, the arrow direction identification device 100 is used by being mounted on a vehicle (not shown). Specifically, based on the direction of the arrow specified by the arrow direction specifying device 100, the type of the auxiliary sign, for example, the auxiliary sign indicating the start of a restricted traffic section, is specified, and the icon indicating such an auxiliary sign is displayed. , is displayed on the display device in the vehicle. Examples of such display devices include a liquid crystal panel in an instrument panel and a head-up display. By displaying such an icon, the driver can know that the vehicle has entered a travel-restricted section.

The CPU 10 functions as an acquisition unit 11 , a sign identification unit 12 , a histogram generation unit 13 , and a direction identification unit 14 by reading and executing control programs pre-stored in the memory 20 .

The acquisition unit 11 acquires image data obtained by imaging the surroundings of the vehicle. Arrow direction identification device 100 is connected to network 16 arranged in the vehicle via external interface 15 . A control device 17 for an imaging device 18 mounted on the vehicle is also connected to the network 16 . Acquisition unit 11 acquires image data obtained by imaging device 18 via control device 17 and network 16 . The imaging device 18 starts imaging when the ignition of the vehicle is turned on and power is supplied. Imaging is performed at a frame rate such as 24 frames per second, for example. Then, each frame image obtained by imaging is obtained by the obtaining unit 11 . In this embodiment, the network 16 is configured by a CAN (Controller Area Network). LIN (Local Interconnect Network) or Ethernet (registered trademark) may be used instead of CAN.

The marker identification unit 12 identifies a marker within the captured image based on the image data acquired by the acquisition unit 11 . Specifically, the sign identification unit 12 calculates the image feature quantity of the region while shifting the scanning window little by little with respect to the entire captured image, and searches for the image feature quantity similar to the feature quantity of the sign. to identify the sign. Such image feature amounts include, for example, size, shape, pixel values (RGB values), edge shape, and the like. In order to specify a sign, the sign identification unit 12 performs main sign recognition, auxiliary sign extraction processing, resizing processing, and binarization processing, as will be described later.

The histogram generating unit 13 counts the number of pixels having pixel values indicating the color of the arrow within the marker identified by the marker identifying unit 12, and generates a histogram. Values indicating the color of such arrows are, for example, RGB values corresponding to red having predetermined hue, saturation and lightness. The pixel value is not limited to a single value (single combination of RGB), and may be any pixel value included in a color space having a predetermined range for each of R, G, and B. The details of the histogram will be explained later.

The direction identification unit 14 identifies the direction of the arrow using the histogram generated by the histogram generation unit 13 . A specific method for identifying the arrow will be described later.

A2. Arrow judgment processing:
The arrow determination process shown in FIG. 2 is a process of specifying the direction of the arrow of the auxiliary sign. The arrow determination process is started when the arrow direction identifying device 100 is powered on and the captured image is acquired by the acquisition unit 11 .

The sign identification unit 12 recognizes the main sign (step S100). As described above, the marker identification unit 12 calculates the image feature amount of the area while gradually shifting the scanning window for the entire image data acquired by the acquisition unit 11, and the image feature amount is the feature of the main marker. Recognize the main sign by looking for something like quantity. As a result, for example, based on the image feature amount in the area W including the main sign 121 indicating the speed limit as shown in FIG. Recognized.

As shown in FIG. 2, the sign identification unit 12 performs auxiliary sign extraction processing (step S110). In the auxiliary sign extraction processing, an area of a predetermined position and size in which an auxiliary sign is estimated to exist (referred to as an auxiliary sign estimation area) is specified as a predetermined area, using the position of the main sign 121 as a reference. This is the cutting process. By this processing, it is possible to narrow down the area in which the auxiliary sign exists.

Specifically, the process of cutting out the estimated auxiliary sign area is performed according to the procedure shown in FIG. The marker identifying unit 12 cuts out a region having a predetermined position, size, and shape under the main marker 121 as a candidate region (step S111). In this embodiment, the shape of the candidate area 120 is rectangular. For example, as shown in FIG. 3, in region W, a rectangular region indicated by a broken line below the main marker 121 identified in step S100 is cut out as a candidate region 120. As shown in FIG. The sizes of the primary and secondary markers, and the relative positions of the secondary markers with respect to the primary marker are determined in advance according to predetermined regulations. Therefore, the position and size of the area to be cut out as the candidate area are determined and set in advance by experiment or the like so that the position and size are such that the auxiliary marker is included.

As shown in FIG. 5, the marker identification unit 12 performs binarization processing on the candidate area 120 (step S112). Specifically, pixels having pixel values within a range predetermined as the pixel value range representing the red color of the auxiliary sign are set to white (numerical value 1), and pixels not included in this range are set to black (numerical value 0). ). Therefore, in the candidate region 120 shown in FIG. 4, each pixel of the ring portion region 121a of the main sign and each pixel of the arrow portion region 123 of the auxiliary sign are colored white (numerical value 1) in step S112. pixels in other areas are set to black (value 0).

As shown in FIG. 5, the marker identification unit 12 performs a process (labeling) of connecting areas showing white (numerical value 1) in the binarized image to form a connected image and labeling each connected image. (Step S113). In the example of FIG. 4, regions 121a and 123 are each labeled as connected images. For example, the label "1" is given to the connected image of the area 121a, and the label "2" is given to the connected image of the area 123, respectively. In FIG. 4, for convenience of illustration, areas other than the area 121a and the area 123 are also represented in white.

As shown in FIG. 5, the marker identifying unit 12 determines whether or not there is a removal target area that does not look like an arrow of an auxiliary marker and meets a predetermined removal condition in the connected image of each label (step S114). Predetermined removal conditions include, for example, conditions 1 to 3 below.
(Condition 1) The long side/short side ratio is small.
(Condition 2) An auxiliary sign arrow with a short vertical or horizontal length.
(Condition 3) When linear regression is performed on the region and the residual from the regression line is large.

Arrows are characterized by elongated shapes. Therefore, when the long side/short side ratio is small, the shape is close to a square, which is a removal condition (Condition 1). If the vertical or horizontal length is short, the region becomes too small and it is difficult to distinguish whether it is noise or an arrow of an auxiliary mark, so it becomes a removal condition (Condition 2). In addition, when the linear regression is performed on the region, the removal condition is also set when the residual from the regression line is large (Condition 3). For example, in FIG. 4, when linear regression is performed on the region 121a to which the label 1 is assigned, a regression line (not shown) is drawn around the dashed line 124. FIG. Also, when linear regression is executed for the region 123 to which the label 2 is assigned, a regression line (not shown) is drawn around the dashed line 125 . In this case, the region 123 labeled 2 and the regression line around the dashed line 125 are nearly parallel. That is, when a vertical line is drawn from the region 123 to the regression line around the dashed line 125, there is almost no residual corresponding to the length of the vertical line. On the other hand, when linear regression is executed for the region 121a to which the label 1 is assigned, a regression line (not shown) is drawn around the dashed line 124. FIG. The area 121a labeled 1 and the regression line around the dashed line 124 are not parallel or substantially parallel, and part of the area 121a is far from the dashed line 124. FIG. That is, when a vertical line is drawn from the region 121a to the regression line around the dashed line 124, the residual corresponding to the length of the vertical line becomes large, so the region 121a assigned the label 1 is the removal target region.

As shown in FIG. 5, when it is determined that there is a region to be removed that satisfies any of the conditions 1 to 3 in the connected image to which each label is assigned (step S114: Yes), the label specifying unit 12 removes the corresponding removal target area (step S115). On the other hand, if it is determined that there is no area to be removed that satisfies any of the above conditions 1 to 3 in the connected image to which each label has been assigned (step S114: No), or the above step S115 is completed After that, the sign specifying unit 12 cuts out the periphery of the narrowed down connected image as an auxiliary sign estimation region (step S116). In the example of FIG. 4, the label identifying unit 12 identifies an area of a predetermined position, size and shape around the connected image 123 to which the label 2 is assigned, and identifies an area of the same position and size as the area. An auxiliary marker estimation region 126 is cut out from the captured image. After execution of step S116, step S120 shown in FIG. 2 is executed. Note that the auxiliary sign estimation region corresponds to a subordinate concept of the arrow-containing region of the present disclosure.

The sign specifying unit 12 performs resizing processing on the estimated auxiliary sign region 126 cut out in step S110 (step S120). The marker specifying unit 12 binarizes the resized area (step S130). In step S130, the same process as the binarization process in step S112 described above is performed. As shown in FIG. 6, for example, in step S120, the sign identification unit 12 resizes the estimated auxiliary sign region 126 to 40 pixels (pix) vertically and 96 pixels (pix) horizontally. The resizing process normalizes the determination target area to a certain size, so that too small areas and too large areas do not coexist, and variations in determination accuracy in the direction of the arrow are avoided. Note that the size after resizing by the sign specifying unit 12 is not limited to 40 pix×96 pix, and may be any size. In FIG. 6, the pixels corresponding to the arrows of the auxiliary signs are represented as white pixels. Horizontally extending white lines are drawn above and below the arrow, and these two white lines will be described later. By executing step S130, the red arrow area becomes white, and the other area becomes black. The binarization process makes it possible to clearly distinguish between the arrow and the background.

As shown in FIG. 2, the histogram generator 13 performs histogram creation processing on the region binarized in step S130 (step S140). Specifically, as shown in FIG. 7, the histogram generator 13 creates a horizontal histogram (step S142). The horizontal direction histogram counts the number of pixels arranged in the horizontal direction for each pixel position along the vertical direction of the estimated auxiliary sign area and having a pixel value of white (numerical value 1). means a histogram showing the horizontal sum of the number of pixels counted per pixel position over the entire vertical length of . Note that white (numerical value 1) can be said to be a pixel value associated with the color of the arrow (red). For example, the histogram generation unit 13 counts the number of pixels having a white value (numerical value 1) among the pixels arranged in the horizontal direction for each pixel position along the vertical direction shown in FIG. Then, a horizontal histogram shown in FIG. 8 is created by plotting the pixel positions in the vertical direction in FIG. 6 on the horizontal axis and plotting the count number of each pixel position on the vertical axis. As shown in FIG. 7, the histogram generator 13 generates a horizontal histogram before generating a vertical histogram.

As shown in FIG. 7, the histogram generation unit 13 identifies the vertical position indicating the maximum value in the horizontal histogram (step S144). In the example of FIG. 8, the vertical position (about 22 pix) showing the maximum value of about 75 is specified. As shown in FIG. 7, the histogram generator 13 identifies a predetermined range in the vertical direction including the vertical position showing the maximum value in the horizontal histogram (step S146). In this embodiment, a range of a predetermined length is specified vertically around the vertical position indicating the maximum value. In the example of FIG. 8, the histogram generation unit 13 specifies a range of 10 pix above and below, ie, a range of 12 pix to 32 pix, with the vertical position of about 22 indicating the maximum value as the center. The identified range corresponds to a region surrounded by two white lines parallel to the horizontal axis in FIG. 6, that is, a white line of 12 pix and a white line of 32 pix. subject to

As shown in FIG. 7, the histogram generator 13 generates a vertical histogram (step S148). The vertical direction histogram counts the number of pixels arranged in the vertical direction for each pixel position along the horizontal direction of the estimated auxiliary sign area and has a pixel value of white (numerical value 1). means a histogram showing the vertical sum of the number of pixels counted per pixel position over the horizontal length of . At this time, the histogram generator 13 generates a vertical histogram with the range specified in step S146 as the processing target. For example, based on the binarized image in FIG. 6, the number of white pixels within the range surrounded by two upper and lower white lines is counted, and the vertical histogram shown in FIG. 9 is generated. After step S148 is completed, the arrow direction specifying process shown in FIG. 2 is executed (step S150).

The direction specifying unit 14 performs arrow direction specifying processing (step S150). Specifically, as shown in FIG. 11, the direction specifying unit 14 determines whether the horizontal width of the input data, that is, the image data acquired by the acquiring unit 11 is equal to or less than the effective horizontal width, as the exclusion determination based on the data length. is determined (step S151). In this embodiment, the effective horizontal width is set to 40 pix, but the value is not limited to this value and any value may be used. If the direction identifying unit 14 determines that the effective width of the input data is equal to or less than the effective width (step S151: Yes), it does not determine the direction (step S152). For example, when the size of the captured image is reduced due to a failure of the imaging device 18, and image data with a horizontal width of 40 pixels or less is acquired, direction identification determination based on such image data is not performed.

When the direction specifying unit 14 determines that the effective horizontal width of the input data is not equal to or smaller than the effective horizontal width (step S151: No), it determines whether or not the arrow is in both directions (step S153). Specifically, the direction specifying unit 14 first obtains the left peak value, the right peak value, and the average value of the central flat portion in the vertical histogram. Here, the peak left is the position where the count number is the largest in the area on the left side of the central flat portion in the vertical histogram. Similarly, the peak right is the position where the count number is the largest in the region on the right side of the central flat portion in the vertical histogram. In this embodiment, as shown in FIGS. 9 and 10, the central flat portion is the central 80% of the vertical histogram of the entire vertical pixel range for which the vertical histogram is created. means the count value in the area. Since the vertical width (height) of the area of the arrow of the auxiliary mark is almost constant, the count value of the white pixels is also constant, and the area is flat on the histogram. The peak left value is the position where the count number is the largest in the left side of the central flat portion and corresponds to 10% of the whole, and the peak right value is the right side of the central flat portion and corresponds to 10% of the whole. After the region, it is the position with the largest count number.

As shown in FIG. 11, the direction specifying unit 14 determines whether or not the right ratio>2.0 and the left ratio>2.0 are satisfied. The right ratio means the ratio of the right peak value to the average value of the central flat portion (right peak value/average value of the central flat portion). The left ratio means the ratio of the left peak value to the average value of the central flat portion (left peak value/average value of the central flat portion). The peak left value and the peak right value are each greater than 2.0 with respect to the average value of the central flat portion, that is, the count number indicating the arrow is large at both ends, which means that the triangle portion at the tip of the arrow is at both ends can be judged to have Therefore, when it is determined that (right ratio)>2.0 and (left ratio)>2.0, the direction specifying unit 14 determines that the arrow is in both directions (step S153: Yes), and determines that the target The indicator is identified as a double arrow (step S154). When determining that (right ratio)>2.0 and (left ratio)>2.0 are not satisfied, the direction specifying unit 14 determines that the arrow is not a double-headed arrow (step S153: No).

If it is determined that the arrow is not bidirectional (step S153: No), the direction specifying unit 14 determines whether the value of the left/right ratio is less than 0.7 (step S155). The left-right ratio means the ratio between the peak left value and the peak right value (peak left value/peak right value). When the direction specifying unit 14 determines that the value of the left/right ratio is less than 0.7 (step S155: Yes), the direction specifying unit 14 specifies that the target image is an arrow pointing to the right (step S156). If the value of the left-to-right ratio shown in the above formula is less than 0.7, it can be inferred that the target image is an arrow pointing to the right because the peak right value is larger than the peak left value. be. In this embodiment, the threshold for determining the one-way arrow is set to 0.7, but other values may be used. When the direction specifying unit 14 determines that the value of the left-right ratio is not less than 0.7, that is, is greater than or equal to 0.7 (step S155: No), the target image is indicated by a leftward arrow or a bidirectional arrow. It specifies that there is (step S157).

After completing steps S152 to S157, as shown in FIG. 2, the arrow direction identification (step S150) is completed, and the process returns to step S100.

The arrow direction identification device 100 of the first embodiment described above includes the sign identification unit 12 that identifies the arrow displayed on the sign in the captured image obtained by the imaging device that captures the surrounding environment, and the identified arrow. Counting the number of vertically aligned pixels for each pixel position along the horizontal direction within an arrow-containing region of a predetermined size including: a histogram generation unit 13 for generating a vertical histogram indicating the total number of pixels counted for each pixel position in the vertical direction over the entire horizontal length of the arrow-containing region; and a direction specifying unit 14 that specifies the direction of the arrow by using. Therefore, according to the arrow direction identifying device 100, the direction of the arrow can be identified without using map information to which a huge amount of data is added. Therefore, the processing power, power consumption, and cost required for the apparatus for identifying the direction of the arrow are not enormous, and the direction of the arrow on the sign displaying the arrow can be easily and accurately identified.

In addition, in the arrow direction identifying apparatus 100 of the present embodiment, the direction identifying unit 14 identifies at least the left peak and the right peak in the vertical histogram, and the ratio of the peak left value to the peak right value (peak left value /peak right value) is less than a predetermined threshold, the target image is identified as a right arrow, while if the value of the ratio is greater than or equal to the predetermined threshold, the target image is Identify it as a left arrow or a double arrow. Therefore, it is possible to easily identify the direction of the arrow in the sign that displays the arrow.

Furthermore, in the arrow direction identification device 100 of the present embodiment, the direction identification unit 14 includes the average value of at least the left peak, the right peak, and the total value of the horizontal central portion of the vertical histogram in the vertical direction histogram, is specified, and it is determined that the first ratio, which is the ratio of the average value to the left peak, and the second ratio, which is the ratio of the average value to the right peak, are greater than a predetermined threshold, the target image is , is specified as a double-headed arrow, it is possible to easily specify whether or not the arrow in the sign displaying the arrow is double-headed.

B. Second embodiment:
Since the configuration of the arrow direction identification device 100 of the second embodiment is the same as that of the arrow direction identification device 100 of the first embodiment, the same components are denoted by the same reference numerals, and detailed description thereof will be omitted. . The arrow direction identifying device 100 of the second embodiment differs from the arrow direction identifying device 100 of the first embodiment in the detailed procedure of the arrow direction identifying process (step S150). Other procedures of the arrow determination process are the same as in the first embodiment, so the same procedures are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

The arrow direction identifying process of the second embodiment shown in FIG. 12 differs from the arrow direction identifying process of the first embodiment shown in FIG. 11 in that steps S166, S167, and S168 are additionally executed. Other procedures in the arrow direction specifying process of the second embodiment are the same as those of the first embodiment, so the same steps are denoted by the same reference numerals, and detailed description thereof will be omitted. The arrow direction identifying device 100 of the second embodiment differs from the arrow direction identifying device 100 of the first embodiment in that the direction is identified using tilt.

In step S155 described above, when it is determined that the value of the left/right ratio is not less than 0.7, that is, is greater than or equal to 0.7 (step S155: No), the direction specifying unit 14 determines that the slope of the regression line described later is It is determined whether or not it is 0.02 or more (step S166). A detailed procedure of this step S166 will be described. The direction identifying unit 14 first obtains the average value of all data using the vertical direction histogram shown in FIG. The direction specifying unit 14 subtracts the average value from the numerical value of each data, ie, the count number of each pixel value, to obtain the deviation. The direction specifying unit 14 obtains a regression line, which is a linear straight line, by the method of least squares using the obtained deviations, and obtains the slope of the regression line. That is, the direction specifying unit 14 approximates the change in the horizontal direction in the vertical direction histogram with a first-order linear straight line. Then, the direction specifying unit 14 determines whether or not the obtained tilt is 0.02 or more.

When it is determined that the inclination is 0.02 or more (step S166: Yes), the direction identification unit 14 identifies the arrow pointing to the right (step S167). If the inclination is less than 0.02 (step S166: No), the direction identifying unit 14 identifies the arrow as a leftward arrow or a double arrow (step S168). When the slope of the regression line is 0.02 or more, it is a straight line that rises to the right, so the number of counts of white numerical values indicating arrows is large on the right side, and a triangular portion indicating an arrow exists at the right end of the target image. is presumed. Therefore, in this case, it is specified as a rightward arrow. On the other hand, if the slope of the regression line is less than 0.02, it is a straight line that rises to the left or is flat in the horizontal direction. It is presumed that there is a triangular portion indicated by arrows, or that there is a triangular portion indicated by arrows at both left and right ends. Therefore, in this case, the leftward arrow or the double-headed arrow is specified. The threshold in step S166 described above is not limited to 0.02, and may be any other value. For example, when the threshold value is 0, the direction specifying unit 14 may specify the direction of the arrow depending on whether the slope of the regression line is positive or negative.

The arrow direction identification device 100 of the second embodiment described above has the same effects as the arrow direction identification device 100 of the first embodiment. In addition, in the arrow direction specifying device 100 of the second embodiment, the direction specifying unit 14 specifies the direction of the arrow using the inclination in addition to the right ratio, the left ratio, and the left/right ratio. The direction of the arrow in the sign displaying the arrow can be specified with higher accuracy.

C. Third embodiment:
Since the configuration of the arrow direction identification device 100 of the third embodiment is the same as that of the arrow direction identification device 100 of the first embodiment, the same components are denoted by the same reference numerals, and detailed description thereof will be omitted. . The arrow direction identifying device 100 of the third embodiment differs from the arrow direction identifying device 100 of the first embodiment in the detailed procedure of the arrow direction identifying process (step S150). Other procedures of the arrow determination process are the same as in the first embodiment, so the same procedures are denoted by the same reference numerals, and detailed descriptions thereof are omitted.

The arrow direction specifying process of the third embodiment shown in FIG. 13 differs from the arrow direction specifying process of the second embodiment shown in FIG. 12 in that steps S153 and S154 are omitted. Other procedures in the arrow direction specifying process of the third embodiment are the same as those of the arrow direction specifying process of the second embodiment. In the arrow direction identification process of the third embodiment, if it is determined in step S151 that the width of the input data is not equal to or less than the effective width (step S151: No), the above-described step S155 is executed, and the left/right ratio value is 0. A determination is made whether it is less than .7.

The arrow direction identifying device 100 of the third embodiment described above has the same effects as the arrow direction identifying devices 100 of the first and second embodiments. In contrast to the arrow direction identification device 100 of the first embodiment, in the arrow direction identification device 100 of the third embodiment, the direction identification unit 14 mainly uses the right ratio and the left ratio for determining the arrow in both directions. The direction of the arrow is specified by using the inclination, omitting the determination process. In this embodiment, the direction specifying unit 14 does not use the right ratio and the left ratio for determining the arrow in both directions. The direction can be easily specified, and the direction on one side can be specified with high accuracy.

D. Other embodiments:
(D1) In each of the embodiments, an auxiliary sign with an arrow attached to the main sign was targeted, but the present disclosure is not limited to this. The label showing the arrow may be the main label. For example, as shown in FIG. 14, an arrow on a main sign 200 indicating a one-way street may be targeted.

(D2) In each embodiment, the direction specifying unit 14 specifies the direction of the arrow using a plurality of feature amounts such as determination based on the left-right ratio and determination based on the inclination, but the present disclosure is not limited to this. . The direction specifying unit 14 may specify the direction of the arrow using only one feature amount. For example, the direction specifying unit 14 may determine the direction of the arrow by determining only the left/right ratio or determining only the inclination. This allows easy determination.

(D3) In each embodiment, the target is the arrow direction identification device 100 mounted on a vehicle, but the present disclosure is not limited to this. The arrow direction identifying device 100 may not be mounted on the vehicle. Alternatively, the arrow may not be displayed on the display device. The direction of the arrow may be guided by voice output. Further, the identified arrow directions may only be stored in memory 20 . Further, although the direction of the arrow specified by the arrow direction specifying device 100 is used to specify the type of the auxiliary sign or the main sign, the present disclosure is not limited to this. The direction of the arrow identified by the arrow direction identification device 100 may be used for storage in the memory 20 .

The present disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from the scope of the present disclosure. For example, the technical features in each embodiment corresponding to the technical features in the form described in the outline of the invention are used to solve some or all of the above problems, or Substitutions and combinations may be made as appropriate to achieve part or all. Also, if the technical features are not described as essential in this specification, they can be deleted as appropriate.

12 sign identification unit 13 histogram generation unit 14 direction identification unit 100 arrow direction identification device

Claims (11)

An arrow direction identification device (100) for identifying the direction of the arrow in a sign displaying an arrow,
a sign identification unit (12) for identifying the arrow displayed on the sign in an image captured by an imaging device that captures an image of the surrounding environment;
Pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size containing the identified arrow, and having a pixel value associated with a color representing the arrow A histogram generator that counts the number of pixels and generates a vertical histogram indicating the total vertical number of the counted pixels per pixel location over the horizontal length of the arrow-containing region. a part (13);
a direction identifying unit (14) that identifies the direction of the arrow using the vertical direction histogram;
with
The direction identifying unit approximates the change in the horizontal direction in the vertical direction histogram with a linear linear straight line, identifies the direction of the arrow as rightward when the slope of the linear linear straight line is positive, and identifies the direction of the arrow as rightward when the slope of the linear linear straight line is positive, if the direction of the arrow is to the left,
Arrow direction identification device. The arrow direction identification device according to claim 1 ,
The direction specifying unit specifies at least a left peak and a right peak in the vertical direction histogram, and specifies the direction of the arrow using the left peak and the right peak.
Arrow direction identification device. In the arrow direction identification device according to claim 1 or claim 2 ,
The direction identifying unit identifies at least a left peak and a right peak in the vertical histogram, and identifies the direction of the arrow based on the ratio of the left peak to the right peak.
Arrow direction identification device. An arrow direction identification device (100) for identifying the direction of the arrow in a sign displaying an arrow,
a sign identification unit (12) for identifying the arrow displayed on the sign in an image captured by an imaging device that captures an image of the surrounding environment;
Pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size containing the identified arrow, and having a pixel value associated with a color representing the arrow A histogram generator that counts the number of pixels and generates a vertical histogram indicating the total vertical number of the counted pixels per pixel location over the horizontal length of the arrow-containing region. a part (13);
a direction identifying unit (14) that identifies the direction of the arrow using the vertical direction histogram;
with
The direction specifying unit specifies, in the vertical direction histogram, at least a left peak, a right peak, and an average value of total values of a horizontal central portion of the vertical direction histogram, and the average value and the left peak Identifying the direction of the arrow based on a first ratio that is the ratio of and a second ratio that is the ratio of the average value to the right peak;
Arrow direction identification device. An arrow direction identification device (100) for identifying the direction of the arrow in a sign displaying an arrow,
a sign identification unit (12) for identifying the arrow displayed on the sign in an image captured by an imaging device that captures an image of the surrounding environment;
Pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size containing the identified arrow, and having a pixel value associated with a color representing the arrow A histogram generator that counts the number of pixels and generates a vertical histogram indicating the total vertical number of the counted pixels per pixel location over the horizontal length of the arrow-containing region. a part (13);
a direction identifying unit (14) that identifies the direction of the arrow using the vertical direction histogram;
with
The label is an auxiliary label attached to the main label,
The sign identification unit recognizes the main sign in the captured image, and cuts out an estimated auxiliary sign region in which the auxiliary sign is estimated to exist from the captured image based on the position of the recognized main sign. , generating the vertical histogram for the clipped auxiliary marker estimation region;
Arrow direction identification device. The arrow direction identification device according to claim 5 ,
The marker identification unit binarizes the auxiliary marker estimation region, connects images having the same value, labels the connected connected images, and determines a predetermined value among the labeled connected images. perform a narrowing process to remove the connected images that meet the conditions;
Arrow direction identification device. An arrow direction identification device (100) for identifying the direction of the arrow in a sign displaying an arrow,
a sign identification unit (12) for identifying the arrow displayed on the sign in an image captured by an imaging device that captures an image of the surrounding environment;
Pixels arranged in the vertical direction for each pixel position along the horizontal direction in an arrow-containing region of a predetermined size containing the identified arrow, and having a pixel value associated with a color representing the arrow A histogram generator that counts the number of pixels and generates a vertical histogram indicating the total vertical number of the counted pixels per pixel location over the horizontal length of the arrow-containing region. a part (13);
a direction identifying unit (14) that identifies the direction of the arrow using the vertical direction histogram;
with
Before generating the vertical direction histogram, the histogram generation unit generates pixels in the arrow-containing region that are pixels arranged in the horizontal direction for each pixel position along the vertical direction and are associated with the color representing the arrow. Counting the number of pixels with values and generating a horizontal histogram showing the total horizontal value of the counted number of pixels per pixel location across the vertical length of the arrow-containing region. and generating the vertical histogram with an area of a predetermined range in the vertical direction including the vertical position indicating the maximum value in the horizontal histogram as a processing target.
Arrow direction identification device. An arrow direction identification method for identifying the direction of the arrow in a sign displaying an arrow using an arrow direction identification device,
(a) in the arrow direction identifying device, identifying the sign in an image captured by an imaging device that captures an image of the surrounding environment;
(b) In the arrow direction identifying device, pixels arranged in the vertical direction for each pixel position along the horizontal direction in the identified sign and having a pixel value associated with the color representing the arrow. and generating a vertical histogram representing the vertical sum of the counted number of pixels per pixel location across the horizontal length of the sign;
(c) using the vertical direction histogram to identify the direction of the arrow in the arrow direction identifying device;
with
The step (c) approximates the change in the horizontal direction in the vertical direction histogram with a linear straight line, and if the slope of the linear straight line is positive, the direction of the arrow is specified as rightward, and the direction of the arrow is specified as negative. if the direction of the arrow is specified to the left,
Arrow direction identification method. An arrow direction identification method for identifying the direction of the arrow in a sign displaying an arrow using an arrow direction identification device,
(a) in the arrow direction identifying device, identifying the sign in an image captured by an imaging device that captures an image of the surrounding environment;
(b) In the arrow direction identifying device, pixels arranged in the vertical direction for each pixel position along the horizontal direction in the identified sign and having a pixel value associated with the color representing the arrow. and generating a vertical histogram representing the vertical sum of the counted number of pixels per pixel location across the horizontal length of the sign;
(c) using the vertical direction histogram to identify the direction of the arrow in the arrow direction identifying device;
with
In the step (c), in the vertical histogram, at least a left peak, a right peak, and an average value of total values in the horizontal central part of the vertical histogram are specified, and the average value and the left identifying the direction of the arrow based on a first ratio that is the ratio of the peak and a second ratio that is the ratio of the average value to the right peak;
Arrow direction identification method. An arrow direction identification method for identifying the direction of the arrow in a sign displaying an arrow using an arrow direction identification device,
(a) in the arrow direction identifying device, identifying the sign in an image captured by an imaging device that captures an image of the surrounding environment;
(b) In the arrow direction identifying device, pixels arranged in the vertical direction for each pixel position along the horizontal direction in the identified sign and having a pixel value associated with the color representing the arrow. and generating a vertical histogram representing the vertical sum of the counted number of pixels per pixel location across the horizontal length of the sign;
(c) using the vertical direction histogram to identify the direction of the arrow in the arrow direction identifying device;
with
The label is an auxiliary label attached to the main label,
The step (a) includes recognizing the main marker in the captured image, and using the position of the recognized main marker as a reference, extracting an estimated auxiliary marker region in which the auxiliary marker is estimated to exist from the captured image. cutting out, and generating the vertical histogram for the cut out estimated auxiliary marker region;
Arrow direction identification method. An arrow direction identification method for identifying the direction of the arrow in a sign displaying an arrow using an arrow direction identification device,
(a) in the arrow direction identifying device, identifying the sign in an image captured by an imaging device that captures an image of the surrounding environment;
(b) In the arrow direction identifying device, pixels arranged in the vertical direction for each pixel position along the horizontal direction in the identified sign and having a pixel value associated with the color representing the arrow. and generating a vertical histogram representing the vertical sum of the counted number of pixels per pixel location across the horizontal length of the sign;
(c) using the vertical direction histogram to identify the direction of the arrow in the arrow direction identifying device;
with
Before generating the vertical histogram, the step (b) associates, in the arrow-containing region, pixels horizontally aligned for each pixel position along the vertical direction with colors representing the arrows. counting the number of pixels with pixel values and creating a horizontal histogram showing the total horizontal number of the counted pixels per pixel location over the vertical length of the arrow-containing region; A step of generating the vertical histogram as a processing target for a region of a predetermined range in the vertical direction including the vertical position showing the maximum value in the horizontal histogram,
Arrow direction identification method.

Images