JP2019109727A - Image processing system and image processing method - Google Patents

Abstract

To provide an image processing system capable of reducing the time and effort of initial setting when recognizing turning on and off of a signal tower by image processing.SOLUTION: An image processing system comprises: an imaging device for imaging a location where a plurality of signal towers 71 to 77 are installed; and an image processing apparatus communicably connected to the imaging device. The image processing system comprises: a storage unit for storing array data representing an array of stacked signal lights 71R, 71Y, 71G, 72R, 72Y, 72G, ... of respective colors in each of the signal towers 71 to 77; and a control unit for determining an image area in image data G2 of the signal lights 71R, 71Y, 71G, 72R, 72Y, 72G,... of the respective colors included in each of the signal towers 71 to 77 on the basis of the image data G2 obtained by imaging and the array data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing system and an image processing method.

  2. Description of the Related Art Conventionally, laminated indicator lights (hereinafter also referred to as "signal towers") for displaying the state of an installation machine and the like are known. The signal tower is configured by stacking a plurality of signal lights.

  For example, Patent Document 1 and Patent Document 2 disclose image processing techniques for recognizing turning on and off of a signal tower by imaging the signal tower with a camera.

JP 2002-358111 A JP, 2007-114967, A

  When lighting and extinguishing of the signal tower are recognized by image processing, color information (pixel value) of each signal lamp constituting the signal tower to be imaged and an area (coordinate value) in image data of each signal lamp as initial setting Needs to be learned by the image processing apparatus. When the lighting and extinguishing of a plurality of signal towers are recognized by image processing, learning processing for a plurality of units is required. For this reason, the time and effort of initial setting increases according to the number of installed signal towers.

  The present invention has been made in view of the above problems, and an object thereof is to reduce the time and effort of initial setting in the case of recognizing on and off of a signal tower by image processing.

  According to an aspect of the present invention, an image processing system includes an imaging device for imaging a location where a plurality of laminated indicator lights are installed, and an image processing device communicably connected to the imaging device. The image processing apparatus is configured based on the storage unit storing the array data representing the array arrangement of the signal lights of each color in each laminated indicator light, the first image data obtained by imaging, and the array data. And a control unit that determines an image area in the first image data of the signal light of each color included in the display light.

  According to the above configuration, the image area in the first image data of the signal light of each color is determined by the control unit with respect to each stacked display light, so that lighting and extinguishing of each stacked display light are recognized by image processing. It is possible to reduce the time and effort of initial setting in the case.

  Preferably, each stacked indicator includes, as a signal light of each color, a first signal light of a first color, and a second signal light of a second color laminated to the first signal light. The array data defines that the second signal light is located above the first signal light in the first image data. The storage unit further stores a first pixel value divided into the first color and a second pixel value divided into the second color. The control unit is configured to, based on the first pixel value, the second pixel value, and the array data, generate a first image area in the first image data of the first signal light and a first image area of the second signal light. A second image area in the image data is determined.

  According to the above configuration, the image area in the first image data of the signal light of the first color and the signal light of the second color is determined by the control unit for each stacked indicator light. It becomes possible to reduce the trouble of the initial setting in the case of recognizing the lighting and extinguishing of the signal lamp and the second signal lamp by image processing.

  Preferably, the control unit is configured to set the pixel value of the first color and the second color based on second image data obtained by imaging one of the plurality of stacked indicator lights with the imaging device. Calculate the pixel value. The control unit stores the calculated pixel value of the first color in the storage unit as a first pixel value, and registers the calculated pixel value of the second color in the storage unit as a second pixel value. .

  According to the above configuration, the first pixel value and the second pixel value used to determine the image area in the first image data of the signal light of each color by imaging of one stacked display light It can be calculated.

Preferably, the image processing device generates array data based on the second image data.
According to the above configuration, it is possible to generate array data used to determine the image area in the first image data of the signal light of each color by imaging one laminated display light.

  Preferably, each of the first pixel value and the second pixel value is defined by hue, saturation, and lightness. The control unit determines a first difference between the hue of the first pixel value and the hue of each pixel in the first image data, and a second difference between the hue of the second pixel value and the hue of each pixel. calculate. The control unit uses the array data to extract the first image area from among the areas formed by the pixels in which the first difference falls within a predetermined range, and the second difference is determined in advance. The second image area is extracted from the area formed by the pixels within the range.

  According to the above configuration, using the hue, the first image area in the first image data of the first signal light and the second image area in the first image data of the second signal light Thus, it is possible to extract an area with high accuracy in accordance with the installation environment of each stacked indicator light.

  Preferably, the control unit is configured to control each of the first image area corresponding to the first signal light when the first signal light of the first laminated signal light of the plurality of laminated signal lights is on. The average value of the pixel values of the pixels is set to the reference color when the first signal light of the first laminated display light is lit. The control unit is configured to set a first average value of pixel values of pixels in a first image area corresponding to the first signal light when the first signal light of the first stacked display light is turned off, It sets to the standard color at the time of light extinction of the 1st signal light of the lamination | stacking indicator light of this.

  According to the above configuration, the reference color when the first signal light of the first laminated indicator light is turned on and the reference color when the first signal light of the first laminated indicator light is turned off are the first laminated It becomes possible to set to the color according to the installation environment of the 1st signal light of an indicator light.

  Preferably, the control unit causes each of the first image areas corresponding to the first signal light to be lit when the first signal light of the second one of the plurality of laminated display lights is on. The average value of the pixel values of the pixels is set to the reference color when the first signal light of the second laminated display light is lit. The control unit is configured to, secondly, calculate an average value of pixel values of respective pixels in a first image area corresponding to the first signal light when the first signal light of the second laminated display light is extinguished. It sets to the standard color at the time of light extinction of the 1st signal light of the lamination | stacking indicator light of this.

  According to the above configuration, the reference color when the first signal light of the second laminated indicator light is lit, and the reference color when the first signal lamp of the second laminated indicator light is unlit, It becomes possible to set to the color according to the installation environment of the 1st signal light of an indicator light.

  Preferably, the control unit turns on and off the first signal light of the first stacked indicator light based on the reference color when the first signal lamp of the first stacked indicator light is turned on and the reference color when the first signal light of the first stacked indicator light is turned off. to decide.

  According to the above configuration, it is possible to accurately determine whether the first signal light of the first stacked display light is on or off.

  According to another aspect of the present invention, an image processing method includes the steps of acquiring image data of a place from an imaging device for imaging a place where a plurality of laminated indicator lights are installed, acquired image data, and each lamination. Determining an image area in the image data of the signal light of each color included in each stacked display light, based on the array data representing the arrangement of the stacked layers of signal lights of each color in the display light.

  According to the above method, for each stacked indicator, the image area in the first image data of the signal lamp of each color is determined by the control unit, so that the lighting and unlit of each stacked indicator can be recognized by image processing. It is possible to reduce the time and effort of initial setting in the case.

  According to the present invention, it is possible to reduce the time and effort of initial setting when recognizing lighting and extinguishing of each stacked display lamp by image processing.

It is a figure showing the image data obtained by imaging production equipment in a factory with a camera. It is a figure showing the example of system composition of an image processing stem. FIG. 2 is a functional block diagram for explaining a functional configuration of the image processing apparatus. FIG. 2 is a block diagram showing hardware included in the image processing apparatus. It is a figure showing the whole flow of processing in initialization. It is a figure for demonstrating the process which produces | generates image data. It is a flowchart for demonstrating the flow of a preset process. It is a figure showing the example of a data structure of learning data. It is a flow figure showing the flow of the search processing of the field at the time of all the lights out. It is a flow figure showing the flow of the search processing of the field at the time of full lighting. It is a figure for demonstrating the operation performed in the case of the search of an area | region. It is a figure showing the image data acquired by the imaging by a camera. It is a flowchart for demonstrating the flow of narrowing-down process. It is a figure showing the field of the red signal light. It is a figure showing the data table obtained after final determination of a field and search object color is completed. It is a figure showing the confirmation screen after main determination of a field and search object color.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description is not repeated about them.

Section 1 Application Example The image processing system of the present embodiment includes a camera that captures an image of a place where a plurality of signal towers (stacked indicator lights) are installed, and an image processing apparatus communicably connected to the camera. Image data obtained by imaging with a camera is stored in the image processing apparatus.

  FIG. 1 is a diagram showing image data G2 obtained by imaging a production facility in a factory with a camera.

  Referring to FIG. 1, image data G2 includes, as an example, a plurality of signal towers 71 to 77 as subjects. The signal tower 71 includes, as an example, a red signal light 71R, a yellow signal light 71Y, and a green signal light 71G. In the signal tower 71, a red signal lamp 71R, a yellow signal lamp 71Y, and a green signal lamp 71G are stacked in order from the top.

  Similarly, the signal tower 72 includes a red traffic light 72R, a yellow traffic light 72Y, and a green traffic light 72G. In the signal tower 72, a red traffic light 72R, a yellow traffic light 72Y, and a green traffic light 72G are stacked in order from the top. In addition, red signal lights, yellow signal lights, and green signal lights are stacked in order from the top of the signal towers 73 to 77.

  The number of signal lights and the arrangement (stacking order) of the signal lights in each of the signal towers 71 to 77 are an example, and the present invention is not limited thereto. For example, in a signal tower, four signal lights of a red light, a yellow light, a green light, and a blue light may be stacked in order from the top.

  The image processing apparatus stores array data representing an array of stacked signal lights of each color in the plurality of signal towers 71-77. For example, the image processing apparatus stores, for the signal tower 71, data representing that the red signal lamp 71R, the yellow signal lamp 71Y, and the green signal lamp 71G are stacked in order from the top. Similarly, the image processing apparatus stores, for the signal tower 72, data representing that the red signal lamp 72R, the yellow signal lamp 72Y, and the green signal lamp 72G are stacked in order from the top.

  The image processing apparatus determines an image area in the image data G2 of the signal lamp of each color included in each signal tower, based on the image data G2 obtained by imaging with the camera and the array data. For example, with regard to the signal tower 71, the image processing apparatus includes an area 711 in the image data G2 of the red traffic light 71R, an area 712 in the image data G2 of the yellow traffic light 71Y, and an image data G2 of the green traffic light 71G. And the area 713 of

  According to such a configuration, the image processing apparatus can determine the area in the image data G2 of the signal lights of the respective colors of the signal towers 71 to 77 based on the array data.

  Therefore, it is possible to omit the process in which the user inputs the area (coordinate values) in the image data G2 of the signal lights of the respective colors of the signal towers 71 to 77 into the image processing apparatus. Therefore, it is possible to reduce the time and effort of initial setting in recognizing the lighting and extinguishing of the signal tower by image processing.

22 Configuration example <A. System configuration>
FIG. 2 is a diagram showing an example of the system configuration of the image processing stem.

  Referring to FIG. 2, the image processing system 1 includes a camera 10, an image processing device 30, a PLC (Programmable Logic Controller) 40, a server device 50, and a display 60. In the factory 90, a plurality of signal towers 71 to 77 are arranged. The number of signal towers is not particularly limited.

  The camera 10 is installed in a factory 90. An object including the signal towers 71 to 77 is imaged by the camera 10. The subject also includes production equipment, wall surfaces in the factory, floor surfaces, and the like.

  The image processing device 30 is communicably connected to the camera 10. The image processing apparatus 30 is communicably connected to the PLC 40. The PLC 40 is communicably connected to a server device 50 as a higher-level device.

  The image processing apparatus 30 displays a predetermined image on a display 60 as an HMI (Human Machine Interface). Typically, the image processing device 30 causes the display 60 to display an image based on image data (for example, the above-described image data G2) obtained by imaging with the camera 10.

  In such a system configuration, the image processing apparatus 30 performs initial setting of the signal tower and image authentication of the signal tower in actual operation after the initial setting, based on the image data acquired from the camera 10. As the image authentication, it is determined whether the signal light of each color is on or off.

  In the following, for convenience of explanation, any one of the plurality of signal towers 71 to 77 is also referred to as a “signal tower 70”.

<B. Image processing device>
Next, details of the image processing apparatus 30 will be described.

(B1. Functional configuration)
FIG. 3 is a functional block diagram for explaining a functional configuration of the image processing apparatus 30. As shown in FIG.

  Referring to FIG. 3, the image processing apparatus 30 includes a control unit 310, a storage unit 320, and an acquisition unit 330.

The acquisition unit 330 acquires image data from the camera 10.
In the storage unit 320, image data G1 (see FIG. 6) and learning data D1 (see FIG. 8) are stored as data related to presetting which is a part of the initial setting. Further, in the storage unit 320, image data G2 (see FIG. 1) and a data table D2 (see FIG. 14) are stored as data related to initial setting other than the presetting.

  Specifically, the learning data D1 is generated based on the image data G1. The image data G1 will be described later. The data table D2 is generated based on the image data G2 and the learning data D1. The learning data D1 includes the above-described sequence data. The learning data D1 and the data table D2 are generated by the control unit 310.

  The control unit 310 controls the overall operation of the image processing apparatus 30. The control unit 310 performs, for example, initial setting of the signal tower and image authentication of the signal tower in actual operation after the initial setting. Further, the control unit 310 determines an image area in the image data G2 of the signal lamp of each color included in each signal tower based on the image data acquired from the camera 10 and the array data.

  The control unit 310 includes a presetting unit 311, a search target color determination unit 312, an area search unit 313, a narrowing processing unit 314, and a display control unit 315. The display control unit 315 causes the display 60 to display various images. The processing executed by each of the pre-setting unit 311, the search target color determination unit 312, and the area search unit 313 will be described later for the convenience of description.

  In the above, the configuration in which the control unit 310 generates the learning data D1 based on the image data G1 has been described as an example, but the learning data D1 may be preset. In this case, the image processing device 30 does not have to acquire the image data G1.

  In the following, after the hardware included in the image processing apparatus 30 is described, acquisition of image data G1, generation of learning data D1, acquisition of image data G2, generation of data table D2, and data table D2 are used. The processes that have been performed will be described in this order.

(B2. Hardware configuration)
FIG. 4 is a block diagram showing hardware included in the image processing apparatus 30. As shown in FIG. Referring to FIG. 4, the image processing device 30 includes an arithmetic processing circuit 350, a memory 360, and a communication interface 370. The arithmetic processing circuit 350 has a main processor 351 and an image processing dedicated processor 352.

  The memory 360 includes, for example, a ROM 361, a RAM 362, and a flash memory 363. The flash memory 363 stores an operating system, an application program, and various data. The memory 360 (specifically, the flash memory 363) corresponds to the storage unit 320 shown in FIG. The memory 360 may be configured to include an HDD (Hard Disk Drive).

  The control unit 310 illustrated in FIG. 3 is realized by the arithmetic processing circuit 350 executing the operating system and application program stored in the memory 360. When the application program is executed, various data (for example, image data sent from the camera 10) stored in the memory 360 are referred to.

  The main processor 351 controls the overall operation of the image processing apparatus 30. The image processing dedicated processor 352 executes predetermined processing on the image data sent from the camera 10. Note that, instead of the image processing dedicated processor 352, an application specific integrated circuit (ASIC) for performing image processing may be provided.

  The communication interface 370 is for communicating with an external device (for example, the camera 10, the PLC 40). The communication interface 370 corresponds to the acquisition unit 330 of FIG.

  The hardware configuration shown in FIG. 4 is an example, and the present invention is not limited to this.

<C. Setting process>
FIG. 5 is a diagram showing an overall flow of processing in initialization.

  Referring to FIG. 5, the presetting unit 311 of the image processing apparatus 30 executes presetting processing in step S1. In step S2, the search target color determination unit 312 of the image processing apparatus 30 temporarily determines a plurality of search target colors (specifically, pixel values) used in actual operation after the initial setting.

  As pixel values, three components in the HSV color space model can be used: hue (Hue), saturation (Saturation (or Chroma)), and lightness (Value (or Lightness, Brightness)). The present invention is not limited to this, and three components of RGB may be used as pixel values. Below, the example which represents the pixel value of one pixel using HSV is demonstrated.

  In step S3, the area search unit 313 of the image processing apparatus 30 searches for an area in the image data G2 using the temporarily determined search target colors. In step S4, the narrowing-down processing unit 314 of the image processing device 30 narrows down the area in the searched area. In step S5, the search target color determination unit 312 of the image processing apparatus 30 determines the coordinates (coordinate values) based on the narrowed area, and determines the search target color on the basis of the pixel value of each pixel in the narrowed area. Do.

The details of steps S1 to S5 will be described below.
(C1. Preset)
The process of step S1 in FIG. 5 will be described.

  FIG. 6 is a diagram for explaining the process of generating the image data G1. Referring to FIG. 6, the user uses camera 10 to image one signal tower 70 as a model. The signal tower 70 has a red signal lamp 70R, a yellow signal lamp 70Y, and a green signal lamp 70G. By the above-described imaging, the image processing apparatus 30 acquires image data G1 including the signal tower 70 as a subject.

  Specifically, the camera 10 captures an image in which the signal lights 70R, 70Y, and 70G are turned on and a state in which the signal lights are turned off. Thereby, the image data G1 at the time of lighting and the image data G1 at the time of lighting off can be obtained. The flow of pre-setting including these imaging processes will be described below.

FIG. 7 is a flowchart for explaining the flow of preset processing.
In step S11, the user captures an image of the signal tower 70 serving as the model with the camera 10 in a state in which the respective signal lights 70R, 70Y, 70G of the signal tower 70 serving as the model are turned off. The image processing apparatus 30 acquires the image data G1 at the time of light-out obtained by the imaging. The signal tower 70 serving as a model may be a signal tower installed in the factory 90, or any other signal tower of the same type.

  In step S12, the image processing apparatus 30 learns color information (pixel values (H, S, V)) of the respective signal lights 70R, 70Y, 70G when the respective light is off. In step S13, the image processing apparatus 30 learns the arrangement (upper and lower positional relationship) of each of the signal lights 70R, 70Y, 70G using the image data G1 at the time of light off.

  In step S14, the user captures an image of the signal tower 70 serving as the model with the camera 10 in a state in which the signal lights 70R, 70Y, and 70G of the signal tower 70 serving as the model are turned on. The image processing apparatus 30 acquires the image data G1 at the time of lighting obtained by the imaging.

  In step S12, the image processing apparatus 30 learns color information (pixel values (H, S, V)) of the respective signal lights 70R, 70Y, 70G when the respective light is on.

The image processing device 30 generates the learning data D1 by the above processing.
When learning the pixel values (H, S, V) of the signal lights 70R, 70Y, 70G, image processing such as Gaussian blurring is performed on the image data G1 at the time of turning off and the image data G1 at the time of turning on. Is preferable. This is because even in the same color area, considerable differences occur in the value of HSV.

FIG. 8 is a view showing an example of the data structure of the learning data D1.
Referring to FIG. 8, in the learning data D1, the manufacturer's name of the signal tower 70 modeled, the model number, the color of each signal light, the order, the state, the pixel value, and the tolerance regarding the hue (H) And are associated with each other. In the case of this example, “red”, “yellow”, and “green” are stored in the learning data D1 as the color of each signal light.

  Further, in the learning data D1, identification information "1" representing the first from the top is stored as the order information for "red". Similarly, in the learning data D1, identification information “2” representing the second from the top is stored as “order information” for “yellow”. Further, in the learning data D1, identification information “3” representing the third from the top is stored as “order information” for “green”.

  In the learning data D1, for each of the signal lights 70R, 70Y, and 70G, pixel values in the HSV color space at the lighting time and at the lighting time out are stored in association with each other. For example, for the red traffic light 70R, HSV = (H1, S1, V1) is associated as a pixel value at lighting, and HSV = (H2, S2, V2) is associated as a pixel value at lighting off .

  In the learning data D1, an allowable error is set for each pixel value. Specifically, in the HSV color space, tolerance is set for the hue of each pixel value. In the case of this example, tolerance values (± Hn) of the same value are associated with the values H1 to H6 of the respective hues. Tolerances are typically input by the user.

  The information on the location of the correspondence between the color and the order in the learning data D1 is an example of the array data.

  As described above, by imaging the signal tower 70 used as a model, it is possible to generate learning data D1 including array data.

  As described above, such data may be preset in the image processing apparatus 30 for each maker and each product (model number) without generating the learning data D1. In this case, the user may select preset data to be used from among a plurality of preset data by performing a user operation of selecting a maker name and a model number on the image processing apparatus 30.

(C2. Temporary determination of search target color)
Next, the process of step S2 in FIG. 5 will be described.

  As shown in FIG. 8, when the learning data D1 is generated, the image processing device 30 temporarily determines the search target color using the learning data D1. The image processing apparatus 30 temporarily determines the search target color based on only the hue (H) among the hue (H), the saturation (S), and the lightness (V).

  Specifically, the image processing apparatus 30 refers to the learning data D1 and sets the red search target color at the time of lighting to a color satisfying the condition of H1-Hn ≦ H ≦ H1 + Hn. Further, the image processing device 30 sets the search target color of red when the light is off as a color satisfying the condition of H2-Hn ≦ H ≦ H2 + Hn.

  Similarly, the image processing apparatus 30 sets the yellow search target color at the time of lighting to a color that satisfies the condition H3-Hn ≦ H ≦ H3 + Hn. Further, the image processing apparatus 30 sets the yellow search target color at the time of turning off as the color satisfying the condition of H4-Hn ≦ H ≦ H4 + Hn.

  Furthermore, the image processing apparatus 30 sets the green search target color at the time of lighting as the color satisfying the condition of H5-Hn ≦ H ≦ H5 + Hn. Further, the image processing device 30 sets the green search target color at the time of turning off as the color satisfying the condition of H6-Hn ≦ H ≦ H6 + Hn.

  The reason why only hue (H) is used for temporary determination of the search target color is that, in a real environment in the factory 90, saturation (S) and lightness (V) may differ greatly from the learning data It is for. By using only the hue, highly accurate area search (extraction) is possible.

The above conditions are summarized as follows.
Search target color of red (when lit): H1-Hn ≦ H ≦ H1 + Hn (condition Q1)
Search target color of red (when off): H2-Hn ≦ H ≦ H2 + Hn (condition Q2)
Yellow search target color (when lit): H3-Hn ≦ H ≦ H3 + Hn (Condition Q3)
Yellow search target color (when not lit): H4-Hn ≦ H ≦ H4 + Hn (condition Q4)
Green search target color (when lit): H5-Hn ≦ H ≦ H5 + Hn (condition Q5)
Search target color of green (when off): H6-Hn ≦ H ≦ H6 + Hn (condition Q6)
(C3. Search of area)
Next, the process of step S3 in FIG. 5 will be described. Below, the case where the signal lights 70R, 70Y, 70G of each color of each signal tower 70 are simultaneously lit and the case where they are individually lit will be described separately.

(1) Simultaneous lighting First, in a state where the signal lights 70R, 70Y and 70G of each color of each signal tower 70 are turned off (all off), a region satisfying the conditions Q2, Q4 and Q6 is searched. The image processing apparatus 30 acquires, from the camera 10, image data captured when all the signal lights are turned off, and searches for an area in the image data.

FIG. 9 is a flow diagram showing a flow of area search processing when all the lights are off.
Referring to FIG. 9, in step S31, the image processing device 30 selects one pixel. In step S32, the image processing apparatus 30 determines whether the pixel value of the selected pixel (specifically, the hue (H)) satisfies any one of the conditions Q2, Q4, and Q6 described above. Specifically, the image processing apparatus 30 determines whether the difference between the hue of the selected pixel and the hues H2, H4 and H6 at the pixel value shown in the learning data D1 is within the range of the tolerance ± Hn. To judge.

  When it is determined that the condition is satisfied (YES in step S32), the image processing device 30 associates and stores coordinates and a pixel value (value of HSV) in step S33. If the image processing apparatus 30 determines that the condition is not satisfied (NO in step S32), the process proceeds to step S34.

  In step S34, the image processing apparatus 30 determines whether the determination of all the pixels is completed. If the image processing apparatus 30 determines that the process has not ended (NO in step S34), the image processing apparatus 30 selects the next pixel in step S35. The selection of pixels is typically performed with regularity. For example, the image processing apparatus 30 selects the pixel next to the right, or when there is no pixel next to the right, selects the pixel on the left side of the next row.

  If the image processing device 30 determines that the determination of all the pixels is completed (YES in step S34), the region is tentatively determined for each of the conditions Q2, Q4, and Q6 in step S36. Typically, image processing device 30 includes an area in which pixels satisfying condition Q2 are collected, an area in which pixels meeting condition Q4 are collected, and an area in which pixels satisfying condition Q6 are collected. decide. Note that “an area in which pixels satisfying the conditions are gathered” typically means an area in which pixels satisfying the conditions are included at a predetermined ratio or more.

  Next, in a state where the signal lights 70R, 70Y and 70G of the respective colors of the signal towers 70 are lit (full lighting), a search for an area satisfying the conditions Q1, Q3 and Q5 is performed. The image processing apparatus 30 acquires, from the camera 10, image data captured when all the signal lights are turned on, and searches for an area in the image data.

  In addition, by receiving the lighting instruction | indication of all the signal lights from said apparatus (server apparatus 50, PLC40) of the image processing apparatus 30, all the signal lights 70R, 70Y, and 70G can be made into a lighting state.

FIG. 10 is a flow diagram showing a flow of area search processing at the time of full lighting.
Referring to FIG. 10, in step S31, the image processing device 30 selects one pixel. In step S32A, the image processing apparatus 30 determines whether the pixel value (specifically, the hue (H)) of the selected pixel satisfies any one of the conditions Q1, Q3, and Q5 described above. Specifically, the image processing apparatus 30 determines whether the difference between the hue of the selected pixel and the hues H1, H3 and H5 at the pixel value shown in the learning data D1 is within the range of the tolerance ± Hn. To judge.

  When it is determined that the condition is satisfied (YES in step S32A), the image processing device 30 associates and stores coordinates and a pixel value (value of HSV) in step S33. If the image processing device 30 determines that the condition is not satisfied (NO in step S32A), the process proceeds to step S34.

  In step S34, the image processing apparatus 30 determines whether the determination of all the pixels is completed. If the image processing apparatus 30 determines that the process has not ended (NO in step S34), the image processing apparatus 30 selects the next pixel in step S35.

  If the image processing apparatus 30 determines that the determination of all the pixels is completed (YES in step S34), in step S36, the area is tentatively determined for each of the conditions Q1, Q3, and Q5. Typically, image processing device 30 includes an area in which pixels satisfying condition Q1 are collected, an area in which pixels meeting condition Q3 are collected, and an area in which pixels satisfying condition Q5 are collected. decide.

  In the case of the configuration in which simultaneous lighting is performed as described above, the area where the pixels meeting the conditions Q1, Q3 and Q5 (or the conditions Q2, Q4 and Q6) are gathered is which signal light 70R of which signal tower 70 , 70Y, and 70G, so as to be distinguishable from each other, it is necessary to perform association by user input or the like.

(2) Individual Lighting Next, instead of simultaneous lighting (collective control), a configuration will be described in which the user turns on each of the signal lights 70R, Y, G individually to determine the area.

FIG. 11 is a diagram for describing an operation performed when searching for a region.
Referring to FIG. 11, recording of an object (image pickup of a moving image) including the signal towers 71 to 77 is started by the camera 10. At the start of recording, it is assumed that the signal lights of the respective colors of the signal towers 71 to 77 are in the off state.

  The user performs an operation of temporarily lighting the red signal light 71R at the installation position of the signal tower 71. Next, the user turns off the red signal light 71R, and then temporarily turns on the yellow signal light 71Y of the signal tower 71. Next, the user turns off the yellow signal light 71Y, and then temporarily turns on the green signal light 71G of the signal tower 71.

  Thereafter, after turning off the green signal light 71G, the user performs an operation to temporarily turn on the red signal light 72R at the installation position of the signal tower 72. Next, the user turns off the red signal light 72R and then temporarily turns on the yellow signal light 72Y of the signal tower 72. Next, the user turns off the yellow signal light 72Y and then temporarily turns on the green signal light 72G of the signal tower 72.

  Thereafter, the user repeats the above-described process at the installation position of the signal tower 73. Furthermore, the user repeats the above process at the installation position of the signal towers 74-77.

  In the case of the example of FIG. 11, the red signal light 71R of the signal tower 71 is in the lighting state in the period T1. That is, the image data of the period T1 in the recording data is at least the image data when the red signal light 71R of the signal tower 71 is lit. Therefore, the image processing apparatus 30 searches for an area satisfying the condition Q1 of the red search target color at the time of lighting in the image data of the period T1. Thereby, the image processing apparatus 30 can determine the area in which the pixels satisfying the condition Q1 are gathered (corresponding to step S36).

  Furthermore, the image processing apparatus 30 can also associate the area where the pixels satisfying the condition Q1 are gathered with the area of the red signal light 71R of the signal tower 71 based on the user input or the like.

  Next, the image processing apparatus 30 searches for an area satisfying the condition Q3 of the yellow search target color at the time of lighting in the image data of the period T2. Thereby, the image processing apparatus 30 can determine the area in which the pixels satisfying the condition Q3 are gathered (corresponding to step S36). Furthermore, the image processing apparatus 30 can also associate the area where the pixels satisfying the condition Q3 are gathered with the area of the red signal light 71Y of the signal tower 71 based on a user input or the like.

  Next, the image processing apparatus 30 searches for an area satisfying the condition Q5 of the green search target color at the time of lighting in the image data of the period T3. Thus, the image processing apparatus 30 can determine an area in which pixels meeting the condition Q5 are collected (corresponding to step S36). Furthermore, the image processing apparatus 30 can also link the area where the pixels satisfying the condition Q5 are gathered to the area of the red signal light 71G of the signal tower 71 based on the user input or the like.

  By sequentially executing such operations in the remaining signal towers 70, it is possible to determine an area in which pixels meeting the conditions Q1, Q3 and Q5 are gathered. Furthermore, it becomes possible to distinguish which of these signal towers 70 corresponds to which signal lights 70R, 70Y and 70G.

  In addition, by using the image data in a period in which none of the signal lights are on, it is possible to determine an area in which pixels meeting the conditions Q2, Q4, and Q6 are gathered. Furthermore, by executing the above-described tying process also in this case, it is possible to distinguish whether these areas correspond to which signal lights 70R, 70Y, 70G of which signal tower 70.

(C4. Narrow down)
(1) Use of Array Data The image processing apparatus 30 determines an area in which pixels meeting the condition Q1 and an area in which the pixels meeting the condition Q3 meet, which are temporarily determined by the above-described area search processing. The area is narrowed down from among the area in which the pixels satisfying the condition Q5 are gathered. First, the reason for narrowing down will be described.

  FIG. 12 is a diagram showing image data G2 'obtained by imaging with the camera 10. As shown in FIG.

  Referring to FIG. 12, the image data G2 'includes a yellow object which is not a signal light of the signal tower 70. Therefore, when the hue (H) of the area of the object satisfies the condition Q3, the image processing apparatus 30 erroneously detects that the area is an area of the yellow signal lamp 70Y. In order to eliminate such erroneous detection, the image processing device 30 uses the array data included in the learning data D1 (see FIG. 8).

FIG. 13 is a flowchart for explaining the flow of the narrowing-down process.
Referring to FIG. 13, in step S41, in the image processing apparatus 30, the area around the area where pixels satisfying the condition Q1 are gathered (hereinafter also referred to as “red area”) is a pixel satisfying the condition Q3. It is determined whether or not it is a gathering area (hereinafter also referred to as "yellow area"). Since there are a plurality of red areas, the image processing apparatus 30 executes the above determination for each area.

  If the image processing apparatus 30 determines that the area around the red area is a yellow area (YES in step S41), the image processing apparatus 30 processes the red area as an area of the red signal lamp 70R in step S42. If the image processing apparatus 30 determines that the area around the red area is not the yellow area (NO in step S41), it determines that the red area is not the area of the red signal light 70R in step S43. In this manner, the area of the red signal light 70R is narrowed from the area (the red area) in which the pixels satisfying the condition Q1 are collected.

  In step S44, the image processing apparatus 30 sets an area (red area) in which the pixels satisfying the condition Q1 are collected and a condition Q5 around the area (yellow area) in which the pixels satisfying the condition Q3 are collected. It is determined whether it is an area in which the filled pixels are gathered (hereinafter also referred to as a "green area"). Since there are a plurality of yellow areas, the image processing apparatus 30 executes the above determination for each area.

  When it is determined that the periphery of the yellow area is the red area and the green area (YES in step S44), image processing device 30 processes the yellow area as the area of yellow signal lamp 70Y in step S45. . When the image processing apparatus 30 determines that the area around the yellow area is not the red area and the green area (NO in step S44), the yellow area is not the area of the yellow signal light 70Y in step S46. to decide. Thus, the area of the yellow traffic light 70Y is narrowed from the area (yellow area) in which the pixels satisfying the condition Q3 are collected.

  In step S47, the image processing apparatus 30 determines whether the periphery of the region (green region) in which the pixels satisfying the condition Q5 are collected is the region (yellow region) in which the pixels satisfying the condition Q3 are collected. To judge. Since there are a plurality of green areas, the image processing apparatus 30 executes the above determination for each area.

  If the image processing apparatus 30 determines that the area around the green area is a yellow area (YES in step S47), the image processing apparatus 30 processes the green area as an area of the green signal lamp 70G in step S42. If the image processing device 30 determines that the area around the green area is not the yellow area (NO in step S47), it determines in step S49 that the green area is not the area of the green signal light 70G. Thus, the area of the green signal light 70G is narrowed from the area (green area) in which the pixels satisfying the condition Q5 are collected.

  In the above, narrowing down of the area from among the area in which the pixels satisfying the condition Q1 are gathered, the area in which the pixels satisfying the condition Q3 are gathered, and the area in which the pixels satisfying the condition Q5 are gathered But it is not limited to this. The image processing apparatus 30 described above from among the region in which the pixels satisfying the condition Q2 are gathered, the region in which the pixels satisfying the condition Q4 are gathered, and the region in which the pixels satisfying the condition Q6 are gathered The area may be narrowed down.

(2) Elimination of non-overlapping regions In order to improve the accuracy of the above-described narrowing process, the following process may be used in combination. Alternatively, instead of the above-described narrowing process, narrowing may be performed by the following process.

  In the above-described area search process (see “(c3. Search of area)”) by the area search unit 313, the image processing apparatus 30 generates an area in which pixels meeting the condition Q2 are gathered when all the lights are off. An area in which pixels satisfying Q4 are gathered and an area in which pixels satisfying the condition Q6 are gathered are determined. Furthermore, the image processing apparatus 30 further includes an area in which pixels satisfying the condition Q1 are collected, an area in which pixels meeting the condition Q3 are collected, and an area in which pixels satisfying the condition Q5 are collected. And decided.

  By the way, most of the area where the pixels satisfying the condition Q2 are gathered should overlap with the area where the pixels satisfying the condition Q1 are gathered. Similarly, the region in which the pixels satisfying the condition Q4 are gathered should be mostly overlapped with the region in which the pixels satisfying the condition Q3 are gathered. Most of the area in which the pixels satisfying the condition Q6 are gathered and the area in which the pixels satisfying the condition Q5 are gathered should overlap.

  Therefore, the overlapping area between the area in which the pixels satisfying the condition Q1 are collected and the area in which the pixels satisfying the condition Q2 are collected may be processed as the area of the red signal lamp 70R. Similarly, an overlapping area between the area in which the pixels satisfying the condition Q3 are collected and the area in which the pixels satisfying the condition Q4 are collected may be processed as the area of the yellow traffic light 70Y. An overlapping area between the area in which the pixels satisfying the condition Q5 are gathered and the area in which the pixels satisfying the condition Q6 are gathered may be processed as the area of the green signal lamp 70G.

(This decision of c5. Area and search target color)
Next, the main determination of the area (coordinates) and the main determination of the search target color will be described.

FIG. 14 is a diagram showing the area of the red signal lamp 70R.
Referring to FIG. 14, an area 1301 represents an area narrowed down as an area of the red signal lamp 70R. Region 1301 usually has an irregular outer shape. For the image processing apparatus 30, the area of the fixed outer shape is more convenient for data processing. Therefore, the image processing apparatus 30 extracts a rectangular area 1302 inscribed in the area 1301 from the area 1301.

  The image processing apparatus 30 determines the extracted area 1302 as a search target area. The area narrowed down as the area of the red signal lamp 70R is the number of the red signal lights 70R (seven in the case of FIG. 1 etc.), so the image processing apparatus 30 has a rectangular shape inscribed in each area. An area is extracted, and the extracted area 1302 is determined as a search target area.

  Similarly, for the area narrowed as the area of the yellow signal lamp 70Y and the area narrowed as the area of the green signal lamp 70G, the processing performed on the area narrowed as the area of the red signal lamp 70R is executed. .

  As a result, the areas (coordinates) of the respective color signal lights 70R, 70G, and 70B in the image data G2 are finally determined (fixed) for each signal tower 70.

  Next, the image processing apparatus 30 finally determines the search target color as follows. The image processing device 30 finally determines the search target color based on the pixel values of the pixels included in the above-described rectangular area. Specifically, the image processing apparatus 30 relates to a search target color (hereinafter also referred to as “search target color when turned off”, “reference color when turned off”) for determining that each signal light is turned off. Finally, a search target color (hereinafter also referred to as “search target color when lighted” and “reference color when light”) is determined.

  More specifically, the image processing apparatus 30 determines a search target color at the time of turning off based on the pixel values stored in association in step S33 in FIG. Further, the image processing apparatus 30 determines the search target color at the time of lighting based on the pixel value stored in association in step S33 of FIG.

  Typically, the image processing device 30 calculates the average of the pixel values of the pixels included in the rectangular area among the pixel values stored in association in step S33 of FIG. Specifically, the image processing device 30 calculates the average of the hue of each pixel in the rectangular area, the average of the lightness of each pixel in the rectangular area, and the average of the saturation of each pixel in the rectangular area. . The image processing device 30 finally determines the average value of the pixel values of the pixels as the search target color when the light is turned off. The image processing apparatus 30 permanently determines the search target color at the time of turning off for each rectangular area.

  Similarly, the image processing apparatus 30 calculates the average of the pixel values of the pixels included in the rectangular area among the pixel values stored in association in step S33 of FIG. Specifically, the image processing device 30 calculates the average of the hue of each pixel in the rectangular area, the average of the lightness of each pixel in the rectangular area, and the average of the saturation of each pixel in the rectangular area. . The image processing device 30 finally determines the average value of the pixel values of the pixels as the search target color at the time of lighting. Note that the image processing apparatus 30 permanently determines the search target color at the time of lighting, for each rectangular area.

  The reason for performing the above processing for each rectangular area is that at least signal towers 70 other than the signal tower 70 serving as a model are placed in an environment different from that at the time of learning. Further, the installation positions of the signal towers 70 are also different from each other.

  In particular, as described above, various factors affecting the installation environment (place) of each signal tower 70 are taken into consideration by actually determining the search target color using not only the hue but also the average of lightness and saturation. It is possible to Therefore, according to the image processing device 30, it is possible to accurately determine whether the signal lights are on or off.

  FIG. 15 is a diagram showing a data table D2 (see FIG. 3) obtained after the main determination of the region and the search target color is completed.

  Referring to FIG. 15, in the data table D2, the identification number of the signal tower 70, the color, the coordinates, the state, the pixel value, and the tolerance are associated.

  The coordinates represent the main determined area. As an example, since the area is a rectangular area, the upper left coordinates of the area and the lower right coordinates of the area are stored in the data table D2.

  The pixel value represents the main determined search target color. In the data table D2, (H, S, V) = (H1 ′, S1) as pixel values for determining lighting of the red signal lamp 70R of the first signal tower 70 (for example, the signal lamp 71R of the signal tower 71). ', V1') is stored. In the data table D2, (H, S, V) = (H2 ', S2', V2 ') are stored as pixel values for determining the turning off of the red signal lamp 70R.

  Similarly, in the data table D2, pixel values (H3 ', S3', V3 ') for determining whether the yellow signal lamp 70Y of the first signal tower 70 is lit and pixel values (H4' for determining whether the yellow signal lamp 70Y is turned off). , S4 ', V4'), the pixel value (H5 ', S5', V5 ') for determining the lighting of the green signal lamp 70G of the first signal tower 70, and the pixel value (H6' for determining the lighting-off) , S6 ', V6') are stored.

  Further, in the data table D2, each tolerance error of hue, lightness and saturation is stored in association with each pixel value. In this example, the case where the tolerance is constant at each pixel value is taken as an example. Specifically, the case where the hue tolerance is ± Hn, the lightness tolerance is ± Sn, and the saturation tolerance is ± Vn is described as an example.

  At the time of actual operation, the image processing apparatus 30 determines whether each signal light 70R, 70Y, 70G of each signal tower 70 is in the on state or in the off state using information in the data table D2. The image processing apparatus 30 also determines whether each of the signal lights 70R, 70Y and 70G is in the blinking state based on the time interval between the lighting state and the lighting state. Furthermore, the image processing device 30 can also identify a light emission pattern based on turning on and off of each signal light in the signal tower 70.

  In the above, whether each of the signal lights 70R, 70Y and 70G of each signal tower 70 is on or off using not only hue but also lightness, saturation and their errors Although the configuration for determining is described as an example, the present invention is not limited to this. The image processing apparatus 30 may be configured to determine the on state and the off state using only hue and hue errors without using lightness and saturation.

(C6. Confirmation operation)
FIG. 16 is a diagram showing a confirmation screen after the area and the search target color are finally determined.

  Referring to FIG. 16, image processing device 30 obtains data shown in data table D2 shown in FIG. 15 through the process of steps S1 to S5 shown in FIG. Perform screen output prompting The image processing device 30 causes the display 60 to display a screen on which the captured image and the frame indicating the position of the identified signal tower are transparently synthesized.

  Specifically, the image processing apparatus 30 displays on the display 60 a screen Z on which the user can perform the determination processing if the identification result is correct, and can perform the correction by the HMI operation if the identification result is not correct. Note that the user may select the enter button 1501 by screen tap or mouse operation at the time of entry processing. The HMI operation for correction includes an operation of tapping an incorrect frame on the screen, or an operation of selecting the frame by a mouse operation including the movement of the cursor 1503.

  Further, the image processing device 30 displays the set tolerance, and the user can arbitrarily change the tolerance. In the case where the setting of the tolerance can not be dealt with because the original color information is largely different or the like, the user can return to the pre-setting of step S1 and perform again by the selection operation of the cancel button 1502.

<D. Appendices>
[Configuration 1]
An imaging device (10) for imaging a place where a plurality of stacked indicator lights are installed;
An image processing device (30) communicably connected to the imaging device (10);
The image processing device (30)
A storage unit (320) storing arrangement data representing an arrangement of laminations of signal lights (70R, 70, Y, 70G) of respective colors in each of the laminated indicator lights (70);
Within the first image data (G2) of the signal light of each color included in each of the stacked indicator lights (70) based on the first image data (G2) obtained by the imaging and the array data And a control unit (310) for determining an image area in the image processing system.

[Configuration 2]
Each of the stacked indicator lamps (70) is a first signal lamp (70Y) of a first color and a second signal lamp of a second color laminated to the first signal lamp (70Y) as the signal lamps of the respective colors. Including signal lights (70R),
The arrangement data defines that the second signal lamp (70R) is positioned above the first signal lamp (70Y) in the first image data,
The storage unit (320) further stores the first pixel value divided into the first color and the second pixel value divided into the second color,
The control unit (310) generates a first signal in the first image data of the first signal light (70Y) based on the first pixel value, the second pixel value, and the arrangement data. The image processing system according to configuration 1, wherein an image area and a second image area in the first image data of the second signal light (70R) are determined.

[Configuration 3]
The control unit (310)
And pixel values of the first color based on second image data (G1) obtained by imaging the one of the plurality of stacked indicator lights (70) with the imaging device (10). Calculating the pixel value of the second color;
The calculated pixel value of the first color is stored in the storage unit (320) as the first pixel value, and the calculated pixel value of the second color is used as the second pixel value. The image processing system according to Configuration 2, which is registered in the storage unit (320).

[Configuration 4]
The image processing system according to Configuration 3, wherein the image processing apparatus (30) generates the array data based on the second image data (G1).

[Configuration 5]
Each of the first pixel value and the second pixel value is defined by hue, saturation, and lightness.
The control unit (310)
A first difference between the hue of the first pixel value and the hue of each pixel in the first image data, and a second difference between the hue of the second pixel value and the hue of each pixel Calculate
The first image area is extracted from among areas formed by the pixels in which the first difference falls within a predetermined range using the array data, and the second difference is determined in advance The image processing system according to any one of Configurations 2 to 4, wherein the second image area is extracted from an area formed by the pixels which falls within a defined range.

[Configuration 6]
The control unit (310)
The first signal lamp (70Y) of the first laminated signal lamp (71) among the plurality of laminated signal lamps (70) is lit, the corresponding to the first signal lamp (70Y) The average value of the pixel values of the respective pixels in the first image area is set to the reference color when the first signal lamp (70Y) of the first laminated indicator lamp (71) is lit,
A pixel of each pixel in the first image area corresponding to the first signal light (70Y) when the first signal light (70Y) of the first stacked display light (71) is extinguished The image processing system according to configuration 5, wherein an average value of the values is set to a reference color when the first signal light (70Y) of the first stacked display light (71) is extinguished.

[Configuration 7]
The control unit (310)
When the first signal light (70Y) of the second stacked display light (72) of the plurality of stacked display lights (70) is on, the above corresponding to the first signal light (70Y) The average value of the pixel values of the respective pixels in the first image area is set to the reference color when the first signal lamp (70Y) of the second laminated indicator lamp (72) is lit,
A pixel of each pixel in the first image area corresponding to the first signal light (70Y) when the first signal light (70Y) of the second stacked display light (72) is extinguished The image processing system according to configuration 6, wherein an average value is set to a reference color when the first signal light (70Y) of the second stacked display light (72) is extinguished.

[Configuration 8]
The control unit (310) is configured to set the first stacked indicator light based on a reference color when the first signal lamp (70Y) of the first stacked indicator light (71) is turned on and a reference color when the first signal 71. The image processing system according to configuration 6 or 7, wherein it is determined whether the first signal light (70Y) is turned on or off.

[Configuration 9]
Acquiring image data of the place from an imaging device (10) for imaging a place where a plurality of stacked indicator lights (70) are installed;
The signal light of each of the colors included in each of the stacked display lights (70) based on the acquired image data and the array data representing the arrangement of stacked layers of the signal lights of each color in each of the stacked display lights (70) Determining an image area within the image data.

  It should be understood that the embodiments disclosed herein are illustrative and non-restrictive in every respect. The scope of the present invention is shown not by the above description of the embodiments but by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

  Reference Signs List 1 image processing system, 10 cameras, 30 image processing devices, 40 PLCs, 50 server devices, 60 displays, 70 to 77 signal towers, 70R, 71R, 72R red signal lights, 70Y, 71Y, 72Y yellow signal lights, 70G, 71G, 72G Yellow traffic light, 310 control unit, 311 preset unit, 312 search target color determination unit, 313 area search unit, 314 narrowing processing unit, 315 display control unit, 320 storage unit, 330 acquisition unit, 350 arithmetic processing circuit, 351 main Processor, 352 processor dedicated to image processing, 360 memory, 361 ROM, 362 RAM, 363 flash memory, 370 communication interface, 711, 712, 713, 1301, 1302 area, 1501 confirmation button, 1502 cancel button, 15 03 Cursor, D1 learning data, D2 data table, G1, G2 image data.

Claims (9)

An imaging device for imaging a place where a plurality of stacked indicator lights are installed;
An image processing apparatus communicably connected to the imaging apparatus;
The image processing apparatus is
A storage unit storing arrangement data representing an arrangement of laminations of signal lights of each color in each of the laminated indicator lights;
A control unit that determines an image area in the first image data of the signal light of each color included in each stacked display light based on the first image data obtained by the imaging and the array data And an image processing system. Each of the stacked indicator lamps includes a first signal lamp of a first color and a second signal lamp of a second color laminated to the first signal lamp as the signal lights of the respective colors;
The arrangement data defines that the second signal light is positioned above the first signal light in the first image data,
The storage unit further stores a first pixel value divided into the first color and a second pixel value divided into the second color,
The control unit is configured to select a first image area in the first image data of the first signal light based on the first pixel value, the second pixel value, and the arrangement data. The image processing system according to claim 1, wherein a second image area in the first image data of two signal lights is determined. The control unit
The pixel value of the first color and the pixel value of the second color based on second image data obtained by the imaging device capturing one of the plurality of stacked display lights Calculate
The calculated pixel value of the first color is stored in the storage unit as the first pixel value, and the calculated pixel value of the second color is stored in the storage unit as the second pixel value. The image processing system according to claim 2, wherein registration is performed.   The image processing system according to claim 3, wherein the image processing device generates the array data based on the second image data. Each of the first pixel value and the second pixel value is defined by hue, saturation, and lightness.
The control unit
A first difference between the hue of the first pixel value and the hue of each pixel in the first image data, and a second difference between the hue of the second pixel value and the hue of each pixel Calculate
The first image area is extracted from among areas formed by the pixels in which the first difference falls within a predetermined range using the array data, and the second difference is determined in advance The image processing system according to any one of claims 2 to 4, wherein the second image area is extracted from among areas formed by the pixels which fall within a defined range. The control unit
The pixel value of each pixel in the first image area corresponding to the first signal light when the first signal light of the first laminated signal light of the plurality of laminated signal lights is on Setting the average value of the first and second signal lights of the first laminated indicator light as a reference color when the first signal light is lit,
An average value of pixel values of pixels in the first image area corresponding to the first signal light when the first signal light of the first stacked display light is extinguished is the first value. The image processing system according to claim 5, wherein the color is set to a reference color when the first signal light of the laminated indicator light is turned off. The control unit
The pixel value of each pixel in the first image area corresponding to the first signal light when the first signal light of the second laminated signal light of the plurality of laminated signal lights is on Setting the average value of the reference color when the first signal light of the second laminated indicator light is on,
An average value of pixel values of pixels in the first image area corresponding to the first signal light when the first signal light of the second stacked display light is extinguished is the second value; The image processing system according to claim 6, wherein the color is set to a reference color when the first signal light of the laminated indicator light is turned off.   The control unit turns on and off the first signal light of the first stacked indicator light based on a reference color when the first signal lamp of the first stacked indicator light is turned on and a reference color when the first signal light is turned off. The image processing system according to claim 6 or 7, wherein Acquiring image data of the place from an imaging device for imaging a place where a plurality of stacked indicator lights are installed;
Based on the acquired image data and array data representing an array of laminations of signal lights of each color in each of the laminated display lights, within the image data of the signal lights of each color included in each of the laminated display lights Determining an image area.