JP6545342B1 - Abnormality detection device and abnormality detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an abnormality detection device and an abnormality detection program capable of accurately and accurately detecting a product having an abnormality (failure) in an abnormality detection device. An abnormality detection apparatus includes a region of interest setting unit, a sequence setting unit, and an abnormality detection unit. The region of interest setting means 211 sets at least one first region of interest and at least one second region of interest at a predetermined position on the frame image. The sequence setting unit 212 performs luminance analysis on the first region of interest set in the frame image, and sets a sequence defining the timing of performing abnormality detection processing. The abnormality detection unit 213 performs luminance analysis on a second region of interest set in the frame images captured during the abnormality detection processing period according to the sequence, and performs abnormality detection processing. [Selected figure] Figure 3

Description

  The present invention relates to a technique for detecting an abnormality of an object, and in particular, an abnormality of an object based on moving image data obtained by imaging an object with a high speed camera capable of imaging at a high frame rate. And an abnormality detection program for detecting

  In order to capture changes in a subject, a high-speed camera capable of continuously capturing images at a high frame rate is used. Such an apparatus causes the display unit to display frame image data captured at a predetermined imaging frame rate by the imaging unit as a frame image at a frame rate different from that at the time of imaging. For example, in a general-purpose video camera, the imaging cycle per image is 1/30 second, that is, the frame rate is a standard speed of 30 fps (frames per second), while the high-speed camera has a frame rate of 60 to One million, for example, as high as 1000 fps. If moving image data is recorded using such a high-speed camera and a series of frame image data captured at a high frame rate is displayed on the display unit at a lower frame rate than at the time of capturing, the movement of the subject is minute It is possible to observe at a slow speed by a series of images captured each time (for example, Patent Documents 1 and 2).

  There is also disclosed an apparatus for recording only frame image data whose motion of a subject is different from normal while evaluating motion of each cycle while capturing moving image data by imaging a subject that moves periodically ( For example, Patent Document 3). This technology captures a scene with a high-speed camera that repeats a periodic operation such as a factory production line that enters and leaves a product, and records it as moving image data, while a plurality of frame image data The abnormal frame image data is extracted and recorded by dividing and analyzing for each cycle.

JP, 2009-141709, A JP, 2009-100326, A JP 2012-99978 A

  However, according to the prior art, the entire frame image data is subjected to luminance analysis to detect abnormal frame image data. In that case, if an abnormality such as burring occurs in the product, the luminance distribution is almost the same as in the normal case where the burring is not attached to the product, so the abnormality may be overlooked There is a case.

  In addition, when an abnormality occurs such as when the product is turned upside down, the luminance distribution is the same as in the case of the product in the normal order, so there is a risk that the abnormality may be missed.

  The present invention has been made in consideration of the above-described circumstances, and is an abnormality detection device capable of accurately and accurately detecting a product having an abnormality (failure) by analyzing only a portion of frame image data. And provide an anomaly detection program.

  An abnormality detection device according to an embodiment of the present invention includes a region of interest setting means, a sequence setting means, and an abnormality detection means. The region of interest setting means sets at least one first region of interest and at least one second region of interest at a predetermined position on the frame image. The sequence setting unit performs a luminance analysis on a first region of interest set in the frame image, and sets a sequence defining a timing at which abnormality detection processing is performed. The abnormality detection means performs luminance analysis on a second region of interest set among the frame images captured during the abnormality detection processing period according to the sequence, and performs abnormality detection processing.

  An abnormality detection device according to an embodiment of the present invention includes a region of interest setting means, a sequence setting means, and an abnormality detection means. The region of interest setting means sets at least one region of interest at a predetermined position on the frame image. The sequence setting unit performs a luminance analysis on a region of interest set in the frame image, and sets a sequence defining a timing at which an abnormality detection process is performed. The abnormality detection means performs luminance analysis on a region of interest set in the frame images captured during the abnormality detection processing period according to the sequence, and performs abnormality detection processing.

  An abnormality detection program according to an embodiment of the present invention has a function of setting at least one first region of interest and at least one second region of interest in a predetermined position on a frame image in a computer. A function of setting a sequence defining a timing at which abnormality detection processing is performed by performing luminance analysis in a first region of interest set out of the frame images, and a frame image captured during an abnormality detection processing period according to the sequence Luminance analysis is performed in the second region of interest set among them, and a function of performing abnormality detection processing is realized.

  An abnormality detection program according to an embodiment of the present invention performs, on a computer, a function of setting at least one region of interest at a predetermined position on a frame image, and brightness analysis on the region of interest set in the frame image. And a function of setting a sequence defining a timing at which abnormality detection processing is performed, and performing luminance analysis on a region of interest set among frame images captured during the abnormality detection processing period according to the sequence, and performing abnormality detection processing. To realize the function.

  According to the abnormality detection apparatus and the abnormality detection program according to the present invention, by analyzing the luminance of only the portion of the frame image data, it is possible to accurately detect a product having an abnormality (failure) accurately.

BRIEF DESCRIPTION OF THE DRAWINGS Schematic which shows one structural example of the abnormality detection system containing the abnormality detection apparatus which concerns on embodiment which concerns on embodiment. BRIEF DESCRIPTION OF THE DRAWINGS Schematic which shows the structural example of the abnormality detection apparatus which concerns on embodiment. The block diagram which shows the function of the abnormality detection apparatus which concerns on embodiment. The figure which shows as a flowchart the operation | movement of the abnormality detection apparatus which concerns on embodiment. The figure which shows operation | movement of the abnormality detection apparatus which concerns on embodiment as a time chart. The figure which shows frame image data at the time of applying operation | movement of the abnormality detection apparatus which concerns on embodiment to monitoring of the sealing material to which the adhesive agent was apply | coated. The figure which shows the example of the luminance analysis performed in the abnormality detection apparatus which concerns on embodiment. The figure which shows frame image data at the time of applying operation | movement of the abnormality detection apparatus which concerns on embodiment to monitoring of circuit elements, such as a capacitor | condenser. The figure which shows transition of the average luminance for every area of interest in the abnormality detection apparatus which concerns on embodiment.

  Embodiments according to the present invention will be described with reference to the attached drawings.

1. Embodiment FIG. 1 is a schematic view showing one configuration example of an abnormality detection system including the abnormality detection device according to the embodiment.

  FIG. 1 shows an abnormality detection system 1 according to the embodiment. The abnormality detection system 1 includes an imaging device 10, an abnormality detection device 20, a sequencer 30, and an operation unit 40. The imaging device 10, the abnormality detection device 20, the sequencer 30, and the operation unit 40 are connected so as to be mutually communicable via a network N such as a wired or wireless LAN (Local Area Network).

  The imaging device 10 includes an image sensor, an amplifier, an A / D (Analog to Digital) converter, and the like (not shown). The image sensor captures a moving product U as a monitoring target via an objective optical system (not shown). The amplifier amplifies the output video signal from the image sensor. The A / D converter converts an analog video signal output from the amplifier into a digital video signal.

  The imaging device 10 is installed in the facility in order to monitor the status of the moving product U in the automated facility. The imaging device 10 may be a general-purpose video camera, but is preferably a high-speed camera. When the imaging device 10 is a general-purpose video camera, the frame rate is about 30 fps, whereas when the imaging device 10 is a high-speed camera, a high frame rate of 60 to 1,000,000, for example, 1000 fps can be realized.

  The imaging device 10 continuously captures an imaging range F including a moving product U and captures moving image data. In addition, although the imaging device 10 demonstrates the case where it is a fixed camera, it is not limited to that case. For example, the imaging device 10 may be a PTZ (pan-tilt-zoom) camera. The moving image data captured by the imaging device 10 is provided to the abnormality detection device 20.

  The abnormality detection device 20 has a general configuration of a PC (Personal Computer). The abnormality detection device 20 may be a desktop PC or a notebook PC, or may be a mobile PC such as a smartphone and a tablet terminal. The abnormality detection apparatus 20 detects abnormal frame image data, that is, an abnormal product U, by inputting and analyzing each frame image data including the moving product U.

  FIG. 2 is a schematic view showing a configuration example of the abnormality detection device 20. As shown in FIG.

  As shown in FIG. 2, the abnormality detection device 20 has a configuration as a computer, and includes a control unit 21, a primary storage unit 22, a secondary storage unit 23, an input unit 24, a display unit 25, and a communication unit 26. The control unit 21 is interconnected to each hardware component constituting the abnormality detection apparatus 20 via a bus as a common signal transmission path. The control unit 21 may be configured by dedicated hardware, or may be configured to realize various functions by software processing by a built-in processor.

  The control unit 21 is a processor such as a dedicated or general-purpose central processing unit (CPU) or a micro processor unit (MPU) that executes a predetermined program, an application specific integrated circuit (ASIC), a programmable logic device, etc. It means a processing circuit. Examples of programmable logic devices include circuits such as Simple Programmable Logic Devices (SPLDs), Complex Programmable Logic Devices (CPLDs), and Field Programmable Gate Arrays (FPGAs).

  Further, the control unit 21 may be configured by a single processor or may be configured by a combination of a plurality of independent processors. In the latter case, a plurality of primary storage units 22 respectively corresponding to a plurality of processors may be provided, and a program executed by each processor may be stored in a storage circuit corresponding to the processors. As another example, one primary storage unit 22 may be configured to collectively store programs corresponding to respective functions of a plurality of processors.

  The primary storage unit 22 is configured by, for example, a semiconductor memory element such as a random access memory (RAM), a flash memory, or the like, a hard disk, an optical disk, or the like. The primary storage unit 22 may be configured as a circuit that allows removable removable media such as Universal Serial Bus (USB) memory and Digital Video Disk (DVD). The primary storage unit 22 stores various programs (including an OS (Operating System) as well as application programs) executed by the control unit 21, character information necessary for the execution of the programs, and image data. The primary storage unit 22 also stores various commands for controlling the OS, and programs of GUI (Graphical User Interface) for supporting input from the touch panel 13.

  The configuration of the secondary storage unit 23 is the same as the configuration of the primary storage unit 22, and thus the description thereof is omitted. The primary storage unit 22 is used when recording all frame image data captured by the imaging device 10. On the other hand, the secondary storage unit 23 is used to store only a plurality of frame image data before and after a frame in which an abnormality is detected among all frame image data (moving image data) captured by the imaging device 10 Be done.

  The input unit 24 includes an input device operable by the operator and an input circuit for inputting a signal from the input device. The input device includes a trackball, a switch, a mouse, a keyboard, a touch pad for performing an input operation by touching an operation surface, a touch screen in which a display screen and a touch pad are unified, a non-contact input device using an optical sensor, And a voice input device. When the operator operates the input device, the input circuit generates a signal corresponding to the operation and outputs the signal to the control unit 21.

  The display unit 25 is configured by a general display output device such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display. The display unit 25 displays various information according to the control of the control unit 21.

  The communication unit 26 is configured by a connector conforming to a parallel connection specification or a serial connection specification. The communication unit 26 connects the abnormality detection device 20 to the network N by performing communication control according to each standard.

  Returning to the description of FIG. 1, the sequencer 30 includes a control unit, a storage unit (not shown), and the like. The control unit has the same configuration as that of the control unit 21 (shown in FIG. 2), and is configured by hardware such as a processor, an FPGA, or an ASIC. The storage unit also has the same configuration as that of the primary storage unit 22 (shown in FIG. 2). The sequencer 30 controls the operation of the operation unit 40 in accordance with the drive control sequence registered in advance in the storage unit.

  The operation unit 40 is configured by an arm that grips and moves the product U, a conveyor that arranges the product U at the top and transports the product U, and the like.

  Subsequently, the function of the abnormality detection device 20 will be described.

  FIG. 3 is a block diagram showing the function of the abnormality detection device 20. As shown in FIG.

  The control unit 21 of the abnormality detection apparatus 20 executes the abnormality detection program, thereby providing the region of interest setting unit 211, the sequence setting unit 212, the abnormality detection unit 213, the output unit 214, the recording control unit 215, and the reproduction control unit 216. Function. Note that all or part of the functions of the units 211 to 116 may be provided in the abnormality detection device 20 in a circuit such as an ASIC.

  The region of interest setting unit 211 has a function of setting a plurality of regions of interest at predetermined positions on the frame image displayed on the display unit 25 via a user interface such as a GUI. Thereby, the positions of the plurality of regions of interest are determined on the displayed moving image. The form of the region of interest may be set as a one-dimensional region as a straight line or a curved line on the frame image, or may be set as a two-dimensional region formed by a rectangle, a circle, or a free curve. In addition, the forms of the plurality of regions of interest may be different from one another.

  The region of interest setting means 221 sets at least one region of interest for timing setting and at least one region of interest for anomaly detection. For example, the region of interest setting unit 221 sets one region of interest for timing setting and one region of interest for anomaly detection. The region of interest for setting the timing is for determining a sequence (hereinafter referred to as "abnormality detection sequence") defining an abnormality detection processing period for performing abnormality detection processing by performing luminance analysis on the relevant region of the frame image data. It is a thing. On the other hand, the region of interest for abnormality detection is for detecting frame image data having abnormality by performing luminance analysis on the region of frame image data during abnormality detection processing period by the abnormality detection sequence. . The plurality of regions of interest are not limited to two regions of interest, that is, one region of interest for setting timing and one region of interest for anomaly detection.

For example, in the case where the plurality of regions of interest are three regions of interest, abnormality detection is performed in each of the two regions of interest for detection during the abnormality detection processing period set in the region of interest for one timing setting. It may be done. Also, for example, in the case where the plurality of regions of interest are three regions of interest, abnormality detection is performed in one region of interest for detection respectively during the abnormality detection processing period determined in the regions of interest for setting two timings. May be performed. Furthermore, for example, in the case where the plurality of regions of interest are four regions of interest, the first region of interest for anomaly detection in the region of interest for abnormality detection determined in the region of interest for setting the first timing. While the abnormality detection is performed, the abnormality detection may be performed in the second area of interest for abnormality detection during the abnormality detection processing period determined in the area of interest for setting the second timing.

  Hereinafter, the case where the plurality of regions of interest are two regions of interest will be described. Further, the region of interest for setting the timing is referred to as a "first region of interest", and the region of interest for abnormality detection is referred to as a "second region of interest".

  The sequence setting unit 212 has a function of performing luminance analysis on the first region of interest set by the region of interest setting unit 211 in the frame image data, and setting an abnormality detection sequence.

  The abnormality detection unit 213 is configured to use, in the second region of interest set by the region of interest setting unit 211, of the frame image data captured during the abnormality detection processing period according to the abnormality detection sequence set by the sequence setting unit 212. It has a function of performing luminance analysis and performing abnormality detection processing.

  The output unit 214 has a function of outputting, via the display unit 25, an indication that an abnormality has been detected or frame image data in which an abnormality has been detected when the abnormality is detected by the abnormality detection unit 213. Further, the output unit 214 has a function of transmitting, to the external device via the communication unit 26 and the network N, the fact that an abnormality has been detected or the frame image data in which the abnormality is detected when the abnormality is detected by the abnormality detection unit 213. May be included. In addition, the output unit 214 has a function of transmitting the result of luminance analysis to be described later (for example, luminance distribution of each frame, time-series transition of edge position, etc.) to an external device via the communication unit 26 and the network N. You may have.

  When an abnormality is detected by the abnormality detection unit 213, the recording control unit 215 acquires, from the primary storage unit 22, a plurality of frame image data within a predetermined period of time before the frame in which the abnormality is detected. It has a function of controlling to record in the unit 23. In addition, when abnormality is detected by the abnormality detection unit 213, the recording control unit 215 appropriately records, in the secondary storage unit 23, a plurality of frame images within a predetermined time after the frame in which the abnormality is detected. Have the ability to control.

  The reproduction control unit 216 has a function of reproducing the frame image data stored in the secondary storage unit 23 by the recording control unit 215 on the display unit 25 in real time or slow reproduction. As a result, if a series of frame image data captured at a high frame rate by a high-speed camera is displayed on the display unit 25 at a lower frame rate than at the time of capturing, the operator captures the movement of the product U every minute time A series of images can be observed in slow motion.

  Subsequently, the operation of the abnormality detection device 20 will be described using FIGS. 4 to 6.

  FIG. 4 is a diagram showing the operation of the abnormality detection device 20 as a flowchart. In FIG. 4, each step of the code | symbol flowchart which attached the number to "ST" is shown. FIG. 5 is a diagram showing the operation of the abnormality detection device 20 as a time chart. FIG. 6 is a diagram showing frame image data in the case where the operation of the abnormality detection device 20 is applied to monitoring of the seal material U (U1 to U3) to which the adhesive is applied. FIG. 6 shows each frame in the case where the imaging device 10 captures a process (the order of (A) to (F) in FIG. 4) in which the sealing material U is transported from right to left by the conveyor 40 as the operation unit 40 Indicates image data.

  In accordance with an instruction signal from the input unit 24 by the operation of the operator, the abnormality detection device 20 starts monitoring of a product (for example, a seal material coated with an adhesive) (step ST1 in FIG. 4). The abnormality detection device 20 causes the display unit 25 to display frame image data (step ST2 in FIG. 4). Here, in step ST2, the abnormality detection device 20 may cause the display unit 25 to reproduce and display frame image data of moving image data that has already been acquired, or sequentially acquired frame image data from the imaging device 10 It is possible to sequentially receive and display it live on the display unit 25. According to the former, the operator can later verify the occurrence of a product abnormality again, and according to the latter, the operator can live-monitor the occurrence of a product abnormality. The latter will be described below.

  The region of interest setting unit 211 of the abnormality detection apparatus 20 determines whether the first region of interest G1 and the second region of interest G2 (shown in FIG. 6) are set in the frame image data (step ST3 in FIG. 4). ). If the determination in step ST3 is NO, that is, if it is determined that two regions of interest are not set in the frame image data, the abnormality detection apparatus 20 receives frame image data for the next frame, and The image data is displayed live (step ST2 in FIG. 4). In addition, when it is judged as NO in step ST3, the present time is a case between t1 frame and t2 frame shown in FIG.

  On the other hand, if the determination in step ST3 is YES, that is, if it is determined that two regions of interest are set in the frame image data, the abnormality detection apparatus 20 causes the display unit 25 to display frame image data for the next frame. Live display is performed (step ST4 in FIG. 4). If YES is determined in step ST3, the current time is the case of the t2 frame shown in FIG. 5, which is the case shown in FIG. 6 (A).

  The sequence setting unit 212 performs luminance analysis in the first region of interest G1 set by the result of step ST3 among the frame image data displayed in step ST4, and determines whether or not it is a reference frame (see FIG. Step 4 of 4). That is, the sequence setting unit 212 analyzes the luminance not for the entire frame image data but for only a partial region. The reference frame means a frame corresponding to the timing when it can be judged that the luminance value in the first region of interest has changed, that is, the timing when each product U comes to a position corresponding to the first region of interest. And the t3 frame of FIG. 6 (B). In FIG. 6B, the position of the first region of interest coincides with the rear end of the seal material U1 obtained by the luminance analysis. In the sealing material U2 subsequent to the sealing material U1, it is assumed that an adhesive sticking out abnormality (broken line) has occurred.

  FIG. 7 is a diagram showing an example of luminance analysis performed in the abnormality detection device 20. As shown in FIG.

  The sequence setting unit 212 performs the luminance analysis based on the periodic change of the luminance value indicated by the pixel group present in the first region of interest among the plurality of frame image data including the past. For example, the sequence setting unit 212 changes the period change of the average luminance (matching with the normal waveform) indicated by the pixel group present in the first region of interest (two-dimensional region) among the plurality of frame image data including the past. 7 (A) and the tracking of the edge (shown in FIG. 7 (B)) position measured from the luminance value indicated by the pixel group present in the first region of interest (one-dimensional region) Perform analysis.

  In addition, the sequence setting unit 212 performs the luminance analysis based on the luminance value indicated by the pixel group in the first region of interest in the frame image data. For example, the sequence setting unit 212 compares the threshold distribution of the brightness distribution indicated by the pixel group present in the first region of interest (one-dimensional region) (shown in FIG. 7C), the first region of interest Comparison with the threshold of the luminance distribution indicated by the pixel group present in the two-dimensional region (shown in FIG. 7D), or the luminance value indicated by the pixel group present in the first region of interest (one-dimensional region) In the histogram of luminance values (shown in FIG. 7F) indicated by the presence or absence of an edge measured from (shown in FIG. 7E) and the pixel group present in the first region of interest (two-dimensional region) Perform brightness analysis based on

  Returning to the description of FIGS. 4 to 6, the determination in step ST5 is NO, that is, when it is determined that the frame image data displayed in step ST4 is not the reference frame, the abnormality detection device 20 proceeds to the next frame. Frame image data is displayed on the display unit (step ST4 in FIG. 4).

  On the other hand, if the determination in step ST5 is YES, that is, if it is determined that the frame image data displayed in step ST4 is the reference frame, the sequence setting unit 212 sets the start timing of the abnormality detection process to the reference frame. The abnormality detection sequence is set by obtaining it as the estimated elapsed time (“R1” in FIG. 5) and determining the end timing of the abnormality detection process as the estimated elapsed time from the reference frame (“R2” in FIG. 5) (FIG. 4) Step ST6). The start timing of the abnormality detection processing means the timing when the front end of the moving seal material U takes on the second region of interest, and the end timing of the abnormality detection processing is the second end of the moving seal material U It means the timing of the area of interest.

  For example, the estimated elapsed time is an actual coordinate of the drive control sequence (for example, the transport speed of the sealing material U by the conveyor 40) held by the sequencer 30, the distance on the screen between the first region of interest and the second region of interest It can be determined by the distance converted to the system, the length and the interval of the sealing material U, and the like.

  The start timing of the abnormality detection process is shown as a t4 frame in FIG. 5 and FIG. 6C, and the end timing of the abnormality detection process is shown as a t5 frame in FIG. 5 and FIG. The inside is shown as the t4 to t5 frames in FIGS. 5 and 6D and 6E.

  The abnormality detection apparatus 20 causes the display unit 25 to live display frame image data for the next frame (step ST7 in FIG. 4). The abnormality detection unit 213 determines whether the frame image data displayed in step ST7 is data within an abnormality detection processing period according to the abnormality detection sequence set in step ST6 (step ST8 in FIG. 4). ). If the determination in step ST8 is NO, that is, if it is determined that the frame image data is not data within the abnormality detection processing period, the abnormality detection apparatus 20 causes frame image data to be displayed live for the next frame (see FIG. Step 4 of 4).

  On the other hand, if the determination in step ST8 is YES, that is, if it is determined that the frame image data is data within the abnormality detection processing period, the abnormality detection unit 213 detects the frame image data displayed in step ST7. Luminance analysis is performed on the second region of interest set by the result of step ST3, and abnormality detection processing is performed (step ST9 in FIG. 4). The method of luminance analysis is as described above with reference to FIG. As can be seen by comparing FIGS. 6 (D) and 6 (E) in the abnormality detection processing period, even if the luminance change is slight in the entire image, the luminance change is performed in the second region of interest G2. It can be easily detected.

  The output means 214 determines whether or not an abnormality is detected in step ST9 (step ST10 in FIG. 4). If the determination in step ST10 is NO, that is, if it is determined in step ST9 that no abnormality is detected, the abnormality detection device 20 proceeds to step ST12.

  On the other hand, if the determination in step ST10 is YES, that is, if it is determined in step ST9 that an abnormality has been detected, the output unit 214 outputs the fact that an abnormality has been detected or the frame image data in which the abnormality is detected (FIG. Step 4 of 4). Further, the output means 214 may cause the display unit 25 to display the fact that an abnormality has been detected or frame image data in which an abnormality has been detected, when the abnormality is detected, or via the communication unit 26 and the network N. It may be output to an external device.

  Subsequently, the abnormality detection device 20 determines whether to end the monitoring of the product (step ST12 in FIG. 4). If the determination in step ST12 is NO, that is, if it is determined that the product monitoring is not ended, the abnormality detection apparatus 20 causes frame image data to be displayed live for the next frame (step ST7 in FIG. 4).

  On the other hand, when it is determined that the monitoring of the product is to be ended according to the determination in step ST12, that is, according to the instruction signal from the input unit 24 by the operation of the operator, the abnormality detection device 20 ends the operation.

  The region of interest may be reset at any timing during the operation of steps ST4 to ST12. In that case, the operation is started again from step ST4 after the timing of resetting.

  FIG. 8 is a diagram showing frame image data when the operation of the abnormality detection apparatus 20 is applied to monitoring of a circuit element such as a capacitor. FIG. 8 is another application example of FIG. FIG. 8 accurately and accurately detects the occurrence of an abnormality such as the case where the product U is turned upside down.

  FIG. 8 shows frame image data when the operation of the abnormality detection device 20 is applied to monitoring of a circuit element U (U1, U2) such as a capacitor. A process in which the placement unit 40A as the operation unit 40 arranges the position of the circuit element U that has moved, and the circuit element U is pulled up by the arm 40B as the operation unit 40 (in order of (A) to (G) in FIG. 8) Each frame image data at the time of the imaging device 10 imaging is shown.

  The sequence setting unit 212 determines the start timing of the abnormality detection process as the estimated elapsed time (“R1” in FIG. 5) from the reference frame, and the end timing of the abnormality detection process as the estimated elapsed time from the reference frame (“D” in FIG. The abnormality detection sequence is set by obtaining as R2 ′ ′). The start timing of the abnormality detection processing in the case shown in FIG. 8 means the timing at which the entire moving circuit element U appears in the frame image data, and the end timing of the abnormality detection processing is the distance between the circuit element U and the arm 40B. Means the timing when the predetermined distance is reached. Since the circuit element U does not move for a fixed period, the abnormality detection processing period may include at least one frame.

  As can be seen by comparing FIGS. 8C and 8F within the abnormality detection processing period, even in the case where there is no abnormality in the luminance in the entire image, the luminance is easy in the second region of interest G2. Can be detected.

  As described above, according to the abnormality detection device 20, since abnormality detection processing is not performed on frame image data outside the abnormality detection processing period due to the setting of the first region of interest, abnormality in frame image data outside the abnormality detection processing period It is possible to ignore the brightness change that is not related to Further, according to the abnormality detection device 20, even a slight luminance change relating to an abnormality can be accurately and accurately detected by the setting of the second region of interest.

  When the imaging device 10 is a high speed camera, the abnormality detection device 20 can collect frame image data at, for example, 1000 fps by the high speed camera. In that case, an advantageous effect is obtained that abnormality detection can be performed with a finer time resolution (1/1000 second unit) compared to the case of frame image data of about 30 fps collected by a general-purpose video camera.

2. First Modified Example Although a plurality of regions of interest have been described, the present invention is not limited to this case, and the number of regions of interest may be one. That is, one region of interest may double as the region of interest for timing setting and the region of interest for anomaly detection. In that case, the region of interest setting unit 211 sets the region of interest (for example, only “G2” in FIG. 6A) at a predetermined position on the frame image data. The sequence setting unit 212 performs luminance analysis in the region of interest G1 of the frame image data, and sets an abnormality detection sequence. In this abnormality detection sequence, since the estimated elapsed time R1 shown in FIG. 5 is "0", only the estimated elapsed time R2 needs to be defined. The abnormality detection unit 213 performs luminance analysis on the region of interest G1 of the frame image captured during the abnormality detection processing period according to the abnormality detection sequence, and performs abnormality detection processing.

  As described above, according to the first modification of the abnormality detection apparatus 20, abnormality detection processing is not performed on frame image data outside the abnormality detection processing period due to the setting of the region of interest. Therefore, the abnormality detection according to the embodiment described above Similar to the effects of the apparatus 20, it is possible to ignore the luminance change that is not related to the abnormality in the frame image data outside the abnormality detection processing period. Further, according to the first modified example of the abnormality detection device 20, setting of the region of interest as well as the effect of the abnormality detection device 20 according to the above-described embodiment, even a slight luminance change related to the abnormality is accurately and accurately It can be detected.

  When the imaging device 10 is a high-speed camera, according to the abnormality detection device 20, as in the case of the effect of the abnormality detection device 20 according to the above-described embodiment, the abnormality detection is performed with fine time resolution (1/1000 second unit). The advantageous effect of being able to do

3. Second Modified Example In step ST6 of FIG. 4, the sequence setting unit 212 sets the drive control sequence held by the sequencer 30 to the estimated elapsed time, and the distance on the screen between the first region of interest and the second region of interest. Has been described which is determined by the distance obtained by converting the above into a real coordinate system, the length and interval of the sealing material U, and the like. However, it is not limited to that case.

  For example, the estimated elapsed time can also use the periodicity determined by the frame image acquired by the prior imaging. The case will be described.

First, the sequence setting unit 212 obtains the transition of the average luminance value for each region of interest by obtaining the average luminance indicated by the pixel group present in each region of interest of the frame image collected by imaging of the product U in advance. . By setting the region of interest in a portion that is likely to trigger an operation in each frame image, it is possible to enhance the calculation accuracy of the estimated elapsed time.

  FIG. 9 is a diagram showing the transition of the average luminance for each region of interest.

  The upper part of FIG. 9 shows the transition of the average luminance indicated by the pixel group present in the first region of interest G1 shown in FIG. The lower part of FIG. 9 shows the transition of the average luminance indicated by the pixel group present in the second region of interest G2 shown in FIG. As shown in FIG. 9, periodicity is recognized in the change of the average brightness related to the first region of interest G1 and the change of the average brightness related to the second region of interest G2. Also, at the start timing t3 of each cycle of the first region of interest G1 (the timing at which the change in luminance is equal to or higher than the threshold) and at the start timing t4 of each cycle of the second region of interest G2 (the timing at which the change in luminance is equal to or higher than the threshold). There is a time lag.

The sequence setting unit 212 obtains the time lag between the start timings as the estimated elapsed time R1. In addition, the sequence setting unit 212 obtains, as an estimated elapsed time R2, a period between the start timing t3 of the first region of interest G1 and the end timing t5 of the second region of interest G2 (the timing when the luminance change is equal to or higher than the threshold). . The abnormality detection unit 213 starts an abnormality detection process in the second region of interest after an elapse of time R1 since the first region of interest G1 detects a change of the average luminance threshold or more in actual imaging. On the other hand, the abnormality detection unit 213 ends the abnormality detection process in the second region of interest after the lapse of time R2 since the first region of interest G1 detects a change of the average luminance threshold or more in actual imaging. .

  Further, by displaying the transition of the average luminance for each region of interest shown in FIG. 9 on the display unit 25, the operator can visually recognize the predicted time of lapse.

  According to at least one embodiment described above, it is possible to accurately and accurately detect a product having an abnormality (failure) by analyzing the luminance of only the portion of the frame image data.

  While certain embodiments of the present invention have been described, these embodiments have been presented by way of example only, and are not intended to limit the scope of the invention. These embodiments can be implemented in other various forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the invention described in the claims and the equivalents thereof as well as included in the scope and the gist of the invention.

DESCRIPTION OF SYMBOLS 1 abnormality detection system 10 imaging device 20 abnormality detection device 21 control unit 22 primary storage unit 23 secondary storage unit 25 display unit 211 region of interest setting unit 212 sequence setting unit 213 abnormality detection unit 214 output unit 215 recording control unit 216 reproduction control unit

Claims (12)

Region of interest setting means for setting at least one first region of interest and at least one second region of interest at a predetermined position on the frame image;
It obtains a reference frame brightness analysis in line Ukoto in the set first regions of interest of the frame image, based on the elapsed expected time from the reference frame, a sequence that defines the timing of the abnormality detecting process Sequence setting means for setting;
An abnormality detection unit that performs an abnormality detection process by performing luminance analysis on the set second region of interest among frame images captured during an abnormality detection process period according to the sequence;
I have a,
The sequence setting means is a distance obtained by converting the distance between the set first region of interest and the set second region of interest on the screen into a real coordinate system; Determined by the transport speed of the imaging object,
Abnormality detection device. The brightness analysis in at least one of the set first region of interest and the set second region of interest is performed on a group of pixels present in a one-dimensional region of the frame image.
The abnormality detection device according to claim 1. The brightness analysis in at least one of the set first region of interest and the set second region of interest is performed on a group of pixels present in a two-dimensional region of the frame image.
The abnormality detection device according to claim 1. When an abnormality is detected by the abnormality detection unit, control is performed so that a plurality of frame images within a predetermined time before the frame in which the abnormality is detected is acquired from the primary storage unit and recorded in the secondary storage unit. And recording control means for controlling so as to timely record a plurality of frame images within a predetermined time after the frame in which the abnormality is detected, within a predetermined time,
The abnormality detection device according to any one of claims 1 to 3, further comprising: Reproduction control means for reproducing a plurality of frame images stored in the secondary storage unit on the display unit according to real time, or performing slow reproduction
The abnormality detection device according to claim 4, further comprising: An output for controlling the communication unit to transmit information indicating the detection of the abnormality and a plurality of frame images stored in the secondary storage unit to the external device when the abnormality is detected by the abnormality detection unit. means,
The abnormality detection device according to claim 4 or 5, further comprising: The output unit controls the communication unit to transmit the result of the luminance analysis to the external device.
The abnormality detection device according to claim 6. It said sequence setting means determines the start timing of the abnormality detection processing from the elapsed expected time from the previous SL reference frame, by obtaining the end timing of the abnormality detection processing from the elapsed expected time from the reference frame, the sequence To set
The abnormality detection device according to any one of claims 1 to 7. The region of interest setting means sets a plurality of first regions of interest respectively at different positions on the frame image;
The sequence setting unit obtains the reference frame in accordance with a combination of analysis results obtained by performing luminance analysis on the plurality of first regions of interest among the frame images.
The abnormality detection device according to any one of claims 1 to 8. The sequence setting means is a distance obtained by converting the distance between the set first region of interest and the set second region of interest on the screen into a real coordinate system; In addition to the transport speed of the imaging target, it is determined by the length and interval of the transport direction of the imaging target,
The abnormality detection device according to any one of claims 1 to 9. The frame image is an image acquired by a high speed camera,
The abnormality detection device according to any one of claims 1 to 10 . On the computer
Setting at least one first region of interest and at least one second region of interest at a predetermined position on the frame image;
It obtains a reference frame brightness analysis in line Ukoto in the set first regions of interest of the frame image, based on the elapsed expected time from the reference frame, a sequence that defines the timing of the abnormality detecting process With the function to set,
A function of performing an abnormality detection process by performing luminance analysis on the set second region of interest among frame images captured during an abnormality detection process period according to the sequence;
To achieve,
The function of setting the sequence is a distance obtained by converting the distance between the set first region of interest and the set second region of interest on the screen into a real coordinate system. And the transport speed of the imaging object,
Abnormality detection program.

Images