CN105391971A - Image processing device, image processing system and image processing method - Google Patents

Image processing device, image processing system and image processing method Download PDF

Info

Publication number
CN105391971A
CN105391971A CN201510494581.3A CN201510494581A CN105391971A CN 105391971 A CN105391971 A CN 105391971A CN 201510494581 A CN201510494581 A CN 201510494581A CN 105391971 A CN105391971 A CN 105391971A
Authority
CN
China
Prior art keywords
image
surveyed area
frame
sudden inspiration
difference
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510494581.3A
Other languages
Chinese (zh)
Other versions
CN105391971B (en
Inventor
大村庆二
长谷川健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ricoh Co Ltd
Original Assignee
Ricoh Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2015041577A external-priority patent/JP6554825B2/en
Application filed by Ricoh Co Ltd filed Critical Ricoh Co Ltd
Publication of CN105391971A publication Critical patent/CN105391971A/en
Application granted granted Critical
Publication of CN105391971B publication Critical patent/CN105391971B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Image Processing (AREA)

Abstract

The invention relates to an image processing device, an image processing system and an image processing method. The invention aims to freely generate a trigger time which is a time benchmark for image judging, etc. The image processing device of the invention comprises a trigger area designation portion (305) designating trigger areas in a display area of video data of a display device (317) so as to decide a trigger time, a changing point detecting portion (306) used for obtaining frames within a set time scope of frames of video data replayed at a time point of a designated trigger area and calculating the difference of pixel values between each frame and the preceding frame. According to the difference, changing points of an image in the difference detection trigger area are detected according to the difference. The image processing device of the invention also comprises a trigger generation portion (313) used for comparing images of the trigger areas of the video data of the difference images. Trigger signals are generated with triggering as timing.

Description

Image processing apparatus, image processing system and image processing method
Technical field
The present invention relates to image processing apparatus, image processing system and image processing method.
Background technology
The video data that image processing system is used to the camera head such as video tape recorder is recorded is analyzed, detect and in evaluating objects object or video image object action and exception or make video.In recent years, the development progress of the image monitoring system of this purposes is remarkable.The video of the disclosed image monitoring system record check object of such as patent documentation 1 (JP Laid-Open 07-248214 publication), the signal (the inspection signal of such as proximity transducer or photoelectric sensor etc.) occurred with testing fixture is for having a sudden inspiration, when occurring when having a sudden inspiration, the frame according to the video data recorded carries out judgement image.Technical scheme disclosed in patent documentation 1 needs carry out image judge time, in the frame of the formation video data that recorded, find from frame in the past check object thing appear at image judge normal position on frame, image judgement is carried out to the frame found.
But what the technical scheme that patent documentation 1 describes judged for image with the signal that testing fixture occurs has a sudden inspiration, and thus exists and is fixed the opportunity of having a sudden inspiration, be difficult to the problem adjusting opportunity flexibly.
Summary of the invention
The object of the invention is to, in view of the above problems, providing a kind of can become image and judges the image processing apparatus on the opportunity of having a sudden inspiration of waiting time benchmark, image processing system and image processing method by freely generated.
In order to achieve the above object, the invention provides a kind of image processing apparatus, wherein possess with lower module: display module; Playback module, the video data for being taken by camera head is reset and is shown to described display module; To have a sudden inspiration Region specification module, in the first viewing area of the described display module of the described video data of display, specify region of having a sudden inspiration; Detection module, have a sudden inspiration the time point in region for benchmark described in specifying with described Region specification module of having a sudden inspiration, from in the described video data in the first specified time limit of this time point, the change point of having a sudden inspiration described in detection in the image in region, and extract the image of having a sudden inspiration in region described in the frame of this change point, as the first image; And, generation module, in the second viewing area for the described display module of the video data by the shooting of display described camera head, described in the image of having a sudden inspiration in region compare with described first image, have a sudden inspiration according to comparative result generation.
Effect of the present invention is, can provide a kind of can become image and judge having a sudden inspiration opportunity of waiting time benchmark by freely generated.
Accompanying drawing explanation
Fig. 1 is that the entirety of image processing system of the present embodiment forms schematic diagram.
Fig. 2 is a routine hardware configuration module map of the information processor of the first execution mode.
Fig. 3 is the functional block diagram of the information processor of present embodiment.
Fig. 4 is the routine key frame shown in the display unit of information processor of the present embodiment, wherein shows multiple live video.
Fig. 5 be display unit in the information processor that relates to of the first execution mode setting screen on the schematic diagram of display video data state.
Fig. 6 is in the information processor related at the first execution mode, the video display part in the setting screen of display unit is specified one of region of having a sudden inspiration illustrate intention.
Fig. 7 is in the information processor related at the first execution mode, the schematic diagram of the example mark display when setting screen shown on the display apparatus finding the change point of having a sudden inspiration in region.
Fig. 8 is that the first execution mode is at the schematic diagram of change detected point action in region of having a sudden inspiration.
Fig. 9 is in the information processor related at the first execution mode, the schematic diagram of example display dialogue when finding 2 change points in region that the setting screen shown on the display apparatus is had a sudden inspiration.
Figure 10 is in the information processor related at the first execution mode, the schematic diagram of example display dialogue when not finding change point in region that the setting screen shown on the display apparatus is had a sudden inspiration.
Figure 11 is used for specifying on video display part one of surveyed area to illustrate intention in the setting screen that the display unit of the information processor that the first execution mode relates to shows.
Figure 12 is used in the setting screen that the display unit of the information processor that the first execution mode relates to shows on video display part, specify one of second surveyed area to illustrate intention.
Figure 13 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and selects one of the state in the region of having a sudden inspiration set with the video data different from the video data that video display part shows to illustrate and is intended to.
Figure 14 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and have selected one of the state after the region of having a sudden inspiration set with the video data different from the video data that video display part shows and illustrates and be intended to.
Figure 15 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and specifies one of the state in the region of having a sudden inspiration set with the video data different from the video data that video display part shows to illustrate and is intended to.
Figure 16 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and video display part sets the attribute information of appointed surveyed area one illustrates intention.
Figure 17 is routine view setting screen that the display unit of the information processor related at the first execution mode shows changing the surveyed area size that video display part is specified.
Figure 18 is routine view setting screen that the display unit of the information processor related at the first execution mode shows changing the area size of having a sudden inspiration that video display part is specified.
Figure 19 be the information processor related at the first execution mode display unit on show, the schematic diagram of the state of display video data in a routine sensitivity determinating preview screen.
Figure 20 be the information processor related at the first execution mode display unit on show, the schematic diagram of a routine judged result of sensitivity determinating preview screen being extracted difference between image and detection reference image.
Figure 21 be the information processor related at the first execution mode display unit on show, in sensitivity determinating preview screen by different sensitivity to the schematic diagram extracting the routine judged result that difference rejudges between image and detection reference image.
Figure 22 be the information processor related at the first execution mode display unit on show, the schematic diagram that confirms the routine the dialog box whether sensitivity after reorganization suitable in sensitivity determinating preview screen.
Figure 23 be the information processor related at the first execution mode display unit on show, an example reflects the schematic diagram of the sensitivity after sensitivity determinating preview screen changes on setting screen.
Figure 24 is the set action flow chart that the example that the display unit of the information processor that the first execution mode relates to shows specifies have a sudden inspiration region and surveyed area on setting screen.
Figure 25 is on the key frame of display unit display of the information processor related at the first execution mode, the schematic diagram of state after have a sudden inspiration region and surveyed area are specified.
Figure 26 is that an illustration picture of the information processor that the first execution mode relates to judges action flow chart.
Figure 27 is on key frame that the display unit of the information processor related at the first execution mode shows, and when having a sudden inspiration in region of having a sudden inspiration, the show state in this region of having a sudden inspiration and surveyed area are at the schematic diagram of the show state on the opportunity of detection.
Figure 28 is the schematic diagram of the routine dialog box failing in the setting screen shown in the display unit of the information processor that the first execution mode relates to occur when benchmark image being detected in appointed area.
Figure 29 is used to change point in the region of having a sudden inspiration of description second execution mode and detects the schematic diagram of action.
Figure 30 is the set action flow chart of setting screen that the display unit of information processor of the second embodiment shows being specified have a sudden inspiration region and surveyed area.
Embodiment
The execution mode of image processing apparatus of the present invention, image processing system and image processing method is described below with reference to accompanying drawing.It is pointed out that following execution mode is not construed as limiting the present invention.Comprise in execution mode the technical staff concerning field of the present invention for apparent, essence is identical and belong to the key element of so-called impartial category.And then, allow in the subject area not departing from following execution mode, omit, replace, change and combine the various key elements in execution mode.
" the first execution mode "
The overall structure > of < image processing system
Fig. 1 is the overall structure schematic diagram of the image processing system that the first execution mode relates to.The overall structure of image processing system 1 of the present embodiment is described below with reference to Fig. 1.
As shown in Figure 1, the image processing system 1 of present embodiment comprises camera head 2a ~ 2f, information processor 3 and network 4.
Camera head 2a ~ 2f is video tape recorder, is used for the light from subject to be converted to the signal of telecommunication, and shooting subject, generates the animated video data being formed (such as 10FPS) with multiple image data.Such as, camera head 2a ~ 2f takes the production equipment of product manufacturing, generating video data, is used for the exception occurred in testing product work.
At this, then when without the need to distinguish camera head 2a ~ 2f, these camera heads are commonly referred to as camera head 2.Image processing system 1 in Fig. 1 has six camera heads, but the quantity of camera head 2 that image processing system of the present invention has is not limited thereto.
Information processor 3 is personal computer or work station etc., is used for judging image based on the video data that camera head 2 is taken.Information processor 3 is connected with peripheral hardwares 10 such as manufacturing equipments, can according to such as Fieldbus specifications communication.Judge that process will in following detailed description about the image based on video data.
Network 4 adopts Ethernet (registered trade mark) specification, is used for connecting camera head 2a ~ 2f and information processor 3.Network 4 carries out data communication according to agreements such as TCP (TransmissionControlProtocol)/IP (InternetProtocol).Camera head 2a ~ 2f has the MAC Address for carrying out communicating according to ICP/IP protocol with information processor 3, but also is endowed the IP addresses such as private ip address.Network 4 be specifically configured to such as star-like wiring mode, with LAN (LocalAreaNetwork) wire camera head 2a ~ 2f and information processor 3 be connected respectively to and have on the switch terminal of multiple port
Network 4 shown in Fig. 1 is for according to tcp/ip communication, but the present invention is not limited thereto, information processor 3 one side such as also can be adopted to be provided with multiple GA (VideoGraphicsArray) terminal or USB (UniversalSerialBus) port, and image of multiple camera 2 is respectively by VGA wire or USB wire link information processing unit.
" hardware configuration of information processor "
Fig. 2 is a routine hardware configuration module map of the information processor of the first execution mode.The hardware configuration of the information processor 3 of present embodiment is described below with reference to Fig. 2.
As shown in Figure 2, information processor 3 possesses CPU (CentralProcessingUnt) 101, ROM (ReadOnlyMemory) 102, RAM (RandomAccessMemory) 103, external memory 104, display 105, network interface 106, keyboard 107, mouse 108, DVD (DigitalVersatileDisc) driver 109 and Peripheral Interface 111.
CPU101 is used for control information processing unit 3 action on the whole.ROM102 is used to the Nonvolatile memory devices of the program of preserving information processor 3.RAM103 is intended for the volatile storage of CPU101 working region.
External memory 104 is used to preserve the storage devices such as the HDD (HardDiskDrive) of video data that camera head 2 takes and the various data such as set information or SSD (SolidStateDrive).
Display 105 is used for the various information such as display highlighting, menu, window, word or image, is also used for display information processor 3 carries out image to judge the picture of process application software.CRT (CathodeRayTube) display, liquid crystal display, plasma display or organic EL (Electroluminesence) display etc. such as can be used as display 105.Can with VGA line or HDMI (registered trade mark, High-DefinitionMultimediaInterface) line etc., or display 105 is connected on information processor 3 by Ethernet line.
Network interface 106 is interfaces that information processor 3 interconnection network 4 carries out data communication, such as, be the NIC (NetworkInterfaceCard) of the protocol communication according to TCP/IP.Be specially information processor 3 and obtain video data via network 4 and network interface 106 from camera head 2.
Keyboard 107 is used to the input unit of input characters and numeral, the various instruction of selection and set information.Mouse 108 is used to the input unit selected and perform various instruction, selection handling object, mobile cursor and select set information.
DVD driver 109 is used for controlling to read or write data from loading and unloading medium such as DVD110.
Peripheral Interface 111 to be connected with peripheral hardware 10 for image processing apparatus 3 and to carry out data communication, for such as according to the interface card of Fieldbus specifications communication.Be specially, information processor 3 carries out data communication via Peripheral Interface 111 in peripheral hardware 10.
Above-mentioned CPU101, ROM102, RAM103, external memory 104, display 105, network interface 106, keyboard 107, mouse 108, DVD driver 109 and Peripheral Interface 111 connect with the bus such as address wire or data wire 112, can communicate mutually.When display 105 connects with Ethernet line, be connected with network interface 106, carry out data communication according to agreements such as TCP/IP.
The modular structure > of < information processor
Fig. 3 is the functional block diagram of the information processor of present embodiment.Below with reference to the modular structure of Fig. 3 descriptor processing unit 3.
As shown in Figure 3, information processor 3 possesses video receiving part 301, storage part 302, input unit 303, first reproduction unit 304 (playback module), to have a sudden inspiration Region specification portion 305 (Region specification module of having a sudden inspiration), change point test section 306 (detection module), surveyed area specifying part 307 (surveyed area designated module), configuration part 308 (setting module), second reproduction unit 309, image zooming-out portion 310, difference judging part 311, video distribution portion 312, generating unit of having a sudden inspiration 313 (generation module), surveyed area judging part 314 (surveyed area judge module), display control unit 315, outside efferent 316 (outside output module), display unit 317 (display module).
Video receiving part 301 carries out data communication via network 4 and camera head 2, accepts the functional module of video data from camera head 2.The video data received is saved in storage unit by video receiving part 301.The function of video receiving part 301 is realized with the network interface 106 shown in Fig. 2.In order to simplify, in Fig. 3, eliminate the explanation of network 4.
Storage part 302 is the storage devices preserving video data and various set informations etc.Storage part 302 can identify that video data that video receiving part 301 receives preserves this data after being the data that generate of which camera head 2.Use-case RAM103 as shown in Figure 2 or external memory 104 realize the function of storage part 302.
Input unit 303 is used for carrying out allowing information processor 3 carry out the operation input of image procossing, is realized by the keyboard 107 shown in Fig. 2 and mouse 108.
First reproduction unit 304 is operation signals of the input unit according to user operation, and the video data obtained from storage part 302 is sent to display control unit 315, allows display control unit 315 be reset by video data the function part be shown in display unit 317.Be specially, video data is reset on the video display part 502 of the setting screen 500 being shown to the display unit display that following Fig. 5 etc. describes by the first reproduction unit 304.Use-case CPU101 executive program as shown in Figure 2 realizes the function of the first reproduction unit 304.
Region specification portion 305 of having a sudden inspiration is the operation signals inputted according to user operation input unit 303, specifies to be used for the functional module in the region of having a sudden inspiration determining the opportunity of having a sudden inspiration in the viewing area (the video display part 502 described in Figure 5 below) of the video data of display unit 317.The information in the region of having a sudden inspiration of specifying is saved in storage part 302 by Region specification portion 305 of having a sudden inspiration.Refer in the information in this region of having a sudden inspiration, such as, in the viewing area of video data, represent the information such as the coordinate of regional location of having a sudden inspiration and the shape in region of having a sudden inspiration.Realize having a sudden inspiration with the CPU101 executive program shown in Fig. 2 the function in Region specification portion 305.
Change point test section 306 is the function parts with following function, namely when Region specification portion 305 of having a sudden inspiration specify have a sudden inspiration region, obtain the first reproduction unit 304 at the appointed time reset display video data frame time front and back specified time limits in frame, and the difference (the first difference) of pixel value between each frame of obtaining of calculating and the previous frame of this frame, and then according to the difference obtained, detect the change point in image in region of having a sudden inspiration.Will at following detailed description about change point.About the calculating of difference, such as, compare the pixel value that two frames are had a sudden inspiration in region, obtain the Volume Composition that pixel value reaches the pixel of the above difference of regulation sensitivity, using this total as difference.When frame is black and white image, pixel value is consistent with gray value, thus can also compare the gray value of having a sudden inspiration in region between two frames, obtains the total that gray value reaches the pixel quantity of the above difference of regulation sensitivity, using this total as difference.And when frame is the image formed with rgb pixel value, compare the pixel value of any one color in R (red), G (green), B (orchid), then same as described above, obtain the Volume Composition that pixel value reaches the pixel of the above difference of regulation sensitivity, using this total as difference.And then, edge detection process is carried out to the image of frame, obtains the total of marginal portion, using this total as difference.With shown in Fig. 2 CPU101 executive program realize the function of change point test section 306.
Surveyed area specifying part 307 is the operation signals inputted according to user operation input unit 303, specify the functional module of surveyed area, this surveyed area detects abnormal image in the viewing area (video display area 502) of video data in display unit 317 to judge handling object.The information of the surveyed area of specifying is saved in storage part 302 by surveyed area specifying part 307.Refer in the information of this surveyed area, such as, in the viewing area of video data, represent the information such as the coordinate of surveyed area position and the shape of surveyed area.In addition, as will be described, surveyed area specifying part 307 can specify multiple surveyed area in the viewing area of video data.With shown in Fig. 2 CPU101 executive program realize the function of surveyed area specifying part 307.
Configuration part 308 is the operation signals inputted according to user operation input unit 303, and set information is also saved in the functional module of storage unit by setting set information.Configuration part 308 also will need the direct information display control unit 315 shown in the set information in storage part 302, allow display control unit 315 be shown in display unit 317 by set information.With shown in Fig. 2 CPU101 executive program realize the function of configuration part 304.
Second reproduction unit 309 is operation signals of the input unit according to user operation, and the video data obtained from storage part 302 is sent to display control unit 315, allows display control unit 315 be reset by video data the function part be shown in display unit 317.Be specially, on the video display part 702 of the sensitivity determinating preview screen 700 in the display unit 317 that the second reproduction unit 309 is used for shown in following Figure 19 etc., playback display video data.With shown in Fig. 2 CPU101 executive program realize the function of the second reproduction unit 309.
Resetting the viewing area (video display part 702) of the video data shown from the second reproduction unit 309 in image zooming-out portion 310, extracts the functional module of the image (extraction image) of the part corresponding with the surveyed area that surveyed area specifying part 307 is specified.With shown in Fig. 2 CPU101 executive program realize the function in image zooming-out portion 310.
Difference judging part 311 is used to calculate the image at the surveyed area of surveyed area fixed time, i.e. the functional module of the difference between the extraction image that detection reference image, and image zooming-out portion 310 extracts.Difference Calculation is carried out according to above-mentioned change point test section 306 computational methods.With shown in Fig. 2 CPU101 executive program realize the function of difference judging part 311.
Video distribution portion 312 is operation signals of the input unit according to user operation, and the live video data that video receiving part 301 receives is sent to display control unit 315, allows the functional module of display control unit 315 display video data in display unit 317.Be specially, video data is assigned to the upper display of video display part 401a to 401f in the display unit 317 shown in following Fig. 4 and Figure 25 in key frame 400 by video distribution portion 312.With shown in Fig. 2 CPU101 executive program realize the function in video distribution portion 312.
Generating unit of having a sudden inspiration 313 be on the key frame 400 shown in following Figure 25 video display part 401a to 401f have a sudden inspiration in region, by the image of the appointed area of the frame of poor partial image and video data, generate the functional module of signal of having a sudden inspiration in the time of having a sudden inspiration.With shown in Fig. 2 CPU101 executive program to realize having a sudden inspiration the function of generating unit 313.
Surveyed area judging part 314 be receive have a sudden inspiration generating unit 313 generate have a sudden inspiration signal time setting-up time before or after setting-up time, the image of the surveyed area of the detection reference image (the second image) in the surveyed area of more above-mentioned video display part 401a to 401f and the frame of video data, judges whether to there is abnormal functional module.With shown in Fig. 2 CPU101 executive program realize the function of surveyed area judging part 314.
Display control unit 315 is used to the functional module controlling display unit 317 display action.What the set information that the video data that the first reproduction unit 304 and the second reproduction unit 309 specifically obtain by display control unit 311, configuration part 308 set, Region specification portion 305 of having a sudden inspiration specified have a sudden inspiration region and the surveyed area etc. that surveyed area specifying part 307 is specified are shown in video distribution portion 312.With shown in Fig. 2 CPU101 executive program (driver etc.) realize the function of display control unit 315.
Outside efferent 316 is when surveyed area judging part 314 judges to occur abnormal, to the functional module of peripheral hardware 10 output abnormality signal.Outside efferent 316 is the Peripheral Interface 111 shown in Fig. 2.Outside efferent 316 output abnormality signal is established at this, but the present invention is not limited thereto, externally efferent 316 can also be adopted to send autoexec, in order to occur so that lettergram mode notice is abnormal, or extremely there is measures such as (such as representing with the flicker of emergency warning lamp icon) in notice on key frame 400.
Display unit 317 is used for showing various data according to the control of display control unit 315.In the present embodiment display unit 312 by shown in Fig. 2 CPU101 executive program show following application software picture 400, setting screen 500 and sensitivity determinating preview screen 70.Display unit 317 is displays 105 of Fig. 2.
In addition to software, part or all function in the first reproduction unit 304 shown in Fig. 3, Region specification portion 305 of having a sudden inspiration, change point test section 306, surveyed area specifying part 307, configuration part 308, second reproduction unit 309, image zooming-out portion 310, difference judging part 311, video distribution portion 312, generating unit of having a sudden inspiration 313, surveyed area judging part 314 and display control unit 315 can also be realized with hardware circuits such as ASIC (ApplicationSpecificIntegratedCircuit) or FPGA (Field-ProgrammableGateArray).
Present embodiment only illustrates the first reproduction unit 304, Region specification portion 305 of having a sudden inspiration, change point test section 306, surveyed area specifying part 307, configuration part 308, second reproduction unit 309, image zooming-out portion 310, difference judging part 311, video distribution portion 312, generating unit of having a sudden inspiration 313, surveyed area judging part 314 and display control unit 315 concept, this structure is not construed as limiting the present invention.
Below conveniently, display control unit 315 is shown to the data received display unit 317 is expressed as simply is shown in display unit 317 (or display frame of display unit 317).Such as, in description first reproduction unit 304, video data is sent to display control unit 315, allow display control unit 315 reset in display unit 317 display time, be expressed as the first reproduction unit 304 simply and video data be sent to display unit 317 and reset display.
< key frame forms >
Fig. 4 is the routine key frame shown in the display unit of information processor of the present embodiment, wherein shows multiple live video.Below with reference to Fig. 4, the formation of key frame 400 and the display action of multiple live video are described.
CPU101 executive program (application software), display unit 317 shows the key frame 400 shown in Fig. 4.The video data that on key frame 400, live display receives from multiple camera head 2, that specifies in Region specification portion 305 of having a sudden inspiration has a sudden inspiration in region, and generating unit of having a sudden inspiration 313 generates at the time point of having a sudden inspiration signal of having a sudden inspiration.Key frame 400 also, before the setting-up time that surveyed area judging part 314 receives when having a sudden inspiration signal or after setting-up time, judges whether exception occurs in surveyed area specifying part 307.As shown in Figure 4, key frame 400 is provided with video display part 401, camera selection portion 401a to 401f, monitors beginning key 411, monitors stop key 412 and monitor setting key 413.
Video display part 401a to 401f is used for the operation signal that display video dispenser 312 inputs according to user operation input unit 303 respectively, the live video data obtained from camera head 2a to 2f by video receiving part 301.When without the need to distinguish video display part 401a to 401f or by general designation, be simply referred to as video display part 401.
User operation input unit 303, press (effective equally by mouse 108 clicking operation) after monitoring beginning key 411, video distribution portion 312 shows live video data respectively on video display part 401a to 401f.And, press and monitor beginning key 411, on video display part 401 after display video data, that specifies in Region specification portion 305 of having a sudden inspiration has a sudden inspiration in region, generating unit of having a sudden inspiration 313 generates at time point of having a sudden inspiration signal of having a sudden inspiration, surveyed area judging part 314, before receiving setting-up time when having a sudden inspiration signal or after setting-up time, judges whether exception occurs in surveyed area.
User operation input unit 303, press after monitoring stop key 412, generating unit of having a sudden inspiration 313 stops at the generation action that the time point of having a sudden inspiration generates signal of having a sudden inspiration, surveyed area judging part 314 stops judging whether abnormal judgement action occurs in surveyed area, video distribution portion 312 stops the display action of live video data.
User operation input unit 303, press after monitoring setting key 412, configuration part 308 shows the setting screen 500 shown in following Fig. 5 in display unit 317.
The formation > of < setting screen
Fig. 5 be display unit in the information processor that relates to of the first execution mode setting screen on the schematic diagram of display video data state.The formation of setting screen 500 is described below with reference to Fig. 5.
Setting screen 500 is used to appointment and has a sudden inspiration region and surveyed area, and sets the picture of sensitivity, threshold value and the time point that in surveyed area, image judges.As shown in Figure 5, setting screen 500 has camera selection portion 501, video display part 502, playback key 503a, Pause key 503b, fast forward button 503c, rewind key 503d, stop key 503e, drag bar 504, slide block 505, time showing portion 506, area information portion 507 of having a sudden inspiration, reference pictures display part 508a, difference picture display part 508b, surveyed area information portion 509, surveyed area key portion 510, reading setting key 511, write setting key 512, setting reflection key, close key 51.
Camera selection portion 501 is used to allow user from the video data of camera head 2a ~ 2f, select some video datas needing to specify have a sudden inspiration region and surveyed area, is shown to the operational module on video display part 502.Such as, the data that camera head 2a ~ 2f takes are correspondence 1 ~ 6 respectively, shows selection 6 in the example shown in Fig. 5, namely selects the video data of camera head 2f.
Video display part 502 is the display modules showing the video data obtained from storage part 302 for the first reproduction unit 304 according to the operation signal playback that user operation input unit 303 inputs.In the example shown in Fig. 5, when after the camera selection portion 501 selecting " 6 ", display unit 317 shows the video data that camera head 2f takes.
User operation input unit 303, after pressing playback key 503a, the first reproduction unit 304 starts the video data that playback of video display part 502 shows.And user operation input unit 303, after pressing Pause key 503b, the first reproduction unit 304 suspends the video data that video display part 502 is being reset.User operation input unit 303, after pressing fast forward button 503c, the first reproduction unit 304 broadcasts the video data that video display part 502 is being reset fast.User operation input unit 303, after pressing rewind key 503d, the first reproduction unit 304 refunds the video data that video display part 502 is being reset fast.User operation input unit 303, after pressing stop key 503e, the video data that the first reproduction unit 304 stops display video display part 502 resetting.
Dragging bar 504 be bar shape, its resetting according to the positional representation video display part 502 of the slide block 405 dragging setting on bar 504 among time corresponding to video data.Slide block 505 is slide mass, and it is reset along with video display part 502 and slides into position corresponding with the shooting time (the shooting moment of the frame shown) of the video data reset on dragging bar 404.And if user operation input unit 303 makes slide block 405 slide, then the first reproduction unit 304 shows the frame of the video data dragging camera time corresponding to bar 504 top shoe 505 position on video display part 502.
Time showing portion 506 be used to show by the first reproduction unit 304 resetting on video display part 502 display the shooting date of video data and the display module of shooting time.
Area information portion 507 of having a sudden inspiration is used to show the title in region of having a sudden inspiration of being specified by Region specification portion 305 of having a sudden inspiration and detects the sensitivity of above-mentioned change point and the display module of threshold value for change point test section 306.Benchmark image display part 508a is used to the display module showing the benchmark image that change point test section 306 extracts.Difference image display part 508b is used to show the display module of difference image that change point test section 306 extracts (will in following description).
Surveyed area information portion 509 is used to show display module and the selection operational module of the method (Fig. 5 is expressed as image and judges) (below these information in surveyed area being called attribute information) that the title of surveyed area, sensitivity, threshold value, testing lag time (being expressed as time of delay in Figure 5), the method for supervisioning (being expressed as supervision in Fig. 5), the presence or absence (being expressed as action in Fig. 5) of action and the image of being specified by surveyed area specifying part 307 judge.
Surveyed area key portion 510 is used to editor and is specified and the key group of the attribute information in the related detection region shown by surveyed area information portion 509 by surveyed area specifying part.Surveyed area key portion 510 comprises action key 510a, image judges options button 510b, delete key 510c, sensitivity judge that key 510d, sensitivity increase key 510e, sensitivity reduces key 510f, threshold value increases key 510g, threshold value reduces key 510h.The function of each key in surveyed area key portion 510 will in following description.
User operation input unit 303, press after reading setting key 511, the set informations such as the information of have a sudden inspiration region and the surveyed area of configuration part 308 reading and saving in storage part 302, on each display part of the setting screen 500 shown by these set informations.User operation input unit 303, after pressing write setting key 512, the set informations such as the information of have a sudden inspiration region and the surveyed area of setting on setting screen 500 are saved in storage part 302 (external memory 104) by configuration part 308.User operation input unit 303, after pressing reflection setting key 513, configuration part 308 makes the set informations such as the information of have a sudden inspiration region and the surveyed area of setting on setting screen 500 remain valid (being such as temporarily kept in RAM103) before the application software such as key frame 400 and setting screen 500 are closed.
User operation input unit 303, after pressing close key 514, setting screen 500 is closed in configuration part 308, and display unit 317 shows key frame 400.
< has a sudden inspiration the appointment in region and the detection > of change point
Fig. 6 is in the information processor related at the first execution mode, the video display part in the setting screen of display unit is specified one of region of having a sudden inspiration illustrate intention.Fig. 7 is in the information processor related at the first execution mode, the schematic diagram of the example mark display when setting screen shown on the display apparatus finding the change point of having a sudden inspiration in region.Fig. 8 is that the first execution mode is at the schematic diagram of change detected point action in region of having a sudden inspiration.Fig. 9 is in the information processor related at the first execution mode, the schematic diagram of example display dialogue when finding 2 change points in region that the setting screen shown on the display apparatus is had a sudden inspiration.Figure 10 is in the information processor related at the first execution mode, the schematic diagram of example display dialogue when not finding change point in region that the setting screen shown on the display apparatus is had a sudden inspiration.Below with reference to Fig. 6 to Figure 10, the have a sudden inspiration appointment in region and the detection of change point are described.
Setting screen 500 shown in Fig. 6 be presented at the first reproduction unit 304 reset display video data video display part 502 on specify the state in region 530 of having a sudden inspiration.Specific as follows.First, the time point that the image section of having a sudden inspiration during display video data that video display part 502 is reset occurs, user operation input unit 303, after pressing Pause key 503b, first reproduction unit 304 stops at display video data that video display part 502 resets, and in time showing portion 506, show shooting date and the shooting time of the frame of this time point.Then, after user operation input unit 303 (as electric shock operating mouse 108), to have a sudden inspiration region 530 in the upper appointment of video display part 502 (the first viewing area) in Region specification portion 305 of having a sudden inspiration.
Setting screen 500 shown in Fig. 7 shows state change point being detected in the region 530 of having a sudden inspiration that change point test section 306 specifies in Region specification portion 305 of having a sudden inspiration.Specific as follows, first, when Region specification portion 305 of having a sudden inspiration specify have a sudden inspiration region 530, change point test section 306 obtain the first reproduction unit 304 at the appointed time reset display video data frame shooting time before and after specified time limit (such as before and after 2 seconds during) (the first stipulated time) in frame, and extract the image in the region 530 of having a sudden inspiration of Region specification time point of having a sudden inspiration, using this image as benchmark image.Refer at this change point and start to occur the time point of predetermined variation at image in region 530 of having a sudden inspiration, this time point generating unit 313 that is used as to have a sudden inspiration generates the benchmark of the time of signal of having a sudden inspiration.The detection method of change point test section 306 change detected point is below described in detail in detail.
As shown in Figure 8, if change point test section 306 obtains 13 frames (frame (0) is to frame (12)) in the stipulated time.Now, change point test section 306 obtain centered by frame (6) front and back specified time limits in frame.For convenience, by the image of the part corresponding to region 530 of having a sudden inspiration in each frame also referred to as frame (0) to frame (12).Then, the difference of pixel value between each frame of obtaining of change point test section 306 calculating and the previous frame of this frame.The computational methods of difference are described above.Fig. 8 (a) shows the frame that change point test section extracts, and Fig. 8 (b) is the schematic diagram of the difference showing each frame obtained according to sequential.In Fig. 8 (b), the difference of display is that the such as difference of frame (1) is the difference between this frame (1) and frame before (0).Be defined as follows as the IN point of change point and OUT point.That is, IN point refers in continuous print frame, and in region 530 of having a sudden inspiration, image starts the time point changed.OUT point refers in continuous print frame, and in region 530 of having a sudden inspiration, image starts the time point terminating change.In addition, the difference between each frame that change point test section 306 calculating obtains and the previous frame of this frame, but the present invention be not limited to calculate with previous frame between difference, such as can also calculate frame and several before frame between difference.
Change point test section 306 finds out the candidate of IN point and the candidate of OUT point according to the difference of each frame.Specific as follows, namely when the difference of continuous two or more frame reaches more than defined threshold (first threshold), initial difference is reached the candidate frame of frame as IN point of more than defined threshold.In the example shown in Fig. 8 (b), the difference of frame (1) does not reach threshold value, and the difference of frame (2) is more than threshold value, and the difference of frame (3) is also more than threshold value.Namely the difference of frame (2) and frame (3) two frames reaches more than threshold value continuously, and thus, initial difference reaches the candidate frame of frame (2) for IN point of more than threshold value.Equally, frame (7) is also the candidate frame of IN point.If not continuous plural frame, and only have the difference of a frame to reach more than threshold value, then may there is impurity, thus not using the candidate frame of this frame as IN point.
On the other hand, change point test section 306, when the difference of continuous plural frame does not reach defined threshold (first threshold), does not reach the candidate frame of initial frame as OUT point of defined threshold using difference.In the example shown in Fig. 8 (b), the difference of frame (4) is more than threshold value, and the difference of frame (5) does not reach threshold value, and the difference of frame (6) does not reach threshold value yet.Namely the difference of frame (5) and continuous two frames of frame (6) does not reach threshold value, and thus, initial difference does not reach the candidate frame of frame (5) for OUT point of threshold value.At this, although the difference of frame (12) does not reach threshold value, frame (13) is not thereafter within the frame of change point test section 306 extraction, and therefore, frame (12) can not as the frame candidate of OUT point.If not continuous plural frame, and the difference of a frame is only had not reach threshold value, then owing to there is the possibility of impurity, thus not using the candidate frame of this frame as OUT point.
As mentioned above, in the example of Fig. 8, change point test section 306 determines the candidate frame of IN point and the candidate frame of OUT point, the candidate frame that frame (2) and frame (7) are IN point, and frame (5) is the candidate frame of OUT point.And when multiple IN point candidate frame, using near the frame of the intermediate frame (6) in the frame obtained and frame (7) frame as IN point.In the example of Fig. 8, the candidate frame of OUT point only has a frame and frame (5), but identical with the candidate of IN point, also has the candidate frame of multiple OUT point.In this case, change point test section 306 is identical with the candidate frame of IN point, the frame of the intermediate frame near the frame obtained is decided to be the frame of OUT.And in the example of Fig. 8, the candidate frame of OUT point only has frame (5), thus frame (5) is just decided to be the frame of OUT point by change point test section 306.As mentioned above, change point test section 306, by determining the frame of IN point and the frame of OUT point from the frame in the specified time limit obtained, carrys out change detected point (IN point and OUT point).In addition, change point test section 306, when change detected point, can also use the previous frame of the frame of change point or a rear frame, or the frame of front and back, detect the action thing of having a sudden inspiration in region 530, expression action thing is saved in storage part 302 from the Vector Message in which direction.In this case, generating unit of having a sudden inspiration 313 Vector Message preserved in storage part 303, confirms that whether the direction of action of action thing is consistent with the direction that Vector Message represents, and generates according to confirmation result the information of having a sudden inspiration.
The detection method detecting change point in region 530 of having a sudden inspiration with change point test section 306 is not limited to said method, as long as can limit the change point of time of origin of having a sudden inspiration, does not need mud to arrest in how change point to be detected.
In addition, as mentioned above, initial difference, when detecting that continuous plural frame reaches more than defined threshold, is reached the candidate frame of frame as IN point of more than defined threshold by change point test section 306.But the present invention is not limited thereto.In other words, also can be the difference of the frame of more than other quantity, instead of the difference of plural frame, reach more than defined threshold continuously.This is applicable to the candidate frame determining OUT point too.
The example of Fig. 8 display is the frame simultaneously finding IN point and OUT point both sides.To this, change point test section 306 also can only find the frame of IN point or the frame of OUT point.Now, these IN points or OUT point are detected as change point.Change point test section 306 extracts in the frame of the change point detected the image in region 530 of having a sudden inspiration, using this image as difference image (the first image), and the difference image of the benchmark image of extraction and the frame of change point is saved in storage part 302, benchmark image display part 508a shows benchmark image, and show difference image on difference image display part 508b.And then, the part corresponding to the time of the frame of the change point that change point test section 306 detects in dragging bar 504 shows mark 540 of having a sudden inspiration.When change point test section 306 detects change point, the information in the region 530 of having a sudden inspiration of specifying is saved in storage part 302 by Region specification portion 305 of having a sudden inspiration.Configuration part have a sudden inspiration area information portion 507 shows the appointment named by Region specification portion 305 of having a sudden inspiration have a sudden inspiration region 530 title and for the regulation sensitivity (such as 50) of change point test section 306 change detected point and defined threshold (such as 20%) (default value).About the name of region 530 title of having a sudden inspiration, as shown in Figure 7, such as configuration part 308 " TR6 " title as the region 530 of having a sudden inspiration of specifying in the video data of the camera head (camera head 2f) of " 6 ".
Setting screen 500 shown in Fig. 9 be change point test section 306 when above-mentioned two change points (IN point and OUT point) being detected in the region 530 of having a sudden inspiration that Region specification portion 305 of having a sudden inspiration specifies the change point of display select the state of dialog box 600.Like this, change point test section 306 is when more than 2 change points being detected, display unit 317 shows change point and selects dialog box 600, be used for selecting whether utilizing putting to generate closest to any one in two change points (IN point and OUT point) of benchmark image extraction time and have a sudden inspiration.Change point is selected dialog box 600 has and is selected IN point key 601 and select OUT point key 602 and close key 603.Close key 603 is used for closing change point and selects dialog box 600.
When user operation input unit 303, when pressing selection IN point key 601, change point test section 306 detects IN point, using this IN point as change point, extracts the image of having a sudden inspiration in region 530 of the frame of IN point, as difference image.Then, difference image in the frame of the benchmark image of extraction and IN point is saved in storage part 302 by change point test section 306, and benchmark image is shown on benchmark image display part 508a, difference image display part 508b shows difference image, and then part corresponding to the time of the frame of the IN point detected in dragging bar 504 shows mark 540 of having a sudden inspiration.
On the contrary, when user operation input unit 303, when pressing selection OUT point key 602, change point test section 306 detects OUT point, using this OUT point as change point, extracts the image of having a sudden inspiration in region 530 of the frame of OUT point, as difference image.Then, difference image in the frame of the benchmark image of extraction and OUT point is saved in storage part 302 by change point test section 306, and benchmark image is shown on benchmark image display part 508a, difference image display part 508b shows difference image, and then part corresponding to the time of the frame of the IN point detected in dragging bar 504 shows mark 540 of having a sudden inspiration.Other actions are identical in the above-mentioned action shown in Fig. 7.
Like this, the IN point detected by change point test section 306 and OUT point are as change point, and any point selecting among above-mentioned 2 by allowing user, user can select the change point really needing to specify.
Setting screen 500 shown in Figure 10 is the states not having change point notification dialog window 610 shown when the change point of having a sudden inspiration in region that change point test section 306 does not find Region specification portion 305 of having a sudden inspiration to specify.In the action that above-mentioned Fig. 8 describes, the difference yet can obtained does not reach defined threshold, and change point test section 306 fails to detect the situation of change point.In this case, change point test section 306 shows unchanged some notification dialog window 610 in display unit 317, represents and does not find change point.
Unchanged some dialog box 610 have OK key 611 and close key 612.Close key 612 is used for closing unchanged some dialog box 610.User operation input unit 303, when pressing 0K key 611, change point test section 306 does not preserve any information because of storage part 302, benchmark image reality portion 508a and difference image display part 508b does not show any image, closes unchanged some notification dialog window 610.
The setting > of the appointment of < surveyed area and the attribute information of surveyed area
Figure 11 is used for specifying on video display part one of surveyed area to illustrate intention in the setting screen that the display unit of the information processor that the first execution mode relates to shows.Figure 12 is used in the setting screen that the display unit of the information processor that the first execution mode relates to shows on video display part, specify one of second surveyed area to illustrate intention.Figure 13 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and selects one of the state in the region of having a sudden inspiration set with the video data different from the video data that video display part shows to illustrate and is intended to.Figure 14 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and have selected one of the state after the region of having a sudden inspiration set with the video data different from the video data that video display part shows and illustrates and be intended to.Figure 15 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and specifies one of the state in the region of having a sudden inspiration set with the video data different from the video data that video display part shows to illustrate and is intended to.Figure 16 is in the setting screen that the display unit of the information processor related at the first execution mode shows, and video display part sets the attribute information of appointed surveyed area one illustrates intention.Below with reference to Figure 11 to Figure 16, the setting of the appointment of surveyed area and the attribute information of surveyed area is described.
Setting screen 500 shown in Figure 11 is presented at reset video display part 502 middle finger of display video data of the first reproduction unit 304 and determines the state of surveyed area 535.Specific as follows.First, on video display part 502 video data playback display during in when there is the image section needing to carry out image judgement, user operation input unit 303, after pressing Pause key 503b, first reproduction unit 304 suspends audio data of heavily making a house call on video display part 502, and in time showing portion 506, show shooting date and the shooting moment of the frame of this time point.Then, after user operation input unit 303 (such as pulling mouse 108), surveyed area specifying part 307 shows surveyed area 535 on video display part 502.But, need when have a sudden inspiration to have have selected region of having a sudden inspiration (being TR6 in fig. 11) of being specified by Region specification portion 305 of having a sudden inspiration in area information portion 507, surveyed area specifying part 307 can specify surveyed area 535.In other words, the surveyed area 535 that surveyed area specifying part 307 is specified is associated with the region of having a sudden inspiration be selected in area information portion 507 of having a sudden inspiration.
Configuration part 308 shows the attribute information of surveyed area 535 in surveyed area information portion 509, the appointed surveyed area 535 namely named by surveyed area specifying part, be by default worth use in following image judges regulation sensitivity (such as " 50 ") and defined threshold (such as " 20% "), supervision method (such as " gray scale "), have attonity (such as "None") and image decision method (such as " unanimously ").When naming surveyed area 535 title, configuration part 308 as shown in figure 11, such as, names the name in first region be associated with region of having a sudden inspiration " TR6 " to be called " K61 ".Configuration part 308 also calculates time of delay (being " 2.14 " in Figure 11) (judging the time) of time of change point that surveyed area specifying part 307 is specified the time of the frame of the time of surveyed area 535, detected relative to change point test section 306, and is presented in surveyed area information portion 509 as attribute information this time of delay.That is, in this case, be named as in the surveyed area 535 of " K61 ", after " 2.14 " second being set in the time of having a sudden inspiration, carry out image judgement.The attribute information of the surveyed area 535 of display in surveyed area information portion 509 is saved in storage part 302 by configuration part 308.
Surveyed area specifying part 307 is extracted in the fixed time of specifying surveyed area 535, and the image of surveyed area 535, as detection reference image, and is saved in storage part 302, preserves the information of the surveyed area 535 of specifying in storage part 302.Setting screen 500 shown in Figure 12 shows, and the video display part 502 of display video data of being reset by the first reproduction unit 304 specifies second surveyed area, the i.e. state of surveyed area 536.Specific as follows, first, during display (time-out) video data that video display part 502 is reset, user operation input unit 303, pull the slide block 505 dragged on bar 504, adjustment needs the displaying time point of the image section carrying out image judgement.The shooting date of the frame of the video data that the first reproduction unit 304 in time showing portion 506, display video display part 502 is suspended and shooting time.
Then, after user operation input unit 303 (such as pulling mouse 108), surveyed area specifying part 307 shows second surveyed area on video display part 502, i.e. surveyed area 536.But, identical with during appointment surveyed area 535, surveyed area specifying part 307 need when have a sudden inspiration to have have selected region of having a sudden inspiration (being TR6 in fig. 12) that specified by Region specification portion 305 of having a sudden inspiration in area information portion 507, surveyed area 536 can be specified.In other words, the surveyed area 536 that surveyed area specifying part 307 is specified is associated with the region of having a sudden inspiration be selected in area information portion 507 of having a sudden inspiration.
Configuration part 308 shows the attribute information of surveyed area 536 in surveyed area information portion 509, the appointed surveyed area 536 namely named by surveyed area specifying part, be by default worth use in following image judges regulation sensitivity (such as " 50 ") and defined threshold (such as " 20% "), supervision method (such as " gray scale "), have attonity (such as "None") and image decision method (such as " unanimously ").When naming surveyed area 536 title, as shown in figure 12, the name nominating in first region that such as will be associated with region of having a sudden inspiration " TR6 " is " K62 " in configuration part 308.Configuration part 308 also calculates surveyed area specifying part 307 and specifies the time of the frame of the time of surveyed area 536, the time of delay of the time of change point detected relative to change point test section 306 (being "-1.86 " in Figure 12), and is presented in surveyed area information portion 509 as attribute information this time of delay.In other words, be now set in and be named as in the surveyed area 536 of " K62 ", before the 1.86 seconds time of having a sudden inspiration, carried out image judgement.The attribute information of the surveyed area 536 of display in surveyed area information portion 509 is saved in storage part 302 by configuration part 308.
Surveyed area specifying part 307 is extracted in the image of the surveyed area 536 of the fixed time of specifying surveyed area 536, and using this image as detection reference image, is saved in storage part 302, preserves the information of the surveyed area 536 of specifying in storage part 302.
Setting screen 500 shown in Figure 13 shows, resetting on the video display part 502 of display video data in the first reproduction unit 304, selecting the state in appointed region of having a sudden inspiration in the video data of the camera head 2 different from the camera head 2 taking the video data shown of resetting.Specific as follows, after pressing the drop-down key 507a in area information portion 507 of having a sudden inspiration, there is the drop-down menu 507b under " zone name of having a sudden inspiration " column in first user operation input unit 303.Area arrangement of having a sudden inspiration appointed in the camera data of multiple camera head 2 is presented in drop-down menu 507b by configuration part 308.
Setting screen 500 shown in Figure 14 shows, user operation input unit 303, selects " TR2 " that have a sudden inspiration in region that show in drop-down menu 507b.At this, the region of having a sudden inspiration that the video data of zone name of the having a sudden inspiration camera head (camera head 2b) that to be the region of having a sudden inspiration of " TR2 " be such as to " 2 " is specified.
Specific as follows.First, user operation input unit 303, the some regions of having a sudden inspiration (being " TR2 " in the example of Figure 14) in the drop-down menu 507b in area information portion 507 of selecting to have a sudden inspiration in the region of having a sudden inspiration (having in the example of Figure 13 " TR2 " and " TR6 ") that shows.Then, configuration part 308 obtains sensitivity and the threshold value in the region of having a sudden inspiration " TR2 " selected by input unit 303 from storage part 302, and is shown to and has a sudden inspiration in area information portion 507.Owing to there is not the surveyed area be associated with region of having a sudden inspiration " TR2 " in the video data of the camera head of " 6 ", therefore, the display of configuration part 308 attribute information of the surveyed area 535 and 536 of display in deletion surveyed area information portion 509.And then change point test section 306 obtains benchmark image and the difference image in region of having a sudden inspiration " TR2 " from storage part 302, and benchmark image and difference image are presented on benchmark image display part 508a and difference image display part 508b respectively.
Setting screen 500 shown in Figure 15 be presented at by the first reproduction unit 304 reset display video data video display part 502 on specify the 3rd surveyed area, the i.e. state of surveyed area 537.Specific as follows.First, during display (time-out) video data that video display part 502 is reset, user operation input unit 303, pulls the slide block 505 dragged on bar 504, and adjustment needs the displaying time point of the image section carrying out image judgement.The shooting date of the frame of the video data that the first reproduction unit 304 in time showing portion 506, display video display part 502 is suspended and shooting time
Then, after user operation input unit 303 (such as pulling mouse 108), surveyed area specifying part 307 shows the 3rd surveyed area, i.e. surveyed area 537 on video display part 502.Now, the surveyed area 537 that surveyed area specifying part 307 is specified is associated by the region of having a sudden inspiration " TR2 " selected with area information portion 507 of having a sudden inspiration.
Configuration part 308 shows the attribute information of surveyed area 536 in surveyed area information portion 509, the appointed surveyed area 537 namely named by surveyed area specifying part, be by default worth use in following image judges regulation sensitivity (such as " 50 ") and defined threshold (such as " 20% "), supervision method (such as " gray scale "), have attonity (such as "None") and image decision method (such as " unanimously ").When naming surveyed area 537 title, configuration part 308 as shown in figure 15, such as, names the name in first region be associated with region of having a sudden inspiration " TR2 " to be called " K21 ".That configuration part 308 also determines change point test section 306 to detect, to be specified the fixed time of surveyed area 537 by surveyed area specifying part 307, relative to the default value of the time of delay (being " 1.50 " in Figure 15) of the time of change point in the region of having a sudden inspiration " TR2 " of the video data of the camera head (camera head 2b) of " 2 " as attribute information, be presented in surveyed area information portion 509.The attribute information of the surveyed area 537 of display in surveyed area information portion 509 is saved in storage part 302 by configuration part 308.
Surveyed area specifying part 307 extracts the image in the appointment moment of specifying surveyed area 537 in surveyed area 537, and using this image as detection reference image, is saved in storage part 302, and in storage part 302, preserve the information of the surveyed area 537 of specifying.
As mentioned above, the region of having a sudden inspiration that the video data of other camera heads 2 with the camera head 2 be different from involved by video data that video display part 502 showing is specified can be associated and specify surveyed area by present embodiment, so just, can launch contact between multiple camera head 2, image is judged, and degree of freedom leap improves.
Setting screen 500 shown in Figure 16 be presented in area information portion 507 of having a sudden inspiration select have a sudden inspiration region " TR6 " (region 530 of having a sudden inspiration) state under, from the surveyed area " K61 " (surveyed area 535) and " K62 " (surveyed area 536) of surveyed area information portion 509 display attribute information selection " K62 " state.The editor of the attribute information of surveyed area is described below with reference to Figure 16.And the sensitivity adjustment of surveyed area after judging key 510d about the sensitivity pressing surveyed area key portion 510 will describe at Figure 19 to Figure 22.
First as shown in figure 16, user's input operating device 303, after attribute information is displayed on any one surveyed area selected in surveyed area information portion 509, the display section of the attribute information of the surveyed area of the surveyed area information portion 509 that configuration part 308 reversion display is selected.In the following description, if as shown in figure 16, select surveyed area 536 (surveyed area name is called " K62 ").
Then, such as, user operation input unit 303, after clicking action key 510a, set point with presence or absence of the action that configuration part 308 switches selected surveyed area 536.Such as when the action of surveyed area 536 have inaction "None", after clicking action key 510a, configuration part 308 by the action of surveyed area 536 with or without being set as " having ".On the contrary, when the action of surveyed area 536 has inaction " to have ", after clicking action key 510a, configuration part 308 by the action of surveyed area 536 with or without being set as "None".As will be described, when action has inaction " to have ", during the image in this surveyed area judges, when surveyed area judging part 314 judges to occur abnormal, outside efferent 316 output abnormality signal.
For another example, user operation input unit 303, after judging options button 510b by motion video, the set point of the method that the picture that configuration part 308 switches chosen surveyed area 536 judges.Such as, when the method that the picture of surveyed area 536 judges is as " unanimously ", after judging options button 510b by motion video, the method that the picture of surveyed area 536 judges is set as " inconsistent " by configuration part 308.On the contrary, when the method that the picture of surveyed area 536 judges is as " inconsistent ", after judging options button 510b by motion video, the method that the picture of surveyed area 536 judges is set as " unanimously " by configuration part 308.
As the following description that will carry out, when the method that picture judges is as " unanimously ", if the difference between the image extracted and detection reference image is more than the threshold value of attribute information, then surveyed area judging part 314 judges exception.Part as " unanimously ", such as, is inserting in the equipment of work package, is inserting the state of work package for detection reference image, carry out image judgement at part to the time that work package shift action terminates with part by the method judged about setting screen.Now, insert the baulk of work package at part, part is failed under the normal state inserted, and surveyed area judging part 314 is carried out image and judged, if the difference between the image extracted and detection reference image is more than the threshold value of attribute information, then judge to occur extremely.
On the contrary, when the method that picture judges is as " inconsistent ", if the difference between the image extracted and detection reference image does not reach the threshold value of attribute information, then surveyed area judging part 314 judges to occur extremely.The method judged about setting screen as " inconsistent ", such as, have and is inserting in the equipment of work package by part, do not insert the state of work package for detection reference image, carry out image judgement at part to the time that work package shift action terminates with part.Now, insert the baulk of work package at part, part is failed under the normal state inserted, and surveyed area judging part 314 is carried out image and judged, and if difference between detection reference image do not reach the threshold value of attribute information, then judge to occur abnormal.Such as be difficult to prepare as finished product insert the situations such as the work package of part under, can be insert the state of work package as detection reference image using part.
When user operation input unit 303, after pressing delete key 510c, the attribute information of the surveyed area 536 of display in surveyed area information portion 509 is deleted in configuration part 308, and deletes the attribute information of the surveyed area 536 preserved in storage part 302.Surveyed area specifying part 307 deletes the surveyed area 536 of display on video display part 502, and deletes the information of detection reference image and the surveyed area 536 preserved in storage part 302.
When user operation input unit 303, press after sensitivity judges key 510d, configuration part 308 shows the sensitivity determinating preview screen 700 that following Figure 19 waits display in display unit 317.
When user operation input unit 303, after pressing sensitivity increase key 510e, configuration part 308 increases the sensitivity of selected surveyed area 536 being carried out to use when image judges.Equally, user operation input unit 303, after pressing sensitivity minimizing key 510f, the sensitivity of selected surveyed area 536 being carried out to use when image judges is reduced in configuration part 308.
When user operation input unit 303, after pressing threshold value increase key 510g, configuration part 308 increases the threshold value of selected surveyed area 536 being carried out to use when image judges with setting.Equally, user operation input unit 303, press threshold value reduce key 510 and after, the threshold value of selected surveyed area 536 being carried out to use when image judges is reduced in configuration part 308 with setting.
< changes surveyed area and the area size > that has a sudden inspiration
Figure 17 is routine view setting screen that the display unit of the information processor related at the first execution mode shows changing the surveyed area size that video display part is specified.Figure 18 is routine view setting screen that the display unit of the information processor related at the first execution mode shows changing the area size of having a sudden inspiration that video display part is specified.Below with reference to Figure 17 and Figure 18 be described in change surveyed area and have a sudden inspiration region time action.
Setting screen 500 shown in Figure 17 is presented at the first reproduction unit 304 and resets on the video display part 502 of display video data, changes the state of surveyed area 535 size of specifying.Specific as follows.First, playback display (time-out) period of video data on video display part 502, user operation input unit 303, the mobile slide block 505 dragged on bar 504, readjusts the displaying time of the part needing the image carrying out image judgement.First reproduction unit 304 shows shooting date and the shooting time of the frame of the video data be suspended in time showing portion 506.
Then, after user operation input unit 303 (such as pulling mouse 108), altered for size surveyed area as the size of video display part 502 is specified second surveyed area and surveyed area 536, and is set as surveyed area 536a by surveyed area specifying part 307 Change Example.Now, the time that frame when surveyed area specifying part 307 changes surveyed area 536a size is recalculated in configuration part 308, the time of delay (being "-0.86 " in Figure 17) of time of change point detected relative to change point test section 306, and this time of delay is presented in surveyed area information portion 509 as attribute information.Configuration part 308 is also saved in surveyed area information portion 509 in storage part 302 through the attribute information of the surveyed area 536a upgrading display.
Surveyed area specifying part 307 is extracted in the time changing surveyed area 536a size, and the image in surveyed area 536a, as detection reference image, and is saved in storage part 302, preserves the information of the surveyed area 536a specified in storage part 302.
Setting screen 500 shown in Figure 18 is presented at the first reproduction unit 304 and resets on the video display part 502 of display video data, changes the state of region 530 size of having a sudden inspiration.Specific as follows.First, playback display (time-out) period of video data on video display part 502, user operation input unit 303, the mobile slide block 505 dragged on bar 504, readjusts the displaying time of the part needing the image carrying out image judgement.First reproduction unit 304 shows shooting date and the shooting time of the frame of the video data be suspended in time showing portion 506.
Then, after user operation input unit 303 (such as pulling mouse 108), Region specification portion 305 of having a sudden inspiration changes the size in appointed region 530 of having a sudden inspiration on video display part 502, and using the region of having a sudden inspiration after size changes as the region 530a that has a sudden inspiration.Size is being changed through Region specification portion 305 of having a sudden inspiration, region 530 of having a sudden inspiration is specified to become when having a sudden inspiration region 530a, frame before and after the camera time that change point test section 306 obtains the frame of display video data of being reset by the first reproduction unit 304 when changing size in specified time limit (such as 2 seconds), and extract the image of the region 530a that has a sudden inspiration changed when having a sudden inspiration area size, using this image as new benchmark image.Then, the difference of the frame that change point test section 306 calculating obtains, carry out change point detection and then extract difference image, these actions are all identical with the action of Fig. 7 to Figure 10.
The difference image of the frame of the benchmark image newly extracted and change point is saved in storage part 302 by change point test section 306, and again on benchmark image display part 508a, shows benchmark image, and difference image display part 508b shows difference image.And then change point test section 306 eliminates the mark 540 of having a sudden inspiration dragged on bar 504, in the part that the time of the frame of the change point newly detected is corresponding, display is had a sudden inspiration and is marked 540a.
When change point test section 306 detects new change point, the information of the region 530a that has a sudden inspiration is saved in storage part 302 by Region specification portion 305 of having a sudden inspiration.
Time of delay (being " 1.64 " in figure 18) of time of the frame of change point that surveyed area specifying part 307 is specified the time of the frame of the fixed time of surveyed area 535, newly detected relative to change point test section 306 is recalculated in configuration part 308, and is shown in surveyed area information portion 509 as attribute information this time of delay.The time that frame when surveyed area 535a size that surveyed area specifying part 307 specifies is changed is recalculated in configuration part 308, the time of delay (being "-1.36 " in figure 18) of time of the frame of change point newly detected relative to change point test section 306, and this time of delay is shown in surveyed area information portion 509 as attribute information.The attribute information of the surveyed area 535 and 536a that surveyed area information portion 509 upgrade display is saved in storage part 302 by configuration part 308.
Sensitivity adjustment > in < sensitivity determinating preview screen
Figure 19 be the information processor related at the first execution mode display unit on show, the schematic diagram of the state of display video data in a routine sensitivity determinating preview screen.Figure 20 be the information processor related at the first execution mode display unit on show, the schematic diagram of a routine judged result of sensitivity determinating preview screen being extracted difference between image and detection reference image.Figure 21 be the information processor related at the first execution mode display unit on show, in sensitivity determinating preview screen by different sensitivity to the schematic diagram extracting the routine judged result that difference rejudges between image and detection reference image.Figure 22 be the information processor related at the first execution mode display unit on show, the schematic diagram that confirms the routine the dialog box whether sensitivity after reorganization suitable in sensitivity determinating preview screen.Figure 23 be the information processor related at the first execution mode display unit on show, an example reflects the schematic diagram of the sensitivity after sensitivity determinating preview screen changes on setting screen.Below with reference to Figure 19 to Figure 23, the sensitivity adjustment in sensitivity determinating preview screen 700 is described.
Sensitivity determinating preview screen 700 is used to the picture adjusting the sensitivity that one of selected surveyed area adjustment attribute information i.e. image judges.As shown in figure 19, sensitivity determinating preview screen 700 has surveyed area information portion 701, video display part 702, detection reference image displaying part 703, extraction image displaying part 704, differential disply portion 705, color code amount detection display part 706, area pixel quantity display part 707, Differential Detection quantity display part 708, Differential Detection ratio display part 709, sensitivity increases key 710a, sensitivity reduces key 710b, judges key 711,0K key 712 and close key 713.
Surveyed area information portion 701 is used to the display part of the attribute information showing the surveyed area selected in the surveyed area information portion 509 of setting screen 500 by configuration part 308.In Figure 19, surveyed area information portion 701 shows camera name is called " CAM6 ", surveyed area name is called " K61 ", sensitivity is " 50 ", threshold value is " 20 ".
On video display part 702, the operation signal that the second reproduction unit 309 sends according to user operation input unit 303, the video data showing and obtain from storage part 302 of resetting.In Figure 19, user operation input unit 303, selects the camera selection portion 501 of " 6 ", and thus the second reproduction unit 309 shows the video data that camera head 2f takes in display unit 317.At this, setting video display part 702 shows the second reproduction unit 309 playback and shows the video data obtained from storage part 302, but the present invention is not limited by this, and also can show the live video data obtained via video receiving part 301 from camera head 2.
Detection reference image displaying part 703 is used for the detection reference image extracted by surveyed area specifying part 307 that display setting portion 308 obtains from storage part 302.Extract image displaying part 704 to be used for showing the extraction image extracted in image zooming-out portion 310.Differential disply portion 705 is used for showing the characteristic image (Differential Characteristics image) representing difference between the extraction image obtained by difference judging part 311 and detection reference image.
Color code amount detection display part 706 is used for display, the detection reference image obtained at difference judging part 311 and extracting in the information (difference information) of difference between image, and difference belongs to the pixel quantity detected in each difference prescribed limit.Surveyed area pixel quantity display part 707 is used for showing, the pixel quantity of surveyed area that difference judging part 311 is obtained when calculating difference, that form the object as difference information.Differential Detection quantity display part 708 is used for display, that difference judging part 311 is obtained when calculating difference, reach more than defined threshold as the difference of difference information pixel quantity.Differential Detection ratio display part 709 shows, that difference judging part 311 is obtained when calculating difference, in the surveyed area becoming object, difference is the pixel quantity of more than defined threshold and the ratio of pixel quantity forming this surveyed area.
User operation input unit 303, after pressing sensitivity increase key 710a, the sensitivity that configuration part 308 will carry out the surveyed area becoming object using when image judges increases setting.Sensitivity increase key 710a function can to increase the function of key 510e identical with above-mentioned sensitivity.User operation input unit 303, after pressing sensitivity minimizing key 710b, the sensitivity that configuration part 308 will carry out the surveyed area becoming object using when image judges reduces setting.Sensitivity reduce key 710b function can to increase the function of key 510f identical with above-mentioned sensitivity.
User operation input unit 303, presses after judging key 711, and image zooming-out portion 310 extracts image, and difference judging part 11 calculates the difference extracted between image and detection reference image, obtains difference information.
User operation input unit 303, after pressing OK key 712, the sensitivity (sensitivity of display in surveyed area information portion 701) that difference judging part 311 uses by configuration part 308 in Difference Calculation is reflected in the surveyed area information portion 509 of setting screen 500.
User operation input unit 303, after pressing close key 713, sensitivity determinating preview screen is closed in configuration part 308, display setting picture 500 in display unit 317.
The sensitivity determinating preview screen 700 of Figure 19 shows, when user operation input unit 303, under the state selecting the surveyed area 535 (surveyed area name is called " K61 ") in surveyed area information portion 509, press the picture shown when sensitivity judges key 510d.Specific as follows.Under the state that user operation input unit 303 selectes surveyed area 535 (surveyed area name is called " K61 "), press after sensitivity judges key 510d, configuration part 308 is display setting picture 500 on the display apparatus.Then, configuration part 308 shows selected surveyed area 535 in surveyed area information portion 701, extract the detection reference image corresponding with the surveyed area 535 that surveyed area specifying part 307 is extracted from storage part 302, and on detection reference image displaying part 703, show this detection reference image.In Figure 19, on detection reference image displaying part 703, the detection reference image of display is the screw of the work package of manufacturing object, and it is in the state being not yet screwed into screw.
After user operation input unit 303, the second reproduction unit 309 obtains the video data of the camera head 2 corresponding with the camera title that surveyed area information portion 701 shows from storage part 302, display that video display part 702 is reset.
Sensitivity determinating preview screen 700 shown in Figure 20 shows, and when user operation input unit 303, press when judging key 711, image zooming-out portion 310 extracts image, and difference judging part 311 calculates the result of the difference between this extraction image and detection reference image.Specific as follows.First, in during video display part 702 display video data, user is when needs carry out Difference Calculation, press after judging key 711, image zooming-out portion 310 is resetting in the video display part 702 of display video data by the second reproduction unit 309, extract the image of the part corresponding to surveyed area 535 that surveyed area specifying part 307 is specified, using this image as extraction image, be shown to and extract on image displaying part 704.In fig. 20, the detection reference image extracting display on image displaying part 704 is the screw of the work mechanism of manufacturing object, is in and twists screw as state wherein.
Then, difference between the extraction image that difference judging part 311 computed image extraction unit 310 is extracted and detection reference image, and on color code amount detection display part 706, be presented at the pixel quantity detected in the prescribed limit (being " more than 70 ", " 50 ~ 69 ", " 20 ~ 49 ", " less than 19 " in fig. 20) belonging to each difference.Difference judging part 311 shows the pixel quantity of the surveyed area 535 forming Difference Calculation object on area pixel quantity display part 707.And then difference judging part 311 also calculates the pixel quantity of difference more than defined threshold extracting and obtain in image, and this quantity is shown on Differential Detection quantity display part 708.Then, difference judging part 311 calculates and extracts the proportion of the pixel quantity of difference more than defined threshold in the pixel quantity of surveyed area 535 forming Difference Calculation object in image, and this ratio is shown on Differential Detection ratio display part 709.At this, the maximum of difference is such as " 255 ".
Difference judging part 311 also shows Differential Characteristics image in differential disply portion 705, represents the feature of the difference obtained.As shown in figure 20, difference judging part 311 generates Differential Characteristics image, is specially and distinguishes with different colours the difference belonging to each different prescribed limit, and in the pixel of answering at the differential pair obtained that the point of the color of answering with this differential pair is overlapped.
Sensitivity determinating preview screen 700 shown in Figure 21 shows on the basis of the difference result that user shows at Figure 20, re-starts Difference Calculation, make sensitivity be reduced to the result of below sensitivity shown in Figure 20 further with difference judging part 311.Specific as follows.First, user operation input unit 303, presses sensitivity and reduces key 710b, the sensitivity of Difference Calculation be reduced to " 40 " from " 50 ".Then, during video display part 702 display video data, user (establishes identical with the opportunity of Figure 20) when needing to carry out Difference Calculation at this, input device 303, presses and judge key 711.Like this, image zooming-out portion 311 is on the video display part 702 of the second reproduction unit 309 display video data, extract the image of the part corresponding to surveyed area 353 that surveyed area specifying part 307 is specified, and this image is shown on extraction image displaying part 704 as extraction image.
At this, judge key 711 because the opportunity being set in Figure 20 presses, therefore, image or the information of extracting display on image displaying part 704, differential disply portion 705, color code amount detection display part 706 and area pixel quantity display part 707 are all identical with Figure 20.
Difference judging part 311 calculates and extracts the pixel quantity that the difference value obtained in image is more than defined threshold, and is shown on Differential Detection quantity display part 708 by this pixel quantity.Now the Difference Calculation sensitivity of difference judging part 311 is lower than sensitivity during Figure 20, and thus now extracting in image the difference value obtained is that the pixel quantity of more than defined threshold will more than the pixel quantity (being increased to " 16 " from " 10 " of Figure 20) during Figure 20.Then, difference judging part 311 calculates and extracts difference in image is proportion in the pixel quantity of more than the defined threshold pixel quantity in the surveyed area 535 of Difference Calculation object, and this ratio is shown on Differential Detection ratio display part 709.Now the Difference Calculation sensitivity of difference judging part 311 is lower than sensitivity during Figure 20, thus now extracting in image the difference value obtained is the pixel quantity proportion of more than the defined threshold ratio (in figure 21, being become " 7.1% " from " 4.4% ") when being greater than Figure 20.
Sensitivity determinating preview screen 700 shown in Figure 22 is the sensitivity in order to use after adjustment shown in Figure 21, and user presses OK key 712, and display sensitivity uses and confirms dialog box 800.Setting screen 500 shown in Figure 23 shows, and surveyed area information portion 509 reflects the state of the sensitivity of the surveyed area (surveyed area in Figure 22 and Figure 23) of the object in sensitivity determinating preview screen 700 after adjustment.
Specific as follows.User operation input unit 303, after pressing 0K key 712, configuration part 308 shows sensitivity and uses confirmation dialog box 800 in display unit 317, be used for being confirmed whether the sensitivity (for " 40 ") used after Figure 21 adjustment, be i.e. whether the surveyed area information portion 509 of setting screen 500 reflect the sensitivity after adjustment.Sensitivity use confirm that dialog box 800 has acknowledgement key 801, do not use key 802, cancel key 803 and close key 804.Close key 804 is used for closing sensitivity and uses confirmation dialog box 800.
User operation input unit 303, after pressing acknowledgement key 801, difference judging part 311 reflects the sensitivity (showing sensitivity in surveyed area information portion 701) after Figure 21 adjustment as shown in figure 23 in the surveyed area information portion 509 of setting screen 500.Then, configuration part 308 is closed sensitivity and is used confirmation dialog box 800 and sensitivity determinating preview screen 700, display setting picture 500 in display unit 317.
When user operation input unit 303, press not after acknowledgement key 802, the sensitivity after Figure 21 adjustment is not reflected in the surveyed area information portion 509 of setting screen 500 by difference judging part 311.Then, configuration part 308 is closed sensitivity and is used confirmation dialog box 800 and sensitivity determinating preview screen 700, display setting picture 500 in display unit 317.
When user operation input unit 303, after pressing cancel key 803, the sensitivity after Figure 21 adjustment is not reflected in the surveyed area information portion 509 of setting screen 500 by difference judging part 311.Then, configuration part 308 is closed sensitivity and is used confirmation dialog box 800, and in display unit 317, retain display sensitivity determinating preview screen 700.
< specifies the set action flow process > of have a sudden inspiration region and surveyed area
Figure 24 is the set action flow chart that the example that the display unit of the information processor that the first execution mode relates to shows specifies have a sudden inspiration region and surveyed area on setting screen.Below with reference to Figure 24, be summarized in the flow process of the set action of setting screen 500 being specified have a sudden inspiration region and surveyed area.
step S11
First reproduction unit 304, according to the input of user operation input unit 303, obtains video data from storage part 302, display that video display part 502 is reset.Then move to step S12.
step S12
On video display part 502 video data reset display during in, user has a sudden inspiration the required image part of occurrence cause for opportunity to occur becoming, input device 303, after pressing Pause key 503b, the first reproduction unit 304 suspends the video data that video display part 502 is heavily visiting display.When after user operation input unit 303 (such as pulling mouse 108), Region specification portion 305 of having a sudden inspiration specifies on video display part 502 has a sudden inspiration region and shows this region of having a sudden inspiration.Then move to step S13.
step S13
When Region specification portion 305 of having a sudden inspiration specify have a sudden inspiration region, change point test section 306 obtains at appointed time point, frame in specified time limit before and after the shooting time of the frame of the video data of the first reproduction unit 304 playback display, and calculate the difference of pixel value between each frame and the previous frame of this frame, and then according to the difference obtained, detect the change point of having a sudden inspiration on the image in region.About the detection method of change point, refer to the description of Fig. 7 to Fig. 9.Then move to step S14.
step S14
When change point test section 306 detects change point (step S14 is), move to step S15, and when change point not detected (step S14's is no), then return step S11.
step S15
When change point test section 306 detects the IN point as change point, extract in IN point frame the image in region 530 of having a sudden inspiration, and using this image as extraction image.And when change point test section 306 detects the OUT point as change point, extract in 0UT point frame the image in region 530 of having a sudden inspiration, and using this image as extraction image.Then the difference image of the benchmark image of extraction and change point frame is saved in storage part 302 by change point test section 306, and on benchmark image display part 508a and difference image display part 508b, shows benchmark image and difference image respectively.The information in the region of having a sudden inspiration of specifying also is saved in storage part 302 by Region specification portion 305 of having a sudden inspiration.Then move to step S16.
step S16
During on video display part 502, the playback of video data shows, user is at the time point needing the image section carrying out image judgement to occur, input device 303, after pressing Pause key 503b, the first reproduction unit 304 suspends the video data that video display part 502 is heavily visiting display.Then move to step S17.
step S17
After user operation input unit 303 (such as pulling mouse 108), surveyed area specifying part 307 is specified and is shown surveyed area on video display part 502.Configuration part 308 shows the attribute information of surveyed area 535 in surveyed area information portion 509, namely surveyed area specifying part 307 to the title of the surveyed area name of specifying and the regulation sensitivity judged for image be worth by default and defined threshold, supervision method, action with or without the method (such as " consistent ") of, image judgement.And, configuration part 308 also calculates surveyed area specifying part 307 and specifies the time of the frame of the fixed time of surveyed area the time of delay of the time of change point to be detected relative to change point test section 306, and is presented in surveyed area information portion 509 as attribute information this time of delay.And then the attribute information of the surveyed area 535 of display in surveyed area information portion 509 is saved in storage part 302 by configuration part 308.Then, step S18 is moved to.
step S18
User operation input unit 303, after selection attribute information has been displayed on certain surveyed area among the surveyed area in surveyed area information portion 509, configuration part 308 has made the display section of the attribute information of selected surveyed area in surveyed area information portion 509 highlight display.Then, user operation input unit 303, after clicking action key 510a, configuration part 308 switches the set point having attonity in selected surveyed area.When user operation input unit 303, after judging options button 510b by motion video, the set point of the method that the picture that configuration part 308 switches selected surveyed area judges.
When user operation input unit 303, after pressing sensitivity increase key 510a, the value of the image judgement sensitivity of selected surveyed area is improved in configuration part 308, makes this value increase setting.And when user operation input unit 303, after pressing sensitivity minimizing key 510a, the value of the image judgement sensitivity of selected surveyed area is reduced setting by configuration part 308.The sensitivity that the image for selected surveyed area judges can be set on above-mentioned sensitivity determinating for the emperor's inspection picture 700.
When user operation input unit 303, after pressing threshold value increase key 510a, the threshold value that the image that selected surveyed area is improved in configuration part 308 judges, makes this value increase setting.And when user operation input unit 303, after pressing sensitivity minimizing key 510a, the threshold value that the image of selected surveyed area judges by configuration part 308 reduces setting.After the attribute information setting of selected surveyed area, move to step S19.
step S19
If all surveyed areas specify complete (step S19 is) in the video display part 502 of display video data, then terminate to specify the set action in region and surveyed area of having a sudden inspiration, if and not yet complete the appointment (S19's is no) of all surveyed areas, then return step S16.
Image on < key frame judges action >
Figure 25 is on the key frame of display unit display of the information processor related at the first execution mode, the schematic diagram of state after have a sudden inspiration region and surveyed area are specified.Figure 26 is that an illustration picture of the information processor that the first execution mode relates to judges action flow chart.Figure 27 is on key frame that the display unit of the information processor related at the first execution mode shows, and when having a sudden inspiration in region of having a sudden inspiration, the show state in this region of having a sudden inspiration and surveyed area are at the schematic diagram of the show state on the opportunity of detection.
First with reference to Figure 25, the image described on key frame 400 judges action.Key frame 400 fact shown in Figure 25 shows the video data that receives from multiple camera head 2, region that video display part 401b has a sudden inspiration be designated as region that " TR2 ", video display part 401f have a sudden inspiration be designated as " TR6 " and surveyed area is designated as " K61 ", " K62 " and " K21 " time state.Image on concrete key frame 400 judges that action is as follows.First, the video data of the live data obtained from video receiving part 301 is assigned to video display part 401a to 401 and is shown by video distribution portion 312.Generating unit of having a sudden inspiration 313 obtains the difference image of the frame of the change point that change point test section 306 detects from storage part 302.Then, the image in the region of having a sudden inspiration of the difference image in region of having a sudden inspiration " TR6 " that the region of having a sudden inspiration " TR2 " that video display part 401b specifies and video display part 401f specify and the frame of video data compares by generating unit of having a sudden inspiration 313 respectively.And then video display part 401b and 401f is corresponding with " the second viewing area ".Generating unit of having a sudden inspiration 313 compares the difference image in the image in region of having a sudden inspiration of the frame of video data and the frame of change point, and reach the total of the different pixels of more than regulation sensitivity and difference (the second difference) not reach defined threshold be opportunity with pixel value, generates signal of having a sudden inspiration.
In the attribute information of surveyed area " K61 " and " K62 ", the testing lag time is set to Figure 23 institute indicating value.The testing lag time in the attribute information of surveyed area " K21 " is set to Figure 15 institute indicating value.Surveyed area judging part 314 obtains the attribute information of specifying surveyed area from storage part 302.Surveyed area judging part 314 receive have a sudden inspiration generating unit 313 generate have a sudden inspiration signal time setting-up time before or after setting-up time, in the surveyed area of specifying, compare image and the detection reference image of the surveyed area of the frame of video data, judge whether to occur extremely.
Specific as follows.Surveyed area judging part 314 is after generating unit 313 of having a sudden inspiration receives the have a sudden inspiration signal corresponding with region of having a sudden inspiration " TR6 ", by the image of the surveyed area of the frame of the video data after testing lag time " 1.5 " second corresponding with surveyed area " K61 ", compare with detection reference image, judge whether image exception occurs.Now, surveyed area judging part 314 carries out image judgement by the image comparing detection reference image and surveyed area, when pixel value reaches the total of the different pixels quantity of more than the sensitivity of the attribute information of surveyed area " K61 ", namely, time more than the difference (the 3rd difference) of the pixel value threshold value (the 3rd threshold value) that is this attribute information, surveyed area " K61 " image abnormity is judged.At this, can start as the press-in state after to press-in of having a sudden inspiration is carried out image and be judged with press-in action with in the equipment such as pressing machine, as the example carrying out image judgement opportunity utilizing surveyed area judging part 314 to receive to have a sudden inspiration signal later.Be specially, the action of press-in element press-in work package starts as having a sudden inspiration with pressing machine by generating unit of having a sudden inspiration 313, generates signal of having a sudden inspiration.Then, pressing machine will be pressed into element press-in work package, and after press-in release, each execution unit of pressing machine returns origin position.The press-in state of surveyed area judging part 314 pairs of work packages after press-in carries out image judgement.This surveyed area judging part 314 is to receive signal the opportunity judged for image through setting-up time and the time after the testing lag time of having a sudden inspiration from generating unit 313 of having a sudden inspiration.Due to pressing machine auto-action, thus from generating the press-in action of signal of having a sudden inspiration, after press-in, be pressed into shape time till image judges is the set time, the testing lag time of having a sudden inspiration to image judges can be set as the set time for this reason.
Surveyed area judging part 314 is after generating unit 313 of having a sudden inspiration receives the have a sudden inspiration signal corresponding with region of having a sudden inspiration " TR6 ", will after testing lag time "-0.86 " second corresponding with surveyed area " K62 ", the image of the surveyed area of the frame of the video data namely before " 0.86 " second, compare with detection reference image, judge whether image exception occurs.At this, can be used in a manual manner by the equipment such as the pressing machine of press-in element press-in work package, operator carries out vanning as having a sudden inspiration using the work package through press-in, image judgement is carried out, as surveyed area judging part 314 to receive the example carrying out image judgement opportunity before signal of having a sudden inspiration to the work package being in the process press-in being placed in state on injector before vanning.Be specially, first, the work package before being pressed into is set on pressing machine by operator.Then, operator presses press-in and starts key, and pressing machine starts to be pressed into action, will be pressed into element press-in work package.After press-in, the work package that operator takes out through press-in is cased.Generating unit of having a sudden inspiration 313 is cased the work package through press-in for having a sudden inspiration with operator, generates signal of having a sudden inspiration.Then, surveyed area judging part 314 with receive signal of having a sudden inspiration setting-up time and before the testing lag time, be namely placed in time point on pressing machine before vanning, as the opportunity of the work package through press-in being carried out to image judgement.Like this, by using have a sudden inspiration signal generate before time point as the opportunity of the work package be placed on pressing machine being carried out to image judgement, following fault can be avoided to occur, such as, after operator arranges work package, because of operation fields away from keyboard such as lunch breaks, but after returning, forget press-in action, and the work package vanning that will be not yet pressed into.Now, if with vanning for having a sudden inspiration, and with before the stipulated time of having a sudden inspiration and testing lag time for image judge opportunity, like this, operator just can find without press-in work package, avoid without be pressed into work package be mixed into vanning.
Surveyed area judging part 314 compares image and the detection reference image of the surveyed area of the frame of video data after testing lag time " 1.5 " second corresponding with surveyed area " K21 " from the signal of having a sudden inspiration that generating unit 313 of having a sudden inspiration accepts for region of having a sudden inspiration " TR2 ", judges whether to exist abnormal.Then, outside efferent 316 when surveyed area judging part 314 judges to occur abnormal, to peripheral hardware 10 output abnormality signal.
Below with reference to Figure 26, general introduction key frame 400 carries out the flow process that image judges action.
step S31
When video distribution portion 312 can obtain video data from camera head 2 via video receiving part 301 (S31 is), move to step S32, and if when can not obtain video data (S31's is no), be then in holding state.
step S32
The live video data obtained from video receiving part is distributed to each video display part 401a to 401f and is shown by video distribution portion 312, is saved in by video data in storage part 302 meanwhile.Then, step S33 is moved to.
step S33
If before the stipulated time of current time, there is the frame (S33 is) in the past as the video data be kept in storage part 302, then move to step S34, contrary (S33's is no) then returns S31.
step S34
The video data of the frame in the past before being kept at the stipulated time of current time in storage part 302 is deleted in video distribution portion 312.Then move to step S35.
step S35
Generating unit of having a sudden inspiration 313 obtains the difference image of the frame of the change point that change point test section 306 detects from storage part 302.Then, the image of having a sudden inspiration in region in the frame of generating unit of having a sudden inspiration 313 poor partial image and video image in the region of having a sudden inspiration of specifying.Then move to step S36.
step S36
Generating unit of having a sudden inspiration 313 compares in the frame of video data the difference image of having a sudden inspiration in the image in region and the frame of change point, and reach at pixel value the time that the pixel quantity total of more than regulation sensitivity and difference (the second difference) be less than defined threshold (Second Threshold), generate signal of having a sudden inspiration.When generating unit 313 of having a sudden inspiration generate and export have a sudden inspiration signal (step S36 is), move to step S37, if do not exported on the contrary (S36's is no), then return step S31.Such as shown in Figure 27 (a), generate in generating unit of having a sudden inspiration 313 region of having a sudden inspiration " TR6 " (have a sudden inspiration region 530a) of specifying in Region specification portion 305 of having a sudden inspiration and export when having a sudden inspiration signal, highlighting the color showing region 530a edge of having a sudden inspiration.Like this, just visually can grasp that generating unit 313 of having a sudden inspiration has a sudden inspiration is when export.
step S37
If in the surveyed area that surveyed area specifying part 307 is specified, there is certain time before generating unit 313 output of having a sudden inspiration is had a sudden inspiration is carried out image judgement (detection) surveyed area (S37 is) by surveyed area judging part, then move to step S38, on the contrary, if there is no (S37's is no) then moves to step S41.
step S38
Surveyed area judging part 314 is by the image of surveyed area in the frame of the video data of stipulated time (testing lag time) corresponding with surveyed area before receiving the signal of having a sudden inspiration corresponding with region of having a sudden inspiration from generating unit 313 of having a sudden inspiration, compare with detection reference image, and judge whether to exist extremely.Then move to step S39.
step S39
When surveyed area judging part 314 detects that method that the picture of the attribute information of surveyed area judges is as " unanimously ", if by the image of surveyed area and comparing of detection reference image, the pixel value obtained is different pixels Volume Composition and the difference of more than the sensitivity of the attribute information of surveyed area, more than the threshold value of this attribute information, then the image judged result of surveyed area is abnormal (abnormality detection).Now (step S39 is), step S40 is moved to.And if image is judged as normally (S39's is no), then return step S37.And when surveyed area judging part 314 detects that method that the picture of the attribute information of surveyed area judges is as " inconsistent ", if by the image of surveyed area and comparing of detection reference image, obtain pixel value and be less than the different pixels Volume Composition of the sensitivity of the attribute information of surveyed area and difference is more than the threshold value of this attribute information, then the image judged result of surveyed area is abnormal (abnormality detection).
step S40
Outside efferent 316 when surveyed area judging part 314 judges abnormal, to peripheral hardware 10 output abnormality signal.Then return step S37.
step S41
In the surveyed area that surveyed area specifying part 307 is specified, if there is the surveyed area that surveyed area judging part 314 carries out image judgement (detection) after generating unit 313 generation of having a sudden inspiration is had a sudden inspiration, and when arriving the time of image judgement (detection) at present (step S41 is), move to step S42.And if do not arrive image judge time (S41's is no), then move to step S45.
step S42
In the frame of the video data after generating unit 313 of having a sudden inspiration is received the signal of having a sudden inspiration corresponding with region of having a sudden inspiration by surveyed area judging part 314 after the stipulated time (testing lag time) corresponding with surveyed area surveyed area image, compare with detection reference image, carry out image judgement, judge whether to occur extremely.Such as shown in figure 27b, surveyed area judging part 314, when the image of the surveyed area of specifying surveyed area specifying part 307 " K61 " (monitored area 535) judges, makes the edge color blast of surveyed area 535.Visually can grasp surveyed area judging part 314 is accordingly when carrying out image judgement.Then move to step S43.
step S43
When surveyed area judging part 314 detects that method that the picture of the attribute information of surveyed area judges is as " unanimously ", if by the image of surveyed area and comparing of detection reference image, the pixel value obtained is different pixels Volume Composition and the difference of more than the sensitivity of the attribute information of surveyed area, more than the threshold value of this attribute information, then the image judged result of surveyed area is abnormal (abnormality detection).Now (step S43 is), step S44 is moved to.And if image is judged as normally (S43's is no), then return step S41.And when surveyed area judging part 314 detects that method that the picture of the attribute information of surveyed area judges is as " inconsistent ", if by the image of surveyed area and comparing of detection reference image, obtain pixel value and be less than the different pixels Volume Composition of the sensitivity of the attribute information of surveyed area and difference is more than the threshold value of this attribute information, then the image judged result of surveyed area is abnormal (abnormality detection).
step S44
Outside efferent 316 when surveyed area judging part 314 judges abnormal, to peripheral hardware 10 output abnormality signal.Then return step S41.
step S45
In the surveyed area that surveyed area specifying part 307 is specified, if there is surveyed area judging part 314 after generating unit 313 generation of having a sudden inspiration is had a sudden inspiration to carry out the surveyed area of image judgement (detection), and exist when not yet carrying out the surveyed area of image judgement (step S45 is), return step S41.And if there is no (S45's is no), then terminate image and judge action.
As mentioned above, after user operation input unit 303, Region specification portion 305 of having a sudden inspiration specifies and shows region of having a sudden inspiration on video display part 502, change point test section 306 obtains the first reproduction unit 304 and is resetting in the video data of display, before and after the shooting time of the frame of fixed time point during the stipulated time in frame, calculate specifying the image that extracts in region of having a sudden inspiration of time of having a sudden inspiration region, i.e. benchmark image, and the difference of pixel value between the image of having a sudden inspiration in region of the frame obtained.And then, change point test section 306 is in the figure of the difference obtained with temporal representation, to the image of having a sudden inspiration in region of frame initial in the frame obtained, be that the total of the different pixels quantity of more than regulation sensitivity is as difference using pixel value, this difference reaches more than defined threshold, and the time point that there is the state of difference compared with benchmark image hardly detects as change point (IN point).Change point test section 306 is also in the figure of the difference obtained with temporal representation, the image of having a sudden inspiration in region of frame initial in the frame obtained, and when there is the image of difference hardly compared with benchmark image, in the image of having a sudden inspiration from initial frame in region, with pixel value be the total of different pixels quantity of more than regulation sensitivity as difference, the time point this difference being reached more than defined threshold detects as change point (OUT point).Then, generating unit of having a sudden inspiration 313 is when change point test section 306 detects that change point is 0UT point, image in view of region of having a sudden inspiration in the frame of video data does not almost have the state of difference compared with benchmark image, is being that the time point that the total of pixel quantity of more than regulation sensitivity and difference reach more than defined threshold generates signal of having a sudden inspiration by pixel value.On the other hand, when at change point test section 306, generating unit 313 of having a sudden inspiration detects that change point is IN point, to the image (difference image) in region of having a sudden inspiration in the frame of video data, be that total and the difference of the different pixels quantity of more than regulation sensitivity reaches more than defined threshold at pixel value, and the time point of the state of difference is not almost had compared with benchmark image, generate signal of having a sudden inspiration.Like this, just freely can generate and become surveyed area judging part 314 and carry out the time point of having a sudden inspiration that image judges time point benchmark.
On video display part 502 video data playback display during in user when needing the image section carrying out image judgement to occur, input device, press Pause key 503b, the first reproduction unit 303 suspend video display part 502 business resetting display video data.Then, user operation input unit 303, surveyed area specifying part 307 specifies display surveyed area on video display part 502.Configuration part 308 calculates surveyed area specifying part and specifies the time of the frame of the fixed time of surveyed area the time of delay of the time of change point to be detected relative to change point test section 306, and using time of delay as attribute information.Then, surveyed area judging part 314 is by before or after time of delay of setting in the configuration part 308 receiving the signal of having a sudden inspiration that generating unit 313 of having a sudden inspiration generates, in the surveyed area of specifying video data frame in the image of surveyed area, with detection reference image with compare, carry out image judgement, judge whether exception.Like this, can not only with the opportunity of having a sudden inspiration generated for benchmark, image judgement is carried out to the image of the surveyed area of the frame after the stipulated time, but also can carry out image judgement to the image of surveyed area in the frame in the past before the stipulated time, thus image is judged, and the design freedom on opportunity obtains tremendous raising.
Present embodiment is on setting screen 500, and the appointment etc. of implementation video display part 502 is had a sudden inspiration region and surveyed area, on key frame 400, based on the image of display on video display part 401, generates signal of having a sudden inspiration, and carry out image judgement.But these do not form restriction to the present invention.That is, the function of setting screen 500 and key frame 400 both sides can be unified, such as using video display part 502 and video display part 401 as same display part, this display part is carried out the implementation that the generation of the appointment of have a sudden inspiration region and surveyed area etc. and signal of having a sudden inspiration and image judge.In this case, video display part 502 and video display part 401 are as same display part, and this display part is " the first display part " and " the second display part ".
The signal of having a sudden inspiration that forming portion 313 of having a sudden inspiration in above-mentioned execution mode generates is used to carry out as surveyed area judging part 314 benchmark on opportunity of image judgement, but it does not produce restriction to the present invention.Such as, signal of having a sudden inspiration also can be used as the benchmark on opportunity carrying out predetermined processing in production equipment, in this case, sends signal at the time point touching signal generation to production equipment.
" the second execution mode "
The difference of the information processor 3 of present embodiment and the information processor 3 of the first execution mode is below described in detail in detail.Especially will describe in detail between the two about the difference of the detection action of change point in region of having a sudden inspiration.At this, the entirety of the image processing system 1 of present embodiment is formed, the hardware configuration of information processor 3 and functional module is formed, key frame 400 and the formation of setting screen 500, the set action of attribute information of the appointment of surveyed area and surveyed area, sensitivity adjustment action in sensitivity determinating preview screen 700 and then key frame 400 epigraph judge action all with the homomorphosis in the first execution mode.
The functional module of < information processor forms >
As mentioned above, the functional module structure of the information processor 3 of present embodiment is identical with the functional module structure of the information processor of the first execution mode 3 shown in Fig. 3, below describes the functional module that action is different.
In present embodiment, when Region specification portion 305 of having a sudden inspiration specify have a sudden inspiration region, change point test section 306 obtain the first reproduction unit 304 at the appointed time reset display video data frame shooting time before frame in specified time limit (such as 3 seconds), then calculate the difference of the pixel value between each frame of obtaining and frame before, and based on the difference obtained, detect the benchmark image of image in region 530 of having a sudden inspiration.Change point test section 306 also obtains from the frame specified time limit (such as 6 seconds) after time of the frame (reference frame) of the benchmark image detected, calculate the difference (the first difference) of the pixel value between each frame of obtaining and reference frame, based on the difference obtained, detect the change point of image in region of having a sudden inspiration.Change point about present embodiment will in following description.About the computational methods of above-mentioned difference, such as can compare the pixel value in region of having a sudden inspiration between two frames, the difference of pixel value is reached the total of the pixel quantity of more than regulation sensitivity, or in the pixel of the image in region of having a sudden inspiration, the difference of pixel value reaches the quantity ratios (area ratio) of the pixel of more than regulation sensitivity as difference.When frame is black and white image, pixel value is consistent with gray value, thus can also compare the gray value of having a sudden inspiration in region between two frames, and the difference obtaining gray value reaches the total of the pixel quantity of more than regulation sensitivity, using this total as difference.And when frame is the image formed with rgb pixel value, compare the pixel value of any one color in R (red), G (green), B (orchid), then same as described above, the difference obtaining pixel value reaches the total of the pixel quantity of more than regulation sensitivity, using this total as difference.And then, edge detection process can also be carried out to the image of frame, obtain the total of marginal portion, using this total as difference.With shown in Fig. 2 CPU101 executive program realize the function of change point test section 306.
< has a sudden inspiration the appointment in region and the detection > of change point
Figure 28 is the schematic diagram of the routine dialog box failing in the setting screen shown in the display unit of the information processor that the first execution mode relates to occur when benchmark image being detected in appointed area.Figure 29 is used to change point in the region of having a sudden inspiration of description second execution mode and detects the schematic diagram of action.Below with reference to the accompanying drawing in Figure 28 and Figure 29 and the first execution mode, describe and touch the appointment in region, the detection of benchmark image and change point detection.
First as shown in Figure 6, during on video display part 502, the playback of video data shows, the time point of the image section of the occurring source of having a sudden inspiration becoming that appearance will be found, user operation input unit 303, after pressing Pause key 503b, first reproduction unit 304 suspends the video data of display that video display part 502 resets, and in time showing portion 506, show shooting date and the shooting time of the frame of this time point.Then, user operation input unit 303 (such as pulling mouse 108), the Region specification portion 305 that makes to have a sudden inspiration specifies and shows region 530 of having a sudden inspiration video display part 502 (the first viewing area) is upper.
Then, after Region specification portion 305 of having a sudden inspiration specifies region 530 of having a sudden inspiration, change point test section 306 obtain the first reproduction unit 304 at the appointed time reset display video data frame shooting time before frame in specified time limit (such as 3 seconds) (the first stipulated time).Then, change point test section 306 calculates the difference of pixel value between each frame obtained and previous frame, and based on the difference obtained, detects the benchmark image of image in region 530 of having a sudden inspiration.Specific as follows.Change point test section 306 calculates in the frame obtained, in the region 530 of having a sudden inspiration of object frame the pixel value of image and the previous frame of this frame region 530 of having a sudden inspiration in image pixel value between difference reach the pixel quantity of more than regulation sensitivity (such as " 30 ") ? have a sudden inspiration region 530 pixel quantity in shared byratio (area ratio), and using this ratio as difference, be used for detecting this difference defined threshold (such as 5%) below time the image in region 530 of having a sudden inspiration of object frame, using this image as benchmark image.
The stipulated time used during said reference image detects, the concrete numerical value of regulation sensitivity and defined threshold are only examples, also can use other numerical value, or be kept at the enactment document can edited in storage part 302.Above-mentioned change point test section 306 obtain the first reproduction unit 304 reset display video data frame shooting time before frame in specified time limit, but the present invention is not limited thereto.Such as change point test section 306 can also obtain the first reproduction unit 304 reset display video data frame shooting time before and after the frame of specified time limit, or to obtain from the frame specified time limit after shooting time.
Setting screen 500 shown in Figure 28 is when change point test section 306 fails benchmark image to be detected in the region 530 of having a sudden inspiration that Region specification portion 305 of having a sudden inspiration specifies, and display does not detect notification dialog window 620 state of benchmark image.Change point test section 306 detects in action at said reference image, and the difference obtained does not reach more than defined threshold, thus fails benchmark image to be detected.Now, change point test section 306 shows without benchmark image notify object window 620 in display unit 317, represents and benchmark image do not detected.
Without benchmark image notification dialog window 620, there is OK key 621 and closedown builds 622.OK key 621 is used for closing without benchmark image notification dialog window 620.User operation input unit 303, after pressing 0K key 621, close without benchmark image notification dialog window 620, now, change point test section 306 does not preserve any information in storage part 302, and benchmark image display part 508a and difference image display part 508b does not also show image.
Secondly, when benchmark image being detected, change point test section 306 obtain the frame of benchmark image and the shooting time of reference frame after frame in specified time limit (such as 6 seconds) (the second specified time limit).Then, change point test section 306 calculates the difference (the first difference) of pixel value between each frame obtained and reference frame, according to the difference obtained, detects the change point in image in region of having a sudden inspiration.About the computational methods of difference, such as, in the frame obtained at change point test section 306, the image in region 530 that object frame is had a sudden inspiration and reference frame have a sudden inspiration region 530 image between the difference of pixel value be the pixel quantity of more than regulation sensitivity (such as " 50 ") shared by the pixel quantity of this surveyed area 530ratio (area ratio) is as difference.The change point detection method that change point test section 306 specifically uses below is described.
As shown in (a) in Figure 29, if amount to 13 frames (frame (0) is to (12)) in change point test section 306 obtains from the shooting time of reference frame specified time limit.Now, change point test section 306 with frame (0) for reference frame, the frame in the specified time limit from this reference frame.In addition, for convenience's sake, the image of the part corresponding to region 530 of having a sudden inspiration in each frame is also called frame (0) to frame (12).Then, change point test section 306 calculates the difference of pixel value between each frame and reference frame.Difference Calculation method as previously discussed.Change point test section 306 extracts frame as shown in (a) of Figure 29, and take temporal representation as (b) of Figure 29 by the difference of each frame obtained.As shown in (b) of Figure 29, such as frame (2) difference be, the difference between frame (2) and reference frame and frame (0).In this as the IN point of the change point of present embodiment and OUT point as given a definition.Namely IN point refers to, by the image of having a sudden inspiration in successive frame in region 530 compared with the image of having a sudden inspiration in reference frame in region 530, change between the two reaches the time point of specified degree.OUT point refers to, from IN point, arrives the image of having a sudden inspiration in region 530 and become the akin time point of the image of having a sudden inspiration in region 530 with reference frame in successive frame.
Change point test section 306, according to the difference of each frame obtained, obtains IN point candidate and OUT point candidate.Specific as follows.Change point test section 306 from defined threshold (such as 20%) (first threshold) state below, is turned into when exceeding this threshold value, using the frame corresponding to the difference exceeding this threshold value as IN point candidate frame in the difference obtained.In (b) of Figure 29, frame (1) is less than threshold value to the difference of frame (3), and the difference of frame (4) exceedes threshold value, and thus, frame (4) is the candidate frame of IN point.Equally, frame (10) is also the candidate frame of IN point.
On the other hand, if the difference that change point test section 306 is obtained is when being turned into below this threshold value from the state being greater than defined threshold (first threshold), using the frame corresponding to the difference below this threshold value as OUT point candidate frame.In (b) of Figure 29, frame (4) is greater than threshold value to the difference of frame (6), and difference is below threshold value from frame (7), and thus, frame (7) is the candidate frame of OUT point.Equally, frame (11) is also the candidate frame of OUT point.
As mentioned above, in the example of Figure 29, change point test section 306 detects that frame (4) and (10) are IN point candidate frame, and frame (7) and (11) are OUT point candidate frame.Like this, when multiple IN point candidate frame being detected, change point test section 306 is by the frame closest to reference frame, and namely frame (4) is decided to be the frame of IN point.Equally, when multiple OUT point candidate frame being detected, change point test section 306 is by the frame closest to reference frame, and namely frame (7) is decided to be the frame of OUT point.As mentioned above, change point test section 306, by determining IN point frame and OUT point frame in the frame in the specified time limit obtained, carrys out change detected point (IN point and OUT point).In addition, change point test section 306, when change detected point, can also use the previous frame of the frame of change point or a rear frame, or the frame of front and back, the action quality testing carrying out having a sudden inspiration in region 530 is surveyed, and the Vector Message of expression action thing since which direction is saved in storage part 302.In this case, generating unit of having a sudden inspiration 313 Vector Message preserved in storage part 303, confirms that whether the direction of action of action thing is consistent with the direction that Vector Message represents, and generates according to confirmation result the information of having a sudden inspiration.
The example of Figure 29 display is the frame simultaneously finding IN point and OUT point both sides, but change point test section 306 also may only find the frame of IN point or the frame of OUT point.Now, these IN points or OUT point are detected as change point.Change point test section 306 extracts the frame of the change point detected at the image in region 530 of having a sudden inspiration, using this image as difference image (the first image), and the difference image of the benchmark image of extraction and the frame of change point is saved in storage part 302, benchmark image display part 508a shows benchmark image, and show difference image on difference image display part 508b.And then change point test section 306 to be had a sudden inspiration mark 540 dragging in bar 504 display in the part corresponding to the time frame of change point being detected.When change point test section 306 detects change point, the information in the region 530 of having a sudden inspiration of specifying is saved in storage part 302 by Region specification portion 305 of having a sudden inspiration.Configuration part have a sudden inspiration area information portion 507 shows the appointment named by Region specification portion 305 of having a sudden inspiration have a sudden inspiration region 530 title and for the regulation sensitivity (such as 50) of change point test section 306 change detected point and defined threshold (such as 20%) (default value).About the title in region 530 of having a sudden inspiration, as shown in Figure 7, configuration part 308 is by region called after " TR6 " of having a sudden inspiration appointed in the video data of the camera head (camera head 2f) of " 6 ".
The stipulated time used during said reference image detects, the concrete numerical value of regulation sensitivity and defined threshold are only examples, also can use other numerical value, or be kept at the enactment document can edited in storage part 302.
Change point test section 306 is described above, 530 kinds, the region of having a sudden inspiration of specifying in Region specification portion 305 of having a sudden inspiration detects two change points (IN point and OUT point), selects dialog box 600 same action with the change point shown in above-mentioned Fig. 9.Like this, change point test section 306 detects IN point as change point and OUT point, and user can select the change point that in these two change points, any point is specified as user.
On the contrary, if change point test section 306 does not have change point to be detected in the region 530 of having a sudden inspiration of specifying in Region specification portion 305 of having a sudden inspiration, then the action of unchanged some notification dialog window 610 of above-mentioned Figure 10 display is carried out.
< specifies the set action flow process > of have a sudden inspiration region and surveyed area
Figure 30 is the set action flow chart of setting screen that the display unit of information processor of the second embodiment shows being specified have a sudden inspiration region and surveyed area.Below, with reference to Figure 30, general introduction setting screen 500 is specified the set action flow process of have a sudden inspiration region and surveyed area.
step S51
First reproduction unit 304, according to user operation input unit 303, obtains the video data preserved in storage part 302, resets and is shown on video display part 502.Then move to step S52.
step S52
During on video display part 502, video data resets display, when the part of the image becoming occurring source of having a sudden inspiration occurs, user operation input unit 303, after pressing Pause key 503b, the first reproduction unit 304 suspends playback display video data.User operation input unit 303 (such as pulling mouse 108), the Region specification portion 305 that makes to have a sudden inspiration specifies and shows region of having a sudden inspiration on video display part 502.Then move to step S53.
step S53
When Region specification portion of having a sudden inspiration specify have a sudden inspiration region 530, change point test section 306 obtain the first reproduction unit 304 at the appointed time reset display video data frame shooting time before specified time limit (such as 3 seconds) in frame.Then, change point test section 306 calculates the difference of pixel value between the frame before each frame obtained and this frame, and based on the difference obtained, detects the benchmark image of image in region 530 of having a sudden inspiration.Concrete benchmark image detection method is same as described above.Then move to step S54.
step S54
When change point test section 306 detects benchmark image (S54 is), move to step S55, otherwise (S54's is no), turn back to step S51.
step S55
When benchmark image being detected, change point test section 306 obtain the shooting time of reference frame after frame in specified time limit (such as 6 seconds).Then, change point test section 306 calculates the difference of the pixel value between each frame and reference frame, and based on this difference, detects the change point in image in region of having a sudden inspiration.Concrete change point detection method is that the description in above-mentioned Figure 28 with Figure 29 etc. is identical.Then move to step S56.
step S56
When change point test section 306 detects change point (S56 is), move to step S57, otherwise (S56's is no), turn back to step S51.
step S57 to S61
The process of step S57 to S61 is identical with the process of the step S15 to S19 shown in the first execution mode midway 24 respectively.
As mentioned above, change point test section 306 is detection reference image first, when benchmark image being detected, frame after obtaining reference frame shooting time in specified time limit, calculate the difference of pixel value between each frame and reference frame, and the change point of the image in region of having a sudden inspiration based on the Differential Detection obtained.According to the change point that said method detects, generating unit of having a sudden inspiration 313 generates signal of having a sudden inspiration.Like this, present embodiment just can freely generate by surveyed area judging part 314 carry out image judgement opportunity benchmark the opportunity of having a sudden inspiration.The effect identical with the first execution mode can be obtained in addition.
If by performing calculation procedure, realize at least some function part functions in the first reproduction unit 304 in above-mentioned various execution mode, Region specification portion 305 of having a sudden inspiration, change point test section 306, surveyed area specifying part 307, configuration part 308, second reproduction unit 309, image zooming-out portion 310, difference judging part 311, video distribution portion 312, generating unit of having a sudden inspiration 313, surveyed area judging part 314 and display control unit 315, this program can be pre-installed in ROM etc. and provide.The program carried out of the information processor of present embodiment also can the file of installation form or practicable form can be installed in the computer-readable recording medium such as CD-ROM, floppy disc (FD), CD-R, DVD and provide.And then the program that the information processor of present embodiment is carried out can also be installed in the computer of the networks such as connecting Internet, is provided by web download.The program that the information processor of present embodiment is carried out also can be provided by networks such as the Internets or promulgate.The program carried out of the information processor of present embodiment can also to comprise the first reproduction unit 304, Region specification portion 305 of having a sudden inspiration, change point test section 306, surveyed area specifying part 307, configuration part 308, second reproduction unit 309, image zooming-out portion 310, difference judging part 311, video distribution portion 312, generating unit of having a sudden inspiration 313, the module composition of at least some function parts in surveyed area judging part 314 and display control unit 315, read and executive program from above-mentioned ROM by the CPU of the hardware of reality, upload in main storage means and generate each function part above-mentioned.

Claims (21)

1. an image processing apparatus, wherein possesses with lower module:
Display module;
Playback module, the video data for being taken by camera head is reset and is shown to described display module;
To have a sudden inspiration Region specification module, in the first viewing area of the described display module of the described video data of display, specify region of having a sudden inspiration;
Detection module, have a sudden inspiration the time point in region for benchmark described in specifying with described Region specification module of having a sudden inspiration, from in the described video data in the first specified time limit of this time point, the change point of having a sudden inspiration described in detection in the image in region, and extract the image of having a sudden inspiration in region described in the frame of this change point, as the first image; And,
Generation module, in the second viewing area for the described display module of the video data by the shooting of display described camera head, described in the image of having a sudden inspiration in region compare with described first image, have a sudden inspiration according to comparative result generation.
2. image processing apparatus according to claim 1, wherein,
Described detection module obtains the frame of the described video data in described first specified time limit, and calculates the difference between each frame obtained and the previous frame of this frame, the benchmark image in region of having a sudden inspiration according to this Differential Detection,
Obtain the frame from the described video data in the second specified time limit after the time point of the frame comprising described benchmark image, and the first difference calculated between each frame obtained and the frame comprising described benchmark image, according to this first difference, the described change point of having a sudden inspiration described in detection in the image in region.
3. image processing apparatus according to claim 2, wherein,
Described detection module detects the time point that described first difference exceedes the frame of described first threshold, or the time point of frame when described first difference is turned into below this first threshold from the state of the frame exceeding first threshold, using this time point as change point.
4. image processing apparatus according to claim 1, wherein,
The appointed area module of having a sudden inspiration described in described detection module obtains specify described in have a sudden inspiration region time point before and after the frame of described video data in described first specified time limit, and the first difference calculated between each frame obtained and the previous frame of this frame, according to this first difference, the described change point in the image of having a sudden inspiration described in detection in region.
5. image processing apparatus according to claim 4, wherein,
According to the comparative result between described first difference and described first threshold, detect described change point.
6. image processing apparatus as claimed in any of claims 1 to 5, wherein,
Described generation module generates described in the benchmark that is used as and carries out image judgement in described second viewing area has a sudden inspiration.
7. image processing apparatus as claimed in any of claims 1 to 6, wherein,
Described generation module calculates second difference of to have a sudden inspiration described in described second viewing area between image in region and described first image, and when this second difference is less than Second Threshold, has a sudden inspiration described in generation.
8. image processing apparatus as claimed in any of claims 1 to 7, wherein possesses further:
Surveyed area designated module, is used for having made to order at described first viewing area middle finger the surveyed area of the object judged for described image;
Surveyed area judge module, is used for carrying out described image judgement to described surveyed area; And,
Setting module, is used in described surveyed area, generates the time point of having a sudden inspiration for benchmark with described generation module, and setting determines that described surveyed area judge module carries out the judgement time of described image judgement.
9. image processing apparatus according to claim 8, wherein,
The described judgement time is set as by described setting module, and the described surveyed area in the frame of the video data before the time point of having a sudden inspiration described in generating described generation module carries out described image judgement.
10. image processing apparatus according to claim 8, wherein,
The described judgement time is set as by described setting module, and the described surveyed area in the frame of the video data after the time point of having a sudden inspiration described in generating described generation module carries out described image judgement.
Image processing apparatus in 11. according to Claim 8 to 10 described in any one, wherein,
The video data that described Region specification module of having a sudden inspiration is taken other camera heads being different from described camera head, in described first viewing area, other regions of having a sudden inspiration in region of having a sudden inspiration described in appointment is different from,
Described in described surveyed area judge module generates with described generation module, the time point of having a sudden inspiration in other regions of having a sudden inspiration is for benchmark, carries out described image judgement to described surveyed area.
Image processing apparatus in 12. according to Claim 8 to 11 described in any one, wherein,
Described surveyed area designated module specifies multiple described surveyed area in described first viewing area.
Image processing apparatus in 13. according to Claim 8 to 12 described in any one, wherein,
Described surveyed area judge module calculates the image of the described surveyed area be subject to when surveyed area designated module is specified, i.e. the second image, and the 3rd difference between the image of surveyed area described in the described frame of described judgement time, when the 3rd difference is more than 3rd threshold value, judge to occur extremely.
Image processing apparatus in 14. according to Claim 8 to 12 described in any one, wherein,
Described surveyed area judge module calculates the image of the described surveyed area be subject to when surveyed area designated module is specified, i.e. the second image, and the 3rd difference between the image of surveyed area described in the described frame of described judgement time, when the 3rd difference is less than more than 3rd threshold value, judge to occur extremely.
15. image processing apparatus according to claim 13 or 14, wherein,
Described surveyed area judge module reaches the pixel quantity of more than regulation sensitivity according to the difference of pixel value between the image of surveyed area described in described second image and the described frame of described judgement time, calculates described 3rd difference,
Described setting module sets described regulation sensitivity.
Image processing apparatus in 16. according to Claim 8 to 15 described in any one, wherein,
The described video data that described surveyed area designated module shows according to described playback module, specifies described surveyed area.
Image processing apparatus in 17. according to Claim 8 to 16 described in any one, wherein,
Possess outside output module further, be used for, when described surveyed area designated module is judged as abnormal in described image judges, exports to peripheral hardware and representing abnormal notice.
18. according to the image processing apparatus in claim 1 to 17 described in any one, wherein,
Described detection module calculates the difference of the pixel value of gray value or pertinent regulations color, using this difference as described first difference.
19. according to the image processing apparatus in claim 1 to 18 described in any one, wherein,
The described described video data of Region specification module according to described playback module playback display of having a sudden inspiration, region of having a sudden inspiration described in appointment.
20. 1 kinds of image processing systems, wherein possess the image processing apparatus described in any one in more than one camera head and claim 1 to 19.
21. 1 kinds of image processing methods, wherein comprise following steps:
Playback step, the video data taken by camera head is reset and is shown on display module;
To have a sudden inspiration region specification step, in the first viewing area of the described display module of the described video data of display, specify region of having a sudden inspiration;
Detecting step, with the time point in region of having a sudden inspiration described in specifying for benchmark, from the described video data in the first specified time limit of this time point, the change point of having a sudden inspiration described in detection in the image in region, and extract the image of having a sudden inspiration in region described in the frame of this change point, as the first image; And,
Generation step, will the shooting of display described camera head video data described display module the second viewing area in, described in the image of having a sudden inspiration in region compare with described first image, have a sudden inspiration according to comparative result generation.
CN201510494581.3A 2014-08-25 2015-08-12 Image processing apparatus, image processing system and image processing method Active CN105391971B (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2014-170735 2014-08-25
JP2014170735 2014-08-25
JP2015041577A JP6554825B2 (en) 2014-08-25 2015-03-03 Image processing apparatus, image processing system, image processing method, and program
JP2015-041577 2015-03-03

Publications (2)

Publication Number Publication Date
CN105391971A true CN105391971A (en) 2016-03-09
CN105391971B CN105391971B (en) 2019-07-12

Family

ID=55423751

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510494581.3A Active CN105391971B (en) 2014-08-25 2015-08-12 Image processing apparatus, image processing system and image processing method

Country Status (1)

Country Link
CN (1) CN105391971B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108540782A (en) * 2017-03-03 2018-09-14 株式会社理光 The method and apparatus for obtaining difference image
CN110853077A (en) * 2019-10-17 2020-02-28 广西电网有限责任公司电力科学研究院 Self-adaptive infrared dynamic frame feature extraction method based on morphological change estimation
CN110933455A (en) * 2019-12-16 2020-03-27 云粒智慧科技有限公司 Video screening method and device, electronic equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101320505A (en) * 2008-07-04 2008-12-10 浙江大学 Partition video monitoring method based on multipath network video stream parallel processing
CN101378498A (en) * 2007-08-28 2009-03-04 微星科技股份有限公司 Image detection method
US20130188044A1 (en) * 2012-01-19 2013-07-25 Utechzone Co., Ltd. Intelligent monitoring system with automatic notification and intelligent monitoring device thereof
CN103327306A (en) * 2013-06-14 2013-09-25 广东威创视讯科技股份有限公司 Method and device for storing video surveillance image
CN103634564A (en) * 2012-08-22 2014-03-12 一二三视股份有限公司 Method for assigning image monitoring area

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101378498A (en) * 2007-08-28 2009-03-04 微星科技股份有限公司 Image detection method
CN101320505A (en) * 2008-07-04 2008-12-10 浙江大学 Partition video monitoring method based on multipath network video stream parallel processing
US20130188044A1 (en) * 2012-01-19 2013-07-25 Utechzone Co., Ltd. Intelligent monitoring system with automatic notification and intelligent monitoring device thereof
CN103634564A (en) * 2012-08-22 2014-03-12 一二三视股份有限公司 Method for assigning image monitoring area
CN103327306A (en) * 2013-06-14 2013-09-25 广东威创视讯科技股份有限公司 Method and device for storing video surveillance image

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108540782A (en) * 2017-03-03 2018-09-14 株式会社理光 The method and apparatus for obtaining difference image
CN110853077A (en) * 2019-10-17 2020-02-28 广西电网有限责任公司电力科学研究院 Self-adaptive infrared dynamic frame feature extraction method based on morphological change estimation
CN110933455A (en) * 2019-12-16 2020-03-27 云粒智慧科技有限公司 Video screening method and device, electronic equipment and storage medium
CN110933455B (en) * 2019-12-16 2023-03-14 云粒智慧科技有限公司 Video screening method and device, electronic equipment and storage medium

Also Published As

Publication number Publication date
CN105391971B (en) 2019-07-12

Similar Documents

Publication Publication Date Title
JP6733785B2 (en) Image processing apparatus, image processing system, image processing method, and program
US10313627B2 (en) Apparatus for playing back recorded video images related to event, and method thereof
US10304032B2 (en) Product monitoring device, product monitoring system, and product monitoring method
US9875409B2 (en) Abnormality detection apparatus, abnormality detection method, and recording medium storing abnormality detection program
US10873719B2 (en) Image processing device, image processing system, image processing method, and computer program product
EP2270772B1 (en) Display control apparatus and display control method
US10699473B2 (en) System and method for generating a virtual viewpoint apparatus
JP5866564B1 (en) MONITORING DEVICE, MONITORING SYSTEM, AND MONITORING METHOD
US10546199B2 (en) Person counting area setting method, person counting area setting program, moving line analysis system, camera device, and person counting program
JP4847165B2 (en) Video recording / reproducing method and video recording / reproducing apparatus
US10235574B2 (en) Image-capturing device, recording device, and video output control device
EP3255603B1 (en) Image processing device, image processing system, and image processing method
US20100189365A1 (en) Imaging apparatus, retrieval method, and program
JP6459256B2 (en) Image storage / playback control apparatus, image storage / playback control method, image storage / playback control system, and program
CN105391971A (en) Image processing device, image processing system and image processing method
KR101251304B1 (en) Picture control device
KR102592231B1 (en) Method for diagnosing fault of camera
CN113766217A (en) Video delay test method and device, electronic equipment and storage medium
JP2021027579A (en) Image determination device and image determination system
JP2018084443A (en) Image processing apparatus, image processing system, image processing method, and image processing program
EP4362457A2 (en) Image-based determination apparatus and image-based determination system
CN105376511A (en) Image processing device, image processing system, and image processing method
JP2018073129A (en) Image processing device, image processing system, image processing method and program
JP2001014471A (en) Image pattern comparing device, image pattern change detector, device and system for monitoring
US11501534B2 (en) Information processing apparatus, information processing system, information processing method, and storage medium

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant