WO2016147652A1 - オブジェクト検出装置、オブジェクト検出方法および記録媒体 - Google Patents
オブジェクト検出装置、オブジェクト検出方法および記録媒体 Download PDFInfo
- Publication number
- WO2016147652A1 WO2016147652A1 PCT/JP2016/001453 JP2016001453W WO2016147652A1 WO 2016147652 A1 WO2016147652 A1 WO 2016147652A1 JP 2016001453 W JP2016001453 W JP 2016001453W WO 2016147652 A1 WO2016147652 A1 WO 2016147652A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- class
- input
- partial area
- image
- object detection
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
- G06F3/03543—Mice or pucks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N99/00—Subject matter not provided for in other groups of this subclass
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
- G06V10/235—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition based on user input or interaction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Definitions
- the present invention relates to an object detection device that detects an object from an image using a dictionary.
- Patent Document 1 discloses a technique for analyzing an image obtained by imaging a substrate and detecting an object such as a scratch on the substrate in a manufacturing process of a substrate such as a printed wiring board.
- Patent Document 2 discloses a technique for detecting an object such as a vehicle by analyzing a monitor video on a road obtained by imaging with a surveillance camera.
- teacher data also called training data
- teacher data is an example of an input / output pair.
- positive cases and negative cases Both positive and negative cases are necessary for the object detection device to learn correctly. However, it takes a lot of time and effort to create appropriate teacher data.
- Patent Document 1 discloses a technique for creating teacher data necessary for detecting an object such as a scratch on a substrate.
- an area having a brightness value different from that of a non-defective product is extracted from an image of a printed wiring board and displayed on a display, and the selection of the area and input of its class (also called category) are used using a keyboard and a mouse.
- the user selects one specific area from among a plurality of existing areas by clicking the mouse, and then selects a desired class from the pull-down menu displayed at the time of selection by clicking the mouse. To do.
- Patent Document 2 discloses a technique for creating teacher data necessary for detecting an object such as a vehicle traveling on a road.
- a region of an object is cut out from an arbitrary captured image using a dictionary, a predetermined feature amount is extracted from the region, and the dictionary is learned based on the extracted feature amount. A series of these operations are automatically performed.
- Patent Document 1 the user needs to select an area used for creating the teacher data and input a class for the selected area by separate operations of the input device. Workability is poor.
- the purpose of the present invention is to solve the above-mentioned problems. That is, it is to provide an object detection device or the like that efficiently creates good quality teacher data.
- the first feature of the present invention is: A detection unit for detecting an object from an input image using a dictionary; A receiving unit that displays an input image on a display device with a display that highlights a partial area of a detected object, and receives a selection of the partial area and an input of a class for the selected partial area by one operation of the input device When, A generation unit that generates teacher data from an image of the selected partial region and the input class; A learning unit for learning a dictionary based on teacher data; Is an object detection device.
- the second feature of the present invention is that Detect objects from input images using a dictionary, Display the input image on the display device with a display that highlights the partial area of the detected object, Accepting selection of a partial area and input of a class for the selected partial area by one operation of the input device, Generate teacher data from the selected partial region image and the input class, This is an object detection method for learning a dictionary from teacher data.
- the third feature of the present invention is that Computer A detection unit for detecting an object from an input image using a dictionary; A receiving unit that displays an input image on a display device with a display that highlights a partial area of a detected object, and receives a selection of the partial area and an input of a class for the selected partial area by one operation of the input device When, A generation unit that generates teacher data from an image of the selected partial region and the input class; A learning unit for learning a dictionary based on teacher data; It is a recording medium for storing a program that functions.
- the present invention has the above-described configuration, it is possible to efficiently create good quality teacher data.
- FIG. 1 is a block diagram of an object detection device according to a first embodiment of the present invention. It is explanatory drawing of the concept which detects the object from the image in the 1st Embodiment of this invention. It is a figure which shows an example of the detection result of the object in the 1st Embodiment of this invention. It is a figure which shows an example of the reception screen in the 1st Embodiment of this invention. It is a flowchart which shows an example of operation
- the object detection apparatus 100 has a function of detecting an object by analyzing a monitor video on a road obtained by being imaged by a monitoring camera 110.
- the object detection device 100 is a kind of multi-class classifier, and detects two specific types of objects from an image.
- the object detection apparatus 100 includes a detection unit 101, a dictionary 102, a detection result DB (detection result database) 103, a reception unit 104, a generation unit 105, a teacher data memory 106, and a learning unit 107 as main functional units. Further, a display device 120 and an input device 130 are connected to the object detection apparatus 100.
- the object detection apparatus 100 may be configured by an information processing apparatus 230 and a storage medium that records a program 240.
- the information processing apparatus 230 includes an arithmetic processing unit 210 such as one or more microprocessors, and a storage unit 220 such as a semiconductor memory or a hard disk.
- the storage unit 220 stores the dictionary 102, the detection result DB 103, the teacher data memory 106, and the like.
- the program 240 is read into the memory from an external computer-readable recording medium when the object detection apparatus 100 is started up, etc., and controls the operation of the arithmetic processing unit 210, so that the detecting unit 101 is displayed on the arithmetic processing unit 210.
- a functional unit such as the reception unit 104, the generation unit 105, and the learning unit 107 is realized.
- the display device 120 includes a screen display device such as an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel), and has a function of displaying various information such as detection results on the screen in accordance with instructions from the object detection device 100. is doing.
- a screen display device such as an LCD (Liquid Crystal Display) or a PDP (Plasma Display Panel)
- LCD Liquid Crystal Display
- PDP Plasma Display Panel
- the input device 130 is an operation input device such as a keyboard or a mouse.
- the input device 130 detects an operator's operation as an input and outputs it to the object detection apparatus 100.
- a mouse is used as the input device 130.
- the mouse to be used can perform two types of mouse gestures, left click and right click.
- the detection unit 101 inputs images obtained by the monitoring camera 110 one by one in chronological order, and detects an object from the input images using the dictionary 102.
- the detection unit 101 has a function of storing detection results in the detection result DB 103.
- One image is an image of one frame captured by the monitoring camera 110.
- a series of frames output from the monitoring camera 110 are given consecutive frame numbers for identification.
- FIG. 2 is an explanatory diagram of the concept of detecting an object from an image.
- the detection unit 101 sets an object detection search window 112 for the image 111 and extracts the feature amount of the image in the search window 112. Furthermore, the detection unit 101 calculates a likelihood indicating the similarity between the extracted feature quantity and the dictionary 102, and determines whether the image in the search window 112 is an object based on the calculated likelihood. To detect.
- the detection unit 101 stores a detection result having information on a partial area where the object is detected and a class described later in association with the identification information of the image 111 in the detection result DB 103. As the information on the partial area of the object, position information on the image 111 of the search window 112 determined to be an object is used.
- the search window 112 is rectangular, for example, the coordinate values of the upper left and lower right vertices are used as the position information.
- the position information there are a total of three classes related to the object: a class 1 representing a class of a two-wheeled vehicle, a class 2 representing a class of a four-wheeled vehicle, and a class 0 representing a class which is none of them.
- the identification information of the image 111 is, for example, the frame number of the image.
- the detecting unit 101 changes the position and size of the search window 112 and repeats the same operation as described above, thereby searching for objects having different positions and sizes in the image 111.
- three objects are detected from the image 111.
- the detection unit 101 detects the first object by the search window having the position and size indicated by reference numeral 113, detects the second object by the search window having the position and size indicated by reference numeral 114, and indicates the position indicated by reference numeral 115.
- the third object is detected by the search window of size.
- the horizontal direction of the image is the X axis and the vertical direction is the Y axis.
- FIG. 3 is an example of the detection result DB 103.
- the detection result DB 103 in this example records the detection result corresponding to the frame number.
- the detection result is composed of partial area information and class of the object.
- the detection result DB 103 records pairs of information and classes of partial areas of three objects corresponding to the frame number 001.
- One is a pair of a partial area and class 2 where the coordinates of the upper left vertex are (x1, y1) and the coordinates of the lower right vertex are (x2, y2), and the other is the coordinate of the upper left vertex ( x3, y3), a pair of a partial region where the coordinates of the lower right vertex are (x4, y4) and class 1, and the other one is the coordinates of the upper left vertex (x5, y5) and the coordinates of the lower right vertex It is a pair of a partial region and class 1 that is (x6, y6).
- the detection result DB 103 shown in FIG. 3 has a column for recording a correction class for each detection result.
- the correction class corresponds to the class input by the reception unit 104.
- all the correction class fields are NULL (indicated by-in FIG. 3).
- the accepting unit 104 visualizes the detection result of the image stored in the detection result DB 103 and displays it on the display device 120, and accepts an input of correction from the operator.
- FIG. 4 shows an example of a reception screen 121 displayed on the display device 120 by the reception unit 104.
- the reception screen 121 displays a graph 122 at the bottom of the screen, a slider 123 at the top, and an image window 124 at the top.
- the graph 122 is a graph showing a relationship between a plurality of images including the image 111 and the number of partial areas in which an object is detected on each screen.
- the horizontal direction of the reception screen 121 is the X axis and the vertical direction is the Y axis
- the X axis of the graph 122 is the frame number of the image
- the Y axis is the number of partial areas in which the object is detected, that is, the number of detected objects. Show. That is, the accepting unit 104 sets a column in which input images corresponding to detection results stored in the detection result DB 103 are arranged in ascending or descending order of frame numbers as the X axis of the graph 122.
- the X axis may be other than the column in which the input images are arranged in ascending order or descending order of the frame numbers.
- the X axis may be a column in which input images are arranged by the number of partial areas of the object (that is, the number of detected objects).
- the reception unit 104 sorts the input images of the detection results 103 stored in the detection result DB 103 in ascending or descending order according to the number of detected objects, and sets the sorted input image column as the X axis of the graph 122.
- the graph 122 is a bar graph, but the type of the graph 122 is not limited to the bar graph, and may be other types such as a line graph.
- the slider 123 is a GUI (Graphical User Interface) for selecting a position on the X axis of the graph 122, that is, an image.
- the slider 123 includes a slide bar 123a, and the position on the X axis of the graph 122 is selected by operating the slide bar 123a left and right with the input device 130.
- the image window 124 displays one image 111 selected by operating the slider 123 among a plurality of images in units of frames.
- a display for emphasizing the partial area of the detected object is added.
- a rectangular frame 125 indicating the outer periphery of the partial area of the object is displayed.
- the display for emphasizing the partial area of the object is not limited to the rectangular frame 125, a display form in which the luminance of the entire partial area is made higher or lower than the other areas, a display form in which the entire partial area is shaded, etc. Any display form may be used.
- the display that highlights the partial area of the detected object is displayed in a different display form depending on the class of the detected object.
- the partial area of the class 1 object is displayed with a broken line frame 125
- the partial area of the class 2 object is displayed with a solid line frame 125.
- the display color of the frame 125 may be changed.
- a numerical value indicating the class may be displayed in the vicinity of the partial image.
- a scale and a label that are used as a guide for the operation of the operator are displayed on the GUI slider.
- the reception screen 121 shown in FIG. 4 displays a graph 122 indicating the number of detected objects for each image instead of the scale and the label. By displaying such a graph 122, the operator can easily select one image from a plurality of images based on the number of detected objects.
- the reception unit 104 has a function of receiving selection of a partial area and input of a class for the selected partial area by operating the input device 130 with respect to the image 111 displayed on the display device 120.
- the accepting unit 104 accepts selection of a partial area based on the position of the image 111 where the operation of the input device 130 is performed, and accepts class input depending on the type of the operation.
- the input device 130 is a mouse, and the types of operations are left click and right click. Left click means an operation of single-clicking the left button of the mouse.
- the right click means an operation of single-clicking the right button of the mouse.
- the accepting unit 104 accepts selection of a partial region having the position of the image 111 left-clicked or right-clicked in the region.
- the accepting unit 104 accepts an input of class 0 for the selected partial area when left-clicked, and accepts an input of a class obtained by incrementing the current class when right-clicked.
- the class obtained by incrementing the current class is class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. means.
- the receiving unit 104 records the received class for the selected partial region in the correction class column of the detection result DB 103.
- the generation unit 105 generates teacher data from the partial region image selected by the reception unit 104 and the input class, and stores the teacher data in the teacher data memory 106.
- the generation unit 105 generates teacher data for each partial region in which any of class 0, class 1, and class 2 is recorded in the correction class column in the detection result DB 103 shown in FIG.
- the teacher data uses the partial region image or the feature amount extracted from the partial region image and the class recorded in the correction class column as an input / output pair.
- the learning unit 107 learns the dictionary 102 using the teacher data stored in the teacher data memory 106. Since the method of learning the dictionary 102 using the teacher data is widely known, the description thereof is omitted.
- the dictionary 102 is upgraded by learning with teacher data.
- the detection unit 101 uses the upgraded dictionary 102 to detect an object from each image input from the monitoring camera 110 again.
- FIG. 5 is a flowchart showing the operation of this embodiment. The operation of this embodiment will be described below with reference to FIG.
- the detection unit 101 of the object detection apparatus 100 inputs images obtained by the monitoring camera 110 one by one in time series (step S101). Next, the detection unit 101 detects an object from the input image using the dictionary 102 by the method described with reference to FIG. 2, and stores the detection result in the detection result DB 103 (step S102). If there is a next image obtained by imaging by the monitoring camera 110 (step S103), the detection unit 101 returns to step S100 and repeats the same processing as described above. Thereby, the detection results of a plurality of images as shown in FIG. 3 are accumulated in the detection result DB 103. When the processing of the detection unit 101 for a series of images captured by the monitoring camera 110 is completed, processing by the reception unit 104 is started.
- the accepting unit 104 displays the accepting screen 121 as shown in FIG. 4 on the display device 120, and accepts an input operation of the input device (step S104). Details of step S104 will be described later.
- the generation unit 105 When the reception by the reception unit 104 ends, the generation unit 105 generates teacher data from the partial region image selected by the reception unit 104 and the input class (step S105). Next, the learning unit 107 learns the dictionary 102 using the teacher data stored in the teacher data memory 106 (step S106).
- FIG. 6 is a flowchart showing details of the operation of the receiving unit 104 performed in step S104. Hereinafter, the operation of the receiving unit 104 will be described in detail with reference to FIG.
- the accepting unit 104 calculates the number of detected objects for each image from the detection result DB 103 (step S111). Referring to FIG. 3, since three partial areas are detected from the image of frame number 001, the receiving unit 104 sets the number of detected objects of frame number 001 to 3. Similarly, the number of detected objects of an image with another frame number is calculated.
- the reception unit 104 displays an initial reception screen 121 on the display device 120 (step S112).
- the initial reception screen 121 displays a graph 122, a slider 123, and an image window 124.
- the graph 122 displays the number of detected objects for each frame number calculated in step S111.
- the slider 123 is obtained by placing a slide bar 123a at a predetermined position.
- the image window 124 displays an image of the object number on the X axis of the graph 122 indicated by the slide bar 123a. Further, the partial area of the object detected on the image displayed in the image window 124 is emphasized by the line type frame 125 corresponding to the class of the partial area.
- the reception unit 104 determines whether or not the reception end condition is satisfied, whether or not the slide bar 123a of the slider 123 has been operated (step S113), whether or not an image on the image window has been clicked (step S114). Whether or not (step S115).
- the reception end condition may be, for example, that the reception end command is input from the operator, or that no input operation is performed for a certain time or more.
- the reception unit 104 determines whether or not a partial region including the click position exists in the image (step S116). This determination is performed by examining whether or not the coordinate value of the clicked position is inside any partial area of the image displayed in the image window 124. If there is no partial region including the click position (NO in step S116), the reception unit 104 ignores the click. On the other hand, if there is a partial region including the click position (YES in step S116), the reception unit 104 determines that the partial region including the click position is selected, and clicks to determine the class input. It is determined whether the type is left click or right click (step S117).
- the accepting unit 104 accepts an input of class 0 for the selected partial area (step S118).
- the reception unit 104 updates the correction class corresponding to the selected partial region of the detection result in the detection result DB 103 to the received class 0 (step S119).
- the reception unit 104 hides the frame 125 displayed in the selected partial area of the image displayed in the image window 124 (step S120).
- the receiving unit 104 receives an input of a class obtained by incrementing the current class of the selected partial area (step S121).
- the current class of the selected partial area is the detection result class when the correction class corresponding to the selected partial area of the detection result in the detection result DB 103 is NULL, and is corrected when the correction class is not NULL. It is a class described in the class.
- the reception unit 104 updates the correction class corresponding to the selected partial region of the detection result in the detection result DB 103 to the class after the increment (step S122).
- the receiving unit 104 updates the display of the frame 125 displayed in the selected partial area of the image displayed in the image window 124 (step S123).
- the reception unit 104 displays the frame 125 with a solid line. If the incremented class is class 0, the reception unit 104 hides the frame 125. Further, if the incremented class is class 1, the reception unit 104 displays the frame 125 with a broken line.
- the reception unit 104 updates the images of the slide bar 123a and the image window 124 on the reception screen 121 (step 124). Specifically, the reception unit 104 moves the display position of the slide bar 123a according to the input operation. In addition, the reception unit 104 displays an image of the object number on the X axis of the graph 122 indicated by the moved slide bar 123a in the image window 124.
- the reception unit 104 determines whether the partial area on the image displayed in the image window 124 has the same position and the same pre-input class as the partial area of the image where the class has been input in the past. When determining that they have the same position and the same pre-input class, the reception unit 104 hides the frame 125 that highlights the partial area. For example, the positions of the third partial images in frame numbers 001, 002, and 003 in FIG. 3 are all rectangles at the same position with (x5, y5) and (x6, y6) as the upper left vertex and the lower right vertex. Yes, the class before input is also the same class 1.
- the receiving unit 104 displays an input image on the display device 120 with a display that highlights the partial area of the detected object, and the partial area can be selected and clicked by one operation of the input device 130. This is to accept an input of a class for the selected partial area.
- one image can be easily selected from a plurality of images based on the number of detected objects.
- the receiving unit 104 displays a graph 122 indicating the relationship between the input image and the number of partial areas in which the object is detected on the display device 120, and according to the position on the graph 122 received by the operation of the slider 123. This is for selecting an input image for displaying the detection result on the display device 120.
- an image to be processed can be selected based on the number of detected objects. For this reason, according to the level and preference of the operator's skill, processing can be performed preferentially from an image in which many objects are detected, and conversely, processing can be performed preferentially from an image with a small number of detected objects. .
- the operator can easily perform processing preferentially from an intermediate number of detected objects.
- the total work time is shortened when processing is performed preferentially from an image in which many objects are detected.
- the accepting unit 104 hides the frame 125 that highlights the partial region having the same position and the same pre-input class as the partial region of the image in which the class has been input in the past. Thereby, the troublesomeness can be eliminated.
- the accepting unit 104 accepts an input of class 0 by a left click of a mouse gesture, and accepts an input of a class increment by a right click.
- various combinations are possible for the gesture type that accepts the input of class 0 and the gesture type that accepts the input of the class increment, as illustrated in the list of FIG.
- the accepting unit 104 accepts an input of class 0 by left clicking the mouse gesture, and accepts an input of class increment by double clicking.
- Double-clicking means an operation of single-clicking the left mouse button twice in succession.
- the receiving unit 104 receives a class 0 input by left-clicking a mouse gesture, and receives a class increment input by dragging and dropping.
- Drag and drop means an operation of moving (dragging) the mouse while holding down the left button of the mouse and releasing the left button (dropping) at another location.
- step S116 for example, a partial region including a drag start point and a drop point is displayed. It is determined whether or not there is.
- the numbers 1-1 and 1-2 use mouse gestures as in the first embodiment.
- numbers 1-3 to 1-6 shown in FIG. 7 use a touch gesture.
- the touch gesture means a touch operation using a part of a human body such as a fingertip performed on the touch panel.
- Touch gestures include taps (flicking with a finger, etc.), double taps (flicking with a finger, etc.), flicks (flicking with a finger, etc.), swipes (flicking with a finger, etc.), pinch in There are many types such as pinching out (operation to spread with a plurality of fingers).
- a touch panel is used as the input device 130.
- the use of the touch panel is also useful when the present invention is mounted on a mobile device such as a smartphone or a tablet terminal.
- the receiving unit 104 receives a class 0 input by flicking and receives a class increment input by swiping. Also, in the numbers 1-4 of FIG. 7, the receiving unit 104 receives a class 0 input by a flicking in the left direction and receives an input of a class increment by a flicking in the right direction. In numbers 1-5 in FIG. 7, the accepting unit 104 accepts class 0 input by pinch-in and accepts class increment input by pinch-out. In the numbers 1 to 6 in FIG. 7, the accepting unit 104 accepts an input of class 0 by a tap and accepts an input of an increment of a class by a double tap. These are examples and any other combination of gestures may be used.
- the receiving unit 104 receives an input of class 0 by a first type gesture, and increments the class by a second type gesture. Accepted input.
- the accepting unit 104 may accept decrement input instead of increment. That is, in the first embodiment and the numbers 1-1 to 1-6 in FIG. 7, the accepting unit 104 accepts an input of class 0 by the first type gesture, and receives the class by the second type gesture. A decrement input may be accepted.
- a class 0 input is accepted by a left click of a mouse gesture, and a class increment input is accepted by a right click.
- the class and the operation may be associated with each other on a one-to-one basis, and the class 0, class 1, and class 2 inputs may be received by a specific one operation. As illustrated in the list of FIG. 8, various combinations are possible for the gesture types that receive the input of each class.
- the accepting unit 104 accepts an input of class 0 by left-clicking a mouse gesture, accepts an input of class 1 by right-clicking, and accepts an input of class 2 by double-clicking.
- the accepting unit 104 accepts class 0 and class 1 inputs with the same mouse gesture as number 1-11, and accepts class 2 inputs with drag and drop.
- numbers 1-13 to 1-15 shown in FIG. 8 use a touch gesture.
- the accepting unit 104 accepts class 0 input by flicking, accepts class 1 input by tapping, and accepts class 2 input by double tapping.
- the accepting unit 104 accepts an input of class 0 by flicking leftward, accepts an input of class 1 by flicking upward, and inputs an input of class 2 by flicking downward. Accept. Also, in the number 1-15 shown in FIG. 8, the accepting unit 104 accepts class 0 input by tapping, accepts class 1 input by pinch-in, and accepts class 2 input by pinch-out.
- the number of classes is three (class 0, class 1, class 2), but a two-class object detection device having two classes (class 0, class 1), and Application to a multi-class object detection apparatus having four or more classes is also possible.
- FIG. 1 is a block diagram of the first embodiment.
- the reception unit 104 has a function of visualizing and displaying the image detection result DB 103 on the display device 120 and receiving an input of correction from the operator.
- FIG. 9 shows an example of a reception screen 126 that the reception unit 104 displays on the display device 120.
- the reception screen 126 of this example displays a rectangular frame indicating the outer periphery of the partial area at the location of the two-wheeled vehicle in the image 111 displayed in the image window 124.
- the reason why the rectangular frame is not displayed is that when the detection unit 101 detects an object from the image 111 using the dictionary 102, the detection of the two-wheeled vehicle at that location has failed.
- the object detection device needs to create teacher data that is an input / output pair of a partial region of a two-wheeled vehicle and its class in the image 111 and learn the dictionary 102. is there.
- the accepting unit 104 has a function of accepting selection of a partial area and input of a class for the selected partial area by operating the input device 130 with respect to the image 111 displayed on the display device 120.
- selection of a partial area means both selection of an existing partial area and creation of a new partial area.
- the accepting unit 104 accepts selection of a partial area based on the position of the image 111 where the operation of the input device 130 is performed, and accepts class input depending on the type of the operation.
- the input device 130 is a mouse, and the operation types are left click, right click, lower right drag and drop, and lower left drag and drop.
- the drag and drop in the lower right direction means that the mouse is moved (dragged) from a certain location (start point) a on the image 111 while the left button of the mouse is pressed. This means an operation of releasing (dropping) the left button at another location (end point) b existing in the lower right direction of a.
- the drag and drop in the lower left direction is, as shown by an arrow in FIG. 10B, moving (dragging) the mouse from a certain place (starting point) a on the image 111 while pressing the left button of the mouse. This means an operation of releasing (dropping) the left button at another location (end point) b existing in the lower left direction of the start point a.
- the accepting unit 104 accepts selection of a partial area having the position of the image 111 left-clicked or right-clicked in the area.
- the accepting unit 104 accepts an input of class 0 for the selected partial area when left-clicked, and accepts an input of a class obtained by incrementing the current class when right-clicked.
- the class obtained by incrementing the current class is class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0.
- the receiving unit 104 records the received class for the selected partial area in the correction class column of the detection result 103. The above is the same as when left-clicking or right-clicking in the first embodiment.
- the receiving unit 104 calculates a rectangular area having the start point a as the upper left vertex and the end point b as the lower right vertex as a partial area, as shown by a broken line in FIG. 10A.
- an input of class 1 is accepted for the partial area.
- the reception unit 104 calculates a rectangular area having the start point a as the upper right vertex and the end point b as the lower left vertex as a partial area, as shown by the broken line in FIG. 10B.
- the class 2 input is accepted for the partial area.
- the receiving unit 104 records the newly calculated partial area and its class information in the detection result DB 103.
- FIG. 11 is a flowchart showing details of the operation of the reception unit 104.
- steps S211 to S224 are the same as steps S111 to S124 of FIG.
- the operation of the reception unit 104 will be described in detail with a focus on differences from FIG.
- the reception unit 104 determines the direction of drag and drop (step S226). In the case of drag and drop in the lower right direction, the accepting unit 104 calculates a rectangular partial region having the start point as the upper left vertex and the end point as the lower right vertex as described with reference to FIG. Is received (step S227). Next, the receiving unit 104 records the calculated partial region / class 1 pair as one detection result in the detection result DB 103 (step S228). Next, the reception unit 104 updates the image window 124 on the reception screen 126 (step S229). Then, the reception unit 104 returns to the process of step S213.
- the reception unit 104 calculates a rectangular partial region having the start point as the upper right vertex and the end point as the lower left vertex as described with reference to FIG. 10B. 2 is received (step S230). Next, the reception unit 104 records the calculated partial region and class 2 pair as one detection result in the detection result DB 103 (step S231). Next, the reception unit 104 updates the image window 124 on the reception screen 126 (step S232). Then, the reception unit 104 returns to the process of step S213.
- FIG. 12A and 12B are explanatory diagrams of step S228 by the receiving unit 104.
- FIG. FIG. 12A shows the detection result DB 103 generated by the detection unit 101
- FIG. 12B shows the detection result DB 103 after adding a record of the partial area and class 1 newly calculated by the reception unit 104.
- the newly added partial area is a rectangle whose upper left vertex is (x3, y3) and lower right vertex is (x4, y4), and its class is class 1.
- the class of the partial area to be added is recorded in the correction class column.
- the generation unit 105 generates teacher data from the added partial area and class.
- the difference is that class 2 is recorded in the correction class.
- FIG. 13A and 13B are explanatory diagrams of step S229 performed by the reception unit 104.
- FIG. FIG. 13A shows the image window 124 before update
- FIG. 13B shows the image window 124 after update.
- a rectangular frame 125 is drawn with a broken line that emphasizes the newly added partial area.
- the updated image window 124 is different in that a rectangular frame that highlights the newly added partial area is drawn with a solid line.
- the receiving unit 104 displays an input image on the display device 120 with a display that highlights the partial area of the detected object, and the partial area can be selected and clicked by one operation of the input device 130. This is to accept an input of a class for the selected partial area. Further, the reception unit 104 detects one operation of drag-and-drop in the lower right direction or the lower left direction on the image, and receives a calculation of a new partial area and an input of class 1 or class 2.
- the accepting unit 104 accepts an input of class 0 by left-clicking a mouse gesture, accepts an input of class increment by right-clicking, and a partial region of class 1 by dragging and dropping in the lower right direction.
- the partial region of class 2 was calculated by drag and drop in the lower left direction.
- the gesture types that accept class 0 input, the gesture types that accept class increment input, the gesture types that accept class 1 partial area calculations, and the gesture type accepts class 2 partial area calculations are listed in FIG. Various combinations are possible as illustrated in FIG.
- the accepting unit 104 accepts an input of class 0 by left-clicking a mouse gesture, and accepts an input of class increment by double-clicking. Further, the reception unit 104 receives calculation of the class 1 partial area by dragging and dropping in the lower right direction, and receives calculation of the class 2 partial area by dragging and dropping in the lower left direction.
- the accepting unit 104 accepts an input of class 0 by left-clicking the mouse gesture, accepts an input of class increment by right-clicking, and accepts an input of class 1 by dragging and dropping in the upper left direction.
- the calculation of the partial area is accepted, and the calculation of the class 2 partial area is accepted by drag and drop in the upper right direction.
- the drag and drop in the upper left direction is drag and drop in the direction opposite to the arrow in FIG. 10A, and the mouse is moved from a certain place (start point) b on the image 111 while the left button of the mouse is pressed.
- the drag and drop in the upper right direction is drag and drop in the direction opposite to the arrow in FIG. 10B, and the mouse is moved from a certain place (start point) b on the image 111 while the left button of the mouse is pressed.
- (Drag) means an operation of releasing (dropping) the left button at another location (end point) c existing in the upper right direction of the start point b.
- the accepting unit 104 accepts calculation of a class 1 partial region by a left double click, and accepts calculation of a class 2 partial region by a right double click.
- the left double-click means that the left button of the mouse is continuously clicked twice.
- the right double-click means that the right button of the mouse is clicked twice in succession.
- FIG. 15 is an explanatory diagram of a method in which the reception unit 104 calculates a partial region of class 2 from the position of the right double click performed on the image 111.
- the reception unit 104 detects that a right double-click has been performed at a point c on the image 111, the center of gravity of a typical class 2 object (four-wheeled vehicle in this embodiment) 131 matches the point c.
- the circumscribed rectangle 132 of the object is calculated as a partial area.
- the circumscribed rectangle of a large object is almost uniquely determined. Furthermore, a circumscribed rectangle of a typical class 1 object (in this embodiment, a two-wheeled vehicle) having a certain point on the image as the center of gravity is also determined almost uniquely. Thereby, the reception unit 104 calculates a partial region of class 1 from the position of the left double click performed on the image 111 by the same method.
- the operator is configured to right double-click or left double-click the center of gravity position of the object via the input device 130.
- the position is not limited to the center of gravity position, and may be any predetermined position.
- the center of the front wheel may be right-clicked or left-double-clicked in the middle of the front and rear wheels.
- the operator can be configured to double left-click the person's head via the input device 130.
- the reception unit 104 estimates a person area from the head position, and calculates a circumscribed rectangle of the estimated person area as a partial area.
- the numbers 2-1 to 2-3 use mouse gestures as in the second embodiment.
- numbers 2-4 to 2-7 shown in FIG. 14 use a touch gesture as follows.
- the accepting unit 104 accepts an input of class 0 by flicking, accepts an input of class increment by tapping, accepts calculation of a partial region of class 1 by swiping in the lower right direction, and lower left The calculation of the class 2 partial area is accepted by swiping the direction.
- a swipe in the lower right direction is, as indicated by an arrow in FIG. 16A, tracing with a fingertip from a certain place (start point) a on the image 111 to another place (end point) b existing in the lower right direction of the start point a.
- Means operation Further, the swipe in the lower left direction is, as indicated by an arrow in FIG. 16B, tracing with a fingertip from a certain place (start point) a on the image 111 to another place (end point) b existing in the lower left direction of the start point a.
- Means operation When the reception unit 104 detects a swipe in the lower right direction, as illustrated by a broken line in FIG.
- the reception unit 104 calculates a rectangular region having the start point a as the upper left vertex and the end point b as the lower right vertex as a partial region, and Class 1 input is accepted for the partial area.
- the reception unit 104 detects a swipe in the lower left direction, as shown by a broken line in FIG. 16B, the reception unit 104 calculates a rectangular region having the start point a as the upper right vertex and the end point b as the lower left vertex as a partial region, and Class 2 input is accepted for the partial area.
- the accepting unit 104 accepts an input of class 0 by flicking in the left direction, accepts an input of class increment by flicking in the right direction, and swipes down in the class 1 partial area. The calculation is accepted and the calculation of the class 2 partial area is accepted by swiping upward.
- the downward swipe is an operation of tracing with a fingertip from a certain place (start point) a on the image 111 to another place (end point) b existing downward from the start point a as shown by an arrow in FIG. 17A. means. Further, as shown by the arrow in FIG. 17B, the upward swipe is performed by tracing with a fingertip from a certain place (start point) b on the image 111 to another place (end point) a existing above the start point b. Means operation.
- the receiving unit 104 detects a downward swipe, the receiving unit 104 calculates a rectangle as indicated by a broken line in FIG. The rectangle is line-symmetric with the swipe line, and the vertical length is equal to the swipe length.
- the length in the left-right direction is obtained by multiplying the swipe length by a predetermined ratio.
- the predetermined ratio for example, the ratio of the length in the left-right direction to the length in the vertical direction of the circumscribed rectangle of a class 1 object (two-wheeled vehicle in this embodiment) having a typical shape can be used.
- the reception unit 104 calculates a rectangle as indicated by a broken line in FIG. 17B as the class 2 partial region.
- the rectangle is line-symmetric with the swipe line, and the vertical length is equal to the swipe length.
- the length in the left-right direction is obtained by multiplying the swipe length by a predetermined ratio.
- the predetermined ratio the ratio of the length in the left-right direction to the length in the vertical direction of the circumscribed rectangle of the class 2 object (four-wheeled vehicle in the present embodiment) having a typical shape can be used.
- the accepting unit 104 accepts input of class 0 by pinch-in, accepts input of class increment by pinch-out, accepts calculation of a partial region of class 1 by swiping in the lower right direction, The calculation of the class 2 partial area is accepted by swiping in the lower left direction.
- the accepting unit 104 accepts an input of class 0 by a tap, accepts an input of an increment of a class by double tap, accepts a calculation of a partial region of the class 1 by swiping in the lower right direction, The calculation of the class 2 partial area is accepted by swiping in the lower left direction.
- the accepting unit 104 accepts an input of class 0 by the first type gesture, and increments the class by the second type gesture. Accepted input. However, the accepting unit 104 may accept decrement input instead of increment. That is, in the second embodiment and numbers 2-1 to 2-7 in FIG. 14, the accepting unit 104 may accept an input of class decrement by a second type of gesture.
- the accepting unit 104 accepts an input of class 0 by left-clicking a mouse gesture, and accepts an input of class increment by right-clicking.
- the class and the operation may be associated with each other on a one-to-one basis, and the class 0, class 1, and class 2 inputs may be received by a specific one operation.
- various combinations are possible for the gesture type that receives the input of each class and the gesture type that receives the calculation of the partial area of each class.
- the accepting unit 104 accepts an input of class 0 by left-clicking a mouse gesture, accepts an input of class 1 by right-clicking, accepts an input of class 2 by double-clicking, Calculation of the class 1 partial area is accepted by dragging and dropping in the lower right direction, and calculation of the class 2 partial area is accepted by dragging and dropping in the lower left direction.
- the number 2-12 shown in FIG. 18 is different from the number 2-11 in the gesture type for accepting the calculation of the class 1 and class 2 partial areas.
- the accepting unit 104 accepts the calculation of the class 1 partial region by dragging and dropping in the upper left direction, and accepts the calculation of the class 2 partial region by dragging and dropping in the upper right direction.
- the reception unit 104 receives a class 0 input by flicking a touch gesture, receives a class 1 input by a tap, receives a class 2 input by a double tap, and moves to the lower right direction.
- the swipe of the class 1 accepts the calculation of the class 1 partial area
- the swipe in the lower left direction accepts the calculation of the class 2 partial area.
- the number 2-14 shown in FIG. 18 differs from the number 2-13 in the gesture type for accepting inputs of classes 0, 1, and 2.
- the reception unit 104 receives a class 0 input by a left flick, receives a class 1 input by an upward flick, and receives a class 2 input by a downward flick.
- the accepting unit 104 accepts class 0 input by tapping, accepts class 1 input by pinch-in, accepts class 2 input by pinch-out, and swipes down to class
- the calculation of the partial area of 1 is accepted, and the calculation of the partial area of class 2 is accepted by swiping upward.
- the number of classes is three (class 0, class 1, class 2), but the two-class object detection device with two classes (class 0, class 1), and Application to a multi-class object detection apparatus having four or more classes is also possible.
- FIG. 1 is a block diagram of the first embodiment.
- the reception unit 104 has a function of visualizing and displaying the image detection result DB 103 on the display device 120 and receiving an input of correction from the operator.
- the accepting unit 104 has a function of accepting selection of a partial area and input of a class for the selected partial area by operating the input device 130 with respect to the image 111 displayed on the display device 120.
- selection of a partial area means both selection of an existing partial area and selection of a new partial area.
- the accepting unit 104 accepts selection of a partial area based on the position of the image 111 where the operation of the input device 130 is performed, and accepts class input depending on the type of the operation. Further, in the present embodiment, different classes of input are accepted even in the same operation depending on whether or not there is an existing partial region having the position of the image on which the operation has been performed.
- the input device 130 is a mouse, and the type of one operation is left click or right click.
- the accepting unit 104 accepts selection of an existing partial area having the position of the image 111 left-clicked or right-clicked in the area.
- the accepting unit 104 accepts an input of class 0 for the selected partial area when left-clicked, and accepts an input of a class obtained by incrementing the current class when right-clicked.
- the class obtained by incrementing the current class is class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0. means.
- the receiving unit 104 records the received class for the selected partial region in the correction class column of the detection result DB 103. The above is the same as when left-clicking or right-clicking in the first embodiment.
- the reception unit 104 calculates a new rectangular partial region based on the clicked position when there is no existing partial region having the position of the left-clicked or right-clicked image 111 in the region, and If the partial area is left-clicked, an input of class 1 is accepted.
- a method for calculating a new partial area based on the clicked position a method similar to the method for calculating the partial area of the object based on the double-clicked position described with reference to FIG. 15 may be used. it can.
- FIG. 19 is a flowchart showing details of the operation of the receiving unit 104.
- steps S211 to S224 and S226 to S232 are substantially the same as steps S211 to S224 and S226 to S232 of FIG.
- FIG. 19 is different from FIG. 11 in that when it is determined that there is no existing partial area including the clicked position in step S216, the process proceeds to step S226.
- step S226 in FIG. 11 determines the direction (type) of drag and drop, but step S226 in FIG. 19 determines the type of click.
- the operation of the receiving unit 104 will be described in detail with a focus on differences from FIG.
- the accepting unit 104 detects left click or right click on the image in the image window 124 (YES in step S214). At this time, if the accepting unit 104 determines that the partial region including the click position does not exist in the image (NO in step S216), the click type is either left click or right click in order to determine the class input. It is determined whether or not there is (step S226).
- the accepting unit 104 calculates a new rectangular partial area and accepts an input of class 1 (step S227).
- the receiving unit 104 records the calculated partial region / class 1 pair as one detection result in the detection result DB 103 (step S228).
- the reception unit 104 updates the image window 124 on the reception screen 126 (step S229). Then, the reception unit 104 returns to the process of step S213.
- the accepting unit 104 is a right click (step S226), it calculates a new rectangular partial region and accepts an input of class 2 (step S230). Next, the reception unit 104 records the calculated partial region and class 2 pair as one detection result in the detection result DB 103 (step S231). Next, the reception unit 104 updates the image window 124 on the reception screen 126 (step S232). Then, the reception unit 104 returns to the process of step S213.
- the receiving unit 104 displays an input image on the display device 120 with a display that highlights the partial area of the detected object, and the partial area can be selected and clicked by one operation of the input device 130. This is to accept an input of a class for the selected partial area.
- the reception unit 104 detects one operation of left click or right click at a position that does not overlap with an existing partial area, and receives a calculation of a new partial area and an input of class 1 or class 2. .
- FIG. 1 is a block diagram of the first embodiment.
- the reception unit 104 has a function of visualizing and displaying the image detection result DB 103 on the display device 120 and receiving an input of correction from the operator.
- FIG. 20 shows an example of a reception screen 127 displayed on the display device 120 by the reception unit 104.
- the reception screen 127 in this example is different from the reception screen 121 shown in FIG. 4 only in the position of a rectangular frame indicating the outer periphery of the partial area of the four-wheeled vehicle in the image 111 displayed in the image window 124. To do.
- the reason why the positions of the rectangular frames are different is that when the detection unit 101 detects an object from the image 111 using the dictionary 102, it erroneously detects a puddle on the road as a part of the partial area of the object. by. In this example, the partial area is erroneously detected, but the class is correctly determined.
- teacher data which is an input / output pair of the erroneously detected partial region in the image 111 and its class 0, the correct partial region of the four-wheeled vehicle in the image 111 and its class 2 It is necessary to create the teacher data that is an input / output pair and to learn the dictionary 102.
- the accepting unit 104 has a function of accepting selection of a partial area and input of a class for the selected partial area by operating the input device 130 with respect to the image 111 displayed on the display device 120.
- selection of a partial area means both selection of an existing partial area and selection of a new partial area.
- the accepting unit 104 accepts selection of a partial area based on the position of the image 111 where the operation of the input device 130 is performed, and accepts class input depending on the type of the operation.
- an input of class 0 is accepted for the existing partial area, and the new partial area Accept class input according to operation type.
- the input device 130 is a mouse, and one operation type is left click, right click, lower right drag and drop, and lower left drag and drop.
- the accepting unit 104 accepts selection of a partial area having the position of the image 111 left-clicked or right-clicked in the area.
- the accepting unit 104 accepts an input of class 0 for the selected partial area when left-clicked, and accepts an input of a class obtained by incrementing the current class when right-clicked.
- the class obtained by incrementing the current class is class 2 when the current class is class 1, class 0 when the current class is class 2, and class 1 when the current class is class 0.
- the receiving unit 104 records the received class for the selected partial region in the correction class column of the detection result DB 103. The above is the same as when left-clicking or right-clicking in the first embodiment.
- the reception unit 104 calculates a rectangular area having the start point a as the upper left vertex and the end point b as the lower right vertex as a partial area, as shown by the broken line in FIG. 10A. In addition, an input of class 1 is accepted for the partial area. Further, when detecting the drag-and-drop in the lower left direction, the receiving unit 104 calculates a rectangular area having the start point a as the upper right vertex and the end point b as the lower left vertex as a partial area, as shown by the broken line in FIG. Class 2 input is accepted for the partial area. In addition, the reception unit 104 records the newly calculated partial area and the information of the class in the detection result DB 103 as one detection result. The above is the same as when dragging and dropping in the lower right direction and the lower left direction in the second embodiment.
- the receiving unit 104 has a function of receiving an input of class 0 for an existing partial area that partially overlaps the newly calculated partial area. For example, as illustrated in FIG. 21, the reception unit 104 performs drag and drop in the lower left direction (or lower right direction) on the image 111 in which the frame 125 that emphasizes the partial region of class 2 is displayed. Thus, when the partial area 125 a partially overlapping the frame 125 is calculated, an input of class 0 is accepted for the existing partial area related to the frame 125.
- FIG. 22 is a flowchart showing details of the operation of the receiving unit 104.
- steps S311 to S327 and S330 are the same as steps S211 to S327 and S230 of FIG.
- the operation of the receiving unit 104 will be described in detail with a focus on differences from FIG.
- the accepting unit 104 determines whether or not an existing partial area that partially overlaps the new partial area exists on the image 111. Is determined (step S341). This determination is performed by investigating whether or not the detection result DB 103 related to the image 111 includes a detection result of an existing partial area that partially overlaps the new partial area. If such an existing partial area exists, the reception unit 104 receives an input of class 0 for the partial area (step S342), and proceeds to step S343. If such an existing partial area does not exist, the reception unit 104 skips the process of step S342 and proceeds to step S343.
- step S343 the reception unit 104 updates the detection result DB 103 according to the reception results of step S327 and step S342. That is, the accepting unit 104 records the newly calculated partial region / class 1 pair in the detection result DB 103 as one detection result based on the acceptance result in step S327. In addition, the reception unit 104 records class 0 in the modification class of the existing partial area based on the reception result in step S342. Next, the reception unit 104 updates the image 111 on the image window 124 in accordance with the update of the detection result DB 103 in step S343 (step S344). Then, the reception unit 104 returns to the process of step S313.
- step S330 when the accepting unit 104 accepts an input of class 2 for the newly calculated partial area (step S330), whether or not an existing partial area partially overlapping with the new partial area exists on the image 111. Is determined (step S345). If such an existing partial area exists, the reception unit 104 receives an input of class 0 for the partial area (step S346), and proceeds to step S347. If such an existing partial area does not exist, the reception unit 104 skips the process of step S346 and proceeds to step S347. In step S347, the reception unit 104 updates the detection result DB 103 according to the reception results of steps S330 and S346.
- the accepting unit 104 records the newly calculated partial area / class 2 pair in the detection result DB 103 as one detection result based on the acceptance result of step S330. Further, the reception unit 104 records class 0 in the modification class of the existing partial area based on the reception result of step S346. Next, the reception unit 104 updates the image 111 on the image window 124 in accordance with the update of the detection result DB 103 in step S347 (step S348). Then, the reception unit 104 returns to the process of step S313.
- FIG. 23A and FIG. 23B are explanatory diagrams of step S347 by the reception unit 104.
- FIG. 23A shows the detection result DB 103 generated by the detection unit 101
- FIG. 23B shows the detection result DB 103 after being updated by the reception unit 104.
- a rectangular partial region in which the upper left vertex is (x7, y7) and the lower right vertex is (x8, y8) is newly added as class 2.
- the class of the existing rectangular partial area in which the upper left vertex is (x1, y1) and the lower right vertex is (x2, y2) is modified from class 2 to class 0.
- step S343 by the accepting unit 104 the difference is that the modified class of the newly added partial area is class 1.
- FIG. 24A and FIG. 24B are explanatory diagrams of step S348 by the reception unit 104.
- FIG. FIG. 24A shows the image window 124 before update
- FIG. 24B shows the image window 124 after update.
- a rectangular frame 125 is drawn with a solid line that emphasizes the newly added partial area, and the frame displayed in the image window 124 before the update partially overlapping therewith is not displayed.
- the updated image window 124 is different in that a rectangular frame that highlights a newly added partial region is drawn by a broken line.
- the receiving unit 104 displays an input image on the display device 120 with a display that highlights the partial area of the detected object, and the partial area can be selected and clicked by one operation of the input device 130. This is to accept an input of a class for the selected partial area. Further, the reception unit 104 detects one operation of drag-and-drop in the lower right direction or the lower left direction on the image, and receives a calculation of a new partial area and an input of class 1 or class 2. Further, the reception unit 104 receives an input of class 0 for an existing partial area partially overlapping with the newly calculated partial area.
- the object detection device 1 has a function of detecting an object from the input image 2.
- the object detection apparatus 1 includes a dictionary 10 and includes a detection unit 11, a reception unit 12, a generation unit 13, and a learning unit 14 as main functional units. Furthermore, the object detection apparatus 1 is connected to the display device 3 and the input device 4.
- the detection unit 11 has a function of detecting an object from the input image 2 using the dictionary 10.
- the receiving unit 12 displays the input image 2 on the display device 3 with a display that highlights the partial region of the object detected by the detection unit 11, and selects and selects the partial region by one operation of the input device 4.
- the generation unit 13 has a function of generating teacher data from the image of the selected partial area and the input class.
- the learning unit 14 has a function of learning the dictionary 10 from the teacher data generated by the generation unit 13 and upgrading the dictionary 10.
- the detection unit 11 of the object detection device 1 detects an object from the input image 2 using the dictionary 10 and notifies the reception unit 12 of the detection result.
- the accepting unit 12 displays the input image 2 on the display device 3 with a display that emphasizes the partial region of the object detected by the detecting unit 11.
- the accepting unit 12 accepts the selection of the partial area and the input of the class for the selected partial area by one operation of the input device 4 and notifies the generation unit 13 of the result.
- the generation unit 13 generates teacher data from the selected partial region image and the input class, and notifies the learning unit 14 of the teacher data.
- the learning unit 14 learns the dictionary 10 based on the teacher data generated by the generation unit 13 and upgrades the dictionary 10.
- the receiving unit 12 displays the input image 2 on the display device 3 with a display that emphasizes the partial area of the detected object. Furthermore, the accepting unit 12 is for accepting selection of a partial region and input of a class for the selected partial region by clicking, which is one operation of the input device 4.
- the accepting unit 12 may be configured to accept the selection of the partial area based on the position of the input image 2 where one operation has been performed.
- the accepting unit 12 accepts selection of a partial area having the position of the input image 2 on which one operation has been performed within the area, and there is a partial area having the position of the input image 2 on which one operation has been performed within the area. If not, the partial area of the object may be calculated based on the position of the input image 2 where one operation has been performed.
- the accepting unit 12 may be configured to accept class input according to the type of one operation.
- the receiving unit 12 receives the input of the first class in advance if the type of one operation is the first type, and inputs the class obtained by incrementing or decrementing the current class of the partial area if the type is the second type. It may be configured to accept.
- the receiving unit 12 may be configured to receive an input of a different class depending on whether or not there is a partial region having the position of the input image 2 where one operation is performed in the region.
- the receiving unit 12 receives the input of the first class when there is a partial region having the position of the input image 2 where one operation has been performed in the region, and receives the input of the second class when there is no partial region. It may be constituted as follows.
- the reception unit 12 receives an input of the first class for the existing partial area and performs one operation.
- a partial area of the object may be calculated based on the position of the input image that has been broken, and an input of the second class may be received for the calculated partial area.
- the receiving unit 12 may be configured to determine the second class according to the type of one operation.
- one operation of the input device 4 is one mouse gesture.
- one operation of the input device 4 may be configured as one touch gesture.
- the object detection apparatus 300 includes a detection unit 301, a reception unit 302, a generation unit 303, and a learning unit 304.
- the detecting unit 301 detects an object from the input image using a dictionary.
- the receiving unit 302 displays an input image on a display device with a display that emphasizes the partial area of the detected object, and selects the partial area and inputs a class for the selected partial area by one operation of the input device. Accept.
- the generating unit 303 generates teacher data from the selected partial region image and the input class.
- the learning unit 304 learns a dictionary from teacher data.
- the receiving unit 302 displays the input image on the display device with a display that highlights the partial area of the detected object, and selects the partial area and the selected partial area by one operation of the input device. This is because it accepts the class input for.
- the present invention can be used for an object detection device that detects an object such as a person or a vehicle from a video on a road obtained by being imaged by a monitoring camera.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Mathematical Physics (AREA)
- Databases & Information Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Description
辞書を用いて入力画像からオブジェクトを検出する検出部と、
検出されたオブジェクトの部分領域を強調する表示を付して入力画像を表示デバイスに表示し、入力デバイスの1操作により、部分領域の選択と該選択した部分領域に対するクラスの入力とを受け付ける受付部と、
選択された部分領域の画像と入力されたクラスとから教師データを生成する生成部と、
教師データにより辞書を学習する学習部と、
を有するオブジェクト検出装置である。
辞書を用いて入力画像からオブジェクトを検出し、
検出されたオブジェクトの部分領域を強調する表示を付して入力画像を表示デバイスに表示し、
入力デバイスの1操作により、部分領域の選択と該選択した部分領域に対するクラスの入力とを受け付け、
選択された部分領域の画像と入力されたクラスとから教師データを生成し、
教師データにより辞書を学習するオブジェクト検出方法である。
コンピュータを、
辞書を用いて入力画像からオブジェクトを検出する検出部と、
検出されたオブジェクトの部分領域を強調する表示を付して入力画像を表示デバイスに表示し、入力デバイスの1操作により、部分領域の選択と該選択した部分領域に対するクラスの入力とを受け付ける受付部と、
選択された部分領域の画像と入力されたクラスとから教師データを生成する生成部と、
教師データにより辞書を学習する学習部と、
して機能させるプログラムを格納する記録媒体である。
[第1の実施形態]
図1を参照すると、本発明の第1の実施形態に係るオブジェクト検出装置100は、監視カメラ110で撮像されて得られた道路上のモニタ映像を解析し、オブジェクトを検出する機能を有する。本実施形態では、検出すべきオブジェクトは2輪自動車と4輪自動車の2種類である。即ち、オブジェクト検出装置100は、多クラス分類器の一種であり、画像から特定の2種類のオブジェクトを検出する。
次に、上記第1の実施形態の構成を変更した各種の変形例について説明する。
本発明の第2の実施形態は、入力画像から機械的に検出できなかったオブジェクトの部分領域の選択とその部分領域に対するクラスの入力とを、入力画像に対する入力デバイスの操作によって行えるようにしたオブジェクト検出装置について説明する。本実施形態は、第1の実施形態と比較して、受付部104の機能が相違し、その他は第1の実施形態と基本的に同じである。以下では、第1の実施形態のブロック図である図1を借用して、第2の実施形態に好適な受付部104の機能について詳細に説明する。
この例の受付画面126は、図4に示した受付画面121と比較して、画像ウインドウ124に表示されている画像111中の2輪自動車の箇所に部分領域の外周を示す矩形の枠が表示されていない点のみ相違する。矩形の枠が表示されない原因は、検出部101が画像111から辞書102を用いてオブジェクトを検出した際、その箇所の2輪自動車の検出に失敗したことによる。このような検出漏れを改善するためには、オブジェクト検出装置は、画像111中の2輪自動車の部分領域とそのクラスとの入出力ペアである教師データを作成し、辞書102を学習する必要がある。
次に、上記第2の実施形態の構成を変更した各種の変形例について説明する。
本発明の第3の実施形態は、操作が行われた画像の位置を領域内に有する既存の部分領域が存在するか否かによって同じ操作であっても異なるクラスの入力を受け付けるようにしたオブジェクト検出装置について説明する。本実施形態は、第2の実施形態と比較して、受付部104の機能が相違し、その他は第2の実施形態と基本的に同じである。以下では、第1の実施形態のブロック図である図1を借用して、第3の実施形態に好適な受付部104の機能について詳細に説明する。
上記第3の実施形態では、左クリックと右クリックという2種類のジェスチャを使用したが、他の種類のマウスジェスチャの組み合わせ、タッチジェスチャの組み合わせを使用することができる。
本発明の第4の実施形態では、誤認識された部分領域に対する正しいクラスの入力およびオブジェクトの正しい部分領域の選択を、入力画像に対する入力デバイスの操作によって行えるようにしたオブジェクト検出装置について説明する。本実施形態は、第2の実施形態と比較して、受付部104の機能が相違し、その他は第2の実施形態と基本的に同じである。以下では、第1の実施形態のブロック図である図1を借用して、第4の実施形態に好適な受付部104の機能について詳細に説明する。
上記第4の実施形態の構成に対しては、上記第2の実施形態の変形例で説明した例と同様の変形を行うことができる。
本発明の第5の実施形態について説明する。図25を参照すると、本実施形態に係るオブジェクト検出装置1は、入力画像2からオブジェクトを検出する機能を有する。オブジェクト検出装置1は、辞書10を有し、また主な機能部として、検出部11、受付部12、生成部13、および学習部14を有する。さらにオブジェクト検出装置1は、表示デバイス3および入力デバイス4に接続されている。
[第6の実施形態]
本発明の第6の実施形態に係るオブジェクト検出装置300について図27を参照して説明する。オブジェクト検出装置300は、検出部301、受付部302、生成部303および学習部304を備える。
2…入力画像
3…表示デバイス
4…入力デバイス
10…辞書
11…検出部
12…受付部
13…生成部
14…学習部
100…オブジェクト検出装置
101…検出部
102…辞書
103…検出結果DB
104…受付部
105…生成部
106…教師データメモリ
107…学習部
111…画像
112、113、114、115…探索窓
120…表示デバイス
121…受付画面
122…グラフ
123…スライダ
123a…スライドバー
124…画像ウインドウ
125…矩形の枠
125a…部分領域
126、127…受付画面
130…入力デバイス
131…オブジェクト
132…外接矩形
Claims (14)
- 辞書を用いて入力画像からオブジェクトを検出する検出手段と、
前記検出されたオブジェクトの部分領域を強調する表示を付して前記入力画像を表示デバイスに表示し、入力デバイスの1操作により、前記部分領域の選択と該選択した前記部分領域に対するクラスの入力とを受け付ける受付手段と、
前記選択された部分領域の画像と前記入力されたクラスとから教師データを生成する生成手段と、
前記教師データにより前記辞書を学習する学習手段と、
を有するオブジェクト検出装置。 - 前記受付手段は、前記1操作が行われた前記入力画像の位置により前記部分領域の選択を受け付ける、
請求項1に記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作が行われた前記入力画像の位置を領域内に有する前記部分領域の選択を受け付け、前記1操作が行われた前記入力画像の位置を領域内に有する前記部分領域が存在しなければ、前記1操作が行われた前記入力画像の位置に基づいて前記オブジェクトの部分領域を算出する、
請求項1または2に記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作の種別により前記クラスの入力を受け付ける、
請求項1乃至3の何れかに記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作の種別が第1の種別であれば予め第1のクラスの入力を受け付け、第2の種別であれば前記部分領域の現在のクラスをインクリメントまたはデクリメントしたクラスの入力を受け付ける、
請求項4に記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作が行われた前記入力画像の位置を領域内に有する前記部分領域が存在するか否かによって異なる前記クラスの入力を受け付ける、
請求項1乃至3の何れかに記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作が行われた前記入力画像の位置を領域内に有する前記部分領域が存在する場合、第1のクラスの入力を受け付け、存在しない場合、第2のクラスの入力を受け付ける、
請求項6に記載のオブジェクト検出装置。 - 前記受付手段は、前記1操作が行われた前記入力画像の位置を領域内に有する前記部分領域が存在する場合、前記存在する部分領域に対して前記第1のクラスの入力を受け付けると共に、前記1操作が行われた前記入力画像の位置に基づいて前記オブジェクトの部分領域を算出し、該算出した部分領域に対して第2のクラスの入力を受け付ける、
請求項6に記載のオブジェクト検出装置。 - 前記受付手段は、前記第2のクラスを前記1操作の種別により決定する、
請求項7または8に記載のオブジェクト検出装置。 - 前記入力デバイスの1操作が1マウスジェスチャである、
請求項1乃至9の何れかに記載のオブジェクト検出装置。 - 前記入力デバイスの1操作が1タッチジェスチャである、
請求項1乃至9の何れかに記載のオブジェクト検出装置。 - 前記受付手段は、前記検出されたオブジェクトの部分領域が、過去にクラスの入力を行った部分領域と同じ位置および同じ入力前クラスを有する場合、前記検出されたオブジェクトの部分領域を強調する表示を非表示にする、
請求項1乃至11の何れかに記載のオブジェクト検出装置。 - 辞書を用いて入力画像からオブジェクトを検出し、
前記検出されたオブジェクトの部分領域を強調する表示を付して前記入力画像を表示デバイスに表示し、
入力デバイスの1操作により、前記部分領域の選択と該選択した前記部分領域に対するクラスの入力とを受け付け、
前記選択された部分領域の画像と前記入力されたクラスとから教師データを生成し、
前記教師データにより前記辞書を学習する、
オブジェクト検出方法。 - 辞書を用いて入力画像からオブジェクトを検出し、
前記検出されたオブジェクトの部分領域を強調する表示を付して前記入力画像を表示デバイスに表示し、入力デバイスの1操作により、前記部分領域の選択と該選択した前記部分領域に対するクラスの入力とを受け付け、
前記選択された部分領域の画像と前記入力されたクラスとから教師データを生成し、
前記教師データにより前記辞書を学習する、
機能をコンピュータに実現させるためのプログラムを記録したコンピュータ読み取り可能な記録媒体。
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/558,652 US10339422B2 (en) | 2015-03-19 | 2016-03-15 | Object detection device, object detection method, and recording medium |
JP2017506093A JP6904249B2 (ja) | 2015-03-19 | 2016-03-15 | オブジェクト検出装置、オブジェクト検出方法およびプログラム |
US16/418,456 US10755143B2 (en) | 2015-03-19 | 2019-05-21 | Object detection device, object detection method, and recording medium |
US16/938,462 US11373061B2 (en) | 2015-03-19 | 2020-07-24 | Object detection device, object detection method, and recording medium |
US17/752,068 US11914851B2 (en) | 2015-03-19 | 2022-05-24 | Object detection device, object detection method, and recording medium |
US18/416,982 US20240160342A1 (en) | 2015-03-19 | 2024-01-19 | Object detection device, object detection method, and recording medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2015055926 | 2015-03-19 | ||
JP2015-055926 | 2015-03-19 |
Related Child Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15588652 A-371-Of-International | 2016-03-15 | ||
US15/558,652 A-371-Of-International US10339422B2 (en) | 2015-03-19 | 2016-03-15 | Object detection device, object detection method, and recording medium |
US16/418,456 Continuation US10755143B2 (en) | 2015-03-19 | 2019-05-21 | Object detection device, object detection method, and recording medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016147652A1 true WO2016147652A1 (ja) | 2016-09-22 |
Family
ID=56918654
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2016/001453 WO2016147652A1 (ja) | 2015-03-19 | 2016-03-15 | オブジェクト検出装置、オブジェクト検出方法および記録媒体 |
Country Status (3)
Country | Link |
---|---|
US (5) | US10339422B2 (ja) |
JP (3) | JP6904249B2 (ja) |
WO (1) | WO2016147652A1 (ja) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2020140518A (ja) * | 2019-02-28 | 2020-09-03 | キヤノン株式会社 | 情報処理装置、情報処理装置の制御方法及びプログラム |
JP2021167990A (ja) * | 2020-04-08 | 2021-10-21 | Arithmer株式会社 | レイアウト解析装置、その解析プログラムおよびその解析方法 |
JP2023001367A (ja) * | 2020-05-29 | 2023-01-04 | 株式会社日立ハイテク | 画像処理システムおよび画像処理方法 |
US11836906B2 (en) | 2017-03-27 | 2023-12-05 | Hitachi High-Tech Corporation | Image processing system and computer program for performing image processing |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9349055B1 (en) * | 2013-06-26 | 2016-05-24 | Google Inc. | Real-time image-based vehicle detection based on a multi-stage classification |
JP6904249B2 (ja) * | 2015-03-19 | 2021-07-14 | 日本電気株式会社 | オブジェクト検出装置、オブジェクト検出方法およびプログラム |
WO2016147653A1 (ja) * | 2015-03-19 | 2016-09-22 | 日本電気株式会社 | オブジェクト検出装置、オブジェクト検出方法および記録媒体 |
US10546195B2 (en) * | 2016-12-02 | 2020-01-28 | Geostat Aerospace & Technology Inc. | Methods and systems for automatic object detection from aerial imagery |
US10699119B2 (en) * | 2016-12-02 | 2020-06-30 | GEOSAT Aerospace & Technology | Methods and systems for automatic object detection from aerial imagery |
US10748033B2 (en) | 2018-12-11 | 2020-08-18 | Industrial Technology Research Institute | Object detection method using CNN model and object detection apparatus using the same |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003317082A (ja) * | 2002-04-25 | 2003-11-07 | Dainippon Screen Mfg Co Ltd | 分類支援装置、分類装置およびプログラム |
JP2005293264A (ja) * | 2004-03-31 | 2005-10-20 | Olympus Corp | 学習型分類装置及び学習型分類方法 |
JP2007280325A (ja) * | 2006-04-12 | 2007-10-25 | Omron Corp | 動画表示装置 |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6266442B1 (en) * | 1998-10-23 | 2001-07-24 | Facet Technology Corp. | Method and apparatus for identifying objects depicted in a videostream |
JP2002032751A (ja) * | 2000-07-18 | 2002-01-31 | Olympus Optical Co Ltd | 学習型画像分類装置及び方法並びにその処理プログラムを記録した記録媒体 |
JP2006012069A (ja) * | 2004-06-29 | 2006-01-12 | Olympus Corp | 分類装置及び分類方法 |
JP2006012089A (ja) | 2004-06-29 | 2006-01-12 | Chugoku Electric Power Co Inc:The | 業務従事者教育管理システム |
JP4266920B2 (ja) * | 2004-12-03 | 2009-05-27 | オリンパス株式会社 | 分類装置及び分類方法 |
JP4050273B2 (ja) * | 2004-12-28 | 2008-02-20 | オリンパス株式会社 | 分類装置及び分類方法 |
JP4516491B2 (ja) | 2005-03-14 | 2010-08-04 | 富士フイルム株式会社 | 感光性組成物及び感光性フィルム、並びに、永久パターン及びその形成方法 |
DE102006047777A1 (de) * | 2006-03-17 | 2007-09-20 | Daimlerchrysler Ag | Virtuelles Spotlight zur Kennzeichnung von interessierenden Objekten in Bilddaten |
JP5581574B2 (ja) | 2008-07-09 | 2014-09-03 | 富士ゼロックス株式会社 | 画像処理装置及び画像処理プログラム |
JP5202148B2 (ja) * | 2008-07-15 | 2013-06-05 | キヤノン株式会社 | 画像処理装置、画像処理方法、及びコンピュータプログラム |
JP2010039795A (ja) | 2008-08-06 | 2010-02-18 | Casio Comput Co Ltd | 認識用データ生成装置及び電子カメラ、認識用データ生成プログラム、並びに認識用データ生成方法 |
JP2009032273A (ja) * | 2008-08-29 | 2009-02-12 | Olympus Corp | 分類装置 |
JP5408143B2 (ja) * | 2009-02-03 | 2014-02-05 | 日本電気株式会社 | パターン認識装置、パターン認識方法およびパターン認識プログラム |
JP5366756B2 (ja) * | 2009-10-19 | 2013-12-11 | キヤノン株式会社 | 情報処理装置及び情報処理方法 |
US8447119B2 (en) * | 2010-03-16 | 2013-05-21 | Nec Laboratories America, Inc. | Method and system for image classification |
US10645344B2 (en) * | 2010-09-10 | 2020-05-05 | Avigilion Analytics Corporation | Video system with intelligent visual display |
JP2014041436A (ja) | 2012-08-21 | 2014-03-06 | Canon Inc | 情報提示方法及び情報提示装置 |
JP2014059729A (ja) | 2012-09-18 | 2014-04-03 | Toshiba Corp | 物体検出・識別装置及びその方法とその物体検出・識別に用いられる辞書データ生成方法 |
US9158996B2 (en) * | 2013-09-12 | 2015-10-13 | Kabushiki Kaisha Toshiba | Learning image collection apparatus, learning apparatus, and target object detection apparatus |
JP6904249B2 (ja) * | 2015-03-19 | 2021-07-14 | 日本電気株式会社 | オブジェクト検出装置、オブジェクト検出方法およびプログラム |
-
2016
- 2016-03-15 JP JP2017506093A patent/JP6904249B2/ja active Active
- 2016-03-15 WO PCT/JP2016/001453 patent/WO2016147652A1/ja active Application Filing
- 2016-03-15 US US15/558,652 patent/US10339422B2/en active Active
-
2019
- 2019-05-21 US US16/418,456 patent/US10755143B2/en active Active
-
2020
- 2020-07-24 US US16/938,462 patent/US11373061B2/en active Active
-
2021
- 2021-06-22 JP JP2021102951A patent/JP7279738B2/ja active Active
-
2022
- 2022-05-24 US US17/752,068 patent/US11914851B2/en active Active
-
2023
- 2023-05-08 JP JP2023076649A patent/JP7491435B2/ja active Active
-
2024
- 2024-01-19 US US18/416,982 patent/US20240160342A1/en active Pending
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003317082A (ja) * | 2002-04-25 | 2003-11-07 | Dainippon Screen Mfg Co Ltd | 分類支援装置、分類装置およびプログラム |
JP2005293264A (ja) * | 2004-03-31 | 2005-10-20 | Olympus Corp | 学習型分類装置及び学習型分類方法 |
JP2007280325A (ja) * | 2006-04-12 | 2007-10-25 | Omron Corp | 動画表示装置 |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11836906B2 (en) | 2017-03-27 | 2023-12-05 | Hitachi High-Tech Corporation | Image processing system and computer program for performing image processing |
JP2020140518A (ja) * | 2019-02-28 | 2020-09-03 | キヤノン株式会社 | 情報処理装置、情報処理装置の制御方法及びプログラム |
JP7233251B2 (ja) | 2019-02-28 | 2023-03-06 | キヤノン株式会社 | 情報処理装置、情報処理装置の制御方法及びプログラム |
JP7496008B2 (ja) | 2019-02-28 | 2024-06-05 | キヤノン株式会社 | 情報処理装置、情報処理装置の制御方法及びプログラム |
JP2021167990A (ja) * | 2020-04-08 | 2021-10-21 | Arithmer株式会社 | レイアウト解析装置、その解析プログラムおよびその解析方法 |
JP2023001367A (ja) * | 2020-05-29 | 2023-01-04 | 株式会社日立ハイテク | 画像処理システムおよび画像処理方法 |
JP7438311B2 (ja) | 2020-05-29 | 2024-02-26 | 株式会社日立ハイテク | 画像処理システムおよび画像処理方法 |
Also Published As
Publication number | Publication date |
---|---|
US20200356814A1 (en) | 2020-11-12 |
JP7279738B2 (ja) | 2023-05-23 |
JPWO2016147652A1 (ja) | 2018-02-01 |
US20240160342A1 (en) | 2024-05-16 |
US20190272448A1 (en) | 2019-09-05 |
US10755143B2 (en) | 2020-08-25 |
US20220284239A1 (en) | 2022-09-08 |
JP7491435B2 (ja) | 2024-05-28 |
US11914851B2 (en) | 2024-02-27 |
JP6904249B2 (ja) | 2021-07-14 |
US10339422B2 (en) | 2019-07-02 |
JP2021168155A (ja) | 2021-10-21 |
JP2023100850A (ja) | 2023-07-19 |
US11373061B2 (en) | 2022-06-28 |
US20180107900A1 (en) | 2018-04-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7279738B2 (ja) | オブジェクト検出装置、オブジェクト検出方法およびプログラム | |
JP7460001B2 (ja) | 情報処理システム、情報処理方法、およびプログラム | |
US8542199B2 (en) | Image processing apparatus, image processing method, and program | |
US7415676B2 (en) | Visual field changing method | |
US9965156B2 (en) | Push-pull type gestures | |
US20160350951A1 (en) | Chart drawing method | |
US6738081B2 (en) | Display for a graphical user interface | |
CN105808129A (zh) | 一种利用手势快速开启软件功能的方法和装置 | |
US20140125588A1 (en) | Electronic device and operation method thereof | |
JP2022019754A (ja) | 情報処理装置、情報処理方法及び情報処理用プログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16764483 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2017506093 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15558652 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 16764483 Country of ref document: EP Kind code of ref document: A1 |