JP7440975B2 - Imaging device and identification method - Google Patents

Description

The present invention relates to an imaging device and an identification method.

In recent years, with the development of technology related to artificial intelligence (AI), automation of manufacturing processes using artificial intelligence is rapidly progressing at manufacturing sites for various products. Particularly in the field of image recognition (image identification), the capabilities of artificial intelligence have exceeded those of humans. Therefore, artificial intelligence is used to automate visual inspections to distinguish between good and defective products (see, for example, Patent Document 1) and identification processing to classify (identify) products by type (see, for example, Patent Document 2). It is being

The technology disclosed in Patent Document 1 performs an external appearance inspection of a product based on training data (image data) for non-defective products and defective products. On the other hand, the technology disclosed in Patent Document 2 performs product classification based on training data for each product type.

Unexamined Japanese Patent Publication No. 2018-5640 Japanese Patent Application Publication No. 05-20426

In these technologies, the amount and quality of training data used to train the artificial intelligence greatly influences image identification accuracy. In other words, in order to perform visual inspection and identification processing with high precision using artificial intelligence, a large amount of high-quality training data is required.

In order to collect a large amount of high-quality training data, it is necessary to prepare an imaging environment for each classification (class) of products, such as good products, defective products, and types, and to prepare imaging samples. These preparations require a huge amount of time and cost. In addition, these preparations are often carried out by companies that introduce equipment using artificial intelligence or consultants who introduce the equipment (introducing companies, etc.). Become.

Furthermore, the absolute number of training data for special types such as defective products and rare species is small, making it difficult to prepare a large amount of training data (imaging samples).

On the other hand, in order for artificial intelligence to learn from training data, the region occupied by the imaging target in the training data needs to be specified with a certain degree of accuracy. Usually, the identification of the same area is performed by human visual observation for each training data. Therefore, identifying the same area requires a huge amount of time and cost.

The present invention was made to solve the problems of the prior art as described above, and aims to reduce the time and cost for collecting a large amount of high-quality teaching data.

The imaging device according to the present invention includes a type input section into which a plurality of types of object information are input, an image sensing section that images the object and outputs a captured image, and a memory in which the type information and the captured image are stored in association with each other. an identification unit that identifies type information corresponding to the object to be identified using identification information for each type information generated based on the captured image stored in the storage unit in association with the type information; and a specific processing unit that executes specific processing on the identified object when the type information of the identified object is specific type information, and the specific processing unit executes specific processing on the identified object when the identified object is specific type information. Make it visible that it has been identified.

According to the present invention, it is possible to reduce the time and cost for collecting a large amount of high-quality teaching data.

FIG. 1 is a functional block diagram showing an embodiment of an imaging device according to the present invention. 3 is a flowchart showing a learning mode included in the imaging method according to the present invention. FIG. 2 is a schematic diagram showing an example of information stored in a storage unit included in the imaging device of FIG. 1. FIG. 3 is a flowchart showing an identification mode included in the imaging method of FIG. 2. FIG. FIG. 2 is a schematic diagram showing an example of the imaging device of FIG. 1. FIG. FIG. 6 is a partial schematic diagram of the imaging device of FIG. 5 in the direction of arrow A. FIG. 6 is a schematic diagram illustrating an example of determination of position information of a subject by the imaging device of FIG. 5. FIG. FIG. 2 is a schematic diagram showing another embodiment of the imaging device shown in FIG. 1; FIG. 2 is a schematic diagram showing still another embodiment of the imaging device shown in FIG. 1. FIG. FIG. 3 is a schematic diagram showing a state in which an object to be identified is irradiated with laser light. 10 is a schematic diagram showing a modification of the imaging device shown in FIG. 9. FIG. FIG. 2 is a schematic diagram showing still another embodiment of the imaging device shown in FIG. 1. FIG. FIG. 2 is a schematic diagram showing still another embodiment of the imaging device shown in FIG. 1. FIG. FIG. 2 is a schematic diagram showing still another embodiment of the imaging device shown in FIG. 1. FIG. FIG. 2 is a schematic diagram showing still another embodiment of the imaging device shown in FIG. 1. FIG.

DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of an imaging device (hereinafter referred to as “this device”) and an imaging method (hereinafter referred to as “this method”) according to the present invention will be described with reference to the drawings. In each figure, the same members and elements are designated by the same reference numerals, and redundant explanations will be omitted.

The present invention captures a plurality of captured images of a single subject with different orientations, postures, shapes, etc. by capturing images of the subject while changing the subject's orientation, posture, shape, etc. The area is identified and acquired as training data for artificial intelligence.

The "subject" is an object to be imaged by this device, and is, for example, a product or product whose good or defective products can be distinguished in an external inspection, or a product (product) which is classified (identified) by type in an identification process. The types will be described later.

The "subject area" is an area surrounding the subject in the captured image.

The "captured image" is, for example, a still image captured by a digital camera or the like. The captured image includes a subject image, which is an image of the subject, and a surrounding image, which is an image of the surroundings of the subject (for example, a mounting surface such as a table or a belt conveyor on which the subject is placed). include.

●Imaging device●
●Configuration of imaging device FIG. 1 is a functional block diagram showing an embodiment of this device.

This device 1 images the same subject while changing the orientation (posture), shape, etc. of the subject. This device 1 includes a type input section 11, an imaging section 12, a storage section 13, a position determination section 14, a form change section 15, a positional relationship change section 16, a subject information acquisition section 17, and a placement section. 18, the detection section 19, the display section 20, the imaging environment change section 21, the identification information generation section 22, the identification section 23, the first identification processing section 24, the second identification processing section 25, and the identified object. It includes a body detection section 26 and a control section 27.

The type input unit 11 is an input medium, such as a keyboard, a touch panel, or a voice input device, into which the user of the apparatus 1 can directly input the type information of the subject. The type information input to the type input section 11 is one or more types.

Note that the type input section is not limited to an input medium that allows direct input by the user. That is, for example, the type input unit is a communication device that receives type information transmitted from a terminal (external device) located at a location physically distant from the device (for example, in the same room, an adjacent room, a remote location, etc.). It can also be a device. Alternatively, the type input section may be an interface (such as a USB terminal or an SD card slot) that can import information from a storage medium such as a USB or an SD card in which type information is stored in advance.

"Type information" is information that indicates the type (label, attribute, class) for classifying multiple objects, such as the type (name) of the object such as an apple or screw, or the condition of the object such as a good product or a defective product. .

The imaging unit 12 images a subject and outputs a captured image. The imaging unit 12 is, for example, a digital camera capable of capturing two-dimensional still images.

Note that the imaging unit may be a device capable of acquiring three-dimensional still images and coordinate information, such as an infrared ToF (Time-of-Flight) camera or a three-dimensional laser scanner.

The storage unit 13 stores information necessary for the device 1 to execute the method described later. The storage unit 13 includes, for example, a recording device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), a semiconductor memory element such as a RAM (Random Access Memory), or a flash memory.

The position determining unit 14 determines position information and stores the same position information in the storage unit 13 in association with the captured image. The specific configuration and operation of the position determining section 14 will be described later.

“Position information” is information that specifies the position of a subject in a captured image. The position information includes coordinate information that specifies relative coordinates from a reference point within the captured image and coordinate information that specifies the subject area. The "reference point" is, for example, the coordinates of a pixel located at the center or four corners of the pixels forming the captured image, or the coordinates of a pixel indicating a reference marker captured in the captured image.

The form changing unit 15 changes the form of the subject, for example, by applying external force or heat to the subject. The form changing unit 15 is, for example, a robot arm that grips and crushes the subject, a needle or blade that destroys the subject, a heater that applies heat to the subject, a cooling device that cools the subject, or the like. The specific configuration and operation of the form changing unit 15 will be described later.

Note that in the present invention, "change in the form of the subject" includes deformation of the shape of the subject and discoloration of the surface of the subject. That is, for example, the shape changing unit may apply external force or heat to the subject to change the shape of the subject, or may change the color of the surface of the subject.

The positional relationship changing unit 16 changes the positional relationship between the subject and the imaging unit 12. The positional relationship changing unit 16 changes the orientation (posture) of the subject by applying an external force to the subject, such as a robot arm that grips the subject, a vibration device that vibrates the subject, or an air gun that blows air (gas) onto the subject. A device (mechanism) for changing the image capturing section 12, a device (mechanism) for three-dimensionally (planarly) moving the image capturing section 12 formed by a combination of a plurality of slide rails, and an orientation (imaging range) of the image capturing section 12 consisting of a hinge, etc. or a combination thereof. The specific configuration and operation of the positional relationship changing section 16 will be described later.

Note that the positional relationship changing section may function as a form changing section. That is, for example, the positional relationship changing unit may be configured with one robot arm that grasps the subject and changes the orientation (posture) of the subject, and also grasps and crushes the subject.

The subject information acquisition unit 17 acquires information regarding the subject (hereinafter referred to as "subject information"). The subject information acquisition unit 17 includes, for example, sensors such as a load cell, a beam sensor, an ultrasonic sensor, and a depth sensor, a device that receives signals from these sensors and calculates subject information, and a mirror that takes a multifaceted view of the subject. It is. The specific configuration and operation of the subject information acquisition unit 17 will be described later.

"Subject information" is information about the subject that is difficult to obtain directly from normal imaging methods or imaging devices (digital cameras), such as the length, weight, and volume of the subject, and multifaceted image information obtained from mirrors. .

The mounting unit 18 is a device/stand on which a subject to be imaged by the imaging unit 12 is placed. The mounting unit 18 is, for example, a device capable of moving a subject, such as a turntable, a belt conveyor, or a slider table. The placement unit 18 includes a placement surface (not shown) on which the subject is placed. The specific configuration and operation of the mounting section 18 will be described later.

The mounting surface can be imaged as a surrounding image. That is, the surrounding image includes an image of the placement surface (hereinafter referred to as "placement surface image"). Therefore, for example, when the placing section 18 is a belt conveyor, by replacing the belt (placement surface) with a belt of a different color, or by painting the belt in different colors in the length direction of the belt ( By connecting belts of different colors, the mounting surface on which the subject is mounted, that is, the surrounding image, can be easily changed.

The detection unit 19 detects the position of the subject placed on the placement unit 18 and outputs a detection signal. The detection unit 19 is, for example, a beam sensor. The specific configuration and operation of the detection unit 19 will be described later. The detection signal is transmitted to the position determining section 14.

The display unit 20 is arranged around the subject imaged by the imaging unit 12 and displays a display image. The display unit 20 is, for example, an organic EL (Electro Luminescence) display or a liquid crystal display that displays a display image captured together with the subject as a background of the subject, or a projector that projects a display image.

A "display image" is, for example, a wallpaper composed of a single color, or a background (such as a belt conveyor) that is imaged together with a product when visual inspection or identification processing is performed on the actual production line of the subject (product). ), all or part of which is captured by the imaging unit 12 as a surrounding image. Images that are candidates for display images (hereinafter referred to as "candidate images") are stored in the storage unit 13 in advance.

The imaging environment changing unit 21 changes the imaging environment when the imaging unit 12 images a subject. The imaging environment changing unit 21 is, for example, a lighting device that illuminates a subject, a filter attached to a lens provided in the imaging unit 12 (covering the lens), or the like.

The "imaging environment" includes, for example, the brightness and color of the lighting device that illuminates the subject, the brightness and color of the light that enters the lens of the imaging section 12, the color of the mounting surface of the mounting section 18, and the color displayed on the display section 20. This includes not only environments (elements) that affect the subject image, such as displayed images, but also environments (elements) that affect surrounding images.

Note that the imaging environment changing unit may be a device that changes the displayed image.

The identification information generation unit 22 generates identification information for each type of information based on the captured image stored in the storage unit 13 in association with the type information. The specific configuration and operation of the identification information generation section 22 will be described later.

"Identification information" is information that can be obtained from a captured image, such as the shape, pattern, and color of a subject, and is information that characterizes the type. The identification information includes information generated by the identification information generation section and information generated by an external device different from the present device.

The identification unit 23 identifies type information corresponding to the object to be identified based on the identification information. The specific configuration and operation of the identification unit 23 will be described later.

The “object to be identified” is, for example, a product that is identified (classified) by type when the device 1 operates in an identification mode described later.

The first identification processing unit 24 executes identification processing on the object to be identified when the type information of the object to be identified identified by the identification unit 23 is specific type information. The first specific processing section is a specific processing section in the present invention. The specific configuration and operation of the first specific processing section 24 will be described later.

"Specific type information" is information indicating a specific type (specific type) of a certain product, such as a defective product, for example, when the device 1 operates in an identification mode described later. That is, the type information includes specific type information. "Identification processing" is, for example, processing to make an identified object identified as a specific type in this device 1 visible with laser light using a laser marker, laser pointer, projector, etc., or processing to make the same identified object visible in this device 1. This includes processing to remove from 1.

The second identification processing unit 25 performs the second identification processing on the object to be identified, on which the first identification processing unit 24 has performed the identification processing.

The "second identification process" is, for example, a process of stopping the operation of the device 1, or changing the manner in which an object identified as a specific type is made visible (for example, the manner in which laser light is irradiated, which will be described later). processing, and processing to emit (notify) a sound such as an alarm.

The identified object detection unit 26 detects the identified object for which the identification process has been performed. The identified object detection unit 26 is, for example, a beam sensor. The specific configuration and operation of the identified object detection section 26 will be described later.

The control unit 27 controls the overall operation of the device 1 . The control unit 27 includes, for example, a processor such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field It is composed of integrated circuits such as Programmable Gate Array (Programmable Gate Array). The operation of the control unit 27 will be described later.

Note that the control unit may function as a subject information acquisition unit, a position determination unit, an imaging environment change unit, an identification information generation unit, an identification unit, a first identification processing unit, and a second identification processing unit. That is, for example, the control section may include a subject information acquisition section, a position determination section, an imaging environment change section, an identification information generation section, an identification section, a first identification processing section, and a second identification processing section. In this case, the subject information acquisition section, the position determination section, the imaging environment change section, the identification information generation section, the identification section, the first identification processing section, and the second identification processing section are the same processor as the control section. It may be constituted by an integrated circuit, or it may be constituted by a separate processor or an integrated circuit.

●Operation of the imaging device Next, the operation of the present device 1 will be explained using an example in which the placing section 18 is provided with a conveying function such as a belt conveyor or a slider table.

The device 1 operates based on two operation modes: a learning mode and an identification mode. The operation mode can be switched, for example, by an operation of the device 1 by the user of the device 1 .

●Learning mode In "Learning mode", the device 1 captures multiple images of the subject with different orientations, postures, shapes, etc., by capturing images of the subject while changing the orientation, posture, shape, etc. This is an operation mode in which a subject area is identified from each captured image and acquired as training data for artificial intelligence.

FIG. 2 is a flowchart of the learning mode.
In this figure, for convenience of illustration, the box indicating "judgment" is replaced with a hexagonal box indicating "preparation" (the same applies hereinafter). That is, for example, the process (S105) described later means "judgment" on the flowchart.

First, a subject is placed on the placement surface (not shown) of the placement unit 18 by the user of the device 1 (S101).

Next, the type input unit 11 is operated by the user of this device 1, so that the type information of the subject to be imaged by this device 1 is input to this device 1 (S102). The input type information is stored in the storage unit 13.

Next, the mounting unit 18 transports the subject to the imaging range of the imaging unit 12 (S103). At this time, the subject information acquisition unit 17 acquires (calculates) information regarding the length of the subject (subject information) based on, for example, the detection signal of the detection unit 19 and the conveyance speed and conveyance amount of the mounting unit 18. ) (S104). The "conveyance amount" is calculated, for example, based on pulses from an encoder (such as a rotary encoder) attached to the belt conveyor or a servo motor that drives the belt conveyor. The acquired subject information is stored in the storage unit 13.

Note that the subject information acquired by the subject information acquisition unit is not limited to information regarding the length of the subject. That is, for example, the subject information acquisition unit may acquire information that can be detected using a sensor, such as the weight, height, volume, and temperature distribution of the subject, as the subject information.

Next, the control unit 27 determines whether the subject has entered the imaging range of the imaging unit 12, for example, based on the detection signal of the detection unit 19 (S105).

When the subject enters the imaging range of the imaging unit 12 (“Yes” in S105), the imaging unit 12 images the subject (S106). At this time, the position determination unit 14 identifies the subject area in the captured image based on the detection signal of the detection unit 19, for example, and determines its coordinate information (position information of the subject) (S107).

On the other hand, when the subject is not within the imaging range of the imaging unit 12 (“No” in S105), the imaging unit 12 waits until the subject enters the imaging range.

Note that the position determination unit in the present invention may, for example, determine the coordinates of two diagonally located points among the coordinates of the four corners of the rectangular subject area as the subject position information.

Further, a mirror capable of displaying a side image of the subject toward the imaging section may be arranged in the imaging range of the imaging section. In this case, the mirror functions as a subject information acquisition unit in the present invention. By arranging the mirror in this manner, the captured image includes an image of the side surface of the subject, which is subject information. That is, this device can acquire subject information associated with a captured image by specifying the mirror area within the captured image in advance.

Next, the position determining unit 14 associates the position information with the captured image and stores it in the storage unit 13 (S108). At this time, the device 1 also stores, for example, the subject information acquired in the process (S104) in the storage unit 13 in association with the captured image.

Next, the mounting unit 18 transports the subject to the subject position change point (S109).

The “subject position change point” is a position where the positional relationship changing unit 16 changes the orientation or posture of the subject. The subject position change point is set, for example, on a device that applies vibration to the subject (vibration device) or within the movable range of a robot arm that grips the subject.

Next, the control unit 27 determines whether the number of times the image capturing unit 12 has imaged the subject has reached the number specified by the number of image capturing information (S110).

"Image capture frequency information" is information that specifies the number of times the device 1 captures an image of a subject by changing the orientation or posture of the subject (image capture frequency). The information on the number of times of imaging is stored in advance in the storage unit 13, for example, by the user of the apparatus 1 operating the input means (for example, the type input unit 11).

When the number of times the image capturing unit 12 images the subject has not reached the number specified by the image capturing frequency information (“No” in S110), the positional relationship changing unit 16 changes the orientation and posture of the subject (S111). .

Note that in the process (S111), the positional relationship changing unit may change the position and orientation of the camera, or may change the orientation and posture of the subject and the position and orientation of the camera at the same time.

Next, the mounting unit 18 transports (returns) the subject to the imaging range of the imaging unit 12 (S112). The process of the device 1 returns to the process (S105). That is, the imaging unit 12 images the subject after the positional relationship changing unit 16 changes the positional relationship between the subject and the imaging unit 12 based on the imaging frequency information.

Note that the present device may acquire subject information each time the subject is returned to the imaging range of the imaging unit.

On the other hand, when the number of times the imaging unit 12 has imaged the subject has reached the number specified by the imaging frequency information (“Yes” in S110), the control unit 27 issues an instruction to change the imaging environment (hereinafter referred to as “environment change The presence or absence of "instruction" is confirmed (S113).

The "environment change instruction" is information that causes the imaging environment changing unit 21 to change the imaging environment of the imaging unit 12 (for example, changing the brightness and color of lighting, changing the background (mounting surface) of the subject, etc.) . The environment change instruction is, for example, set in advance by the user of the device 1 and stored in the storage unit 13.

When there is an environment change instruction (“Yes” in S113), the imaging environment changing unit 21 changes the imaging environment (S114).

Next, the control unit 27 resets the number of times the imaging unit 12 images the subject (S115). The process of the device 1 returns to the process (S112). When the imaging environment is changed, the surrounding image of the captured image captured after the imaging environment changing unit 21 changes the imaging environment is the surrounding image of the captured image captured before the imaging environment changing unit 21 changes the imaging environment. Different from the image.

On the other hand, when there is no environment change instruction ("No" in S113), the control unit 27 checks whether there is an instruction to change the form of the subject (hereinafter referred to as "form change instruction") (S116).

The "shape change instruction" is information that causes the shape change unit 15 to perform processing such as changing the aspect ratio of the subject by pressing, cutting/piercing the subject, deforming/discoloring the subject by heat, etc. The format change instruction is set based on, for example, information on the number of times of imaging. For example, the format change instruction is set in advance by the user of the device 1 and stored in the storage unit 13.

When there is a form change instruction ("Yes" in S116), the form changing unit 15 changes the form of the subject based on the form change instruction (S117).

Next, the control unit 27 resets the number of times the imaging unit 12 images the subject (S118). At this time, type information indicating that the form of the subject has been changed is input by the control unit 27 or the user of the apparatus 1 as necessary (S119). For example, if the type information is "good watermelon" and "defective watermelon" and a good watermelon is imaged with the type information "good watermelon" and then transformed into a defective watermelon, then the type information " "Watermelon defective product" is input. The process of the device 1 returns to the process (S112). That is, when there is a form change instruction, the imaging unit 12 images the subject after the form changing unit 15 changes the form of the subject.

On the other hand, when there is no form change instruction ("No" in S116), the control unit 27 inputs each captured image captured in the process (S106) with the corresponding type information (in the process (S102) or the process (S119)). (S120).

Next, the identification information generation unit 22 reads the type information and the captured image stored in association with the type information from the storage unit 13, and generates identification information for each type of information based on the same type information and the captured image. is generated (S121). The identification information is stored in the storage unit 13 in association with the type information (S122).

Note that each process of the learning mode executed by this device is not limited to this embodiment. That is, for example, the present device does not need to determine position information or acquire subject information in the learning mode. Even in this case, the present device can acquire a plurality of captured images of one subject. Further, for example, the present device does not need to reset the imaging number information in the learning mode. In this case, the imaging number information may include, for example, an individual number of imaging times set for each change in imaging environment (environment change instruction) and each change in form (form change instruction).

Furthermore, the present device does not need to generate identification information for each type of information in the learning mode. In this case, for example, a user of the present device may generate identification information for each type of information using the present device or an external device, and store the generated identification information in the storage section of the present device.

Furthermore, after capturing a captured image, this device generates an image obtained by rotating the captured image at a predetermined angle with type information corresponding to the captured image (type information input from the type input section before capturing the captured image). information) and may be stored in the storage unit.

Furthermore, the input of the type information is not limited to before the captured image is captured (processing S102). That is, for example, the type information may be input after the captured image is captured by changing the number of times specified by the imaging frequency information, the orientation (posture) of the subject (before processing S113), or when all captured images are input to the corresponding type. The information may be input before being associated with the information and stored in the storage unit (processing S120).

Furthermore, in the process (S121), the identification information generation unit may determine whether to use the captured image to generate the identification information based on the identification information stored in the storage unit. That is, for example, when the accuracy (described later) of the identification information generated based on the captured image is sufficiently high (for example, accuracy "90" or higher) and there is no need to update the identification information, the identification information generation unit generates the captured image. is not used to generate identification information (does not generate identification information). On the other hand, when the accuracy of the identification information is low (for example, the accuracy is less than "90"), the identification information generation unit uses the captured image to generate the identification information (generates the identification information).

Furthermore, when a mark is attached to the subject, the position determination unit in the present invention determines position information that specifies the position of the mark attached to the subject in the captured image, and associates the same position information with the captured image. It may be stored in the storage unit. In this case, the "mark" is, for example, paint painted on the subject to indicate a specific location on the subject with a special color or shape. Information indicating the landmark is stored in advance in the storage unit as information that can be recognized by the position determining unit. In this way, by storing the position information of the landmark in association with the captured image, the location to be learned by the artificial intelligence (program) becomes clear. As a result, for example, by attaching marks to surround a defect that a craftsman identifies based on experience or intuition (a defect determined as tacit knowledge), the present device can easily identify the location of the defect.

Furthermore, the "mark" may be made of paint that is recognizable only under special lighting, such as an ultraviolet marker. In this case, the device may be equipped with an imaging device that can recognize the paint, or it may be equipped with special lighting that makes the paint visible. When imaging a subject by switching imaging devices or lighting, a shift in the subject area within the captured image (shift in position information: position shift) may occur depending on the conveyance speed of the mounting section. Therefore, this device may temporarily stop the operation of the mounting unit when switching the imaging equipment or lighting, and may move the mounting unit backwards depending on the amount of positional deviation to bring the subject into the imaging range. Alternatively, correction information may be calculated by taking into account the amount of positional deviation from the conveyance speed.

FIG. 3 is a schematic diagram showing an example of information stored in the storage unit 13.
The figure shows that "imaging number", "captured image", "position information", and "subject information" are associated and stored in the storage unit 13 for each "type information". The "imaging number" is a number indicating the number of times (in order) the imaging unit 12 images the subject based on the imaging frequency information. In the same figure, for example, the type information "Watermelon in good condition" is associated with the image capture number "01", the captured image "p1", the position information "x1y1z1", and the subject information "A1" and stored in the storage unit 13. indicates that it is stored in The figure also shows that the type information "Watermelon in good condition" is associated with the imaging numbers "01", "02", . . . "nn", and the captured images, position information, and subject information corresponding to each imaging number. This indicates that the information is stored in the storage unit 13.

By referring to the type information, the device 1 can read from the storage unit 13 the captured image stored in association with the type information, and the position information and subject information associated with the captured image. . Further, for example, by referring to the subject information, the device 1 can read out type information stored in association with the subject information.

●Identification mode "Identification mode" uses identification information generated by this device 1 in learning mode, or identification information generated by the user of this device 1 from captured images acquired by this device 1 in learning mode. This is an operation mode for identifying products. The identification mode is an example of an identification method according to the present invention.

FIG. 4 is a flowchart of the identification mode.

First, identification information corresponding to an object to be identified is read out from the storage unit 13 by a user of the device 1 or the like (S201).

Next, the object to be identified is placed on the placing section 18 by, for example, the user of the device 1 (S202).

Next, the mounting unit 18 transports the object to be identified to the imaging range of the imaging unit 12 (S203). At this time, the subject information acquisition unit 17 acquires information regarding the length of the object to be identified (identified object information) based on, for example, the detection signal of the detection unit 19 and the conveyance speed of the mounting unit 18 ( calculation) (S204). The acquired object information is stored in the storage unit 13.

"Identified object information" includes, for example, the length, color, shape, weight, volume, and multifaceted image information obtained from mirrors of the object to be identified, directly from normal imaging methods and imaging devices (digital cameras). This is information about the object that is difficult to obtain.

Next, the control unit 27 determines whether the object to be identified has entered the imaging range of the imaging unit 12, for example, based on the detection signal of the detection unit 19 (S205).

When the object to be identified enters the imaging range of the imaging unit 12 (“Yes” in S205), the imaging unit 12 images the object to be identified (S206). At this time, the position determination unit 14 determines the position information of the object to be identified within the captured image, for example, based on the detection signal of the detection unit 19 (S207).

On the other hand, when the object to be identified does not enter the imaging range of the imaging unit 12 (“No” in S205), the imaging unit 12 waits until the object to be identified enters the imaging range.

Next, the position determining unit 14 associates the position information with the captured image and stores it in the storage unit 13 (S208). At this time, the device 1 also stores, for example, the subject information acquired in the process (S204) in the storage unit 13 in association with the captured image.

Next, the identification unit 23 uses the captured image of the object to be identified, the position information stored in association with the captured image, and the identification information, which are stored in the storage unit 13, to identify the object to be identified. The corresponding type information is identified (specified) (S209). That is, for example, when the identification information is information regarding pixel values, the identification unit 23 calculates the pixel value of the range in which the object to be identified is imaged in the captured image, based on the captured image of the subject and the position information. Then, the calculated pixel value is compared with the identification information for each type of information. The identification unit 23 identifies the type information of the identification information corresponding to the calculated pixel value as the type information of the object to be identified. The identification information corresponding to the object to be identified is stored in the storage unit 13 in association with the captured image of the object to be identified.

Next, the first identification processing unit 24 determines whether the type information of the object to be identified identified by the identification unit 23 is specific type information (S210).

When the type information of the object to be identified is specific type information ("Yes" in S210), the first identification processing unit 24 performs a specific process on the object to be identified (for example, irradiating a defective product with a laser beam). process) is executed (S211).

Next, the second identification processing unit 25 checks whether the identification object detection unit 26 has detected the identification object for which the identification process has been performed (S212).

When the identified object detection unit 26 detects the identified object for which the identification process has been performed (“Yes” in S212), the second identification processing unit 25 performs the second identification process (for example, removing defective products or applying laser light). (such as changing the irradiation mode) (S213). The "laser light irradiation mode" is the mode in which the object to be identified is irradiated with the laser light, such as the color of the laser light and the irradiation interval of the laser light such as lighting or blinking.

On the other hand, when the type information of the identified object is not specific type information ("No" in S210), or when the identified object detection unit 26 does not detect the identified object for which the identification process has been performed ("No" in S212) ), the device 1 ends the identification mode.

Note that each process of the identification mode executed by this device is not limited to this embodiment. That is, for example, the present device does not need to execute the specifying process or the second specifying process in the identification mode, or does not need to acquire subject information.

The identification unit may also read subject information stored in association with the captured image from the storage unit, compare the subject information with the object information, and identify type information corresponding to the object. Alternatively, the type information corresponding to the object to be identified may be used as reference information for identification.

Furthermore, when the type information of the object to be identified is specific type information, the first identification processing unit

●Example (1)●
Next, an embodiment of the imaging device according to the present invention will be described, focusing on the differences from the previously described embodiment. The imaging device in this example differs from the embodiment described above in that it does not include the form changing section, the display section, the first identification processing section, the second identification processing section, and the identification object detection section. It differs from the imaging device in its form.

●Configuration of Example (1) FIG. 5 is a schematic diagram showing an example of the present device.

This device 100 includes a housing 101, a touch panel 111, a camera 112, a vibration device 116, a weight scale 117a, a pair of mirrors 117b1 and 117b2 (see FIG. 6), a belt conveyor 118, and four parts. 1 sensor 119a1, 119a2, 119a3, 119a4, eight second sensors 119b1, 119b2, 119b3, 119b4, 119b5, 119b6, 119b7, 119b8 (see FIG. 6), lighting 121, and control device 127. It will be done.

The housing 101 includes a camera 112, a vibration device 116, a weight scale 117a, a pair of mirrors 117b1 and 117b2, a belt conveyor 118, first sensors 119a1-119a4, second sensors 119b1-119b8, and lighting 121. and a control device 127. The housing 101 is, for example, an aluminum frame.

The touch panel 111 is an input device through which the user of the apparatus 100 inputs object type information. The touch panel 111 is an example of a type input section in the present invention. Touch panel 111 is attached to the outer surface of housing 101.

The camera 112 is arranged above the center of the belt conveyor 118 and images the subject being conveyed by the belt conveyor 118. Camera 112 is an example of an imaging unit in the present invention. Camera 112 is, for example, a line scan camera.

The vibration device 116 is arranged on the rear side of the conveyor belt 118 in the conveyance direction (hereinafter referred to as the "conveyance direction"), and applies vibration to the subject to change the orientation (posture) of the subject, and also changes the orientation of the subject. Later, the subject is returned to the belt conveyor 118 by tilting toward the belt conveyor 118. That is, the vibration device 116 is an example of a positional relationship changing unit in the present invention.

The weight scale 117a is disposed below the belt conveyor 118, and measures the weight of the belt conveyor 118 and the subject placed on the placement surface 118a of the belt conveyor 118. The weight scale 117a is an example of a subject information acquisition unit in the present invention.

FIG. 6 is a partial schematic diagram of the present device 100 in FIG. 5 in the direction of arrow A.
In the figure, the imaging range of the camera 112 is shown by a dashed-dotted line, and the light fluxes of the first sensors 119a1-119a3 and the second sensors 119b1-119b8 are shown by dashed-double lines.

Returning to FIGS. 5 and 6.
The mirror 117b1 is disposed to the right of the center of the belt conveyor 118 (above the paper surface in FIG. 6), and displays the right side of the subject being conveyed by the belt conveyor 118 toward the camera 112. The mirror 117b2 is arranged to the left of the center of the belt conveyor 118 (at the bottom of the paper in FIG. 6), and displays the left side of the subject being conveyed by the belt conveyor 118 toward the camera 112. The mirrors 117b1 and 117b2 are examples of subject information acquisition units in the present invention.

The belt conveyor 118 changes the position of the subject relative to the camera 112 by conveying the subject placed on the belt conveyor 118. The belt conveyor 118 is an example of a placement section and a positional relationship changing section in the present invention. The belt conveyor 118 includes a placement surface 118a on which a subject is placed. The belt conveyor 118 can transport the subject in the forward direction (to the left in the paper of FIG. 6) and in the reverse direction (to the right in the paper of FIG. 6). The mounting surface 118a of the belt conveyor 118 is made of, for example, monochrome synthetic resin.

The first sensors 119a1 to 119a4 detect the presence or absence of a subject being transported by the belt conveyor 118 in the transport direction (left-right direction in the paper of FIG. 6). The first sensors 119a1-119a4 are beam sensors composed of a pair of light projecting section and light receiving section. The first sensors 119a1-119a4 are examples of detection units in the present invention. The first sensors 119a1-119a4 are arranged at the carry-in section of the belt conveyor 118, in front and behind the mirrors 117b1 and 117b2 in the conveyance direction, and at the carry-out section of the belt conveyor 118. The direction of the light beams of the first sensors 119a1-119a4 is parallel to the mounting surface 118a and perpendicular to the conveying direction (crossing the belt conveyor 118).

The second sensors 119b1 to 119b8 are narrow beam sensors that include a pair of light projecting section and light receiving section. The second sensors 119b1-119b8 are examples of detection units in the present invention. The second sensors 119b1-119b8 are attached to, for example, a lifting mechanism composed of a rail and an actuator, and are movable in the vertical direction above the belt conveyor 118. The direction of the light beams of the second sensors 119b1-119b8 is along the transport direction.

Note that the numbers of the first sensors and the second sensors are not limited to those in this embodiment. That is, for example, the number of first sensors may be "1" or "2", and the number of second sensors may exceed "10".

The illumination 121 illuminates the subject conveyed within the imaging range of the camera 112. The illumination 121 is an example of an imaging environment changing unit in the present invention.

The control device 127 controls the overall operation of the present device 100, stores information necessary for the operation of this method, determines position information of a subject, obtains subject information, and identifies a subject to be identified. That is, the control device 127 is an example of a storage section, a position determination section, a subject information acquisition section, an identification section, and a control section in the present invention. The control device 127 is realized by, for example, a personal computer.

Here, among the members included in the device 100, the members that are driven to generate vibrations (the vibrating device 116 and the belt conveyor 118) are the members that are easily affected by the same vibrations (the housing 101, the camera 112, the mirror 117b1, 117b2, first sensors 119a1-119a4, second sensors 119b1-119b8, lighting 121, control device 127, etc.). Therefore, vibrations from the vibration device 116 and the belt conveyor 118 are not transmitted to the camera 112, mirrors 117b1, 117b2, first sensors 119a1-119a4, second sensors 119b1-119b8, lighting 121, control device 127, and the like.

Note that the member that is driven to generate vibration may be physically separated from the member that is easily affected by the same vibration, or may be provided with a vibration isolator made of an elastic material such as rubber, They may be connected and arranged with a damping material or the like interposed therebetween.

The device may also include a mechanism that allows the camera to move in a three-dimensional trajectory, such as a mechanism that combines multiple slide rails, or a mechanism that can move the camera along a three-dimensional trajectory, such as a hinge. A changeable mechanism may also be provided. In this case, the mechanism functions as a positional relationship changing unit.

Furthermore, the present device may include a plurality of cameras (imaging section in the present invention) and lights (imaging environment changing section in the present invention).

●Operation of Example (1) Next, the operation of the apparatus 100 in Example (1) will be described with reference to FIGS. 2, 5, and 6.

This device 100 measures the weight and length of the object placed on the placement surface 118a of the belt conveyor 118 (S104), images the object while conveying it (S106), and positions the object in the captured image. The information is determined (S107), the orientation (posture) of the subject is changed by the vibration device 116 (S111), and the subject whose orientation has been changed is imaged (S106). The present device 100 can acquire a plurality of captured images for each type of information by repeating changing the orientation of the subject and capturing the image.

The apparatus 100 measures the length of the subject conveyed on the belt conveyor 118, for example, based on the detection signal of the first sensor 119a1 and the conveyance speed of the belt conveyor 118. That is, for example, the present device 100 uses the control device 127 to measure the time during which the first sensor 119a1 detects the subject and the conveyance speed of the belt conveyor 118, and determines whether the same time and the same conveyance speed. Calculate (measure) the length of the subject from At this time, the control device 127 functions as a subject information acquisition unit in the present invention.

When the subject is being transported in the forward direction, the camera 112 starts capturing images of the subject when the first sensor 119a2 detects the subject, and ends capturing images of the subject when the first sensor 119a3 no longer detects the subject. do. On the other hand, when the subject is being transported in the opposite direction, the camera 112 starts capturing images of the subject when the first sensor 119a3 detects the subject, and captures images of the subject when the first sensor 119a2 no longer detects the subject. end. That is, the camera 112 starts and ends imaging the subject based on the detection signals of the first sensors 119a2 and 119a3.

Note that the camera (the imaging unit in the present invention) images the subject while changing the imaging time interval (timing rate) based on, for example, the detection signal of the first sensor 119a1 and the conveyance speed of the belt conveyor. Good too.

As described above, the camera 112 is a line scan camera, and generates a captured image by combining a series of image data (line-shaped image data) acquired from the start to the end of imaging. Therefore, by narrowing the distance between the imaging range of the camera 112 and the first sensors 119a2 and 119a3, the length of the captured image in the transport direction and the length of the subject captured in the same captured image become approximately the same. In other words, the apparatus 100 can determine the position information of the subject in the captured image by specifying the position of the subject in the direction perpendicular to the transport direction in the captured image.

In this embodiment, the control device 127 determines the position information of the subject within the captured image. That is, the control device 127 functions as a position determining section in the present invention. The control device 127 (position determining unit 14) determines the position of the subject in the captured image by specifying the position of the subject in the captured image in the direction orthogonal to the transport direction based on the detection signals of the second sensors 119b1-119b8. Determine location information.

FIG. 7 is a schematic diagram showing an example of determining position information of a subject by the present device 100.
The figure schematically shows an example of a captured image captured by the camera 112. In the same figure, the light fluxes of the second sensors 119b1-119b8 are indicated by two-dot chain lines. This figure shows that the light beams of the second sensors 119b2-119b5 are blocked by the subject.

Here, the imaging range of the camera 112 and the detection range (position of the light flux) of the second sensors 119b1-119b8 are stored in advance in the control device 127 (storage unit 13) in association with each other. Therefore, the control device 127 (position determining unit 14) can determine the position information of the subject based on the position of the light flux blocked by the subject among the light fluxes of the second sensors 119b1-119b8. That is, for example, in the example shown in FIG. 7, the control device 127 (position determining unit 14) is configured by the light beams of the second sensors 119b1 and 119b6 and both ends of the captured image in the transport direction in the captured image. It can be specified that the subject is located within a rectangular area (subject area). That is, the control device 127 (position determining unit 14) determines the coordinates of the subject area within the captured image as the subject position information.

Note that the position determination unit in the present invention calculates the distance between the subject and the first sensor 119a2 based on the time from when the subject passes the first sensor 119a2 until the camera captures the captured image, and locates the subject. The area may be further limited. Further, the position determination unit in the present invention may determine the position information of the subject based on the position of a reference marker captured in the surrounding image, a mirror, or the like.

The figure also shows that the mirrors 117b1 and 117b2 and the side surfaces of the subject reflected in the mirrors 117b1 and 117b2 are captured in the captured image. By arranging the mirrors 117b1 and 117b2 in the imaging range in this way, the device 100 can acquire a side image of the subject as subject information every time the subject is imaged.

Furthermore, the camera (imaging unit in the present invention) may be an area camera. In this case, for example, the present device may determine the position information of the subject within the captured image based on the detection signal of the second sensor arranged in a grid pattern.

Furthermore, the user of the present device may change the orientation and posture of the subject together with or in place of the positional relationship changing unit. Even in this case, the camera can output captured images in which the positional relationship between the subject and the camera differs each time the image is captured. According to this configuration, the user can efficiently acquire unobtained teacher data or teacher data with few acquisition opportunities.

Furthermore, the control device (storage unit) may store a plurality of pieces of identification information and a captured image that is the basis of the identification information. In this case, the control device connects the identification information (or captured image) stored in the control device (storage unit) to the type information input by the user to a display device (e.g., touch panel, or this device). (e.g., a monitor). As a result, the user can change the orientation and shape of the subject based on the identification information (or the captured image) displayed on the display device. That is, for example, by checking the display device, the user can grasp the orientation and shape of a subject that has not been imaged or has few opportunities to be imaged, and can efficiently acquire training data.

Furthermore, the subject type information may be transmitted from an external device connected to the present device via a network. Further, the identification information and the captured image that the present device outputs to the display device may be transmitted from an external device connected to the present device via a network.

●Example (2)●
Next, another embodiment of the imaging device according to the present invention will be described, focusing on the differences from the previously described embodiment. The imaging device in this embodiment differs from the imaging device in the previously described embodiments in the configuration of the belt conveyor and the control device, and in that it includes a display section.

●Configuration of Example (2) FIG. 8 is a schematic diagram showing another example of the present device.

This device 100A includes a touch panel 111, a camera 112, a vibration device 116, a weight scale 117a, a pair of mirrors 117b1 and 117b2 (see FIG. 6), a belt conveyor 118A, and four first sensors 119a1-119a4. , eight second sensors 119b1-119b8 (see FIG. 6), a liquid crystal display 120A, a lighting 121, and a control device 127A.

The belt conveyor 118A conveys the subject placed on the placement surface 118aA of the belt conveyor 118A. Belt conveyor 118A is an example of a mounting section in the present invention. The belt conveyor 118A can transport the subject in the forward direction (to the left in the paper of FIG. 8) and in the reverse direction (to the right in the paper of FIG. 8). The mounting surface 118aA of the belt conveyor 118A is, for example, a metal mesh.

Note that the mounting surface of the belt conveyor may be made of, for example, transparent synthetic resin.

The liquid crystal display 120A is arranged in the imaging range of the camera 112 and below the belt conveyor 118A, and displays a display image toward the camera 112. The liquid crystal display 120A is an example of a display section in the present invention.

The control device 127A controls the overall operation of the present device 100A, stores information necessary for the operation of this method, determines position information of a subject, selects a display image, obtains subject information, and identifies a subject to be identified. Execute. That is, the control device 127A is an example of a storage section, a position determination section, a subject information acquisition section, an imaging environment change section, an identification section, and a control section in the present invention. The control device 127A is realized by, for example, a personal computer.

●Operation of Example (2) Next, the operation of the apparatus 100A in Example (2) will be described with reference to FIGS. 2 and 8.

The device 100A changes the image displayed on the liquid crystal display 120A, which is captured as a surrounding image when the camera 112 captures an image of the subject, based on the imaging environment instruction (S114).

The control device 127A (storage unit 13) stores a plurality of candidate images in advance. The control device 127A selects a display image from among the plurality of candidate images. That is, the control device 127A (imaging environment changing unit 21) functions as an imaging environment changing unit in the present invention that changes the display image displayed on the liquid crystal display 120A. Selection of a display image is performed according to an instruction from a user of the device 100A or an instruction (environment change instruction) from a program stored in the control device 127A (storage unit 13). The candidate images include, for example, an image of an actual production line, an image of a product of the same type as the subject, a wallpaper with a color that makes the contrast of the subject clear, and the like.

The liquid crystal display 120A displays a display image toward the camera 112 via the mesh placement surface 118aA of the belt conveyor 118A. The liquid crystal display 120A changes the displayed image based on the selection by the control device 127A (imaging environment changing unit 21). In other words, the liquid crystal display 120A changes the imaging environment (surrounding image) under the control of the control device 127A (imaging environment changing unit 21). As a result, the surrounding image of the captured image captured after the control device 127A (imaging environment changing unit 21) changes the imaging environment is not captured before the control device 127A (imaging environment changing unit 21) changes the imaging environment. This is different from the surrounding image of the captured image.

Note that the liquid crystal display may be placed upright on the side of the belt conveyor instead of below the belt conveyor. In this case, the camera images the subject from the side so that the liquid crystal display is captured in the surrounding image. Moreover, in this case, the mounting surface of the belt conveyor does not need to be mesh or transparent.

Further, the present device may include a rear projection device (a device that projects an image from the back of a screen that looks like a screen) or a super single focus projector (hereinafter referred to as "projector") in place of the liquid crystal display 120A. . In the former case, the rear projection device is arranged below the belt conveyor (on the back side of the mounting surface). The mounting surface of the belt conveyor is made of, for example, a transparent synthetic resin onto which images from a rear projection device can be projected. In the latter case, the projector may be placed below the belt conveyor (on the back side of the mounting surface) or on the side of the belt conveyor. The mounting surface of the belt conveyor is, for example, made of (transparent) synthetic resin on which images from a projector can be projected.

Further, the imaging environment changing unit in the present invention may replace (synthesize) the surrounding image with a replacement image selected from candidate images after capturing the captured image. A "replacement image" is, for example, an image of the background of a product that can be captured by a camera on an actual manufacturing line, images of multiple non-defective products (defective products), or images of multiple non-defective products (defective products). This is an image rotated by an angle. In this case, the image before rotation is the first candidate image in the present invention, and the image after rotation is the second candidate image in the present invention. By rotating the replacement image, this device can, for example, use a small number of images of non-defective products to replace surrounding images with images of the non-defective products from various angles as replacement images, and detect defects that exist among a large number of non-defective products. Captured images of good products (subjects) can be easily combined. When replacing the surrounding image with the replacement image, the imaging environment changing unit stores the position information (coordinates) of the subject image in the captured image before replacement in the storage unit in association with the captured image after replacement with the replacement image.

●Example (3)●
Next, still another embodiment of the imaging device according to the present invention will be described, focusing on the differences from the previously described embodiments (1) and (2). The imaging device in this embodiment differs from the imaging devices in the previously described embodiments (1) and (2) in that it includes a first identification processing section, a second identification processing section, and an identification object detection section.

●Configuration of Example (3) FIG. 9 is a schematic diagram showing yet another example of the present device.

This device 100B includes a housing 101, a touch panel 111, a camera 112, a vibration device 116, a weighing scale 117a, a pair of mirrors 117b1 and 117b2 (see FIG. 6), a belt conveyor 118, and four parts. 1 sensor 119a1-119a4, eight second sensors 119b1-119b8 (see FIG. 6), illumination 121, laser pointer 124B, object detection sensor 126B, and control device 127B. .

When the device 100B is operating in the identification mode, the laser pointer 124B irradiates a laser beam to an object to be identified that has been identified as a specific type, so that it can be visually recognized that the object to be identified has been identified as a specific type. do. The laser pointer 124B is an example of a specific processing unit in the present invention.

The object detection sensor 126B detects the presence or absence of an object irradiated with laser light at a predetermined position on the belt conveyor 118. The object detection sensor 126B is, for example, a beam sensor. The object detection sensor 126B is an example of an object detection section in the present invention.

The "predetermined position" is, for example, a position between the first sensor 119a3 and the first sensor 119a4, where the user of the apparatus 1 can remove the object to be identified that is irradiated with the laser beam.

Note that, for example, the first sensor (corresponding to the first sensor 119a4) may function as an identification object detection sensor.

The control device 127B controls the overall operation of the present device 100B, stores information necessary for the operation of this method, determines position information of a subject, obtains subject information, and identifies a subject to be identified. That is, the control device 127B is an example of a storage section, a position determination section, a subject information acquisition section, an identification section, and a control section in the present invention. The control device 127B is realized by, for example, a personal computer.

●Operation of Example (3) Next, the operation of the apparatus 100B in Example (3) will be described with reference to FIGS. 4 and 9.

The device 100B identifies the type information corresponding to the object to be identified (S209), determines whether the same type information is specific type information (S210), and determines whether the type information of the object to be identified is specific type information. At some point, the object to be identified is irradiated with a laser beam (S211), the presence or absence of the object to be identified that has been irradiated with the laser beam is detected (S212), and the irradiation mode of the laser beam is changed (S213).

The control device 127B (identification unit 23) identifies the type information of the object to be identified based on the captured image of the object to be identified and the identification information, and determines whether the same type information is specific type information. Confirm (S210).

If the type information of the object to be identified is specific type information ("Yes" in S210), the laser pointer 124B irradiates the object to be identified with laser light to highlight the object and make it visible. (S211). At this time, the control device 127B (control unit 27) predicts the movement path of the object to be identified based on the position information of the object to be identified in the captured image and the moving speed of the belt conveyor 118, and obtains a prediction result. is transmitted to the laser pointer 124B. That is, the laser pointer 124B irradiates the object to be identified with laser light so as to track the object being transported by the belt conveyor 118.

FIG. 10 is a schematic diagram showing a state in which an object to be identified is irradiated with laser light.
In the figure, arrows indicate how the object to be identified and the laser beam move. In the figure, lit laser beams are shown as solid-line circles, and blinking laser beams are shown as broken-line circles.

In this way, the laser pointer 124B irradiates the object identified as a specific type with a laser beam based on the prediction result of the control device 127B (control unit 27), so that the object to be identified is identified as the specific type. This allows the user of the device 100B to visually recognize what has been done. Laser light irradiation is an example of specific processing in the present invention.

Note that the control device may control the operation of the laser pointer. In this case, the control device also functions as an imaging environment changing section.

Next, the control device 127B detects the object detected by the object detection sensor 126B based on the detection signal of the object detection sensor 126B, the predicted moving route of the object, and the conveyance speed of the belt conveyor 118. It is confirmed whether the object to be identified is the object to be identified that is irradiated with laser light (S212).

When the object to be identified sensor 126B detects the object to be identified that has been irradiated with the laser light ("Yes" in S212), the laser pointer 124B changes the irradiation mode of the laser light (for example, changes the irradiation mode of the laser light to (changes from flashing to flashing) (S213). In other words, changing the laser beam irradiation mode by the laser pointer 124B is an example of the second specifying process in the present invention. That is, the laser pointer 124B functions as a second specific processing section in the present invention.

Note that the present apparatus may stop the operation of the belt conveyor when the object to be identified sensor detects the object to be identified that has been irradiated with the laser beam. In this case, the belt conveyor functions as the second specific processing section in the present invention. The operation of the belt conveyor by the present device may be stopped immediately or under predetermined conditions (for example, when the object to be identified is conveyed to a position where it can be removed by the user, the object to be identified is detected by the object detection sensor). (when a predetermined period of time has elapsed since).

In addition, for example, in place of the identification object sensor, this device may be equipped with a sensor (area sensor, etc.) that detects an operation of a user of this device to remove an object to be identified that has been irradiated with laser light; Laser light irradiation may be terminated based on a detection result of a sensor (such as a second camera) that detects that the object's hand covers the surface of the object to be irradiated with laser light.

Further, the present device may remove the object to be identified from the belt conveyor when the object to be identified sensor detects the object to be identified that has been irradiated with the laser beam. In this case, the means for removing the identification object from the belt conveyor (for example, a robot arm, a sorting machine placed on the belt conveyor, etc.) functions as the second identification processing section in the present invention.

Furthermore, the laser pointer (the specific processing section in the present invention) may change the laser beam irradiation mode (perform the second specific processing) when a predetermined time has elapsed since the laser pointer was irradiated with the laser beam. In this case, the laser pointer functions as a second specific processing section in the present invention.

Furthermore, when the object detection sensor detects the object to be identified that has been irradiated with the laser beam, or when a predetermined period of time has elapsed since the object has been irradiated with the laser beam, this device notifies the user with a sound. It may also include a notification section. In this case, the notification section functions as a second specific processing section in the present invention.

Furthermore, the present device may perform different specific processing depending on the content of the specific type information. That is, for example, when the type information of the object to be identified is specific type information corresponding to a defective product in shape or color, this device irradiates the object with red laser light and detects that the same type information corresponds to a defective product due to foreign matter. When it is the corresponding specific type information, the object to be identified may be irradiated with blue laser light.

Furthermore, instead of the laser pointer, the present device may include, for example, a projector capable of projecting projection mapping onto the object to be identified. In this case, the projector may change the mapping projection mode depending on the content of the specific type information. That is, for example, when the type information of the object to be identified is specific type information corresponding to a defective product in shape or color, the projector projects a red circle onto the object to be identified, and the projector projects a red circle onto the object to be identified when the type information of the object to be identified corresponds to a defective product in terms of shape or color. When the information is of a specific type, a character such as "!" or a figure that emphasizes a circle may be additionally projected. In this way, by changing the mapping projection mode according to the content of the specific type information, the user of this device can improve the efficiency of the identification work of the object to be identified (such as the removal of defective products). At the same time, the same user can select an object to be identified to be removed with priority.

Furthermore, the present device may include an adjusting means for adjusting the projection position of the projector to the mounting surface of the mounting section. The adjusting means includes, for example, a positioning mark placed on the mounting surface and a moving mechanism that moves the projector. In this case, the user, for example, projects the positioning image from the projector onto the mounting surface and adjusts the position of the projector by operating the moving mechanism so that the mark on the mounting surface and the positioning image overlap. do. The mark on the placement surface may be directly attached to the placement surface, or may be removable from the placement surface. Further, the touch panel may be linked with a projection correction function (for example, keystone correction, zoom, focus, etc.) of the projector, and the present apparatus may be provided with a projection correction function by touch panel operation.

In this way, by making it possible to visually recognize an object that has been identified as a specific type (for example, a defective product), the user of the device 100B can very easily remove the same object from the production line, etc. becomes possible.

Furthermore, in this embodiment, the apparatus may calculate the identification accuracy of the identification unit and classify the object to be identified based on the calculated accuracy. "Accuracy" is a degree indicating the accuracy of the type information identified by the identification unit. The accuracy is calculated by the identification unit using, for example, a known method. In this case, the identification unit is an example of the specific processing unit in the present invention. The classification of objects to be identified is performed, for example, by a robot arm included in the present device. The robot arm is an example of the second specific processing unit in the present invention.

FIG. 11 is a schematic diagram showing a modification of yet another embodiment of the present device.
This figure shows that the objects to be identified that are transported by the belt conveyor 118 are classified by the robot arm 125B1.

The robot arm 125B1 is arranged, for example, at a position where it can classify objects that have passed through the object detection sensor 126B. The robot arm 125B1 classifies the object detected by the object detection sensor 126B based on the accuracy calculated by the control device 127B1 (identifier 23), for example. That is, for example, when the accuracy of the type information "defective product" is "80 or more", the robot arm 125B1 moves the object to be identified to the left in the traveling direction of the belt conveyor 118 (to the upper side of the paper in FIG. 11). On the other hand, when the accuracy is "less than 79", the robot arm 125B1 moves the object to be identified to the right in the traveling direction of the belt conveyor 118 (toward the bottom of the paper in FIG. 11). In other words, the robot arm 125B1 allows the user to visually confirm that the object to be identified has been identified by the specific type information and its accuracy. According to this configuration, by inspecting only the object to be identified located on the right side in the direction of movement of the belt conveyor 118, the user can reliably inspect only the object to be identified that is vaguely identified by the present apparatus. As a result, the user can use this device to identify the object with high precision even if there is a mixture of type information with insufficient identification accuracy.

●Example (4)●
Next, still another embodiment of the imaging device according to the present invention will be described, focusing on the differences from the previously described embodiments (1), (2), and (3). The imaging device in this example differs from the imaging devices in Examples (1), (2), and (3) described above in that it includes a form changing section.

●Configuration of Example (4) FIG. 12 is a schematic diagram showing yet another example of the present device.

This device 100C includes a touch panel 111, a camera 112, a pressing device 115C, a vibration device 116, a weight scale 117a, a pair of mirrors 117b1 and 117b2 (see FIG. 6), a belt conveyor 118, and four 1 sensor 119a1-119a4, eight second sensors 119b1-119b8 (see FIG. 6), illumination 121, and control device 127C.

The pressing device 115C is disposed on the vibrating device 116 and presses the subject carried into the vibrating device 116 to deform the shape of the subject. That is, the pressing device 115C is an example of a shape changing section in the present invention.

In addition, instead of or in addition to the pressing device, this device can be used to change the shape of the object, such as a heating device that applies heat to the object, a cutting device that cuts the object, and a piercing device that makes a hole in the object. Other devices may also be included.

In addition, instead of the pressing device and the vibration device, this device is capable of changing the orientation of the subject and deforming the shape of the subject, such as a robot arm (the function and form of the positional relationship changing section). A device that also has the function of a changing section) may be provided.

The control device 127C controls the overall operation of the present device 100C, stores information necessary for the operation of this method, determines position information of the subject, acquires subject information, and identifies the subject to be identified. That is, the control device 127C is an example of a storage section, a position determination section, a subject information acquisition section, an identification section, and a control section in the present invention. The control device 127C is realized by, for example, a personal computer.

●Operation of Example (4) Next, the operation of the apparatus 100C in Example (4) will be described with reference to FIGS. 2 and 12.

The pressing device 115C presses the object conveyed to the vibration device 116 to deform the shape of the object based on the above-mentioned deformation instruction (S117). Next, the vibrating device 116 returns the deformed object to the belt conveyor 118. The belt conveyor 118 conveys the deformed subject to the imaging range of the camera 112 (S112).

In this manner, the present device 100C is equipped with the pressing device 115C (shape changing unit in the present invention), and thereby uses one subject to capture a plurality of captured images of the same subject, and also captures a plurality of captured images of the same subject. A plurality of captured images of a subject are captured. In other words, the device 100C acquires training data for a plurality of non-defective products and training data for a plurality of defective products from one product.

Note that the user of the apparatus may change the form of the subject together with or in place of the form changing unit. Even in this case, the camera can output captured images of subjects with different shapes. According to this configuration, the user can obtain training data in which the form of the subject is arbitrarily changed.

●Example (5)●
Next, still another embodiment of the imaging device according to the present invention will be described, focusing on the differences from the previously described embodiments (1), (2), (3), and (4). The imaging device in this embodiment is similar to the previously described embodiments (1), (2), and (3) in that it can be attached to an existing belt conveyor (it does not include a vibration device, a weighing scale, and a belt conveyor). , (4).

●Configuration of Example (5) FIG. 13 is a schematic diagram showing yet another example of the present device.

This device 100D includes a touch panel 111, a camera 112, a pair of mirrors 117b1 and 117b2, four first sensors 119a1-119a4, eight second sensors 119b1-119b8 (see FIG. 6), and lighting 121. and a control device 127D.

This device 100D is attached to an existing belt conveyor (belt conveyor placed on an actual production line) X. This device 100D does not include means for changing the shape of the subject (shape changing unit in the present invention) and means for changing the orientation (posture) of the subject (positional relationship changing unit). Therefore, for example, when the user of the device 100D changes the direction of one type of product (product) many times and puts it into the device 100D, the device 100D can capture multiple images corresponding to one type of information. Capture an image. In this way, the present device 100D can collect a plurality of pieces of training data and identify product type information using the existing belt conveyor X by simply being attached to the existing belt conveyor X. do. Even in this case, the camera 112 can output a captured image in which either the positional relationship between the subject and the camera 112 or the form of the subject differs for each image capture.

Note that this device controls the operation of an existing belt conveyor so that, for example, the product (subject) that the user of this device throws into the device returns to the position where the product was thrown after being captured by the camera. It's okay. That is, for example, the present device may rotate the belt conveyor in reverse at the timing when the first sensor 119a4 detects the product, and return the rotation of the belt conveyor at the timing when the first sensor 119a1 detects the product. Alternatively, the operation of the belt conveyor may be temporarily stopped. According to this configuration, the efficiency of the work in which the user of the device changes the orientation of one type of product many times and feeds it into the device is improved.

Further, the present device may include a mechanism for changing the position of the camera, that is, a positional relationship changing unit according to the present invention.

Furthermore, the control device (storage unit) may store a plurality of pieces of identification information and a captured image that is the basis of the identification information. In this case, the control device may output identification information (or a captured image) stored in the control device (storage unit) in association with the type information input by the user to the display device. As a result, the user can change the orientation and shape of the subject based on the identification information (or the captured image) displayed on the display device. That is, for example, by checking the display device, the user can grasp the orientation and shape of a subject that has not been imaged or has few opportunities to be imaged, and can efficiently collect training data.

●Example (6)●
Next, still another embodiment of the imaging device according to the present invention will be described, focusing on the parts that are different from the previously described embodiments (1), (2), (3), (4), and (5). . The imaging device in this example differs from the imaging devices in Examples (1), (2), (3), (4), and (5) described above in that it includes a turntable instead of a belt conveyor. .

●Configuration of Example (6) FIG. 14 is a schematic diagram showing yet another example of the present device.

The device 200 includes a housing 201, a touch panel 211, a camera 212, a camera support member 216, a load cell 217, a turntable 218, a light 221, and a control board 227.

The housing 201 houses a camera 212, a camera support member 216, a load cell 217, a turntable 218, a light 221, and a control board 227.

The touch panel 211 is an input device through which the user of the apparatus 200 inputs object type information. The touch panel 211 is an example of a type input section in the present invention. Touch panel 211 is attached to the outer surface of housing 201.

The camera 212 is attached to the camera support member 216 and images a subject placed on a placement surface 218a of the turntable 218, which will be described later, from above the turntable 218. Camera 212 is an example of an imaging unit in the present invention. Camera 212 is, for example, an area camera.

Camera support member 216 is disposed on the side of turntable 218 to change the position and orientation of camera 212. A portion of the camera support member 216 to which the camera 212 is attached is vertically slidable and swingable. That is, the camera support member 216 is an example of a positional relationship changing unit in the present invention that changes the position and orientation of the camera with respect to the subject.

The load cell 217 is disposed below the turntable 218 and measures the weight of the turntable 218 and the weight of the subject placed on the turntable 218. The load cell 217 is an example of a subject information acquisition unit in the present invention.

The turntable 218 changes the position and orientation of the subject with respect to the camera 212 by rotating. The turntable 218 includes a placement surface 218a on which a subject is placed. The turntable 218 is an example of a mounting section and a positional relationship changing section in the present invention.

Illumination 221 illuminates a subject placed within the imaging range of camera 212. The illumination 221 is an example of an imaging environment changing unit in the present invention.

The control board 227 controls the overall operation of the present apparatus 200, stores information necessary for the operation of this method, determines position information of the subject, and identifies the subject. The control board 227 mounts, for example, a CPU and a flash memory. The control board 227 is an example of a storage section, a position determination section, an identification section, and a control section in the present invention.

Note that this device functions as a positional relationship changing unit or a shape changing unit in the present invention, such as a mechanism that vibrates a turntable or a mechanism that applies external force to a subject placed on a turntable by touching the subject. A mechanism may also be provided.

Further, in this device, a liquid crystal display functioning as a display section in the present invention may be arranged, for example, on the side of the turntable and in the imaging range of the camera.

Further, in this device, a mirror functioning as the subject information acquisition unit in the present invention may be arranged, for example, on the side of the turntable and in the imaging range of the camera.

●Operation of Example (6) Next, the operation of the device 200 in Example (6) will be described with reference to FIGS. 2 and 14.

The device 200 measures the weight of the object placed on the mounting surface 218a of the turntable 218 by the user of the device 200 (S104), and images the object while rotating the turntable 218 (S106). , determines the position information of the subject in the captured image (S107), changes the position or direction of the camera 212 (S111), and repeats imaging of the subject (S106). The device 200 acquires a plurality of captured images for each type of information by repeatedly changing the orientation of the subject and changing the position and orientation of the camera 212 with respect to the subject.

This device 200 uses, for example, a control board 227 to recognize the outer edge of the turntable 218 by image recognition or the like, and obtains position information of the turntable 218 in the captured image and position information of the subject within the turntable 218. and decide. At this time, the control board 227 functions as a position determining section in the present invention.

Note that the present device may include an area sensor that is movable in the space above the turntable, and determine the position information of the subject based on the detection result of the area sensor. In this case, the area sensor functions as a position determination unit in the present invention.

FIG. 15 is a schematic diagram showing yet another embodiment of the present invention.
In the figure, the present device 200A includes a housing 201, a touch panel 211, a camera 212, a camera support member 216, a load cell 217, a turntable 218, an area sensor 219A, a lighting 221, and a control board 227A. .

Area sensor 219A determines position information of the subject placed on turntable 218. Area sensor 219A is an example of a position determining unit in the present invention. Area sensor 219A is arranged above turntable 218. The area sensor 219A is movable in the vertical direction (vertical direction in the paper of FIG. 15). The imaging range of the camera 212 and the detection range (position of the light flux) of the area sensor 219A are stored in advance in the control board 227A (storage unit 13) in association with each other. When the area sensor 219A moves relative to the subject placed on the turntable 218, the subject blocks part of the light detected by the area sensor 219A. The detection result of the area sensor 219A is transmitted to the control board 227A. The control board 227A determines the position information of the subject within the captured image based on the detection result of the area sensor 219A. At this time, the control board 227A functions as a position determining section in the present invention.

Note that the area sensor may be placed on the side of the turntable. In this case, the area sensor is above the turntable and is movable in the horizontal direction (in the horizontal direction on the paper of FIG. 15).

●Summary●
According to the embodiment described above, the present device 1 includes a type input unit 11 into which type information of a subject is input, an imaging unit 12 which captures an image of the subject, and a memory that stores the type information and the captured image in association with each other. 13. Therefore, the device 1 can capture a plurality of captured images for each type of information and store them in association with each other. That is, the present device 1 reduces the time and cost for collecting a large amount of teacher data.

The apparatus 1 also includes at least one of a positional relationship changing unit 16 that changes the positional relationship between the subject and the imaging unit 12, and a form changing unit 15 that changes the shape of the subject. The imaging unit 12 images the subject after the positional relationship changing unit 16 changes the positional relationship between the subject and the imaging unit 12, or after the form changing unit 15 changes the form of the subject. Therefore, the device 1 can change the type information and capture a plurality of captured images for each type of information and store them in association with each other. That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the positional relationship changing unit 16 or the form changing unit 15 changes the positional relationship between the subject and the imaging unit 12 or changes the form of the subject based on the information on the number of times of imaging. Therefore, this device 1 automatically changes the positional relationship between the subject and the imaging unit 12 and images the subject the number of times set in the imaging frequency information, and also automatically changes the form of the subject to capture the subject. Take an image. That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the device 1 includes a subject information acquisition unit 17 that acquires subject information (weight, length, volume, multifaceted image, etc. of the subject). The storage unit 13 stores subject information in association with the captured image. Therefore, the present device 1 can acquire and generate identification information that takes into account not only identification information for each type of information generated based on a captured image but also subject information. As a result, the device 1 can, for example, not only identify the type information from the captured image of the object to be identified, but also identify the type information from the object information (weight, length, etc. of the object to be identified) of the object to be identified. can do. That is, the present device 1 can collect not only a captured image but also subject information associated with the captured image as training data. Therefore, the identification accuracy of the artificial intelligence that learns using this training data is improved. In this way, the present device 1 reduces the time and cost for collecting a large amount of high-quality teaching data.

Furthermore, the device 1 includes a position determination unit 14 that determines positional information of a subject within a captured image and stores the same positional information in association with the captured image of the subject. Therefore, the present device 1 can collect a large amount of high-quality training data in which the position information of the subject in the captured image is determined (specified). That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the device 1 includes a placement section 18 on which a subject is placed, and a detection section 19 that detects the position of the subject placed on the placement section 18 and outputs a detection signal. The position determination unit 14 determines the position information of the subject within the captured image based on the detection signal. Therefore, the present device 1 can easily determine the position information of the subject within the captured image. As a result, the present device 1 reduces the time and cost for collecting a large amount of high-quality teaching data.

Furthermore, the position determination unit 14 determines position information based on surrounding images. The position information includes coordinate information that specifies relative coordinates from a reference point within the captured image. Therefore, the present device 1 can easily determine position information by, for example, capturing a reference point such as a reference marker in the surrounding image.

Furthermore, the position determining unit 14 determines position information that specifies the position of the mark attached to the subject in the captured image, and stores the same position information in the storage unit 13 in association with the captured image. In this way, by storing the position information of the landmark in association with the captured image, the location to be trained by the artificial intelligence becomes clear. As a result, for example, by placing marks surrounding a defect that a craftsman identifies based on experience or intuition (defects identified as tacit knowledge), the device 1 can easily identify the location of the defect. .

Furthermore, the apparatus 1 includes an imaging environment changing unit 21 that changes the imaging environment when the imaging unit 12 images a subject. The surrounding image of the captured image captured after the imaging environment changing unit 21 changes the imaging environment is different from the known image of the captured image captured before the imaging environment changing unit 21 changes the imaging environment. Therefore, this device 1 changes the imaging environment such as the brightness of the lighting, and changes the imaging environment such as the background of the subject (the mounting surface of the mounting section 18 and the display section 20) that is imaged as the surrounding image. By doing so, it is possible to obtain a large amount of training data in which the imaging environment has been changed. That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the surrounding image includes a placement surface image of the placement surface. Therefore, for example, the user of this device 1 can change the color of the mounting surface to a color that provides a clear contrast depending on the subject, or the color of the actual production line, thereby improving the identification accuracy of artificial intelligence. It is possible to collect high-quality teacher data that increases the That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the imaging environment changing unit 21 selects and changes the display image displayed on the display unit 20 arranged around the subject from among the plurality of candidate images. Therefore, the present device 1 can easily collect captured images including various surrounding images just by changing the displayed image. That is, the present device 1 reduces the time and cost for collecting a large amount of high-quality training data.

Furthermore, the imaging environment changing unit 21 replaces the surrounding image with a replacement image selected from a plurality of candidate images. The candidate images include a first candidate image and a second candidate image obtained by rotating the first candidate image by a predetermined angle. The device 1 can generate a plurality of second candidate images from one first candidate image, for example. As a result, this device 1 can, for example, combine an image of an actual production line with a subject image, combine an image of a good product rotated at multiple angles with a subject image of a defective product, and arbitrarily create an image of a defective product. It is possible to combine various captured images, such as by rotating and combining them. Therefore, the present device 1 reduces the time and cost for collecting a large amount of high-quality teaching data.

Furthermore, the present device 1 uses identification information for each type information generated based on the captured image stored in the storage unit 13 in association with the type information to identify the type information corresponding to the object to be identified. A section 23 is provided. In other words, this device 1 not only acquires a large amount of training data, but also uses artificial intelligence (identification unit) trained based on the training data to perform visual inspection and inspection of objects to be identified (products, products, etc.). Identification processing can be performed. Generally, the practicality of visual inspection and identification processing using artificial intelligence cannot be understood until the verification work is performed in a work environment such as an actual production line. Therefore, if the result is that it is not practical, the time and money spent on collecting teacher data and investing in equipment may be wasted. However, the present device 1 can perform switching between collection of teacher data (learning mode) and identification processing using identification information generated based on the teacher data (identification mode). Therefore, this device 1 can reduce the time and cost for collecting a large amount of high-quality teaching data, and can confirm the usefulness of the teaching data collected by this device 1. As a result, the present device 1 temporarily improves the accuracy of identification and enables highly accurate (highly accurate) identification.

Furthermore, the device 1 includes an identification information generation section 22 that generates identification information for each type of information. Therefore, the user of the device 1 can generate identification information using the device 1 without requesting an artificial intelligence vendor or the like to generate the identification information. As a result, this device 1 reduces the time and cost required to collect a large amount of high-quality teaching data, and also reduces the time and cost required to confirm the usefulness of the teaching data collected by this device 1. reduce

Furthermore, in the present device 1, when the type information of the object to be identified identified by the identification section 23 is specific type information (defective product, etc.), the first identification processing section 24 executes a specifying process for the object to be identified. Equipped with. The identification process is, for example, a process that makes it possible to visually recognize that the object to be identified has been identified by specific type information using a laser pointer or the like. According to this configuration, the device 1 can track defective products, foreign objects, etc. with a laser beam, and notify the user of the device 1 of their presence and location. As a result, this device 1 can perform the process of inspecting for the presence of defective products and foreign objects, which is a process that conventionally required manual inspection and removal of defective products and foreign objects. It is possible to reduce the number of personnel required for inspection.

Furthermore, the device 1 includes a second identification processing section 25 that performs a second identification processing on the object to be identified on which the first identification processing section 24 has performed identification processing. The second identification processing unit 25 executes the second identification processing when a predetermined time has elapsed since the first identification processing unit 24 executed the identification processing. The second identification process is, for example, a process of changing the laser beam irradiation mode, removing the object to be identified, and the like. According to this configuration, the device 1 can make the user of the device 1 aware that defective products, foreign matter, etc. have not been removed by changing the laser beam irradiation mode, etc. .

Furthermore, the apparatus 1 includes an identified object detection unit 26 that detects the identified object for which the identifying process has been performed. The second identification processing unit 25 executes a second identification process when the identification object detection unit 26 detects an identification object. According to this configuration, the apparatus 1 can perform the second identification process only when removal of defective products, foreign substances, etc. has not been performed.

Furthermore, the embodiments described above do not unduly limit the content of the invention described in the claims. Furthermore, not all of the configurations described in the embodiments described above are essential configuration requirements of the present disclosure.

1 Imaging device 11 Type input unit 12 Imaging unit 13 Storage unit 14 Position determination unit 15 Form change unit 16 Positional relationship change unit 17 Subject information acquisition unit 18 Placement unit 19 Detection unit 20 Display unit 21 Imaging environment change unit 22 Identification information generation Section 23 Identification section 24 First specific processing section (specific processing section)
25 Second identification processing section 26 Identification object detection section 27 Control section 100 Imaging device 111 Touch panel (type input section)
112 Camera (imaging unit)
116 Vibration device (positional relationship change unit)
117a Weight scale (subject information acquisition unit)
117b1 Mirror (subject information acquisition unit)
117b2 Mirror (subject information acquisition unit)
118 Belt conveyor (loading section)
118a Placement surface 119a1-119a4 First sensor (detection section)
119b1-119b8 2nd sensor (detection part)
121 Lighting (imaging environment change unit)
127 Control device (storage unit, position determination unit, subject information acquisition unit, identification unit)
100A Imaging device 118A Belt conveyor 118aA Placement surface 120A Liquid crystal display (display section)
127A Control device (storage unit, position determination unit, subject information acquisition unit, environment change unit, identification unit)
100B Imaging device 124B Laser pointer (specific processing unit)
126B Identification object detection sensor (identification object detection section)
127B Control device (storage unit, position determination unit, subject information acquisition unit, identification unit)
125B1 Robot arm (second specific processing section)
127B1 Control device (storage unit, position determination unit, subject information acquisition unit, identification unit, specific processing unit)
100C Imaging device 115C Pressing device (shape changing unit)
127C Control device (storage unit, position determination unit, subject information acquisition unit, identification unit)
100D Imaging device 200 Imaging device 211 Touch panel (type input section)
212 Camera 216 Camera support member (positional relationship changing unit)
217 Load cell (subject information acquisition unit)
218 Turntable (mounting section)
218a Placement surface 221 Lighting (imaging environment change unit)
227 Control board (storage unit, position determination unit, identification unit)

Claims (8)

a type input section into which multiple types of subject information are input;
an imaging unit that images the subject and outputs a captured image;
a storage unit in which the type information and the captured image are stored in association with each other;
an identification unit that identifies the type information corresponding to the object to be identified using identification information for each type information generated based on the captured image stored in the storage unit in association with the type information;
a specific processing unit that performs specific processing on the identified object when the type information of the identified object identified by the identification unit is specific type information;
It has
The identification processing unit makes it visible that the object to be identified is identified by the specific type information.
An imaging device characterized by: an identification information generation unit that generates the identification information for each type of information;
Equipped with
The imaging device according to claim 1. a second identification processing unit that performs a second identification process on the object to be identified on which the identification processing unit has performed the identification processing;
Equipped with
The imaging device according to claim 1. The second specific processing unit executes the second specific processing when a predetermined time has elapsed since the specific processing unit executed the specific processing.
The imaging device according to claim 3. an identified object detection unit that detects the identified object on which the identification process has been performed;
Equipped with
The second identification processing unit executes the second identification processing when the identification object detection unit detects the identification object.
The imaging device according to claim 3. The identification processing unit calculates the accuracy with which the identification unit has identified the object to be identified.
The imaging device according to claim 1. a second identification processing unit that classifies the object to be identified based on the accuracy calculated by the identification processing unit;
Equipped with
The imaging device according to claim 6. a type input section into which multiple types of subject information are input;
an imaging unit that images the subject and outputs a captured image;
a storage unit in which the type information and the captured image are stored in association with each other;
an identification unit that identifies the type information corresponding to the object to be identified using identification information for each type information generated based on the captured image stored in the storage unit in association with the type information;
a specific processing unit that performs specific processing on the identified object when the type information of the identified object identified by the identification unit is specific type information;
An identification method performed by an imaging device comprising:
a step in which the identification unit identifies the type information corresponding to the object to be identified based on the captured image of the object to be identified and the identification information;
a step in which the identification processing unit determines whether the type information of the object to be identified identified by the identification unit is the specific type information;
when the type information of the object to be identified identified by the identification unit is the specific type information, the identification processing unit executes the identification process on the object to be identified;
It has
The identification processing unit, as the identification processing, makes it visible that the object to be identified has been identified by the specific type information.
An identification method characterized by:

Images