CN110248086B - Image processing apparatus, mounting apparatus, image processing method, and storage medium - Google Patents
Image processing apparatus, mounting apparatus, image processing method, and storage medium Download PDFInfo
- Publication number
- CN110248086B CN110248086B CN201910171309.XA CN201910171309A CN110248086B CN 110248086 B CN110248086 B CN 110248086B CN 201910171309 A CN201910171309 A CN 201910171309A CN 110248086 B CN110248086 B CN 110248086B
- Authority
- CN
- China
- Prior art keywords
- image
- mode
- standard
- quality
- imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 111
- 238000003672 processing method Methods 0.000 title claims abstract description 12
- 238000003860 storage Methods 0.000 title description 4
- 238000003384 imaging method Methods 0.000 claims abstract description 139
- 238000006243 chemical reaction Methods 0.000 claims abstract description 46
- 238000012546 transfer Methods 0.000 claims description 64
- 230000010354 integration Effects 0.000 claims 1
- 239000000758 substrate Substances 0.000 description 27
- 238000000034 method Methods 0.000 description 15
- 230000008569 process Effects 0.000 description 13
- 238000001514 detection method Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 229910000679 solder Inorganic materials 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000006835 compression Effects 0.000 description 2
- 238000007906 compression Methods 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000010008 shearing Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 230000006866 deterioration Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000155 melt Substances 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004904 shortening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/32—Normalisation of the pattern dimensions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2628—Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
Abstract
The invention provides an image processing apparatus, a mounting apparatus, and an image processing method, which can appropriately shoot shooting objects with different sizes while suppressing equipment cost. An image processing device (30) performs image recognition on a component based on shot data input from a single shooting device (32), the image processing device (30) being configured to include: a mode selection unit (41) that selects either the standard mode or the image quality priority mode; an image generation unit (45) that generates a standard image from the imaging data under the normal imaging conditions; an image conversion unit (46) that performs image conversion from a standard image to a high-quality image in an image quality priority mode; and an image recognition unit (47) that performs image recognition on the component based on the standard image or the high-quality image.
Description
Technical Field
The present invention relates to an image processing apparatus, an image processing method, and a mounting apparatus for use in image recognition of a captured image.
Background
Conventionally, as an imaging system mounted on various manufacturing apparatuses, an imaging system is known in which an imaging field of view of an imaging apparatus is used in a different manner according to the size of an object to be imaged (for example, see patent document 1). In the imaging system described in patent document 1, an imaging device with a standard field of view is used to capture an image of an imaging object with a standard size, and an imaging device with a high resolution field of view is used to capture an image of an imaging object with a minute shape. The captured images captured by the respective imaging devices are transmitted to an acquisition card, respectively, and then digitized, and then subjected to various image processing by a processor or the like, thereby recognizing the image of the subject.
Patent document 1: japanese laid-open patent publication No. 2005-064048
However, in the imaging system described in patent literature 1, a plurality of imaging systems must be prepared in order to change the imaging field of view, and the equipment cost of the imaging system increases. It is also conceivable to image an object by a 1-system imaging system, but if only an imaging device with a standard field of view is passed, a recognition error occurs or recognition accuracy deteriorates due to insufficient resolution with respect to an object having a minute shape. On the other hand, if the imaging apparatus only passes through a high-resolution field of view, it is necessary to perform division recognition according to the size of the object to be imaged, which causes a problem that the processing time is long.
Disclosure of Invention
The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus, an image processing method, and a program that can appropriately capture images of subjects having different sizes while suppressing the facility cost.
An image processing apparatus according to an aspect of the present invention is an image processing apparatus for recognizing an image of a subject based on imaging data input from a single imaging apparatus, the image processing apparatus including: a mode selection unit that selects a standard mode or an image quality priority mode; an image generation unit that generates a standard image from the imaging data under the normal imaging conditions; an image conversion unit that performs image conversion from a standard image to a high-quality image in an image quality priority mode; and an image recognition unit that performs image recognition on the subject based on the standard image or the high-quality image.
An image processing method according to an aspect of the present invention is an image processing method for recognizing an image of a subject based on captured data input from a single imaging device, the image processing method including: selecting a standard mode or an image quality priority mode; generating a standard image from the imaging data under the normal imaging conditions; a step of performing image conversion from a standard image to a high-quality image in a picture quality priority mode; and a step of performing image recognition on the photographic subject based on the standard image or the high-quality image.
According to these configurations, the subject is image-recognized by the standard image in the standard mode, and the subject is image-recognized by the high-quality image in the image quality priority mode. Thus, the standard mode and the image quality priority mode are used separately according to the size of the subject, and the subject can be appropriately recognized. Further, since the high-quality image is generated from the standard image by the processing on the software module side, it is not necessary to prepare a plurality of imaging devices, and the facility cost can be reduced.
In the above-described image processing apparatus, the mode selection unit selects the standard mode or the image quality priority mode according to a size of the subject. According to this configuration, a mode suitable for the size of the photographic subject is automatically selected.
In the above-described image processing apparatus, the image conversion unit performs image conversion by super resolution processing. According to this configuration, by using super-resolution processing, the resolution can be sufficiently improved even in processing on the software module side, and a decrease in recognition accuracy due to a lack of resolution can be suppressed.
The image processing apparatus described above receives captured data from the image capturing apparatus via an acquisition card (capture board), and includes a transfer condition setting unit that sets transfer conditions for the captured data for the acquisition card. According to this configuration, the transfer condition of the capture card is set on the image processing device side, and the transfer speed of the captured data from the capture card to the image processing device can be increased. The shortened amount of the transfer time of the shot data is used as the processing time of the image conversion, whereby the increase in the tact time can be suppressed.
In the above-described image processing apparatus, the transfer condition setting unit may set the presence or absence of the cut-out transfer as the transfer condition according to an outer dimension of the subject. According to this configuration, the transfer speed can be increased by reducing the data size of the shot data by cut-transfer.
The image processing apparatus described above includes an imaging condition setting unit that sets imaging conditions for the imaging apparatus. According to this configuration, the image processing apparatus can set the imaging conditions of the imaging apparatus, and the transfer speed of the captured image from the imaging apparatus to the image processing apparatus can be increased.
In the above-described image processing apparatus, the mode selection unit selects any one of a standard mode, an image quality priority mode, and a speed priority mode, the image capturing condition setting unit sets a normal image capturing condition in the standard mode and the image quality priority mode, and sets an image capturing condition having a lower image quality than the normal image capturing condition in the speed priority mode, the image generation unit generates a standard image from normal image capturing data in the standard mode and the image quality priority mode, and generates a low-quality image from low-quality image capturing data in the speed priority mode, the image conversion unit performs image conversion from the standard image to the high-quality image in the image quality priority mode, and the image recognition unit performs image recognition on the image to be captured from the standard image, the high-quality image, and the low-quality image. According to this configuration, since the image quality is reduced and the image is captured in the speed priority mode, the transfer speed of the captured image from the image capturing apparatus to the image processing apparatus can be increased.
In the above-described image processing apparatus, the imaging condition setting unit may set the merge process of the imaging apparatus to an imaging condition of low image quality in the speed priority mode. According to this configuration, by causing the image pickup device to perform the merge operation, the transfer speed of the picked-up image from the image pickup device to the image processing device can be increased by reducing the data size of the merged image.
The image processing apparatus described above includes a mode specifying unit that receives specification of a standard mode, an image quality priority mode, and a speed priority mode from a user, and that gives priority to the mode specified by the mode specifying unit over the mode selected by the mode selecting unit. With this configuration, the mode automatically selected by the mode selection unit can be changed according to the user's judgment.
In the above-described image processing apparatus, the image recognition by the image recognition unit and the image conversion by the image conversion unit are executed in parallel by a multiprocessor or a multicore processor. With this configuration, the processor cores can be allocated for the image recognition and the image conversion, respectively, and the image recognition and the image conversion can be operated in parallel to increase the processing speed.
A mounting device according to an aspect of the present invention includes: the image processing apparatus described above; and an imaging device that inputs the component image as a captured image to the image processing device, and performs image recognition on the component from the component image by the image processing device. According to this configuration, since the image recognition of the component can be appropriately performed in accordance with the size of the component, the component can be appropriately mounted in accordance with the recognition result.
A program according to an aspect of the present invention is a program for an image processing apparatus that recognizes an image of a subject based on captured data input from a single imaging apparatus, the program causing the image processing apparatus to execute: selecting a standard mode or an image quality priority mode; generating a standard image from the imaging data under the normal imaging conditions; a step of performing image conversion from a standard image to a high-quality image in a picture quality priority mode; and a step of performing image recognition on the photographic subject based on the standard image or the high-quality image. According to this configuration, by installing a program in the image processing apparatus, it is possible to appropriately capture images of imaging subjects having different sizes while suppressing the equipment cost.
ADVANTAGEOUS EFFECTS OF INVENTION
According to the present invention, the standard mode and the image quality priority mode are used separately, whereby it is possible to appropriately image subjects having different sizes while suppressing the equipment cost.
Drawings
Fig. 1 is a schematic view showing the entire mounting device of the present embodiment.
Fig. 2 is a schematic view showing the periphery of the mounting head of the present embodiment.
Fig. 3 is a block diagram of the imaging apparatus and the image processing apparatus according to the present embodiment.
Fig. 4 is a flowchart of the mode selection processing of the present embodiment.
Description of the reference numerals
1: mounting device
30: image processing apparatus
31: collection card
32: image capturing apparatus
41: mode selection unit
42: mode designating unit
43: transfer condition setting unit
44: imaging condition setting unit
45: image generation unit
46: image conversion unit
47: image recognition unit
51: processor core
52: processor core
P: component (shooting object)
W: substrate
Detailed Description
Next, the mounting device of the present embodiment will be described with reference to the drawings. Fig. 1 is a schematic view showing the entire mounting device of the present embodiment. Fig. 2 is a schematic view showing the periphery of the mounting head of the present embodiment. The mounting device of the present embodiment is merely an example, and can be modified as appropriate.
As shown in fig. 1, the mounting apparatus 1 is configured to mount a component P (see fig. 2) supplied from a supplier 13 to a predetermined position on a substrate W by a mounting head 12. A substrate conveying unit 11 for conveying the substrate W in the X-axis direction is disposed on the base 10 of the mounting apparatus 1. The substrate transfer unit 11 carries in and positions the substrate W before component mounting from one end side in the X axis direction below the mounting head 12, and carries out the substrate W after component mounting from the other end side in the X axis direction to the outside of the apparatus. A feeder holder 14 in which a plurality of feeders 13 are arranged in a lateral direction in the X-axis direction is detachably connected to both sides of the substrate conveying unit 11.
A reel 15 is loaded in the feeder 13 in a freely detachable manner, and a carrier tape in which a large number of parts P are enclosed is wound around the reel 15. The feeder 13 sequentially feeds out the components P toward a feeding position to be picked up by the mounting head 12 by rotation of a sprocket in the device. At the supply position of the mounting head 12, the outer tape on the surface is peeled off from the carrier tape, and the components P in the pockets of the carrier tape are exposed to the outside. In the present embodiment, a tape feeder is exemplified as the feeder 13, but another feeder may be provided.
The base 10 is provided with a moving mechanism 16 for horizontally moving the pair of mounting heads 12 in the X-axis direction and the Y-axis direction. The moving mechanism 16 includes a pair of Y-axis driving units 17 extending in the Y-axis direction and an X-axis driving unit 18 extending in the X-axis direction. The pair of Y-axis driving units 17 are supported by support units (not shown) provided upright at four corners of the base 10, and the X-axis driving unit 18 is provided to the pair of Y-axis driving units 17 so as to be movable in the Y-axis direction. The mounting head 12 is movably provided in the X-axis direction on an X-axis driving unit 18, and the mounting head 12 is horizontally moved by the X-axis driving unit 18 and a Y-axis driving unit 17 to mount the component P picked up from the feeder 13 on a desired position of the substrate W.
As shown in fig. 2, the mounting head 12 is configured by providing a plurality of suction nozzles 22 (only 1 in the present embodiment) in a mounting head body 21 supported by an X-axis driving unit 18 (see fig. 1). Each suction nozzle 22 is supported by the mounting head main body 21 via a nozzle driving unit 23, and is moved up and down in the Z-axis direction by the nozzle driving unit 23 and rotated about the Z-axis. Each suction nozzle 22 is connected to a suction source (not shown), and holds the component P by a suction force from the suction source. The suction nozzle 22 is provided with a coil spring, and the component P sucked by the suction nozzle 22 is mounted on the substrate W while contracting the coil spring.
The head body 21 is provided with a height measurement sensor 24 (see fig. 1) for detecting a height from the substrate W, and a component detection unit 25 for performing image recognition of a component shape and detection of a suction error. The height measurement sensor 24 detects the distance from the substrate W to the suction nozzle 22 by detecting the reflection of light, and controls the vertical movement amount of the suction nozzle 22 based on the detection result. In the component detection unit 25, the light emitting unit 26 and the light receiving unit 27 are opposed to each other in the horizontal direction, and the shape of the component, the suction error, and the like are inspected based on the light shielding state in which the light from the light emitting unit 26 is shielded by the component P. The component detection unit 25 may emit LED light from the light emitting unit toward the light receiving unit, or may emit laser light from the light emitting unit toward the light receiving unit.
The mounting head body 21 is provided with a substrate imaging unit 28 (see fig. 1) for imaging the BOC mark on the substrate W from directly above, and a component imaging unit 29 for imaging the mounting operation of the component P by the suction nozzle 22 from obliquely above. The substrate imaging unit 28 performs image recognition of the position, warpage, and the like of the substrate W based on the image captured by the BOC mark, and corrects the mounting position of the component P on the substrate W based on the recognition result. The component imaging unit 29 inspects the presence or absence of the component P sucked by the suction nozzle 22 and the presence or absence of the component P mounted on the substrate W, in accordance with the captured image of the component P. The imaging data is input to the image processing device 30 (see fig. 3) from the imaging device 32 such as the component imaging unit 29, and the image processing device 30 recognizes the image of the component P based on the imaging data.
In addition, in a typical mounting apparatus, various components are mounted on a substrate, but it is necessary to change the field of view of the imaging apparatus according to the size of the component and perform imaging. Therefore, in addition to the imaging device with the standard field of view, an imaging device and a lens corresponding to the high-resolution field of view are provided as options, and imaging is performed while switching the standard field of view and the high-resolution field of view according to the size of the component. However, it is necessary to prepare an imaging device and a lens with a high-resolution field of view, which causes a problem of an increase in equipment cost. On the other hand, with only an imaging device with a high-resolution field of view, it is necessary to perform division recognition according to the size of the part, and the imaging process is repeated, so that the processing time becomes long.
Therefore, in the image processing apparatus according to the present embodiment, image conversion such as super-resolution conversion is performed on a standard image acquired from an imaging apparatus with a standard field of view by a software module instead of the imaging apparatus with a super-resolution field of view as an option, thereby improving image quality. By creating a high-quality image corresponding to a super-resolution field of view by a software module, it is possible to combine an imaging device with a standard field of view and an imaging device with a high-resolution field of view, thereby reducing the equipment cost. In addition, the transfer speed of the shot data is increased, and the shortening time of the transfer time is used as the processing time of the image conversion on the software module side, thereby suppressing the increase of the tact time.
Next, an imaging device and an image processing device will be described with reference to fig. 3. Fig. 3 is a block diagram of the imaging apparatus and the image processing apparatus according to the present embodiment. In the block diagram of fig. 3, the imaging device and the image processing device are described in a simplified manner, but the imaging device and the image processing device have the configurations that are normally provided. The captured image and the operation of the image processing apparatus in fig. 3 are shown as an example, and are not limited to this configuration.
As shown in fig. 3, an imaging device 32 is connected to the image processing device 30 via an acquisition card 31, and imaging data imaged by the imaging device 32 is digitized by the acquisition card 31 and input to the image processing device 30. In the image processing apparatus 30, various processes are performed on the shot data in accordance with the component size, thereby performing image recognition of the component P from the shot image, and positioning, inspection, measurement, and the like are performed. The image processing apparatus 30 includes a mode selection unit 41, a mode specification unit 42, a transfer condition setting unit 43, an imaging condition setting unit 44, an image generation unit 45, an image conversion unit 46, and an image recognition unit 47.
The mode selection unit 41 automatically selects the standard mode or the image quality priority mode according to the component size. In this case, the outside dimensions or electrode dimensions of the component are compared with the reference values as component dimensions. For example, if the short side size of the component is greater than or equal to the reference value, the standard mode is selected, and if the short side size of the component is less than the reference value, the picture-quality-priority mode is selected. In addition, for example, if the electrode length of the part P is greater than or equal to the reference length, the standard mode is selected, and if the electrode length of the part P is less than the reference length, the picture-quality-priority mode is selected. The mode selection unit 41 is not limited to a configuration in which the mode selection is performed according to the component size, and may perform the mode selection according to the component name or other selection conditions.
The mode selection unit 41 may select the speed priority mode in addition to the standard mode and the image quality priority mode. The standard mode is a mode using a standard image captured under a normal imaging condition, and is mainly used in imaging processing of a medium-sized component to a large-sized component. The image quality priority mode is a mode in which a high-quality image having an improved image quality compared to a standard image is used, and is mainly used in an imaging process of a small component. The speed priority mode is a mode in which a low-quality image whose image quality is lower than that of a standard image is used, and is used when the processing speed is prioritized over the recognition accuracy of a component regardless of the size of the component.
The mode specification unit 42 receives the mode specification of the standard mode, the image quality priority mode, and the speed priority mode from the user. If a mode is specified from the user in the mode specifying unit 42, the mode specified by the mode specifying unit 42 is prioritized over the mode selected by the mode selecting unit 41. Therefore, the mode automatically selected by the mode selection unit 41 according to the component size can be changed according to the user's judgment. For example, if the accuracy of the error level is lowered, the speed priority mode can be designated, and if the speed of the error level is lowered, the accuracy priority mode can be used, and the mode can be flexibly changed according to the use of the user.
The transfer condition setting unit 43 sets transfer conditions of the captured data to the acquisition card 31. The transfer condition is set in the standard mode and the image quality priority mode according to the external dimensions of the component, and whether or not the cut transfer is performed is set. If the transfer condition is input from the transfer condition setting unit 43 to the acquisition card 31, the transfer condition is reflected on the acquisition card 31. If the cut transfer is set to "invalid", the entire captured data is transferred from the capture card 31 to the image processing device 30, and if the cut transfer is set to "valid", a part of the captured data is cut from the capture card 31 and transferred to the image processing device 30.
For example, if the component size is greater than or equal to the reference value, the cut-and-forward is set to "invalid" and is forwarded without cutting the shot data by the acquisition card 31. On the other hand, if the component size is smaller than the reference value, the cut transfer is set to "effective", and the central area of the shot data is cut by the acquisition card 31 in accordance with the transfer size and transferred. Since the imaging device 32 captures an image with the component accommodated in the center, the component image is not lost by cutting out the central area of the captured data. The transfer speed can be increased by reducing the data size of the shot data by cut-transfer. Therefore, even in the standard mode, the speed can be prioritized by the cut-and-forward.
The imaging condition setting unit 44 sets imaging conditions for the imaging device 32. As the shooting conditions, "normal" is set in the standard mode and the image quality priority mode, and "low image quality" in which the image quality is reduced from "normal" is set in the speed priority mode. If the imaging conditions are input to the imaging device 32 from the imaging condition setting unit 44 via the acquisition card 31, the imaging conditions are reflected on the imaging device 32. When the imaging condition is "normal", the imaging device 32 images the component in the standard specification, and when the imaging condition is "low image quality", the imaging device 32 reduces the resolution and compression rate compared with the standard specification to image the component.
For example, in the standard mode and the image quality priority mode, since the component is imaged in the standard specification, the imaging is performed without reducing the data size. On the other hand, in the speed priority mode, since the binning processing is performed at the time of shooting, a plurality of pixels are read as 1 pixel, thereby shooting in a state of reducing the data size. By causing the image pickup device 32 to perform the merge operation in the speed priority mode, the transfer speed can be increased by the merged image having a reduced data size. In addition to the binning processing, a thinning processing for thinning out pixels at a predetermined ratio may be set as a condition for capturing a low-quality image.
The image generating unit 45 generates a standard image from the captured data under the normal imaging conditions in the standard mode and the image quality priority mode, and generates a low-quality image from the captured data under the low-quality imaging conditions in the speed priority mode. In the image conversion unit 46, the standard image is converted into a high-quality image in the image quality priority mode. The standard image is an image of a standard size and a standard pixel rate, the low-resolution image is an image of a reduced size and a high pixel rate, and the high-resolution image is an image of an enlarged size and a low pixel rate. As described above, the standard image, the low-resolution image, and the high-resolution image are generated for each mode.
In addition, the image conversion by the image conversion unit 46 of the present embodiment uses super-resolution processing, estimates an accurate pixel value from a standard image, and increases the number of pixels, thereby achieving high resolution. By using super-resolution processing, the resolution can be sufficiently improved even in processing on the software module side, and a decrease in recognition accuracy due to a lack of resolution can be suppressed. The image conversion is not limited to the super-resolution processing, and for example, it is possible to use, as the image conversion, bilinear interpolation that performs interpolation using a weighted average of the luminance values of 4 pixels around the target pixel, or bicubic interpolation that performs interpolation by approximating the luminance values of 16 pixels around the target pixel by a 3-degree expression.
The image recognition unit 47 performs image recognition on the part P from the standard image in the standard mode, performs image recognition on the part P from the high-quality image in the image quality priority mode, and performs image recognition on the part P from the low-quality image in the speed priority mode. The mode is selected according to the size of the component, and the image is recognized according to the proper image quality, so that the generation of recognition error and the deterioration of recognition accuracy caused by insufficient resolution are suppressed. For example, in the small component, a high-quality image is generated in the high-quality mode, and the small component is subjected to image recognition with high accuracy from the high-quality image. Various processes such as a positioning process, an inspection process, and a measurement process are performed based on the recognition result of the image recognition unit 47.
The mode selection unit 41, the mode specification unit 42, the transfer condition setting unit 43, the imaging condition setting unit 44, the image generation unit 45, the image conversion unit 46, and the image recognition unit 47 are configured by a processor, a memory, and the like that execute various processes. As the processor, a multi-core processor or a multiprocessor having a plurality of processor cores is provided. The memory is composed of one or more storage media such as rom (readonly memory), ram (random access memory) and the like according to the use. A program for causing the image processing apparatus 30 to execute the image processing method is stored in the memory.
Here, the processor core 51 is assigned to image recognition or the like performed by a normal image processing apparatus, and the processor core 52 is assigned to newly added image conversion or the like. The image recognition and the image conversion are operated in parallel, thereby improving the processing speed. The processor core 52 is assigned with any or all of the processes of mode selection, mode designation, transfer condition setting, and imaging condition setting, as well as image conversion. The number of processor cores is not limited to 2, and may be 3 or more.
Next, a flow of processing operation of the image processing apparatus 30 will be described. First, the mode selection unit 41 selects a mode according to the component size. As described above, the outer dimensions and the electrode lengths, which are the dimensions of the components, are compared with the reference values, and thereby any one of the standard mode and the image quality priority mode is selected. In addition, if a mode is specified by the user through the mode specifying section 42, the mode selection by the mode selecting section 41 is overridden. For example, when the recognition accuracy is less degraded even if the image quality is degraded, the speed priority mode is manually selected. The details of the mode selection by the mode selection unit 41 according to the component size will be described later.
If the mode of the image processing apparatus 30 is determined, the transfer condition is set for the acquisition card 31 by the transfer condition setting portion 43, and the imaging condition is set for the imaging apparatus 32 by the imaging condition setting portion 44. The transfer condition setting unit 43 compares the outer dimension with a reference value to set the cut transfer to "invalid" or "valid". The imaging condition setting unit 44 sets the mode to "normal" in the standard mode and the image quality priority mode, and sets the mode to "low image quality" in the speed priority mode. Further, if the component is photographed by the photographing device 32 with a standard view (e.g., 54mm view), the photographing data is transferred from the photographing device 32 to the image processing device 30 via the acquisition card 31.
At the time of selection of the standard mode, the shooting condition is set to "normal", so that shooting data of a standard size (for example, 1024 × 1024) is forwarded from the shooting device 32 to the acquisition card 31. When the cut transfer by the acquisition card 31 is set to "invalid", the captured data of the standard size is transferred from the acquisition card 31 to the image processing device 30. On the other hand, when the cut-and-forward operation of the acquisition card 31 is set to "effective", the acquisition card 31 forwards the captured data cut out from the captured data only in the central area (for example, 512 × 512) to the image processing device 30.
In the selection of the image quality priority mode, the image data of a standard size (for example, 1024 × 1024) is transferred from the image capturing device 32 to the acquisition card 31, similarly to the selection of the standard mode. In addition, the shot data of the standard size or the central area is cut from the shot data and transferred according to the presence or absence of the cut transfer of the acquisition card 31. In the selection of the speed priority mode, the shooting condition is set to "low quality image", and therefore the shooting data of a reduced size (for example, 512 × 512) is transferred from the shooting device 32 to the acquisition card 31. Then, the capture data of the reduced size is transferred from the capture card 31 to the image processing device 30.
If the captured data is input to the image processing apparatus 30, the captured image is generated by the image generating section 45. In the standard mode, a standard image of a standard pixel rate (e.g., 56 μm) is generated in a standard size (e.g., 1024 × 1024) from the captured data. In the image quality priority mode, the standard image is subjected to image conversion by the image conversion unit 46, and a high-quality image with a low pixel rate (for example, 28 μm) is generated in an enlarged size (for example, 2048 × 2048). In the speed priority mode, a low-quality image with a high pixel rate (e.g., 112 μm) is generated in a reduced size (e.g., 512 × 512) from the captured data.
Then, if the standard image, the high-quality image, and the low-quality image are input to the image recognition unit 47, the component is image-recognized from the standard image, the high-quality image, and the low-quality image. As described above, in the image quality priority mode, since the high-quality image is generated from the standard image by the image conversion, it is not necessary to prepare the imaging device for high-quality image as an option. In addition, if the cut transfer is set to "valid" as the transfer condition, the data size of the captured data can be reduced by the cut transfer, and the reduced amount of the transfer time can be used as the processing time for image conversion. Thus, the image recognition of the component can be appropriately performed without increasing the facility cost and the overall processing time.
Next, the mode selection processing will be described with reference to fig. 4. Fig. 4 is a flowchart of the mode selection processing of the present embodiment. In the flowchart of fig. 4, the short side size is used for mode selection as the outer size of the component, but the flowchart shows an example and is not limited to this configuration. In fig. 4, the explanation will be given by using the reference numerals in fig. 3 as appropriate.
As shown in fig. 4, if the mode selection process is started by the mode selection unit 41, the presence or absence of a mode designation from the user is determined by the mode designation unit 42 (step S01). If it is determined that "there is a mode designation" (Yes at step S01), the mode designated by the user is preferentially set (step S02). This enables the mode to be manually set regardless of the size of the component. If it is determined that "No mode has been designated" (No at step S01), the mode selector 41 refers to the size data of the component to determine whether or not the electrode of the component is present (step S03).
If it is determined that there is No electrode of the component (No at step S03), it is determined whether or not the short side size of the component is equal to or larger than a reference value (for example, 0.5mm) (step S04). When it is determined that the short side size of the member is equal to or larger than the reference value (Yes at step S04), a standard mode is selected in which the imaging condition is set to "normal" and the cut transfer of the transfer condition is set to "invalid" (step S05). When it is determined that the short side size of the component is smaller than the reference value (No at step S04), the image quality priority mode in which the imaging condition is set to "normal" and the cut transfer of the transfer condition is set to "effective" is selected (step S06).
If it is determined that there is "an electrode of a component" (Yes at step S03), it is determined whether or not the electrode length of the component is equal to or greater than a reference value (for example, 0.4mm) (step S07). When it is determined that the electrode length of the member is smaller than the reference value (No at step S07), the image quality priority mode in which the imaging condition is set to "normal" and the cut transfer of the transfer condition is set to "invalid" is selected (step S08). When it is determined that the electrode length of the component is equal to or greater than the reference value (Yes at step S07), it is determined whether or not the short side size of the component is equal to or greater than the reference value (for example, 0.5mm) (step S09).
When it is determined that the short side size of the member is equal to or larger than the reference value (Yes at step S9), a standard mode is selected in which the imaging condition is set to "normal" and the cut transfer of the transfer condition is set to "invalid" (step S10). When it is determined that the short side size of the member is smaller than the reference value (No at step S9), a standard mode in which the imaging condition is set to "normal" and the cut transfer of the transfer condition is set to "effective" is selected (step S11). In addition, the speed priority mode in which the imaging condition is "low image quality" is manually specified by the user through the mode specifying unit 42 in step S02, regardless of the outer dimensions and electrode lengths of the members.
As described above, in the image processing apparatus 30 according to the present embodiment, the component P is image-recognized by the standard image in the standard mode, and the component P is image-recognized by the high-quality image in the image quality priority mode. Thus, the standard mode and the image quality priority mode are used separately according to the size of the subject, and the component P can be appropriately image-recognized. Further, since the high-quality image is generated from the standard image by the processing on the software module side, it is not necessary to prepare a plurality of imaging devices, and the facility cost can be reduced.
In the present embodiment, a configuration in which a component to be imaged is imaged by an imaging device of a component imaging unit is described, but the configuration is not limited to this configuration. For example, the substrate mark as the object of imaging may be imaged by an imaging device of the substrate imaging unit.
In the present embodiment, the high-quality image is not necessarily determined by the size of the resolution. The high-quality image includes an image that appears good even at the same resolution as the standard image. Therefore, the image conversion unit is not limited to image conversion with a high resolution.
In the present embodiment, the low-quality image is not necessarily determined by the size of the resolution. The low-quality image includes an image that is apparently deteriorated even at the same resolution as the standard image. Therefore, the imaging conditions for low image quality are not limited to imaging conditions in which the resolution is low.
In the present embodiment, the mode is selected by the mode selection unit based on the external dimensions of the component, the electrode length, and other component dimensions, but the configuration is not limited to this configuration. The mode selection unit may be configured to select the mode based on the selection condition, and may be configured to automatically select the mode according to the type of the component, for example, using the name of the component as the selection condition.
In the present embodiment, the standard mode, the image quality priority mode, and the speed priority mode can be set in the image processing apparatus, but the present invention is not limited to this configuration. The image processing apparatus may be capable of setting at least a standard mode and an image quality priority mode.
In the present embodiment, the mode selection unit is configured to select the standard mode and the image quality priority mode, but the configuration is not limited to this configuration. The mode selection unit may select the speed priority mode in addition to the standard mode and the image quality priority mode.
In the present embodiment, the presence or absence of cut transfer is set as a transfer condition, but the present invention is not limited to this configuration. The transfer condition may be a condition for transferring the captured image data, and may be, for example, a transfer speed.
In the present embodiment, the shearing amount is set in accordance with the outer dimensions of the member at the time of the shearing transfer, but the present invention is not limited to this configuration. In the cut-and-forward, the central area may be cut to a predetermined size from the shot data.
In the present embodiment, the conditions for degrading the image quality by the resolution of the merge process, the thinning process, and the like are set as the imaging conditions of the low image quality, but the present invention is not limited to this configuration. The imaging condition for low image quality may be set as a condition for reducing the image quality by compression processing.
In the present embodiment, the mode designated by the mode designating unit is configured to be prioritized over the mode selected by the mode selecting unit, but the present invention is not limited to this configuration. The image processing apparatus may not be provided with the mode designating unit.
In the present embodiment, the components are described as an example of the object to be captured, but the present invention is not limited to this configuration. The subject to be photographed is not particularly limited as long as it is a subject to be photographed by the photographing device.
In addition, although the present embodiment has been described with respect to the configuration in which the image processing apparatus is provided in the mounting apparatus, the image processing apparatus can be applied to other apparatuses including an imaging apparatus. For example, the image processing apparatus can be applied to a printing apparatus that prints solder in paste form on a substrate, an inspection apparatus that inspects a printed state of the solder, a reflow apparatus that melts the solder and attaches the solder to a component, and the like.
In the present embodiment, the suction nozzle is exemplified as the nozzle, but the configuration is not limited to this. The suction nozzle may be a suction nozzle as long as it can hold a component, for example, it may be a clamp nozzle.
In the present embodiment, the substrate may be a substrate on which various components can be mounted, and is not limited to a printed substrate, and may be a flexible substrate mounted on a tool substrate.
The program of the present embodiment may be stored in a storage medium. The recording medium is not particularly limited, and may be a nonvolatile recording medium such as an optical disk, a magneto-optical disk, and a flash memory.
Further, although the embodiment and the modification of the present invention have been described, the above embodiment and modification may be combined wholly or partially as another embodiment of the present invention.
The embodiment of the present invention is not limited to the above-described embodiments and modifications, and various changes, substitutions, and alterations can be made without departing from the spirit and scope of the technical idea of the present invention. And if the technical idea of the present invention can be implemented in other ways by technical advances or other derived techniques, it can be implemented using this method. Therefore, the claims cover all the embodiments that can be included in the technical idea of the present invention.
In the above-described embodiment, an image processing apparatus for recognizing an image of a subject based on imaging data input from a single imaging apparatus includes: a mode selection unit that selects a standard mode or an image quality priority mode; an image generation unit that generates a standard image from the imaging data under the normal imaging conditions; an image conversion unit that performs image conversion from a standard image to a high-quality image in an image quality priority mode; and an image recognition unit that performs image recognition on the subject based on the standard image or the high-quality image. According to this configuration, the image recognition of the imaging target is performed based on the standard image in the standard mode, and the image recognition of the imaging target is performed based on the high-quality image in the image quality priority mode. Thus, the standard mode and the image quality priority mode are used separately according to the size of the subject, and the subject can be appropriately recognized. Further, since the high-quality image is generated from the standard image by the processing on the software module side, it is not necessary to prepare a plurality of imaging devices, and the facility cost can be reduced.
Industrial applicability
As described above, the present invention has an effect that it is possible to appropriately photograph subjects having different sizes while suppressing the facility cost, and is particularly suitable for an image processing apparatus, a mounting apparatus, an image processing method, and a program used for image recognition of components of the mounting apparatus.
Claims (12)
1. An image processing apparatus which performs image recognition on a photographic subject based on photographic data input from a single photographic apparatus,
the image processing apparatus is characterized by comprising:
a mode selection unit that selects a standard mode or an image quality priority mode;
an image generation unit that generates a standard image from the imaging data under the normal imaging conditions;
an image conversion unit that performs image conversion from a standard image to a high-quality image in an image quality priority mode; and
and an image recognition unit that performs image recognition on the subject based on the standard image or the high-quality image.
2. The image processing apparatus according to claim 1,
the mode selection unit selects the standard mode or the image quality priority mode according to the size of the subject.
3. The image processing apparatus according to claim 1 or 2,
the image conversion unit performs image conversion by super-resolution processing.
4. The image processing apparatus according to claim 1 or 2,
shooting data is input from the shooting device via an acquisition card,
the transfer condition setting unit sets transfer conditions of the captured data for the capture card.
5. The image processing apparatus according to claim 4,
the transfer condition setting unit sets the presence or absence of shear transfer as a transfer condition according to the outer dimension of the imaging target.
6. The image processing apparatus according to claim 1 or 2,
the imaging apparatus includes an imaging condition setting unit that sets imaging conditions for the imaging apparatus.
7. The image processing apparatus according to claim 6,
the mode selection unit selects any one of a standard mode, an image quality priority mode, and a speed priority mode,
the imaging condition setting unit sets a normal imaging condition in the standard mode and the image quality priority mode, sets an imaging condition having a lower image quality than the normal imaging condition in the speed priority mode,
the image generation unit generates a standard image from normal captured data in the standard mode and the image quality priority mode, generates a low-quality image from low-quality captured data in the speed priority mode,
the image conversion unit performs image conversion from a standard image to a high-quality image in the image quality priority mode,
the image recognition unit recognizes an image of a subject based on the standard image, the high-quality image, and the low-quality image.
8. The image processing apparatus according to claim 7,
the imaging condition setting unit sets the integration processing of the imaging device to imaging conditions of low image quality in the speed priority mode.
9. The image processing apparatus according to claim 7,
the image processing device includes a mode specifying unit for receiving specification of a standard mode, an image quality priority mode, and a speed priority mode from a user,
the mode designated by the mode designating unit is prioritized over the mode selected by the mode selecting unit.
10. The image processing apparatus according to claim 1 or 2,
the image recognition by the image recognition unit and the image conversion by the image conversion unit are executed in parallel by a multiprocessor or a multicore processor.
11. A mounting device, comprising:
the image processing apparatus of any one of claims 1 to 10; and
a photographing device which inputs the component image as a photographed image to the image processing device,
image recognition of the component is performed from the component image by the image processing apparatus.
12. An image processing method for performing image recognition on a photographic subject based on photographic data input from a single photographic device,
the image processing method is characterized by comprising the following steps:
selecting a standard mode or an image quality priority mode;
generating a standard image from the imaging data under the normal imaging conditions;
a step of performing image conversion from a standard image to a high-quality image in a picture quality priority mode; and
and recognizing the image of the object based on the standard image or the high-quality image.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2018-040504 | 2018-03-07 | ||
| JP2018040504A JP2019161249A (en) | 2018-03-07 | 2018-03-07 | Image processing device, mounting device, image processing method, and program |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110248086A CN110248086A (en) | 2019-09-17 |
| CN110248086B true CN110248086B (en) | 2022-03-11 |
Family
ID=67883006
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910171309.XA Active CN110248086B (en) | 2018-03-07 | 2019-03-07 | Image processing apparatus, mounting apparatus, image processing method, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP2019161249A (en) |
| CN (1) | CN110248086B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2024057433A1 (en) * | 2022-09-14 | 2024-03-21 | ヤマハ発動機株式会社 | Component mounter and component imaging method |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009246710A (en) * | 2008-03-31 | 2009-10-22 | Fujifilm Corp | Image processing method and apparatus therefor |
| CN101626445A (en) * | 2008-07-08 | 2010-01-13 | 京瓷美达株式会社 | Image reading apparatus, and image forming apparatus |
| CN202918393U (en) * | 2011-10-04 | 2013-05-01 | 富士机械制造株式会社 | Camera device |
| CN104052926A (en) * | 2013-03-14 | 2014-09-17 | Juki株式会社 | Image processing device and image processing method |
| CN105989361A (en) * | 2015-03-19 | 2016-10-05 | 卡西欧计算机株式会社 | Information processor and face recognition method |
| CN107079620A (en) * | 2014-11-13 | 2017-08-18 | 富士机械制造株式会社 | Fitting machine and used fitting machine electronic component absorption posture inspection method |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0973510A (en) * | 1995-09-06 | 1997-03-18 | Hitachi Ltd | Bar code recognition processor |
| JP4262832B2 (en) * | 1999-05-20 | 2009-05-13 | Ｊｕｋｉ株式会社 | Image input method and apparatus |
| JP4828741B2 (en) * | 2001-08-23 | 2011-11-30 | 大日本スクリーン製造株式会社 | Probe mark measuring method and probe mark measuring apparatus |
| JP5030261B2 (en) * | 2005-11-16 | 2012-09-19 | キヤノン株式会社 | VIDEO DISTRIBUTION DEVICE, VIEWER DEVICE, VIDEO DISTRIBUTION DEVICE CONTROL METHOD, VIEWER DEVICE CONTROL METHOD, AND COMPUTER PROGRAM |
| JP2009206354A (en) * | 2008-02-28 | 2009-09-10 | Fuji Mach Mfg Co Ltd | Image recognition apparatus and image recognition method of electronic component mounting machine |
| JP5673833B2 (en) * | 2011-08-30 | 2015-02-18 | 富士通株式会社 | Imaging device, work support program, and information providing program |
| WO2013187033A1 (en) * | 2012-06-12 | 2013-12-19 | 日本電気株式会社 | Control device, image transmission method, and control program |
| JP6655422B2 (en) * | 2016-02-23 | 2020-02-26 | Juki株式会社 | Image processing apparatus, mounting apparatus, image processing method, and program |
| JP6751144B2 (en) * | 2016-07-28 | 2020-09-02 | 株式会社Ｆｕｊｉ | Imaging device, imaging system, and imaging processing method |
-
2018
- 2018-03-07 JP JP2018040504A patent/JP2019161249A/en active Pending
-
2019
- 2019-03-07 CN CN201910171309.XA patent/CN110248086B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2009246710A (en) * | 2008-03-31 | 2009-10-22 | Fujifilm Corp | Image processing method and apparatus therefor |
| CN101626445A (en) * | 2008-07-08 | 2010-01-13 | 京瓷美达株式会社 | Image reading apparatus, and image forming apparatus |
| CN202918393U (en) * | 2011-10-04 | 2013-05-01 | 富士机械制造株式会社 | Camera device |
| CN104052926A (en) * | 2013-03-14 | 2014-09-17 | Juki株式会社 | Image processing device and image processing method |
| CN107079620A (en) * | 2014-11-13 | 2017-08-18 | 富士机械制造株式会社 | Fitting machine and used fitting machine electronic component absorption posture inspection method |
| CN105989361A (en) * | 2015-03-19 | 2016-10-05 | 卡西欧计算机株式会社 | Information processor and face recognition method |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2019161249A (en) | 2019-09-19 |
| CN110248086A (en) | 2019-09-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7089656B2 (en) | Electric parts mounting apparatus | |
| JP5588331B2 (en) | 3D shape recognition device | |
| CN103685952B (en) | Terminal and image processing method | |
| JP6310058B2 (en) | Image processing apparatus and substrate production system | |
| CN108029243B (en) | Mounting device, shooting processing method and shooting unit | |
| JP3953988B2 (en) | Inspection apparatus and inspection method | |
| CN107006148B (en) | Component mounting device and component mounting system | |
| CN110248086B (en) | Image processing apparatus, mounting apparatus, image processing method, and storage medium | |
| JPWO2014076836A1 (en) | Component mounting machine and mounting inspection machine | |
| JP5759271B2 (en) | Electronic component mounting equipment | |
| JP2015023176A (en) | Part mount device and part mount method | |
| EP3407696B1 (en) | Mounting apparatus and image processing method | |
| KR20060129791A (en) | System and method for optical automated inspection | |
| JP6309962B2 (en) | Assembly machine | |
| JP2000124683A (en) | Image pickup of electronic component and electronic component mounting equipment | |
| JPH05198994A (en) | Method and device for mounting part | |
| CN107079620B (en) | Mounting machine and method for inspecting suction posture of electronic component using mounting machine | |
| JP2022104646A (en) | Cargo inspection device, cargo inspection method, program, and cargo inspection system | |
| JP2006140391A (en) | Component recognition device and component mounting apparatus | |
| JP2009212373A (en) | Component mounting apparatus | |
| EP4002977A1 (en) | Mounting device, mounting system, and inspection and mounting method | |
| US20030017758A1 (en) | Apparatus for and method of mounting electronic parts | |
| JP6837138B2 (en) | Mounting device, information processing device, mounting system, mounting method and information processing method | |
| CN106937525B (en) | Image generation device, installation device, and image generation method | |
| CN109257921B (en) | Electronic component mounting apparatus and electronic component mounting method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |