JP2021067555A - Method and device for creating three-dimensional component data - Google Patents

Abstract

To provide a method and a device for creating three-dimensional component data capable of creating three-dimensional component data.SOLUTION: In a method of creating three-dimensional component data, images obtained by imaging an image of a component PT to be measured and a mark MK serving as a mark by one camera 1 are respectively created at first and second moving points MP1, MP2 where a component PT and the camera 1 are moved relatively, and based on a plurality of created images and the predetermined real length information on the actual length, the three-dimensional component data is obtained, which is predetermined three-dimensional data related to a shape of the component PT. Then, a position of the mark MK is invariable with respect to the component PT while being imaged by the camera 1 at the first and second moving points MP1 and MP2, respectively.SELECTED DRAWING: Figure 2

Description

The present invention relates to a three-dimensional part data creation method for creating three-dimensional part data and a three-dimensional part data creation device.

For example, in order to automatically mount components such as integrated circuits, capacitors, and resistance elements by a machine, component data (component recognition data) representing the shape of the component is important. If such component data is manually input to the component mounting device, man-hours will be required. Therefore, a technique for automatically inputting component data to a component mounting device is desired. As such a technique, for example, Patent Document 1 discloses a component recognition data creation device.

The component recognition data creation device disclosed in Patent Document 1 obtains image data by imaging an electronic component with an imaging device, and extracts the body of the electronic component and the edge of an electrode from the obtained image data. By doing so, component recognition data such as the shape and center position of the body and electrodes of the electronic component is extracted (see paragraphs [0998] to [0100]).

Japanese Patent No. 3877501 (Japanese Unexamined Patent Publication No. 2002-24804)

As described above, the component recognition data creating device disclosed in Patent Document 1 extracts component recognition data such as the shape and center position of the body and electrodes of the electronic component, and the component recognition data is 2 Since it is dimensional data, height (thickness) data cannot be extracted.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a three-dimensional component data creation method and a three-dimensional component data creation device capable of creating three-dimensional component data.

As a result of various studies, the present inventor has found that the above object can be achieved by the following invention. That is, in the three-dimensional component data creation method according to one aspect of the present invention, an image obtained by capturing an image of a component to be measured and one or a plurality of marks as marks by one imaging device is captured by the component and the imaging device. The shape of the part is based on an image generation step generated at each of a plurality of moving points in which and are relatively moved, and a predetermined actual length information regarding the plurality of images generated in the image generation step and the actual length. A component data processing step for obtaining three-dimensional component data, which is predetermined three-dimensional data relating to the above, is provided, and the position of the mark does not change with respect to the component while being imaged by the imaging device at each of the plurality of moving points. Is.

In such a three-dimensional component data creation method, the position of the mark is invariant with respect to the component while being imaged by the imaging device at each of the plurality of moving points. Therefore, since the relative movement amount of the movement is obtained based on the position of the mark in each image before and after the movement, the three-dimensional part data creation method is based on the principle of so-called triangular survey, and each of the movements before and after the movement. Based on the image and the predetermined actual length information regarding the actual length, the predetermined three-dimensional data regarding the shape of the part can be obtained, and therefore the three-dimensional part data can be created. Generally, a so-called stereo camera using two image pickup devices is used when obtaining three-dimensional data, but the above-mentioned three-dimensional component data creation method is cheaper than a stereo camera because one image pickup device is sufficient. Three-dimensional component data can be created with various device configurations.

In another aspect, in the above-mentioned three-dimensional component data creation method, the mark is a plurality of marks, and the position is unchanged with respect to the component while being imaged by the imaging device at each of the plurality of moving points. The mark is at least one of the plurality of marks, at least two of the plurality of marks have a known distance between marks, and the actual length information is the distance between marks. Preferably, in the above-mentioned three-dimensional part data creation method, the distance between marks is set before the image generation step is performed, during the movement in the image generation step, or after the image generation step is performed. , Measured and made known before performing the component data processing step.

In such a three-dimensional part data creation method, there are a plurality of the marks, and at least two of the plurality of marks have known mark-to-mark distances as actual length information, so that the three-dimensional part data As a creation method, three-dimensional part data can be created in actual dimensions using this known distance between marks. Since there are a plurality of marks, the above-mentioned three-dimensional component data creation method can also consider a change in an angle relative to the imaging device with respect to the plurality of marks.

In another aspect, in the above-mentioned three-dimensional component data creation method, the actual length information is the known actual length of the subject reflected in one pixel of the image pickup apparatus.

According to this, it is possible to provide a known three-dimensional component data creation method using the actual length (actual length, actual size) of a subject reflected in one pixel of an imaging device as actual length information.

In another aspect, in the above-mentioned three-dimensional component data creation method, the movement in the image generation step is a movement in the horizontal direction, and the plurality of marks are at least two or more.

In general, at least three or more marks are required in a three-dimensional space to specify the posture (orientation of the optical axis) of the imaging device with respect to the component to be measured or a plurality of marks. In the above three-dimensional data creation method, since the relative movement of the imaging device is restricted in the horizontal direction, at least two or more marks are sufficient, and three-dimensional component data can be created with fewer marks. When the number of marks is three or more, the above-mentioned three-dimensional data creation method can create three-dimensional component data with higher accuracy due to the redundant number of marks.

In another aspect, in these three-dimensional component data creation methods described above, at least three of the plurality of marks are linearly independent while being imaged by the imaging device at each of the plurality of moving points. At the position, the plurality of marks are at least three or more.

Since such a three-dimensional data creation method uses at least three marks, there are no restrictions on the movement of the imaging device, so that the imaging device can be held by hand for imaging. When the number of marks is four or more, the above-mentioned three-dimensional data creation method can create three-dimensional component data with higher accuracy due to the redundant number of marks.

In another aspect, in the above-mentioned three-dimensional component data creation method, the image generation step includes an image display step and a generation step for each of the plurality of moving points, and the image display step is performed at the moving points. A temporary image temporarily captured by the imaging device is displayed on the display device, and a guide display for guiding the proper position of the mark on the image according to the moving location is displayed on the display device so as to be superimposed on the temporary image. Then, the generation step generates the image used in the component data processing step at the moving location.

In such a three-dimensional component data creation method, the guide display is displayed on the display device together with the temporary image, so that the imaging device can be easily moved to the component at each moving location by referring to the mark and the guide display. , The position of the image pickup device with respect to the component at each moving point can be easily optimized. The above-mentioned three-dimensional component data creation method optimizes the posture of the imaging device with respect to the component by referring to at least three or more marks located at each linearly independent position and each guide display corresponding to each mark. it can.

In another aspect, in the above-mentioned three-dimensional part data creation method, the image generation step includes a determination step of determining whether or not the position of the mark is appropriate for the guide display in the temporary image. It further includes an output step for outputting the determination result of the determination step.

Since such a three-dimensional component data creation method further includes a determination process and an output process, the operator (user) can easily position the image pickup device with respect to the component at each moving location by taking the determination result into consideration. Can be optimized to.

In another aspect, in the above-mentioned three-dimensional part data creation method, the image generation step further includes a determination step of determining whether or not the position of the mark is appropriate for the guide display in the temporary image. In addition, the generation step is automatically performed when the determination result of the determination step is appropriate.

In such a three-dimensional component data creation method, the generation process is automatically performed when the determination result of the determination process is appropriate, so that the position shift of the imaging device with respect to the component due to the shutter pressing operation by the operator (user). And camera shake can be reduced.

In another aspect, in these three-dimensional component data creation methods described above, the image pickup device is an image pickup device mounted on a mobile terminal device with a shooting function.

In such a three-dimensional component data creation method, for example, a mobile terminal device with a shooting function such as a smartphone with a camera or a tablet with a camera can be diverted, and therefore, it is cheaper and easier to take an image, and three-dimensional. There is no need for a dedicated device to create component data.

In another aspect, in these three-dimensional part data creation methods described above, the mark has a shape capable of measuring the position of the mark based on the image.

In such a three-dimensional data creation method, the position of the mark can be measured by image processing.

The three-dimensional component data creation device according to another aspect of the present invention has a plurality of moving points in which a component to be measured and one or a plurality of marks as marks are relatively moved between the components. Based on one imaging unit to be imaged by each, a plurality of images captured by the imaging unit at each of the plurality of moving points, and predetermined actual length information regarding the actual length, a predetermined value regarding the shape of the component is obtained. It includes a component data processing unit that obtains three-dimensional component data that is three-dimensional data, and the position of the mark is unchanged with respect to the component while being imaged by the imaging device at each of the plurality of moving points.

In such a three-dimensional component data creation device, the position of the mark is invariant with respect to the component while being imaged by the imaging unit at each of the plurality of moving points. Therefore, since the relative movement amount of the movement is obtained based on the position of the mark in each image before and after the movement, the three-dimensional part data creation device is based on the principle of so-called triangular survey, and each of the movements before and after the movement. Based on the image and the predetermined actual length information regarding the actual length, the predetermined three-dimensional data regarding the shape of the part can be obtained, and therefore the three-dimensional part data can be created. Generally, a so-called stereo camera using two image pickup devices is used when obtaining three-dimensional data, but the above-mentioned three-dimensional component data creation device is cheaper than a stereo camera because one image pickup device is sufficient. Three-dimensional component data can be created with various device configurations.

The three-dimensional component data creation method and the three-dimensional component data creation device according to the present invention can create three-dimensional component data.

It is a block diagram which shows the structure of the 3D component data creation apparatus in 1st Embodiment. It is the schematic which shows the appearance structure of the 3D component data creation apparatus shown in FIG. It is a figure for demonstrating the mark of each aspect used in the 3D component data creation apparatus shown in FIG. It is a figure for demonstrating the calculation method of the height (thickness) of 3D component data in the 3D component data creation apparatus shown in FIG. It is a flowchart which shows the operation of the 3D component data creation apparatus shown in FIG. It is a block diagram which shows the structure of the 3D component data creation apparatus in 2nd Embodiment. It is the schematic which shows the appearance structure of the 3D component data creation apparatus shown in FIG. It is a flowchart which shows the operation of the 3D component data creation apparatus shown in FIG.

Hereinafter, one or more embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the disclosed embodiments. It should be noted that the configurations with the same reference numerals in the respective drawings indicate that they are the same configurations, and the description thereof will be omitted as appropriate. In the present specification, the generic term is indicated by a reference code with the subscript omitted, and the individual configuration is indicated by a reference code with a subscript.

In the three-dimensional component data creation method in the present embodiment, an image obtained by capturing an image of a component to be measured and one or a plurality of marks serving as a mark with one imaging device is obtained by making the component and the imaging device relatively relative to each other. A predetermined three-dimensional shape relating to the shape of the part based on an image generation step generated at each of the plurality of moved locations, a plurality of images generated in the image generation step, and predetermined actual length information regarding the actual length. It includes a part data processing process for obtaining three-dimensional part data which is data. Then, in the three-dimensional component data creation method, the position of the mark is unchanged with respect to the component while being imaged by the imaging device at each of the plurality of moving points. In one example, the mark is a plurality of marks, and the mark whose position is unchanged with respect to the component while being imaged by the imaging device at each of the plurality of moving points is at least one of the plurality of marks. The distance between marks is known for at least two of the plurality of marks, and the actual length information is the distance between marks. Such a three-dimensional component data creation method will be described in more detail below by using the three-dimensional component data creation apparatus in the first and second embodiments in which the method is implemented.

(First Embodiment)
FIG. 1 is a block diagram showing a configuration of a three-dimensional component data creation device according to the first embodiment. FIG. 2 is a schematic view showing an external configuration of the three-dimensional component data creation device shown in FIG. FIG. 2A is a side view of the three-dimensional component data creation device Sa, and FIG. 2B is a top view of the mounting table ST on which the component PT is mounted. FIG. 3 is a diagram for explaining the marks of each aspect used in the three-dimensional component data creating apparatus shown in FIG. 3A to 3G show the first to seventh marks MKa to MKg of the first to seventh aspects, respectively. FIG. 4 is a diagram for explaining a method of calculating the height (thickness) of the three-dimensional component data in the three-dimensional component data creating apparatus shown in FIG. In FIGS. 2, 4 and 7, which will be described later, an XYZ Cartesian coordinate system is shown for clarifying the directional relationship. The X direction (X axis) is, for example, a direction parallel to the horizontal plane, the Y direction (Y axis) is a direction orthogonal to the X direction in the horizontal plane, and the Z direction (Z axis) is the X direction. And Y directions, respectively (vertical direction, normal direction of the horizontal plane).

The three-dimensional component data creation device Sa in the embodiment is an apparatus that creates three-dimensional component data in the component PT to be measured, and is controlled by, for example, one imaging unit 1 as shown in FIGS. 1 and 2. It includes a processing unit 2a, an input unit 3, an output unit 4a, an interface unit 5, a storage unit 6a, and a mounting table (stage) ST.

The component PT may be any member, preferably a member that is automatically mounted by a machine in order to reduce man-hours. For example, when three-dimensional component data is used in a component mounting device, the component PT is, for example, an electronic component such as an integrated circuit (IC), a capacitor, and a resistance element. The three-dimensional component data is predetermined data regarding the shape of the component PT and is represented in three dimensions. When the component PT includes a component body PB and an accessory (for example, an electrode) PE extending outward from the component body PB, the three-dimensional component data is the outer shape of the component PT (the outer shape of the component body PB and the entire accessory PE). ), Horizontal dimension (length in X direction), vertical dimension (length in Y direction) and height dimension (thickness dimension, length in Z direction), horizontal dimension, vertical dimension and height dimension in the component body PB. , Horizontal dimension (length), vertical dimension (width) and height dimension in the accessory PE, and at least one of the number and spacing of the accessory PEs when there are a plurality of accessory PEs.

The mounting table ST is a member for mounting the component PT to be measured. As shown in FIG. 2, the mounting table ST is, for example, a rectangular flat plate-like member in a plan view.

On the one-sided (main surface) SF on which the component PT to be measured on the mounting table ST is placed, there are a plurality of mark MKs serving as markers, and the position of at least one of the plurality of mark MKs in the mark MK is While being imaged by the imaging unit 1 at each of the plurality of moving points MP in which the component PT and the imaging unit 1 are relatively moved in order to generate a plurality of images for creating the three-dimensional component data. Be in position. In the example shown in FIG. 2, the plurality of mark MKs include four first to fourth marks MK1 to MK4 located at each of the four corners (corners) in the rectangular mounting table ST. These four first to fourth marks MK1 to MK4 may be in a fixed position while at least one of them is imaged by the imaging unit 1 at each of the plurality of moving points MP, but the present embodiment Then, all of them are in the fixed positions. Such first to fourth marks MK1 to MK4 are, for example, each member (small piece) formed in a predetermined shape having a color different from the color of the main surface SF of the mounting table ST, and the mounting table ST. Placed at each corner. In the present embodiment, the mark MK is detected by extracting an edge from the image as described later, so that the contrast between the mark MK and the main surface SF of the mounting table ST as the background is higher. Is preferable. The first to fourth marks MK1 to MK4 may be in a fixed position while being imaged by the imaging unit 1 at each of the plurality of moving points MP by their own weight, and may be, for example, an adhesive or a fastener (for example, a screw). Etc.), etc., may be fixed (or semi-fixed) to each corner of the mounting table ST. Alternatively, for example, the first to fourth marks MK1 to MK4 are formed by a sticker and are attached to each corner of the mounting table ST to be formed on the main surface SF of the mounting table ST. Alternatively, for example, the first to fourth marks MK1 to MK4 are formed on the main surface SF of the mounting table ST by being drawn on the main surface SF of the mounting table ST.

The mark MK may have any shape as long as the position of the mark can be measured based on the image. For example, the shape of the mark is a circular shape in a plan view as shown in FIGS. 2 and 3A (first aspect). In this case, the position of the center of the circular mark MKa is the position of the mark. Alternatively, for example, the shape of the mark is, as shown in FIG. 3B, a regular polygon such as a square in a plan view (second aspect). In this case, the center position (diagonal intersection) of the regular polygon mark MKb is set as the mark position. Alternatively, for example, the shape of the mark is, as shown in FIG. 3C, a cross shape formed from two orthogonal line segments in a plan view (third aspect). In this case, the position of the intersection in the cross-shaped mark MKc is the position of the mark.

The mark MK is not limited to the above, and may be configured as follows. Alternatively, for example, as shown in FIG. 3D, the mark MK is a mark MKd composed of intersections of two orthogonal line segments MKd1 and MKd2 in a plan view (fourth aspect), and the position of the intersection is a mark. It is said to be the position of. Alternatively, for example, as shown in FIG. 3E, the mark MK is a mark MKe composed of one end of the line segment MKe1 in a plan view (fifth aspect), and the position of the one end is the position of the mark. .. Alternatively, for example, the mark MK is, as shown in FIG. 3F, a mark MKf composed of a recess or a through hole formed in a mounting table ST in a predetermined shape (sixth aspect). Alternatively, for example, as shown in FIG. 3G, the mark MK is a mark MKg including the tip of a corner in the mounting table ST (seventh aspect), and the position of the tip is the position of the mark. Here, one mark MKa to MKg is shown in each of FIGS. 3A to 3G (for this reason, in the case of the mark MKe of the fifth aspect, two mark MKe with one line segment). , MKe is formed). Further, the plurality of mark MKs used in the three-dimensional component data creation device Sa may be composed of the same type, or may be composed of a plurality of different types.

A component PT to be measured is mounted on the main surface SF of a mounting table ST having a plurality of mark MKs, and the mounted component PT is based on its own weight at each of the plurality of moving points MP. While being imaged by the imaging unit 1, the first to fourth marks MK1 to MK4 (at least one of the plurality of marks) are invariant because they are in the mounted positions. The position) is invariant with respect to the component PT because it is in a fixed position and immobile while being imaged by the imaging unit 1 at each of the plurality of moving points MP.

Then, at least two of the plurality of mark MKs on the main surface SF of the mounting table ST are measured in advance, and the distance between the marks is known. For example, in the example shown in FIG. 2, the distance between any two marks among the four first to fourth marks MK1 to MK4 is actually measured and is known. For example, the distance between the marks between the first and second marks MK1 and MK2 is actually measured and is known. Alternatively, for example, the distance between the marks between the first and third marks MK1 and MK3 is actually measured and made known. Alternatively, for example, the distance between the marks between the first and fourth marks MK1 and MK4 is actually measured and made known. The distance between one mark may be actually measured and known, but in the present embodiment, the distance between the first and second marks MK1 and MK2 (distance between vertical marks, distance between Y-direction marks). ), And the distance between the marks (distance between the lateral marks and the distance between the marks in the X direction) between the first and third marks MK1 and MK3 are actually measured and known. As a result, the vertical dimension of the measurement target is the distance between the first and second marks MK1 and MK2 in the image (image sensor) (distance between the vertical marks on the image, the first and second marks MK1, Calculated by multiplying the ratio (aspect ratio) of the vertical length (the number of pixels in the vertical direction of the measurement target) of the measurement target in the image to the number of pixels between MK2) by the distance between the vertical marks of the actual size. ((Vertical dimension of measurement target) = ((Vertical length of measurement target in image) / (Distance between vertical marks on image)) x (Distance between vertical marks in actual size)). The horizontal dimension of the measurement target is the distance between marks between the first and third marks MK1 and MK3 in the image (distance between vertical marks on the image, the number of pixels between the first and third marks MK1 and MK3). ) To the ratio (horizontal ratio) of the horizontal length of the measurement target (the number of pixels in the horizontal direction of the measurement target) in the image, multiplied by the distance between the horizontal marks of the actual dimensions ((measurement target) Horizontal dimension) = ((horizontal length of the measurement target in the image) / (distance between horizontal marks on the image)) x (distance between horizontal marks in actual dimensions)). Since the actual size of the measurement target is obtained in this way, the longer the known inter-mark distance is, the more preferable (the two mark MKs giving the known inter-mark distance are preferably separated from each other). If the distance between one mark is actually measured and known, the distance between the vertical marks of the actual size and the distance between the horizontal marks of the actual size are respectively based on the distance between the marks of the actual size. (For example, if the distance between the vertical marks of the actual size is known, the distance between the vertical marks of the actual size is defined as the distance between the horizontal marks of the actual size ((horizontal of the actual size). Distance between direction marks) = (distance between vertical marks in actual dimensions))), as described above, when the distance between vertical marks in actual dimensions and the distance between horizontal marks in actual dimensions are known, respectively. For example, even if the imaging optical system of the imaging unit 1 is distorted due to product variation or the like, the actual dimensions of the parts can be more accurately measured than when the distance between the marks of one actual dimension is known. Desired. Such a distance between marks corresponds to an example of predetermined actual length information regarding the actual length for obtaining the actual size from the image.

The imaging unit 1 is a device that is connected to the control processing unit 2a, images a predetermined subject under the control of the control processing unit 2a, and generates an image (image data) of the subject. In the present embodiment, the imaging unit 1 images the component PT to be measured and the plurality of mark MKs to be marks at each of the plurality of moving points MP that are relatively moved between the component PT. Each image at each of the plurality of moving points MP is output to the control processing unit 2a. Such an imaging unit 1 is, for example, an imaging optical system that forms an optical image of a subject on a predetermined imaging surface, and is arranged so that a light receiving surface is aligned with the imaging surface to obtain an optical image of the subject. It is a digital camera including an image sensor that converts an electrical signal, an image processing unit that generates image data that is data representing the image of the subject by image processing the output of the image sensor, and the like.

Such an imaging unit 1 is held by, for example, a holder (camera stand) (not shown) that holds the imaging unit 1 above the mounting table ST in order to relatively move the component PT and the imaging unit 1. Will be done. The holder is attached to, for example, a columnar rod and one end (lower end) of the rod, and has a plate shape in which the rod is erected along the normal direction (Z direction) in the main surface SF of the mounting table ST. The support base is provided with an arm attached to the other end (upper end) of the rod and extending upward of the mounting base ST, and the imaging unit 1 is attached to the tip of the arm. The arm may be omitted, and the imaging unit 1 may be attached to the other end of the rod.
When an image is generated at each of the plurality of moving points, for example, as shown in FIG. 2A, by manually moving the holder, the imaging unit 1 is moved to the first moving point MP1. When an image is taken and an image is generated by imaging at the first moving point MP1, the imaging unit 1 moves to the second moving point MP2 which has moved along the lateral direction (X direction) by an appropriate distance from the first moving point MP1. And image.

In the above description, the component PT and the imaging unit 1 are relatively moved by moving the imaging unit 1 with respect to the mounting table ST, but the mounting table ST is manually moved with respect to the imaging unit 1. As a result, the component PT and the imaging unit 1 may move relatively, or the imaging unit 1 and the mounting table ST move by different amounts of movement, so that the component PT and the imaging unit 1 are relative to each other. You may move to.

The input unit 3 is a device connected to the control processing unit 2a and inputting various commands and various data to the three-dimensional component data creation device Sa, for example, a plurality of input switches to which a predetermined function is assigned. The various commands are, for example, a command for instructing the start of creation of three-dimensional component data, a command for instructing the imaging unit 1 to start imaging, and the like. The various data include, for example, the identifier of the part for which the three-dimensional part data is created (for example, the product name of the part), the actual length information, the distance between marks of the actual dimensions known in the present embodiment, and the like. This data is necessary for creating component data. The output unit 4a is connected to the control processing unit 2a, and according to the control of the control processing unit 2a, commands and data input from the input unit 3, three-dimensional component data created by the three-dimensional component data creating device Sa, and the like. Is a device that outputs data, such as a display device such as a CRT display, a liquid crystal display (LCD), and an organic EL display, a printing device such as a printer, and the like.

The touch panel may be composed of the input unit 3 and the output unit 4a. In the case of configuring this touch panel, the input unit 3 is a position input device that detects and inputs an operation position such as a resistance film method or a capacitance method, and the output unit 4a is a display device. In this touch panel, a position input device is provided on the display surface of the display device, candidates for one or more input contents that can be input to the display device are displayed, and the user touches the display position displaying the input contents to be input. Then, the position is detected by the position input device, and the display content displayed at the detected position is input to the three-dimensional component data creation device Sa as the operation input content of the user. With such a touch panel, since the user can intuitively understand the input operation, the three-dimensional component data creation device Sa that is easy for the user to handle is provided.

The IF unit 5 is a circuit that is connected to the control processing unit 2a and inputs / outputs data to / from an external device according to the control of the control processing unit 2a. For example, an interface circuit of RS-232C which is a serial communication method. , An interface circuit using the Bluetooth (registered trademark) standard, an interface circuit for performing infrared communication such as the IrDA (Infrared Data Association) standard, and an interface circuit using the USB (Universal Serial Bus) standard. The IF unit 5 is a circuit that communicates with an external device, and may be, for example, a data communication card, a communication interface circuit according to the IEEE802.11 standard, or the like.

The storage unit 6a is a circuit that is connected to the control processing unit 2a and stores various predetermined programs and various predetermined data under the control of the control processing unit 2a. The various predetermined programs include, for example, a control processing program and the like. This control processing program includes a control program, an imaging processing program, and a component data processing program. The control program is a program that controls each part 1, 3 to 6a of the three-dimensional component data creation device Sa according to the function of each part. The image pickup processing program generates an image obtained by capturing an image of the component PT to be measured and a plurality of mark MKs serving as markers at each of the plurality of moving points MP in which the component PT and the imaging unit 1 are relatively moved. As described above, when the input unit 3 accepts the shutter pressing operation by the operator (user) at each of the plurality of moving points MP, the image pickup unit 1 is made to take an image to generate an image, and the generated image is generated. This is a program acquired from the image pickup unit 1. The component data processing program is a program that obtains three-dimensional component data based on a plurality of images and actual length information generated by the imaging unit 1 by the imaging processing program, and in the present embodiment, the distance between marks. The various predetermined data include actual length information, data necessary for executing each program, such as a distance between marks of actual dimensions actually measured in advance in the present embodiment. The storage unit 6a functionally includes a mark-to-mark distance information storage unit 61 that stores mark-to-mark distance information relating to the mark-to-mark distance. The mark-to-mark distance information includes two marks that give the mark-to-mark distance and their actual dimensions. Such a storage unit 6a includes, for example, a ROM (Read Only Memory) which is a non-volatile storage element, an EEPROM (Electrically Erasable Programmable Read Only Memory) which is a rewritable non-volatile storage element, and the like. The storage unit 6a includes a RAM (Random Access Memory) or the like that serves as a working memory of the so-called control processing unit 2a that stores data or the like generated during execution of the predetermined program. The storage unit 6a may be provided with a hard disk device having a relatively large storage capacity.

The control processing unit 2a is a circuit for creating three-dimensional part data by controlling each part 1, 3 to 6a of the three-dimensional part data creation device Sa according to the function of each part. The control processing unit 2a is configured to include, for example, a CPU (Central Processing Unit) and peripheral circuits thereof. The control processing unit 2a functionally includes the control unit 41, the imaging processing unit 22a, and the component data processing unit 23 by executing the control processing program.

The control unit 21 controls each unit 1, 3 to 6a of the three-dimensional component data creation device Sa according to the function of each unit, and controls the entire control of the three-dimensional component data creation device Sa.

The image pickup processing unit 22a provides guidance so that an image obtained by capturing the image of the component PT and the mark MK is generated at each of the plurality of moving points MP in which the component PT and the image pickup unit 1 are relatively moved, and a plurality of images are generated. When the input unit 3 receives a shutter pressing operation by the operator (user) at each of the moving points MP, the image pickup unit 1 is made to image and an image is generated, and the generated image is acquired from the image pickup unit 1. .. For example, when the image pickup processing unit 22a receives an instruction to start creating three-dimensional component data at the input unit 3, the image pickup unit 1 is moved to the first movement location MP1, and after the movement, the shutter pressing operation is performed by the input unit 3. A message (first message) prompting the user is output from the output unit 4a. When the input unit 3 receives the shutter pressing operation, the image pickup processing unit 22a causes the image pickup unit 1 to take an image (first image), acquires the generated first image from the image pickup unit 1, and takes an image. The unit 1 is moved to the second moving location MP2, and after the movement, a message (second message) prompting the input unit 3 to perform the shutter pressing operation is output from the output unit 4a. When the shutter pressing operation is received by the input unit 3, the image pickup processing unit 22a causes the image pickup unit 1 to take an image (second image), and acquires the generated second image from the image pickup unit 1. In this example, instead of moving the image pickup unit 1, the image pickup processing unit 22a may move the mounting table ST as described above, or may move each of the imaging unit 1 and the mounting table ST. ..

The component data processing unit 23 obtains three-dimensional component data based on a plurality of images and actual length information generated by the imaging unit 1 by the imaging processing unit 22a, and in the present embodiment, the distance between marks.

More specifically, the component data processing unit 23 obtains three-dimensional component data based on, for example, the first and second images imaged and generated at the first and second moving points MP1 and MP2, respectively.

More specifically, the component data processing unit 23 first extracts (detects and recognizes) the contours of the mark MK and the component PT from the first image. For example, the component data processing unit 23 extracts an edge by performing image processing on the first image with a so-called edge filter, and extracts the contours of the mark MK and the component PT from the extracted edge. Next, the component data processing unit 23 obtains the position of the mark in the first image by simulating the edge caused by the mark MK as the shape of the mark MK. For example, when the mark MK has a circular shape, the component data processing unit 23 regards the edge caused by the mark MK as a circle and obtains the center position of the circle as the position of the mark. Next, the component data processing unit 23 obtains the distance between the horizontal marks on the image and the distance between the vertical marks on the image, respectively, from the positions of the obtained marks in the present embodiment. Next, the component data processing unit 23 obtains each horizontally long and each vertically long of the component PT in the image, which is applied to the three-dimensional component data, based on the edge caused by the component PT. For example, as shown in FIG. 2B, the component PT includes the component body PB and 14 first to 14th electrodes PE1 to PE14, and the three-dimensional component data includes the horizontal and vertical dimensions in the component body PB and the horizontal dimension in the electrode PE. When the dimensions and the vertical dimensions are included, the component data processing unit 23 obtains the horizontal and vertical directions of the component main body PB and the horizontal and vertical directions of the electrode PE in the image based on the edges caused by the component PT. Next, the component data processing unit 23 stores the obtained distances between the horizontal marks on the image, the distances between the vertical marks on the image, each horizontal and vertical length of the component PT in the image, and the storage unit 6a. The horizontal and vertical dimensions of the component PT are obtained from the distance between the horizontal marks and the distance between the vertical marks. In the above description, the horizontal and vertical dimensions of the component PT are obtained from the first image, but the horizontal and vertical dimensions of the component PT may be obtained from the second image, or from the first and second images, respectively. The horizontal and vertical dimensions of the component PT are obtained, and their average values may be the horizontal and vertical dimensions of the component PT. Further, when extracting the outline of the mark MK or the component PT, image processing such as Hough transform (Hough transform) of a straight line or a curved line may be used.

Next, in order for the component data processing unit 23 to obtain the height dimension of the component PT, the component data processing unit 23 extracts the outlines of the mark MK and the component PT from the second image in the same manner as described above, and the second Find the position of the mark in the image. The height dimension Oh of the component PT in the three-dimensional component data can be obtained by using the following equation 1. As shown in FIG. 4, the height dimension of the part of the component PT for which the dimension is to be obtained, for example, the component PT itself is Oh, and the distance between the first moving point MP1 and the second moving point MP2 is Cx. The distance (camera height) between the imaging unit 1 and the main surface SF of the mounting table ST is set to Ch, and the first image in the portion for which dimensions are desired to be obtained for the same mark MK (for example, the third mark MK3). Assuming that the amount of displacement between the second image and the image is Ox, the equation 1; Oh = Ox × (Ch / (Cx + Ox)) is obtained from the similarity of the triangles. The height dimension Oh of the component PT itself is such that in the case of the component PT in which the tip of the electrode PE protrudes from the bottom surface of the component body PB in the Z direction as shown in FIG. 2, the upper surface of the component body PB is the main surface of the mounting table ST. When the component PT is placed on the mounting table ST so as to come into contact with the SF, the height dimension of the tip of the electrode PE is obtained. In order to obtain the height dimension Oh from Equation 1, the component data processing unit 23 then determines the amount of displacement between the first image and the second image in the same mark MK (for example, the third mark MK3). Relative movement amount between the PT and the image pickup unit 1, in the example shown in FIG. 2, the movement amount Cx of the image pickup unit 1 (that is, the distance between the first movement point MP1 and the second movement point MP2). Obtained as Cx). Next, the component data processing unit 23 determines the distance between the mark MK (in this example, the third mark MK3) and the portion for which dimensions are to be obtained for the same mark MK (for example, the third mark MK3). It is obtained for each of the first image and the second image, and the difference between the obtained distances is obtained as the displacement amount Ox between the first image and the second image in the portion for which the dimensions are to be obtained. Then, the component data processing unit 23 uses the obtained values Cx and Ox in the above equation 1 to obtain the height dimension Oh of the component PT. The camera height Ch is measured in advance and stored in the storage unit 6a. Alternatively, the focal distance of the imaging unit 1 is f, and the distance between two marks that give the actual dimension distance between the vertical marks (distance between the horizontal marks) in the image sensor is s, and the actual dimension vertical mark. Assuming that the distance (distance between the horizontal marks) is m, the camera height Ch can be obtained from Ch = m × f / s.

In this way, the component data processing unit 23 obtains the three-dimensional component data of the component PT.

The control processing unit 2a, the input unit 3, the output unit 4a, the IF unit 5, and the storage unit 6a can be configured by, for example, a computer such as a desktop type, a node type, or a tablet type. Alternatively, the imaging unit 1, the control processing unit 2a, the input unit 3, the output unit 4a, the IF unit 5 and the storage unit 6a are three-dimensional components of a mobile terminal device with a shooting function such as a smartphone with a camera or a tablet with a camera. The image pickup device, which is diverted as the data creation device Sa and mounted on such a mobile terminal device, is used as the image pickup unit 1.

Next, the operation of this embodiment will be described. FIG. 5 is a flowchart showing the operation of the three-dimensional component data creation device shown in FIG.

When the power of the three-dimensional component data creation device Sa having such a configuration is turned on, the necessary initialization of each part is executed and the operation is started. By executing the control processing program, the control processing unit 2a is functionally configured with the control unit 21, the imaging processing unit 22a, and the component data processing unit 23.

In FIG. 5, first, preparation for measurement is carried out (S11). More specifically, the operator (user) arranges the component PT to be measured on the main surface SF of the mounting table ST. In the examples shown in FIGS. 2 and 3, the component PT is arranged on the mounting table ST so that the upper surface of the component body PB abuts on the main surface SF of the mounting table ST in order to obtain the height dimension of the component PT. .. Then, the operator inputs an instruction to start creating the three-dimensional component data to the input unit 3. When the input unit 3 receives the instruction to start the creation, the three-dimensional component data creation device Sa outputs the first message from the output unit 4a by the image pickup processing unit 22a of the control processing unit 2a. By referring to the first message, the operator moves the imaging unit 1 to the first moving point MP1 and inputs the shutter pressing operation to the input unit 3. Alternatively, the three-dimensional component data creation device Sa outputs the third message from the output unit 4a after moving the image pickup unit 1 to the first movement location MP1 by the image pickup processing unit 22a. By referring to the third message, the operator inputs the shutter pressing operation to the input unit 3.

When the shutter pressing operation is received by the input unit 3, the three-dimensional component data creation device Sa causes the image pickup unit 1 to take an image by the image pickup processing unit 22a to generate a first image, and the generated first image is used in the image pickup unit. It is taken into the control processing unit 2a from 1 (S12). When the first image is generated at the first moving location MP1, the image pickup processing unit 22a outputs the second message from the output unit 4a. By referring to the second message, the operator moves the imaging unit 1 to the second moving point MP2 and inputs the shutter pressing operation to the input unit 3. Alternatively, the image pickup processing unit 22a outputs the fourth message from the output unit 4a after moving the image pickup unit 1 to the second moving point MP2. By referring to the fourth message, the operator inputs the shutter pressing operation to the input unit 3.

When the shutter pressing operation is received by the input unit 3, the three-dimensional component data creation device Sa causes the image pickup unit 1 to take an image by the image pickup processing unit 22a to generate a second image, and the generated second image is used in the image pickup unit. It is taken into the control processing unit 2a from 1 (S13).

When the second image is generated at the second moving location MP2, the three-dimensional component data creating device Sa is imaged and generated at each of the first and second moving locations MP1 and MP2 by the component data processing unit 23 of the control processing unit 2a. Based on the first and second images, the three-dimensional component data is obtained by each of the above processes (S14). Here, the distance between marks required for creating the three-dimensional part data may be known before the process S14 (part data processing step). For example, the distance between the marks is actually measured before performing the image generation step of the process S12 or the process S13. More specifically, when the mark MK is arranged on the mounting table ST, the inter-mark distance is actually measured and stored in advance in the inter-mark distance information storage unit 61. Alternatively, for example, in the process S11 (before the execution of the process S12), the inter-mark distance is actually measured, input from the input unit 3, and stored in the inter-mark distance information storage unit 61. Further, for example, during the movement in the image generation step, in this example, the inter-mark distance is actually measured between the processes S12 and S13, input from the input unit 3, and stored in the inter-mark distance information storage unit 61. Will be done. Further, for example, after performing the image generation step, the inter-mark distance is actually measured, input from the input unit 3, and stored in the inter-mark distance information storage unit 61.

When the three-dimensional part data is obtained, the three-dimensional part data creation device Sa outputs the obtained three-dimensional part data from the output unit 4a by the control unit 21 of the control processing unit 2a (S15), and ends this processing. .. The three-dimensional component data may be stored in the storage unit 6a and stored.

As described above, in the three-dimensional component data creation device Sa according to the first embodiment and the three-dimensional component data creation method implemented therein, the positions at at least one of the four marks MK1 to MK4 at the mark MK. (Each position of all the marks MK1 to MK4 in this embodiment) is invariant to the component PT while being imaged by the imaging unit 1 at each of the first and second moving points MP1 and MP2. Therefore, since the relative movement amount of the movement is obtained based on the position of the mark in each image before and after the movement, the three-dimensional component data creation device Sa and its method are based on the principle of so-called triangular survey (FIG. 4). From the similarity of triangles in the example shown in (1), it is possible to obtain predetermined three-dimensional data regarding the shape of the component PT based on each image before and after the movement and the actual length information, and in the present embodiment, the distance between marks. Three-dimensional component data can be created. Since the above-mentioned three-dimensional component data creation device Sa and its method require only one imaging unit 1, it is possible to create three-dimensional component data with a device configuration that is cheaper than that of a stereo camera. When using one image pickup unit 1 and actually measuring the movement amount of the image pickup unit 1 due to the relative movement, it is necessary to strictly control the movement amount of the image pickup unit 1, but the above three-dimensional As described above, the component data creation device Sa and its method determine the relative movement amount of the movement based on the position of the mark in each image before and after the movement, so that the movement amount of the imaging unit 1 is strictly determined. There is no need to manage it. Since at least two of the four marks MK1 to MK4 have known mark-to-mark distances (horizontal mark-to-mark distance and vertical-direction mark-to-mark distance in the present embodiment), the three-dimensional component data creation is performed. The device Sa and its method can create three-dimensional part data in actual dimensions. Since there are a plurality of marks, the above-mentioned three-dimensional component data creation device Sa and its method can also consider a change in an angle relative to the image pickup device with respect to the plurality of marks.

In general, at least three or more mark MKs are required in a three-dimensional space to specify the posture (direction of the optical axis) of the imaging unit 1 with respect to the component PT to be measured or a plurality of mark MKs. In the above-mentioned three-dimensional component data creation device Sa and its method, since the relative movement of the imaging unit 1 is restricted in the horizontal direction by the support member, at least two or more marks are sufficient, and three-dimensional marks are required with fewer marks. Part data can be created. When the number of marks is three or more, the above-mentioned three-dimensional component data creation device Sa and its method can create three-dimensional component data with higher accuracy due to the redundant number of marks. For example, as described above, by using the distance between the horizontal marks and the distance between the vertical marks, it is possible to create more accurate three-dimensional component data. Alternatively, for example, for each of the plurality of mark MKs, a plurality of displacement amounts Ox between the first image and the second image in the portion for which the dimensions are desired are obtained, and the average value thereof is obtained to obtain a more accurate 3 It is possible to create dimensional component data.

Since the above-mentioned three-dimensional component data creation device Sa and its method use a mark MK having an appropriate shape, the position of the mark can be measured by image processing. The circular mark MKa and the regular polygonal mark MKb are more preferable because the center position can be obtained from the edges of a plurality of points.

Next, another embodiment will be described.
(Second Embodiment)
In the three-dimensional component data creation device Sa in the first embodiment, the imaging unit 1 is supported by the support member, and therefore, at least two or more mark MKs are used, but the three-dimensional component in the second embodiment. The data creation device Sb uses three or more mark MKs including at least three marks at each linearly independent position while being imaged by the imaging unit 1 at each of the plurality of moving points MP.

FIG. 6 is a block diagram showing a configuration of a three-dimensional component data creation device according to the second embodiment. FIG. 7 is a schematic view showing an external configuration of the three-dimensional component data creation device shown in FIG. FIG. 7A shows a three-dimensional component data creation device Sb in which the temporary image and the guide display at the first moving point MP1 are displayed on the display unit, and FIG. 7B shows the temporary image and the guide display at the second moving point MP2. The three-dimensional component data creation device Sb displayed in the section is shown. FIG. 7C shows three marks MK1, MK3, and MK4 located at each linearly independent position on the mounting table ST.

As shown in FIGS. 6 and 7, for example, the three-dimensional component data creating device Sb in the second embodiment includes an imaging unit 1, a control processing unit 2b, an input unit 3, an output unit 4b, and the like. It includes an IF unit 5, a storage unit 6b, and a mobile communication unit 7. Functionally, the imaging unit 1, the input unit 3, and the IF unit 5 in the three-dimensional component data creating device Sb in the second embodiment are functionally the imaging unit 1 in the three-dimensional component data creating device Sa in the first embodiment. , Since it is the same as the input unit 3 and the IF unit 5, the description thereof will be omitted. Since the output unit 4b is a device for displaying a guide, which will be described later, the output unit 4b is the same as the output unit 4a except that the display device is always included.

Similar to the first embodiment, the one-sided SF on which the component PT to be measured in the mounting table ST is mounted has a plurality of mark MKs. Then, in the second embodiment, at least three marks out of the plurality of mark MKs are linearly independent at each position (on two straight lines intersecting each other) while being imaged by the imaging unit 1 at each of the plurality of moving points MP. At each position (at least the first and second positions of two points on one straight line and the third position of one point on the other straight line). Each of the linearly independent positions comprises at least three, including at least three marks. More specifically, the plurality of mark MKs in the second embodiment are the same as those in the first embodiment. As shown in 7A and 7B, the rectangular mounting table ST includes four first to fourth marks MK1 to MK4 located at each of the four corners. Therefore, in the present embodiment, for example, it is shown in FIG. 7C. As described above, the first, third and fourth marks MK1, MK3 and MK4 are located at positions orthogonal to each other, and pass through a straight line passing through the first and third marks MK1 and passing through the third and fourth marks MK3 and MK4. The straight line is orthogonal to the position of the third mark MK3.

The three-dimensional component data creation device Sb in the second embodiment may be a dedicated machine, but in the present embodiment, for example, a mobile terminal device with a shooting function such as a smartphone with a camera or a tablet with a camera creates three-dimensional component data. It is diverted as the device Sb, and the image pickup device and the display device mounted on such a mobile terminal device are used as the image pickup unit 1 and the output unit (display unit) 4b, respectively. In the examples shown in FIGS. 6 and 7, a smartphone with a camera is diverted as the three-dimensional component data creation device Sb. Therefore, the three-dimensional component data creation device Sb in the second embodiment includes the mobile communication unit 7. The mobile communication unit 7 is a circuit that is connected to the control processing unit 2b and communicates with another communication device via the mobile communication network under the control of the control processing unit 2b.

The storage unit 6b is a circuit that is connected to the control processing unit 2b and stores various predetermined programs and various predetermined data under the control of the control processing unit 2b, and includes, for example, a ROM, EEPROM, RAM, and the like.

The various predetermined programs include, for example, a control processing program and the like. This control processing program (three-dimensional component data creation application) includes a control program, an imaging processing program, and a component data processing program. Since the control program and the component data processing program in the second embodiment are the same as the control program and the component data processing program in the first embodiment, respectively, the description thereof will be omitted. The imaging processing program provides guidance so that an image obtained by capturing an image of the component PT and the plurality of mark MKs is generated at each of the plurality of moving points MP in which the component PT and the imaging unit 1 are relatively moved. This is a program in which when the input unit 3 receives a shutter pressing operation by the operator at each of the plurality of moving points MP, the image pickup unit 1 is made to image and an image is generated, and the generated image is acquired from the image pickup unit 1.

In the present embodiment, the imaging processing program provides the guidance by using a guide display GD that guides the proper position of the mark on the image according to the movement location MP, and further, in the present embodiment, the guide display. It is determined whether or not the position of the mark is appropriate for the GD, and the determination result is output from the output unit 4b. The guide display GD is provided corresponding to each of the plurality of moving points MP. When the guide display GD is displayed on the output unit (display unit) 4b at each of the plurality of moving location MPs, the guide display GD is arbitrary as long as it can guide the appropriate position of the mark on the image according to the moving location MP. It may be a display. The guide display GD may be, for example, a point shape (・), a circular shape (●), an arrow (→), or the like indicating an appropriate position of a mark on an image, but in the present embodiment, the guide display GD may be used. Is, for example, a ring-shaped (ring-shaped) figure in which the proper position of the mark is defined as the central position, as shown in FIGS. 7A and 7B. In the guide display GD having such a shape, the image pickup unit 1 (that is, the three-dimensional component data creation device Sb) is moved so that the mark MK on the image is positioned at the center position of the guide display GD. At the moving location MP, the position of the imaging unit 1 (that is, the three-dimensional component data creating device Sb) with respect to the component PT is optimized. Preferably, the ring shape of the guide display GD is larger than the mark MK and similar to the shape of the mark. In such a guide display GD, even if the mark MK on the image is positioned at the center position of the guide display GD, the guide display GD is not shielded by the mark MK, and the guide display is almost always displayed. The position of the mark on the image can be moved while referring to the GD, and the position can be easily optimized.

Then, in the second embodiment, as described above, the plurality of mark MKs have at least three mark MKs at each linearly independent position while being imaged by the imaging unit 1 at each of the plurality of moving points MP. Including. Therefore, the three marks MK at each of such linearly independent positions and the three guide display GDs corresponding to each of these are referred to, and the three marks for each of the three guide display GDs are referred to. By optimizing all the positions of each of the MKs, the posture of the imaging unit 1 with respect to the component PT can be optimized.

The various predetermined data include actual length information, the distance between marks of actual dimensions actually measured in advance in the present embodiment, data necessary for executing each program such as the guide display GD, and the like. .. The storage unit 6b functionally includes a mark-to-mark distance information storage unit 61 that stores mark-to-mark distance information related to the mark-to-mark distance, and a guide display information storage unit 62 that stores guide display information represented by the guide display GD. .. The guide display information includes, for example, a plurality of guide display images corresponding to each of the plurality of movement location MPs, and the guide display image is an image including a guide display GD displayed at a position corresponding to the movement location MP. Is. Alternatively, for example, the guide display information includes an image (guide image) representing the guide display GD and a guide display GD (guide image) on the output unit (display unit) 4b corresponding to each of the plurality of moving points MP. Includes each position to be displayed.

The control processing unit 2b is a circuit for creating three-dimensional part data by controlling each part 1, 3 to 6b of the three-dimensional part data creation device Sb according to the function of each part. The control processing unit 2b includes, for example, a CPU and peripheral circuits thereof. The control processing unit 2b functionally includes the control unit 21, the imaging processing unit 22b, and the component data processing unit 23 by executing the control processing program. Since the control unit 21 and the component data processing unit 23 in the second embodiment are the same as the control unit 21 and the component data processing unit 23 in the first embodiment, respectively, the description thereof will be omitted.

The image pickup processing unit 22b provides guidance so that an image obtained by capturing the image of the component PT and the mark MK is generated at each of the plurality of moving points MP in which the component PT and the image pickup unit 1 are relatively moved, and a plurality of images are generated. When the shutter pressing operation by the operator is received by the input unit 3 at each of the moving points MP, the image pickup unit 1 is made to image and an image is generated, and the generated image is acquired from the image pickup unit 1. More specifically, in the second embodiment, the image pickup processing unit 22b functionally includes an image display processing unit 221, a determination processing unit 222, and a generation processing unit 223.

The image display processing unit 221 displays a temporary image tentatively captured by the imaging unit 1 on the output unit (display unit) 4b in the moving location MP for each of the plurality of moving location MPs, and responds to the moving location MP. The guide display GD is displayed on the output unit (display unit) 4b so as to be superimposed on the temporary image. For example, when the plurality of moving points MP are two first and second moving points MP1 and MP2, first, in the first moving point MP1, the image display processing unit 221 outputs an image to the imaging unit 1 as a temporary image ( A guide corresponding to the first moving point MP1 by capturing an image as a first temporary image), acquiring the captured first temporary image from the imaging unit 1, displaying the acquired first temporary image on the display unit 4b, and so on. The display GD is read from the guide display information storage unit 62 and displayed on the display unit 4b so as to be superimposed on the temporary image. As a result, for example, the image shown in FIG. 7A is displayed on the display unit 4b of the smartphone Sb as the three-dimensional component data creation device Sb. Then, in the second moving location MP2, the image display processing unit 221 causes the imaging unit 1 to image an image as a temporary image (second temporary image), acquires the captured second temporary image from the imaging unit 1, and obtains the image. The acquired second temporary image is displayed on the display unit 4b, and the guide display GD corresponding to the second movement location MP2 is read from the guide display information storage unit 62 and displayed on the display unit 4b so as to be superimposed on the temporary image. To do. As a result, for example, the image shown in FIG. 7B is displayed on the display unit 4b of the smartphone Sb as the three-dimensional component data creation device Sb. In the example shown in FIGS. 7A and 7B, in the first moving location MP1, as shown in FIG. 7A, four first to fourth guides correspond to each of the four first to fourth marks MK1 to MK4. Display GD1 to GD4 are displayed on the display unit 4b, and in the second moving location MP2, as shown in FIG. 7B, four fifth to fifth marks corresponding to each of the four first to fourth marks MK1 to MK4. 8 Guide display GD5 to GD8 are displayed on the display unit 4b. The positions of the first to fourth guide displays GD1 to GD4 and the positions of the fifth to eighth guide displays GD5 to GD8 are the case of the first moving point MP1 shown in FIG. 7A and the second moving point shown in FIG. 7B. Different from each other in the case of MP2, each position of the fifth to eighth guide displays GD5 to GD8 in the case of the second moving point MP2 shown in FIG. 7B is the first to the case of the first moving point MP1 shown in FIG. 7A. The distances between the first moving point MP1 and the second moving point MP2 are shifted (shifted) by the distance in the image according to the distance Cx from each position of the fourth guide display GD1 to GD4.

The determination processing unit 222 determines whether or not the position of the mark MK is appropriate with respect to the guide display GD in the temporary image, and outputs this determination result from the output unit (display unit) 4b. More specifically, the determination processing unit 222 obtains an appropriate index indicating the appropriate degree of the mark position with respect to the guide display GD, and is based on the obtained appropriate index and a preset threshold value (judgment threshold value). , Judge whether or not it is appropriate. The appropriate index is, for example, the amount of deviation in the image between the position of the guide display and the position of the mark. The determination threshold is appropriately set in advance based on a plurality of samples. More specifically, the determination processing unit 222 obtains the center position of the mark MK in the temporary image, obtains the center position of the guide display GD in the temporary image (appropriate position of the mark represented by the guide display GD), and obtains these. The absolute value of the difference is obtained as an aptitude index, and it is determined whether or not the obtained appropriate index (absolute value of the difference) is equal to or less than the determination threshold value. As a result of this determination, when the appropriate index (absolute value of the difference) is equal to or less than the determination threshold value, the determination processing unit 222 determines that the position of the mark is appropriate with respect to the guide display GD, while the appropriate index. When (absolute value of difference) exceeds the determination threshold value, the determination processing unit 222 determines that the position of the mark is not appropriate (inappropriate) with respect to the guide display GD.

Since there are a plurality of guide display GDs corresponding to each of the plurality of mark MKs, the determination processing unit 222 sets the determination processing unit 222 for each of the plurality of guide display GDs (each set of the mark MK and the guide display GD). It is determined whether or not the position of the mark is appropriate for the guide display GD. Then, the determination processing unit 222 may individually output each determination result determined for each of the plurality of guide display GDs (each set of the mark MK and the guide display GD) from the output unit 4b, but in the present embodiment, the determination processing unit 222 may output each determination result individually. In order to determine the appropriateness of the posture of the imaging unit 1 with respect to the component PT to be measured, the determination results for a plurality of guide display GDs corresponding to a plurality of linearly independent mark MKs are collectively collected from the output unit (display unit) 4b. Output. More specifically, the determination processing unit 222 receives an image pickup unit for the component PT to be measured when all the determination results for the plurality of guide display GDs corresponding to the plurality of linearly independent mark MKs are appropriate. The output unit 4b outputs that the position and posture of 1 are finally appropriate. For example, as shown in FIG. 7A, a message that the position and orientation of the imaging unit 1 indicating that it is ultimately properly (e.g., ^"1 st Position OK" proper display such as MS1) is displayed on the display unit 4b .. On the other hand, if any one of the determination results for the plurality of guide display GDs corresponding to the plurality of linearly independent mark MKs is not appropriate, the determination processing unit 222 takes an image on the component PT to be measured. The output unit 4b outputs that at least one of the position and the posture of the unit 1 is not appropriate (inappropriate). For example, as shown in FIG. 7B, a message indicating that at least one of the position and orientation of the imaging unit 1 is not proper (e.g., ^"2 nd Position NG" improper display such as MS2 (error display)) is It is displayed on the display unit 4b.

The generation processing unit 223 generates an image to be used by the component data processing unit 23 at the movement location MP for each of the plurality of movement location MPs. More specifically, for example, when the generation processing unit 223 receives the instruction to start creating the three-dimensional component data in the input unit 3, the image display processing unit 221 receives a temporary image and a guide display GD1 at the first movement location MP1. ~ GD4 is displayed on the display unit 4b, the judgment processing unit 222 is made to judge the appropriateness of the position of the mark with respect to the guide display GD, and the judgment result (for example, individual judgment result or the whole judgment result) is displayed on the display unit 4b. Display it. When the shutter pressing operation is received by the input unit 3, the generation processing unit 223 causes the image pickup unit 1 to take an image to generate a first image, and acquires the generated first image from the image pickup unit 1. Then, when the first image is acquired, the generation processing unit 223 causes the image display processing unit 221 to display the temporary image and the guide displays GD5 to GD8 at the second moving point MP2 on the display unit 4b, and causes the determination processing unit 222 to display the temporary image and the guide displays GD5 to GD8. The appropriateness of the position of the mark with respect to the guide display GD is determined, and the determination result (for example, individual determination result or overall determination result) is displayed on the display unit 4b. When the shutter pressing operation is received by the input unit 3, the generation processing unit 223 causes the image pickup unit 1 to take an image to generate a second image, and acquires the generated second image from the image pickup unit 1.

Next, the operation of this embodiment will be described. FIG. 8 is a flowchart showing the operation of the three-dimensional component data creation device shown in FIG.

When the power of the three-dimensional component data creation device Sb having such a configuration is turned on, the necessary initialization of each part is executed and the operation is started. By executing the control processing program (three-dimensional component data creation application), the control processing unit 2b is functionally configured with the control unit 21, the imaging processing unit 22b, and the component data processing unit 23, and the imaging processing unit 22b is configured. , Image display processing unit 221 and determination processing unit 222 and generation processing unit 223 are functionally configured.

In FIG. 8, first, preparation for measurement is carried out (S21). More specifically, the operator (user) arranges the component PT to be measured on the main surface SF of the mounting table ST as in the first embodiment. Then, the operator inputs an instruction to start creating the three-dimensional component data to the input unit 3.

When the input unit 3 receives the instruction to start the creation, the three-dimensional component data creation device Sb causes the image pickup unit 1 to take an image as a first temporary image by the image pickup processing unit 22b of the control processing unit 2b to generate the image ( S22). This first temporary image is output from the imaging unit 1 to the control processing unit 22b.

Next, the three-dimensional component data creation device Sb displays the first temporary image on the display unit 4b by the image display processing unit 221 of the imaging processing unit 22b, for example, as shown in FIG. 7A, and the first moving location MP1 The first to fourth guide displays GD1 to GD4 corresponding to the above are read from the guide display information storage unit 62 and displayed on the display unit 4b so as to be superimposed on the temporary image (S23).

Next, the three-dimensional component data creation device Sb causes the determination processing unit 222 of the imaging processing unit 22b to determine the appropriateness of the position of the mark with respect to the guide display GD, and the determination result, for example, as shown in FIG. 7A, shows. The entire determination result is displayed on the display unit 4b (S24).

Next, the three-dimensional component data creation device Sb determines whether or not the shutter pressing operation is input from the input unit 3 by the image pickup processing unit 22b (S25). As a result of this determination, when the shutter pressing operation is received by the input unit 3 within a predetermined time set in advance (Yes), the image pickup processing unit 22b then executes the process S26, while the shutter. If the pressing operation is not received by the input unit 3 within the time (No), the imaging processing unit 22b returns the processing to the processing S22.

In this process S26, the three-dimensional component data creation device Sb causes the image pickup unit 1 to take an image by the image pickup process unit 22b to generate a first image, and the generated first image is generated from the image pickup unit 1 to the control process unit 2b. Incorporate into.

At the first moving location MP1, for example, the operator recognizes and determines the appropriateness of the position and orientation of the imaging unit 1 with respect to the component PT to be measured by referring to the temporary image, the guide displays GD1 to GD4, and the determination result MS1. To do. When the operator determines that the image is inappropriate, the image pickup unit 1 (three-dimensional component data creation device Sb, in this example, the smartphone Sb) is referred to the component PT while referring to the temporary image and the first to fourth guide displays GD1 to GD4. To move. Therefore, each process of process S22 to process S25 is repeated until the position and orientation of the imaging unit 1 with respect to the component PT to be measured are appropriate. On the other hand, when the operator determines that it is appropriate, the shutter pressing operation is input from the input unit 3, and the process S26 is executed by this.

Following the process S26, when the first image is generated at the first moving location MP1, the three-dimensional component data creation device Sb causes the image pickup process unit 22b of the control process section 2b to output an image to the image pickup section 1 as a second temporary image. (S27). This second temporary image is output from the imaging unit 1 to the control processing unit 22b.

Next, the three-dimensional component data creation device Sb displays the second temporary image on the display unit 4b by the image display processing unit 221 of the imaging processing unit 22b, for example, as shown in FIG. 7B, and the second moving location MP2. The fifth to eighth guide displays GD5 to GD8 corresponding to the above are read from the guide display information storage unit 62 and displayed on the display unit 4b so as to be superimposed on the temporary image (S28).

Next, the three-dimensional component data creation device Sb causes the determination processing unit 222 of the imaging processing unit 22b to determine the appropriateness of the position of the mark with respect to the guide display GD, and the determination result, for example, as shown in FIG. 7B, shows. The entire determination result is displayed on the display unit 4b (S29).

Next, the three-dimensional component data creation device Sb determines whether or not the shutter pressing operation is input from the input unit 3 by the image pickup processing unit 22b (S30). As a result of this determination, when the shutter pressing operation is received by the input unit 3 within the time (Yes), the imaging processing unit 22b then executes the process S31, while the shutter pressing operation is performed during the time. If the input unit 3 does not accept the process (No), the image pickup processing unit 22b returns the processing to the processing S27.

In this process S31, the three-dimensional component data creation device Sb causes the image pickup unit 1 to take an image by the image pickup process unit 22b to generate a second image, and the generated second image is generated from the image pickup unit 1 to the control process unit 2b. Incorporate into.

At the second moving point MP2, for example, the operator recognizes and determines the appropriateness of the position and orientation of the imaging unit 1 with respect to the component PT to be measured by referring to the temporary image, the guide displays GD5 to GD8, and the determination result. .. If it is determined to be inappropriate, the operator moves the imaging unit 1 with respect to the component PT while referring to the temporary image and the guide display. Therefore, each process of process S27 to process S30 is repeated until the position and orientation of the imaging unit 1 with respect to the component PT to be measured are appropriate. On the other hand, if it is determined to be appropriate, the operator inputs the shutter pressing operation from the input unit 3, and the process S31 is executed by this.

When the second image is generated at the second moving location MP2, the three-dimensional component data creating device Sb is imaged and generated at each of the first and second moving locations MP1 and MP2 by the component data processing unit 23 of the control processing unit 2b. Based on the first and second images, the three-dimensional component data is obtained by each of the above processes (S32).

When the three-dimensional part data is obtained, the three-dimensional part data creation device Sb outputs the obtained three-dimensional part data from the output unit 4b by the control unit 21 of the control processing unit 2b (S33), and ends this processing. ..

As described above, the three-dimensional component data creation device Sb in the second embodiment and the three-dimensional component data creation method implemented therein have the same operations and effects as those in the first embodiment, and the three-dimensional component data. Can be created.

The three-dimensional component data creation device Sb and its method include at least three mark MKs (for example, linearly independent positions) while being imaged by the imaging unit 1 at the first and second moving points MP1 and MP2, respectively. Four first to fourth marks MK1 to MK4, which are at least three or more, including the first, third and fourth marks MK1, MK3, MK4 and the first to third marks MK1 to MK3, etc. are used. Therefore, since the above-mentioned three-dimensional component data creating device Sb and its method do not have restrictions on the movement of the imaging unit 1 as in the first embodiment, for example, the imaging unit 1 (three-dimensional component data creating device Sb, the above-mentioned example). Then, it becomes possible to hold the smartphone Sb) by hand and take an image. When the number of mark MKs is 4 or more, the above-mentioned 3D component data creation device Sb and its method can create 3D component data with higher accuracy due to the redundant number of marks.

Since the guide display GD is displayed on the display unit 4b together with the temporary image in the above-mentioned three-dimensional component data creation device Sb and its method, by referring to the mark MK and the guide display GD, the component PT is displayed at each moving location MP. On the other hand, the image pickup unit 1 can be easily moved, and the position of the image pickup section 1 with respect to the component PT at each movement location MP can be easily optimized. The three-dimensional component data creation device Sb and its method refer to at least three or more mark MKs located at each linearly independent position and each guide display GD corresponding to each mark MK, thereby PT component PT. The posture of the imaging unit 1 with respect to the image can be optimized.

Since the above-mentioned three-dimensional component data creation device Sb and its method determine the appropriateness of the mark position with respect to the guide display GD and output the determination result, the operator can take into consideration the determination result to MP each moving location. The position of the imaging unit 1 with respect to the component PT in each can be easily optimized.

As the above-mentioned three-dimensional component data creation device Sb and its method, for example, a mobile terminal device with a shooting function such as a smartphone with a camera or a tablet with a camera can be diverted, and therefore, it is cheaper and easier to take an image. There is no need for a dedicated device to create 3D component data.

In the above-mentioned second embodiment, the generation of the first and second images is executed by the operation of pressing the shutter down by the operator, but it may be automatically executed according to the determination result. In such a case, the generation processing unit 223 in the above-mentioned three-dimensional component data creation device Sb further causes the imaging unit 1 to take an image and generate an image when the determination result of the determination processing unit 222 is appropriate. More specifically, in the above-mentioned process S25, the generation processing unit 223 determines whether or not the determination result of the determination processing unit 222 is appropriate instead of the input determination of the shutter pressing operation. As a result of this determination, if the determination result is appropriate (Yes), the imaging processing unit 22b then executes the process S26, while if the determination result is not appropriate (No), the imaging processing unit 22b executes the process S26. The image pickup processing unit 22b returns the processing to the processing S22. As a result, the process S26 for generating the first image is automatically executed. Similarly, in the above-mentioned process S30, the generation processing unit 223 determines whether or not the determination result of the determination processing unit 222 is appropriate instead of the input determination of the shutter pressing operation. As a result of this determination, if the determination result is appropriate (Yes), the imaging processing unit 22b then executes the process S31, while if the determination result is not appropriate (No), the imaging processing unit 22b executes the process S31. The image pickup processing unit 22b returns the processing to the processing S27. As a result, the process S31 for generating the second image is automatically executed.

Since such a three-dimensional component data creation device Sb and its method automatically generate an image when the determination result is appropriate, the position of the imaging unit 1 with respect to the component PT accompanying the shutter pressing operation by the operator. You can reduce slippage and camera shake.

Further, in the above-described second embodiment, the determination result of the determination processing unit 222 is displayed on the display unit 4b, but instead of or in addition to this, the determination result of the determination processing unit 222 is output by voice. You may. In this case, the output unit 4b includes a speaker that outputs audio and peripheral circuits thereof.

Further, in the first and second embodiments, the mounting table ST is used, but since the component and the imaging unit 1 are relatively moved by moving the imaging unit 1, the mounting table ST is not always necessary. , The plurality of mark MKs and component PTs may be arranged directly on, for example, a work desk or the like.

Further, in the first and second embodiments, the plurality of moving points MP are two first and second moving points MP1 and MP2, but may be three or more. By obtaining a plurality of three-dimensional component data based on each image at two moving points among the plurality of moving points MP and obtaining the average value thereof, it is possible to create more accurate three-dimensional component data.

Further, in the first and second embodiments, the distance between marks is used as an example of predetermined actual length information regarding the actual length, but the actual length information is not limited to this.

The actual length information is, for example, a known actual length of a subject reflected in one pixel of the imaging device (actual length (actual size) of the subject reflected in one pixel) (for example, a length in the vertical direction (actual size). ) And the length in the horizontal direction (actual size)), and the actual size per pixel (for example, the actual size in the vertical direction and the actual size in the horizontal direction) may be used. The actual size per pixel is measured in advance and stored in the storage unit 6 (6a, 6b) at least before the actual size per pixel is used. When such actual length information is used, the component data processing unit 23 obtains the length (number of pixels) of the measurement target in the image, which is applied to the three-dimensional component data, based on the edge caused by the measurement target. The actual size of the measurement target is obtained by multiplying the obtained number of pixels by the actual size per pixel. For example, the vertical dimension of the measurement target is obtained by multiplying the vertically long (number of pixels in the vertical direction) of the measurement target in the image by the actual dimension in the vertical direction per pixel. The number of pixels may be counted in pixel units such as 1 pixel, 2 pixels and 3 pixels, or subpixels such as 1.4 pixels, 2.3 pixels and 3.7 pixels. It may be counted in units.

Alternatively, for example, the actual length information may be the actual size of the mark MK itself (the actual size of the mark MK). The actual size of the mark MK itself is measured in advance and stored in the storage unit 6 (6a, 6b) at least before using the actual size of the mark MK itself. When such actual length information is used, first, the component data processing unit 23 determines the length of the mark MK itself from the image to the image (for example, the mark length on the image (for example, the horizontal pixel of the mark MK itself). (Number) and the vertically long mark on the image (the number of pixels in the vertical direction of the mark MK itself)). Next, the component data processing unit 23 applies the three-dimensional component data based on the edge caused by the measurement target, the image. The length of the measurement target in the above (the length of the measurement target on the image (for example, the horizontal length of the measurement target on the image (the number of pixels in the horizontal direction) and the vertical length of the measurement target on the image (the number of pixels in the vertical direction)) is obtained. The unit 23 obtains the actual size of the measurement target from the obtained mark length on the image, the measurement target length on the image, and the actual size of the mark MK itself. For example, the horizontal dimension of the measurement target is on the image with respect to the mark horizontal length on the image. It is obtained by multiplying the ratio of the horizontal length of the measurement target (horizontal ratio) by the actual size of the mark MK itself in the horizontal direction. The actual length information is obtained from the actual size of the mark MK itself and the first and second embodiments described above. It may be a predetermined actual size related to the mark MK, such as the actual size of the distance between marks in.

When the actual length information is the actual size per pixel or the actual size of the mark MK itself, the number of mark MKs may be plural or one. In such a case, the height dimension is obtained in the same manner as in the first and second embodiments.

In order to express the present invention, the present invention has been appropriately and sufficiently described through the embodiments with reference to the drawings above, but those skilled in the art can easily change and / or improve the above embodiments. It should be recognized that it can be done. Therefore, unless the modified or improved form implemented by a person skilled in the art is at a level that deviates from the scope of rights of the claims stated in the claims, the modified form or the improved form is the scope of rights of the claims. It is interpreted as being comprehensively included in.

Sa, Sb 3D parts data generator PT parts MK, MK1 to MK4, MKa to MKg Mark MP, MP1, MP2 Moving points GD, GD1 to GD4, GD5 to GD8 Guide display 1 Imaging unit 2a, 2b Control processing unit 6a, 6b Storage unit 7 Mobile communication unit 21 Control unit 22a, 22b Imaging processing unit 23 Parts data processing unit 221 Image display processing unit 222 Judgment processing unit 223 Generation processing unit 61 Mark-to-mark distance information storage unit 62 Guide display information storage unit

Claims (11)

An image obtained by capturing an image of a component to be measured and one or a plurality of marks as marks with one imaging device at each of a plurality of moving points in which the component and the imaging device are relatively moved. Generation process and
A component data processing step for obtaining three-dimensional component data, which is predetermined three-dimensional data regarding the shape of the component, is provided based on a plurality of images generated in the image generation step and predetermined actual length information regarding the actual length. ,
The position of the mark is invariant with respect to the component while being imaged by the imaging apparatus at each of the plurality of moving points.
3D part data creation method. The mark is plural,
The mark whose position is unchanged with respect to the component while being imaged by the imaging device at each of the plurality of moving points is at least one of the plurality of marks.
At least two of the plurality of marks have known mark-to-mark distances.
The actual length information is the distance between the marks.
The three-dimensional part data creation method according to claim 1. The actual length information is the known actual length of the subject reflected in one pixel of the image pickup apparatus.
The three-dimensional part data creation method according to claim 1. The movement in the image generation step is a movement in the horizontal direction.
The plurality of marks are at least two or more.
The three-dimensional data creation method according to claim 2 or 3. At least three of the plurality of marks are in linearly independent positions while being imaged by the imaging device at each of the plurality of moving points.
The plurality of marks are at least three or more.
The three-dimensional data creation method according to claim 2 or 3. The image generation step includes an image display step and a generation step for each of the plurality of moving points.
In the image display step, a temporary image temporarily captured by the imaging device is displayed on the display device at the moving location, and a guide display for guiding the appropriate position of the mark on the image according to the moving location is displayed. Displayed on the display device so as to be superimposed on the image,
The generation step generates the image used in the part data processing step at the moving location.
The three-dimensional part data creation method according to claim 5. The image generation step further includes a determination step of determining whether or not the position of the mark is appropriate with respect to the guide display in the temporary image, and an output step of outputting the determination result of the determination step.
The three-dimensional part data creation method according to claim 6. The image generation step further includes a determination step of determining whether or not the position of the mark is appropriate for the guide display in the temporary image.
The generation step is automatically carried out when the judgment result of the judgment step is appropriate.
The three-dimensional part data creation method according to claim 6. The image pickup device is an image pickup device mounted on a mobile terminal device with a shooting function.
The three-dimensional part data creation method according to any one of claims 1 to 8. The mark has a shape capable of measuring the position of the mark based on the image.
The three-dimensional data creation method according to any one of claims 1 to 9. One imaging unit that captures images of a component to be measured and one or a plurality of marks that serve as markers at each of a plurality of moving points that are relatively moved from the component.
Based on a plurality of images captured by the imaging unit at each of the plurality of moving points and predetermined actual length information regarding the actual length, three-dimensional component data which is predetermined three-dimensional data regarding the shape of the component is obtained. Equipped with a parts data processing unit
The position of the mark is invariant with respect to the component while being imaged by the imaging apparatus at each of the plurality of moving points.
3D part data creation device.

Images