JP6987537B2 - Image processing equipment, imaging equipment, work support systems, control methods and programs - Google Patents

Description

The present invention relates to an image processing device, an image pickup device, a work support system, a control method and a program.

A remote work support system has been proposed in which a supporter in a remote location provides remote support via a network to an inexperienced worker who works at a site such as printer repair or construction. Patent Document 1 discloses the following remote support system. While the worker has a camera terminal with a shooting function and a display function, the supporter sends a live view image of the work situation to the work support terminal owned by the supporter, and the supporter sends the live view image displayed on the work support terminal. Confirm. Then, the work support terminal receives the instruction data instructed by the supporter at the predetermined position in the live view image, and transmits the data in which the instruction data and the instruction position are associated with each other to the camera terminal. The camera terminal of the worker displays the received instruction data, so that the worker confirms the instruction contents of the supporter.

Japanese Unexamined Patent Publication No. 2011-34315 Japanese Unexamined Patent Publication No. 2007-06533 Japanese Unexamined Patent Publication No. 2010-74315

The remote support system disclosed in Patent Document 1 requires a display device that displays the content of instructions from the supporter on the camera terminal on the worker side. In addition, the work support terminal needs a function to process the instruction contents of the supporter, which increases the cost of constructing the remote work support system. In addition, since the worker must check the instruction data transmitted from the supporter while holding the camera terminal, the camera terminal hinders the work. An object of the present invention is to provide an image processing device that enables a work supporter to effectively support a worker's work even if the worker does not have a display device.

The image processing apparatus according to the embodiment of the present invention is an image processing apparatus that communicates with the information processing apparatus, and is a transmission means for transmitting an captured image to the information processing apparatus and a work designated by the information processing apparatus. A work object corresponding to the position information based on the acquisition means for acquiring the position information of the object from the information processing apparatus, the captured image stored in the first storage unit, and the acquired position information. The first recognition means for recognizing the subject and the second recognition means for recognizing the subject used for designating the work object based on the captured image stored in the first storage unit. It is provided with a tracking means for tracking a work object and the subject, and a notification means for notifying based on the distance between the tracked work object and the subject, and a plurality of work targets are provided by the information processing apparatus. When an object is designated, the notification means gives the notification based on the distance between the plurality of work objects and the subject, and the image processing apparatus further designates the plurality of work objects. A second storage unit for storing the order is provided, the tracking means tracks the work object and the subject in the designated order stored in the second storage unit, and the notification means is the first. The notification content is changed according to the designated order stored in the storage unit of 2 .

According to the present invention, the work supporter can effectively support the work of the worker even if the worker does not have a display device.

It is a figure which shows the system configuration of Example 1. FIG. It is a figure explaining the operation of matching a work object and a designated subject. It is a flowchart explaining the operation of matching a work object and a designated subject. It is a flowchart explaining the operation of matching a work object and a designated subject. It is a flowchart explaining the process of a work support terminal. It is a figure which shows the system configuration of Example 2. It is a figure explaining the operation of matching a plurality of work objects with a designated subject. It is a flowchart explaining the operation of matching a plurality of work objects with a designated subject. It is a flowchart explaining the operation of matching a plurality of work objects with a designated subject.

(Example 1)
FIG. 1 is a diagram showing a system configuration of the first embodiment.
The work support processing system shown in FIG. 1 includes a camera terminal 100 and a work support terminal 120. The camera terminal 100 is an image pickup device and is provided on the operator side. The camera terminal 100 is installed on, for example, a helmet or the like worn by a worker who works on a work object such as a printer. The work support terminal 120 is an information processing device operated by a work supporter who supports the work of the worker.

The camera terminal 100 includes an image pickup optical system 101, an image pickup element 102, an A / D conversion unit 103, a system control unit 104, a memory unit 105, a microphone / speaker unit 106, an image processing unit 107, a work object recognition unit 108, and a designated subject. It has a recognition unit 109. Further, the camera terminal 100 has a tracking unit 110, a distance image generation unit 111, a distance calculation unit 112, a notification control unit 113, a notification unit 114, a communication unit 115, and a bus 116. The present invention is also applicable to an image processing apparatus having a system control unit 104 to a bus 116.

The system control unit 104 controls the entire camera terminal 100. The camera terminal 100 has, for example, a programmable processor (hereinafter, CPU), a ROM, and a RAM. The CPU executes a program stored in the ROM and controls each part to realize the function of the camera terminal 100. The memory unit 105 is a volatile storage unit (first storage unit). The memory unit 105 functions as a buffer memory for temporarily storing image data, distance information, and the like based on a request from each block.

The image pickup optical system 101 has a lens, a shutter, and an aperture, and controls the focusing distance, the aperture amount of the aperture, and the opening / closing of the shutter based on the instruction of the system control unit 104. Based on the control of the system control unit 104, the image pickup element 102 photoelectrically converts the subject light that has passed through the image pickup optical system 101 to generate an analog signal of the subject image (captured image). Further, the image pickup device 102 includes an element having a distance measuring function. For example, using the technique described in Patent Document 2, the light flux that has passed through different regions on the pupil of the lens of the image pickup optical system 101 is exposed to the image pickup element 102 in which the pixel having the distance measuring function is arranged. The system control unit 104 acquires the image shift amount of the exposed light image, and controls the distance image generation unit 111 to calculate the defocus amount.

The A / D conversion unit 103 performs A / D conversion on the analog signal output from the image pickup element 102 based on the control of the system control unit 104, and outputs a digital signal (hereinafter referred to as “RAW data”). Generate and store in the memory unit 105.

The microphone / speaker unit 106 is an acoustic input / output unit having a commonly used microphone device and speaker device. The microphone / speaker unit 106 has a function of picking up the content spoken by the operator and surrounding voices, and a function of outputting the voice picked up by the microphone / speaker unit 123 of the work support terminal 120.

Based on the control of the system control unit 104, the image processing unit 107 generates image data by performing noise removal, gamma correction processing, white balance adjustment processing, and the like on the RAW data stored in the memory unit 105. Further, the image processing unit 107 performs coding processing and decoding processing, and stores the obtained image data in the memory unit 105. The image data is updated at a predetermined frame rate under the control of the system control unit 104. Then, the image data is transmitted to the work support terminal 120 via the network 130 and displayed on the display unit 126 as a live view image.

The work object recognition unit 108 functions as a first recognition means that executes the following processing under the control of the system control unit 104. The work object recognition unit 108 is a work object designated by the work supporter based on the image data stored in the memory unit 105 and the coordinates of the position information received by the communication unit 115 (hereinafter referred to as “designated coordinates”). Performs object recognition processing. Specifically, the work object recognition unit 108 cuts out an area around the designated coordinates and stores the recognized work object as a work object template image in the memory unit 105. Then, the work object recognition unit 108 supplies information including the feature amount and position of the work object (hereinafter referred to as “work object tracking information”) to the tracking unit 110 by analyzing the work object template image. ..

The designated subject recognition unit 109 functions as a second recognition means that executes the following processing based on the control of the system control unit 104. The designated subject recognition unit 109 uses the template image of the designated subject, which will be described later, with respect to the image data stored in the memory unit 105, and the designated subject recognition unit 109 is a subject used by the worker to specify a work object such as a worker's finger. (Hereinafter referred to as "designated subject") recognition processing is performed. The template image of the designated subject is image data of the designated subject previously taken by the camera terminal 100 or the like. The designated subject recognition unit 109 performs a matching process for analyzing the subject in the image data based on the template image of the designated subject. During the matching process, the designated subject recognition unit 109 compares the feature amounts of the subjects in the specified image data with the feature amounts of the template image of the designated subject and calculates the similarity. The designated subject recognition unit 109 identifies the designated subject based on the degree of similarity. Then, the designated subject recognition unit 109 supplies information including the similarity and position of the designated subject (hereinafter, referred to as “designated subject tracking information”) to the tracking unit 110.

The tracking unit 110 executes the following processing based on the control of the system control unit 104. The tracking unit 110 performs tracking processing of the work object and the designated subject existing in the image data based on the work object tracking information and the designated subject tracking information. The tracking unit 110 calculates the degree of correlation in the subject region specified by the information for tracking between image frames, for example, by using the method disclosed in Patent Document 3. After that, the subject area having a follow-up relationship with the subject area of the previous frame is detected from the current frame, and its motion vector is calculated. Then, the coordinates of moving the subject area by the amount of the motion vector are calculated, and the position information of the work object and the designated subject is updated. The tracking unit 110 supplies the updated position information of the work object and the designated subject to the distance calculation unit 112.

The distance image generation unit 111 calculates the defocus amount for each subject area from the image shift amount obtained by the image pickup device 102 based on the control of the system control unit 104. The distance image generation unit 111 generates a distance image showing the distance in the depth direction for each area based on the calculated defocus amount, and stores it in the memory unit 105. The distance calculation unit 112 executes the following processing based on the control of the system control unit 104. The distance calculation unit 112 determines the distance of the work object and the designated subject based on the position information of the work object and the designated subject obtained from the tracking unit 110 and the distance image for each area generated by the distance image generation unit 111. And the distance of. In the present embodiment, the distance calculation unit 112 calculates the distance of the work object and the distance of the designated subject in the depth direction. The distance calculation unit 112 calculates the horizontal distance information between the work object and the designated subject based on the position information of the work object and the designated subject. Then, the distance calculation unit 112 is based on the distance information in the horizontal direction, the distance of the work object in the depth direction, and the distance of the designated subject, and the three-dimensional distance information between the work object and the designated subject (hereinafter referred to as “3D distance information”). Evaluation distance) is calculated. The distance calculation unit 112 may calculate the horizontal distance calculated from the position information of the work object and the designated subject as the evaluation distance.

The notification unit 114 outputs sound according to the distance between the work object and the designated subject. In the present embodiment, the notification unit 114 is a speaker. The notification unit 114 may be a vibration motor for transmitting vibration, a light emitting diode for outputting light, or the like. The notification control unit 113 controls the notification unit 114 based on the evaluation distance obtained by the distance calculation unit 112 based on the control of the system control unit 104. In this embodiment, the notification control unit 113 controls the volume of the notification unit 114 according to the evaluation distance. For example, the notification control unit 113 increases the volume emitted by the notification unit 114 as the evaluation distance becomes closer. The communication unit 115 transmits / receives data to / from the work support terminal 120 via the network 130 under the control of the system control unit 104. Bus 116 connects each module.

The work support terminal 120 includes a system control unit 121, a memory unit 122, a microphone / speaker unit 123, a work object position acquisition unit 124, a communication unit 125, a display unit 126, an operation unit 127, and a bus 128.

The system control unit 121 controls the entire work support terminal 120. The system control unit 121 includes, for example, a programmable processor (hereinafter, CPU), a ROM, and a RAM. The CPU executes the program stored in the ROM and controls each part to realize the function of the work support terminal 120. The memory unit 122 is a volatile storage unit. The memory unit 105 functions as a buffer memory for temporarily storing image data and the like based on a request from each block.

The microphone / speaker unit 123 is an acoustic input / output unit. The microphone / speaker unit 123 has a microphone device and a speaker device that are widely used, and has a function of collecting sound of the content spoken by a supporter and surrounding voice, and sound collection by the microphone / speaker unit 106 of the camera terminal 100. It has a function to output the sound. The operation unit 127 is an input device device such as a keyboard and a mouse that receives an operation instruction from a supporter.

Based on the control of the system control unit 121, the work object position acquisition unit 124 acquires the coordinates designated by the operation unit 127 as the designated coordinates and stores them in the memory unit 122. The communication unit 125 transmits / receives voice, designated coordinates, and image data to and from the camera terminal 100 via the network 130 under the control of the system control unit 121. The display unit 126 has a display and a display controller, and based on the control of the system control unit 121, reads out the contents of the image data stored in the memory unit 122 and displays the contents on the display. Bus 128 is a bus that connects each module. The network 130 is a wide area network (WAN: Wide Area Network) that connects a plurality of terminals so that they can communicate with each other. The system configuration described in FIG. 1 is an example, and the system configuration according to the present invention is not limited to the configuration shown in FIG. 1 as long as the operation described below can be executed.

FIG. 2 is a diagram illustrating an operation of matching a work object with a designated subject.
FIG. 2A shows a live view image 200 taken by the camera terminal 100 displayed on the display unit 126 of the work support terminal 120. The work supporter communicates with the worker through the microphone / speaker unit 123 while observing the live view image 200. FIG. 2B shows a live view image 210 when the work supporter specifies the designated coordinates 211. The work support terminal 120 acquires the designated coordinates of the designated coordinates 211 designated as the work target by the work supporter and transmits them to the camera terminal 100.

FIG. 2C shows a distance image 220 generated by the distance image generation unit 111. The distance image 220 is color-coded according to the distance from the camera terminal 100. The color becomes darker when the distance from the camera terminal 100 is short, and the color becomes lighter when the distance from the camera terminal 100 is short. The distance calculation unit 112 sets the designated coordinates 221 and the designated subject 222 based on the distance information on the image between the coordinates of the designated subject 221 specified by the designated subject tracking information and the designated coordinates 221 and the distance image 220. The evaluation distance 223 between them is calculated.

FIG. 2D shows the correspondence between the evaluation distance 223 calculated by the distance calculation unit 112 and the volume notified by the notification unit 114. The notification unit 114 increases the volume of notification as the evaluation distance 223 gets closer. In the state shown in FIG. 2D, the volume notified by the notification unit 114 is low. FIG. 2E shows a state in which the worker finds the designated coordinates 211 designated as the work target by the work supporter. In the state of the live view image 230, the evaluation distance 223 between the designated subject 232 and the designated coordinates 211 is 0. FIG. 2F shows the correspondence between the evaluation distance 223 calculated by the distance calculation unit 112 and the volume notified by the notification unit 114. Since the evaluation distance 223 becomes 0, the notification unit 114 stops the notification.

3 and 4 are flowcharts illustrating an operation of matching a work object with a designated subject.
In S301, the camera terminal 100 captures the operator's finger as a designated subject for designating the work object and stores it as a template image of the designated subject. Subsequently, in S302, the system control unit 104 controls the image pickup optical system 101, the image pickup element 102, the A / D conversion unit 103, and the image processing unit 107 to generate image data related to the image pickup image at a predetermined frame rate. , Stored in the memory unit 105.

Next, in S303, the distance image generation unit 111 calculates the defocus amount for each subject area based on the image shift amount, and generates a distance image for each area. The distance image generation unit 111 stores the distance image for each generated area in the memory unit 105. In S304, the communication unit 115 transmits the image data stored in the memory unit 105 to the work support terminal 120.

Next, in S305, the work support terminal 120 performs processing based on the image data transmitted from the camera terminal 100. The processing in S305 will be described later. Subsequently, in S306, the communication unit 115 determines whether or not the designated coordinates have been received from the work support terminal 120. When the designated coordinates are received, the process proceeds to S307. If the designated coordinates have not been received, the process proceeds to S309 in FIG.

In S307, the communication unit 115 acquires the designated coordinates received from the work support terminal 120. The acquired designated coordinates are stored in the memory unit 105. Subsequently, in S308 of FIG. 4, the work object recognition unit 108 performs the work object recognition process based on the designated coordinates and the image data stored in the memory unit 105, and obtains the work object tracking information. calculate. The work object recognition unit 108 stores the calculated work object tracking information in the memory unit 105.

Next, in S309, the system control unit 104 determines whether or not the work object has been recognized. Specifically, the system control unit 104 determines whether the work object has been recognized even once in the past. If the work object is recognized, the process proceeds to S310. If the work object is not recognized, the process returns to S302 in FIG. In S310, the designated subject recognition unit 109 performs matching processing on the image data generated in S302 based on the template image of the designated subject, recognizes the designated subject, and generates the designated subject tracking information. The designated subject recognition unit 109 stores the generated designated subject tracking information in the memory unit 105.

In S311 the tracking unit 110 performs tracking processing of the designated subject and the work object based on the designated subject tracking information and the work object tracking information stored in the memory unit 105. The tracking unit 110 updates the position information of the designated subject and the work object stored in the memory unit 105 based on the tracking processing result.

Next, in S312, the distance calculation unit 112 determines the distance corresponding to the position of the designated subject and the work object based on the position information of the designated subject and the work object updated in S311 and the distance image for each area. Get information. The distance calculation unit 112 calculates the evaluation distance based on the distance of the work object and the distance of the designated subject.

Next, in S313, the notification control unit 113 determines whether the evaluation distance calculated by the distance calculation unit 112 is within a predetermined range (whether it is equal to or less than the threshold value). If the evaluation distance is equal to or less than the threshold value, the process proceeds to S314. If the evaluation distance is not equal to or less than the threshold value, the process returns to S302 in FIG.

In S314, the notification control unit 113 controls the sound emitted by the notification unit 114 according to the evaluation distance. For example, the notification control unit 113 increases the volume emitted by the notification unit 114 as the evaluation distance decreases, and decreases the volume emitted by the notification unit 114 as the evaluation distance increases.

Subsequently, in S315, the system control unit 104 determines whether the work object and the designated subject match based on the evaluation distance. If the work object and the designated subject do not match, the process returns to S302 in FIG. If the work object and the designated subject match, the process ends.

FIG. 5 is a flowchart illustrating the processing of the work support terminal in S305 of FIG.
In S401, the communication unit 125 receives the image data transmitted from the communication unit 115 via the network 130. The communication unit 125 stores the received image data in the memory unit 122. Subsequently, in S402, the display unit 126 displays the image data stored in the memory unit 122.

Next, in S403, the system control unit 121 determines whether or not the designated coordinates have been designated through the operation unit 127. If the specified coordinates are not specified, the process ends. If the designated coordinates are specified, the process proceeds to S404.

In S404, the work object position acquisition unit 124 acquires the designated coordinates and stores them in the memory unit 122. Then, in S405, the communication unit 125 transmits the designated coordinates stored in the memory unit 122 to the communication unit 115 via the network 130.

According to the work support system of this embodiment, it is possible to notify the worker when the distance between the designated subject and the work object is within a predetermined range. As a result, even if the worker does not have a display device, the work object designated by the work supporter can be accurately notified to the worker, and the work of the worker can be effectively supported.

(Example 2)
FIG. 6 is a diagram showing a system configuration of the second embodiment.
In FIG. 6, the same components as those of the work support system of the first embodiment are assigned the same numbers, and the description thereof will be omitted. In the second embodiment, the work supporter designates a plurality of work objects in order.

The camera terminal 500 includes an image sensor 102, an A / D conversion unit 103, a system control unit 104, a memory unit 105, a microphone / speaker unit 106, an image processing unit 107, a work object recognition unit 508, and a designated subject recognition unit 109. .. Further, the camera terminal 500 has a tracking unit 510, a distance image generation unit 111, a distance calculation unit 112, a notification control unit 513, a notification unit 114, a communication unit 115, a bus 116, and a work object designation order storage unit 517.

The work object recognition unit 508 performs recognition processing of the work object designated by the work supporter on the image data stored in the memory unit 105 based on the designated coordinates received by the communication unit 115. The work object recognition unit 508 recognizes the work object by the number of received designated coordinates. The work object designated order storage unit 517 functions as a second storage unit that stores the designated order of the designated coordinates received by the communication unit 115.

The tracking unit 510 acquires work object tracking information from the work object recognition unit 508. Further, the tracking unit 510 acquires the designated subject tracking information from the designated subject recognition unit 109. The tracking unit 510 performs tracking processing of the work object and the designated subject existing in the image data based on the work object tracking information and the designated subject tracking information. The tracking unit 510 performs tracking processing using the work object tracking information corresponding to the head, that is, the earliest designated order among the designated orders stored in the work object designation order storage unit 517.

The notification control unit 513 controls the notification unit 114 based on the evaluation distance obtained by the distance calculation unit 112 and the designated order of the designated coordinates stored in the work object designation order storage unit 517. The notification control unit 513 changes the content of the notification by the notification unit 114 according to the evaluation distance. The notification control unit 513 may change the notification content according to the designated order stored in the work object designation order storage unit 517. For example, the notification control unit 513 changes the cycle of sounding according to the designated order.

FIG. 7 is a diagram illustrating an operation of matching a plurality of work objects and a designated subject in a designated order and therefore in order.
The same contents as those of the first embodiment will be omitted in the present embodiment. FIG. 7A shows a live view image 200 taken by the camera terminal 500 displayed on the display unit 126 of the work support terminal 120, similarly to FIG. 2A. FIG. 7B shows a live view image 600 in a state where the work supporter has specified the designated coordinates 601 and then the designated coordinates 602. The work support terminal 120 acquires the designated coordinates 601 and the designated coordinates 602 designated by the work supporter as work targets and transmits them to the camera terminal 100. The notification control unit 513 controls the notification unit 114 according to the evaluation distance in the order in which the designated coordinates are designated.

FIG. 7C shows a distance image 220 generated by the distance image generation unit 111, similarly to FIG. 2C. FIG. 7D shows the correspondence between the evaluation distance 223 calculated by the distance calculation unit 112 and the volume notified by the notification unit 114, similarly to FIG. 2D. Further, FIG. 7 (E) shows an evaluation distance 223 between the designated subject 232 and the designated coordinates 211 in a state where the operator has searched for the first work object, as in FIG. 2 (E). FIG. 7F shows the correspondence between the evaluation distance 223 calculated by the distance calculation unit 112 and the volume notified by the notification unit 114, similarly to FIG. 2F.

FIG. 6G shows a state after the designated coordinates 601 designated as the work target by the work supporter and the designated subject 232 match. When the system control unit 104 determines that the designated coordinates 601 and the designated subject 232 match, the designated coordinates 602, which is the next designated order after the designated coordinates 601 are to be tracked. Then, the designated subject 232 is detected, and the notification unit 114 gives a notification with a sound corresponding to the evaluation distance 612 between the designated coordinates 602 and the designated subject 232. FIG. 6H is a diagram showing the correspondence between the evaluation distance 612 calculated by the distance calculation unit 112 and the volume notified by the notification unit 114. In the example shown in FIG. 7 (H), since the evaluation distance 612 is large, the notification unit 114 does not notify.

8 and 9 are flowcharts illustrating an operation of sequentially matching a plurality of work objects and a designated subject in the second embodiment.
The processing on the work support terminal side is the same as the processing described with reference to FIG. Further, the same processing as the processing in the flowcharts of FIGS. 3 and 4 in the processing in this flowchart is assigned the same number, and the description thereof will be omitted.

In S701, the system control unit 104 determines whether all the transmission of the designated coordinates from the work support terminal 120 is completed. For example, a work supporter sends a completion command after selecting all work objects. When the communication unit 115 receives the completion command, the system control unit 104 determines that all transmission of the designated coordinates from the work support terminal 120 has been completed.

When all the transmission of the designated coordinates from the work support terminal 120 is completed, the process proceeds to S703. If the transmission of the designated coordinates from the work support terminal 120 is not completed, the process proceeds to S306.

In S702, the system control unit 104 stores in the work object designation order storage unit 517 the designated order indicating the order in which the work object recognized in S308 is the designated work object. Then, the process returns to S302. In S703, the system control unit 104 sets the work object tracking information of the work object corresponding to the first designated order among the designated orders stored in the work object designation order storage unit 517 as the tracking target.

In S704 of FIG. 9, the notification control unit 513 determines whether the evaluation distance is within a predetermined distance range. If the evaluation distance is not within the predetermined distance range, the process returns to S302. If the evaluation distance is within a predetermined distance range, the process proceeds to S314. Then, in S314, the notification control unit 513 controls the notification unit 114 to perform notification at a volume corresponding to the evaluation distance. The notification control unit 513 may change the cycle of sounding the sound according to the designated order stored in the work object designation order storage unit 517.

In S705, the system control unit 104 determines whether or not the processing has been completed for all the work objects. If there is a work object for which the process has not been completed, the process proceeds to S706. When the processing is completed for all the work objects, the processing is terminated. In S706, the system control unit 104 designates the work object in the following designated order at the beginning of the designated order. Then, the process returns to S302 in FIG. According to the work support system of the second embodiment, the work supporter can notify the worker of a plurality of designated work objects in a designated order.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Camera terminal 120 Work support terminal

Claims (10)

An image processing device that communicates with an information processing device.
A transmission means for transmitting an captured image to the information processing device, and
An acquisition means for acquiring the position information of a work object designated by the information processing device from the information processing device, and
A first recognition means for recognizing a work object corresponding to the position information based on the captured image stored in the first storage unit and the acquired position information.
A second recognition means for recognizing a subject used for designating the work object based on the captured image stored in the first storage unit, and
A tracking means for tracking the recognized work object and the subject,
It is provided with a notification means for notifying based on the distance between the tracked work object and the subject .
When a plurality of work objects are designated by the information processing apparatus, the notification means performs the notification based on the distance between the plurality of work objects and the subject.
The image processing device further includes a second storage unit that stores the designated order of the plurality of work objects.
The tracking means tracks the work object and the subject in the designated order stored in the second storage unit.
The notification means is an image processing device characterized in that the notification content is changed according to the designated order stored in the second storage unit. The image processing apparatus according to claim 1 , wherein the notification means performs the notification when the distance between the work object and the subject becomes equal to or less than a threshold value. The image processing apparatus according to claim 1 or 2 , wherein the notification means changes the notification content according to the distance between the work object and the subject. The image processing apparatus according to any one of claims 1 to 3 , wherein the notification means performs the notification by outputting sound. The image processing apparatus according to any one of claims 1 to 3 , wherein the notification means performs the notification by vibration. The image processing apparatus according to any one of claims 1 to 5 , wherein the notification means performs the notification by outputting light. An image pickup apparatus comprising: an image pickup element that photoelectrically converts subject light to capture an image pickup image, and an image processing apparatus according to any one of claims 1 to 6. A system having an image processing device and an information processing device that communicates with the image processing device.
The image processing device is
A transmission means for transmitting an captured image to the information processing device, and
An acquisition means for acquiring position information of a work object from the information processing device, and
A first recognition means for recognizing a work object corresponding to the position information based on the captured image stored in the first storage unit and the acquired position information.
A second recognition means for recognizing a subject used for designating the work object based on the captured image stored in the first storage unit, and
A tracking means for tracking the recognized work object and the subject,
A notification means for notifying based on the distance between the tracked work object and the subject is provided.
When a plurality of work objects are designated by the information processing apparatus, the notification means performs the notification based on the distance between the plurality of work objects and the subject.
The image processing device further includes a second storage unit that stores the designated order of the plurality of work objects.
The tracking means tracks the work object and the subject in the designated order stored in the second storage unit.
The notification means changes the notification content according to the designated order stored in the second storage unit.
The information processing device is
A display means for displaying an captured image transmitted from the image processing device, and
Designating means for designating the position information of the work object and
A work support system comprising: a transmission means for transmitting the designated position information to the image processing device. It is a control method of an image processing device that communicates with an information processing device.
The process of transmitting the captured image to the information processing device and
The process of acquiring the position information of the work object designated by the information processing device from the information processing device, and
A step of recognizing a work object corresponding to the position information based on the captured image stored in the first storage unit and the acquired position information.
A step of recognizing a subject used for designating a work object based on an captured image stored in the first storage unit, and a step of recognizing the subject.
The process of tracking the recognized work object and the subject,
Based on the distance between the workpiece and the object being the tracking, possess and performing notification,
When a plurality of work objects are designated by the information processing apparatus, the step of performing the notification performs the notification based on the distance between the plurality of work objects and the subject.
The image processing device further includes a second storage unit that stores the designated order of the plurality of work objects.
In the tracking step, the work object and the subject are tracked in the designated order stored in the second storage unit.
The step of performing the notification is a control method characterized in that the content of the notification is changed according to the designated order stored in the second storage unit. A program characterized in that a computer functions as each means provided in the image processing apparatus according to any one of claims 1 to 6.

Images