JP2017120615A - Image processing device, measuring device, image processing system, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To receive or transmit data without using a radio transmitter/receiver.SOLUTION: An image processing device 300 comprises acquisition means and a communication unit 302. The acquisition means acquires moving image information including a visible light signal that includes measurement information on a measurement object measured by a measuring device 100 attached to the measurement object and is transmitted by the measuring device 100, and video of the measurement object. The communication unit 302 acquires the measurement information from the visible light signal included in the moving image information acquired by the acquisition means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus, a measurement apparatus, an image processing system, an image processing method, and a program.

  In order to analyze and provide guidance for sports, dance, and the like, a moving image to be measured is photographed using a photographing device. The use of moving images has the advantage that the movement can be easily grasped even if there is a lack of specialized knowledge, and the situation around the measurement target is easy to understand. On the other hand, there is a demerit that it can only grasp the rough movements that are visible, and it is difficult to quantitatively evaluate.

  In addition, a wearable sensor is also used in the same manner as the photographing apparatus. Using a wearable sensor has the advantage that it is easy to acquire invisible information and quantitatively evaluate movement. On the other hand, there is a demerit that analysis of data requires specialized knowledge and it is difficult to grasp the situation around the measurement target.

  As described above, the imaging device and the wearable sensor can complement each other's disadvantages, and by using them in combination, it is possible to effectively analyze and instruct the movement. In Patent Document 1, a subject is equipped with an acceleration sensor, and a video image showing a subject's swing is captured with a video camera, and the maximum peak of the time waveform of acceleration detected by the acceleration sensor is marked on the video image. An apparatus for displaying on a display together with an image is described.

JP 2009-106323 A

  In the device of Patent Document 1, a photographing device and a wearable sensor are combined, image processed, and displayed. The apparatus of Patent Document 1 has a problem that the image processing apparatus must receive data acquired by the wearable sensor using a wireless receiver.

  The present invention has been made in view of such a problem, and provides an image processing apparatus, a measurement apparatus, an image processing system, an image processing method, and a program capable of performing data reception or transmission without using a wireless transceiver. The purpose is to provide.

In order to achieve the above object, an aspect of the image processing apparatus according to the present invention is as follows.
An obtaining means for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and including a visible light signal transmitted by the measurement apparatus and an image of the measurement object; ,
Obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means;
Comprising
It is characterized by that.

In order to achieve the above object, an aspect of the measuring apparatus according to the present invention is as follows:
Measuring means attached to the measuring object and measuring the measurement information of the measuring object;
A light emitting means for transmitting a visible light signal having the measurement information measured by the measuring means;
With
The measuring means measures the geomagnetism to obtain the angular displacement of the measurement object,
The light emitting means emits the visible light signal when the angular displacement does not exceed a threshold, and does not emit the visible light signal when the angular displacement exceeds the threshold.
It is characterized by that.

In order to achieve the above object, an aspect of the image processing system according to the present invention is as follows.
A measurement device attached to a measurement object to measure measurement information of the measurement object, and transmitting a visible light signal having the measured measurement information;
Obtaining means for obtaining moving image information including the visible light signal and the measurement target image; obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means; An image processing apparatus having
Comprising
It is characterized by that.

In order to achieve the above object, an aspect of the image processing method according to the present invention is as follows:
Obtaining moving image information including the visible light signal transmitted by the measurement device having the measurement information of the measurement target measured by the measurement device attached to the measurement target, and the video of the measurement target,
Obtaining the measurement information from the visible light signal included in the obtained moving image information;
Including that,
It is characterized by that.

In order to achieve the above object, an aspect of the program according to the present invention is as follows:
On the computer,
An acquisition process for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and a visible light signal transmitted by the measurement apparatus, and the image of the measurement object;
An acquisition process for acquiring the measurement information from the visible light signal included in the obtained moving image information;
To execute,
It is characterized by that.

  According to the present invention, data can be received or transmitted without using a wireless transceiver.

1 is a schematic diagram illustrating a configuration of an image processing system according to a first embodiment. 1 is a block diagram illustrating a configuration of an image processing system according to a first embodiment. It is a figure explaining the method for the acquisition part which concerns on Embodiment 1 to search for (a) signal, and (b) to specify a signal. It is a figure which shows the structure of the visible light signal which concerns on Embodiment 1. FIG. It is a figure explaining the time synchronization which the acquisition part which concerns on Embodiment 1 performs. 3 is a flowchart illustrating synchronization processing according to the first embodiment. It is a figure which shows the screen which the display part which concerns on Embodiment 1 displays. It is a figure which shows the example which displays the example which selects (a) measurement object and (b) evaluation value in the screen which the display part which concerns on Embodiment 1 displays. FIG. 5 is a diagram illustrating an example in which (a) a measurement target is selected and (b) data in the screen displayed by the display unit according to the first embodiment. In the image processing system which concerns on Embodiment 2, it is a figure explaining the example at the time of (a) measurement start, and (b) angular displacement exceeded the threshold value. It is the schematic which shows the structure of the image processing system of a modification.

(Embodiment 1)
Hereinafter, an image processing system 1 according to Embodiment 1 of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

  FIG. 1 is a schematic diagram illustrating a configuration of an image processing system 1 according to the first embodiment of the present invention, and FIG. 2 is a block diagram illustrating a configuration of the image processing system 1. As illustrated in FIGS. 1 and 2, the image processing system 1 includes a measurement device 100, a photographing device 200, and an image processing device 300.

  The measuring device 100 is a wearable sensor attached to a measurement target. The measuring apparatus 100 includes a control unit 101, a sensor unit 102, an analysis unit 103, an encoding unit 104, a storage unit 105, a light emitting unit 106, and a communication unit 107.

  The control unit 101 is a processing device that controls the measuring apparatus 100 by executing a program. The control unit 101 can include a CPU (Central Processing Unit).

  The sensor unit 102 is a sensor that measures exercise information or biological information of a measurement target, or environmental information around the measurement target, and acquires data (measurement information). The sensor unit 102 may include a speed sensor, an acceleration sensor, an angular velocity sensor, a heart rate sensor, a blood pressure sensor, and an atmospheric pressure sensor. The sensor unit 102 functions as a measurement unit.

  The analysis unit 103 analyzes the data acquired by the sensor unit 102 and evaluates the exercise information or biological information of the measurement target or the environmental information around the measurement target (evaluation value of the exercise information, evaluation value of the biological information) , Environmental information evaluation value). The analysis unit 103 can include a CPU. The evaluation value of the exercise information calculated by the analysis unit 103 includes the speed of the measurement target, the amount of activity (energy) calculated from the weight and speed of the measurement target, the calorie consumption calculated from the amount of activity, the smallness of speed and acceleration May be included. The analysis unit 103 functions as an analysis unit.

  The encoder 104 encodes information to generate a signal. The encoding unit 104 can include a CPU. The visible light signal 400 encoded and generated by the encoding unit 104 includes the measurement apparatus 100, the time when the sensor unit 102 performs measurement, and the evaluation value calculated by the analysis unit 103. Details will be described later.

  The storage unit 105 is a storage device that stores the data acquired by the sensor unit 102, the evaluation value calculated by the analysis unit 103, and the signal generated by the encoding unit 104. The storage unit 105 also functions as a storage area when the control unit 101 executes processing. The storage unit 105 can include a RAM (Random Access Memory), a flash memory, and a magnetic disk.

  The light emitting unit 106 is a visible light emitting device serving as a transmitter for visible light communication. The light emitting unit 106 transmits the visible light signal 400 encoded by the encoding unit 104 using, for example, a blinking pattern that repeatedly turns on and off with visible light. The light emitting unit 106 may include an LED (Light Emitting Diode). The light emitting unit 106 functions as a light emitting unit. The light emitting unit 106 may transmit the visible light signal 400 encoded by the encoding unit 104 using a plurality of LEDs that emit light at different wavelengths. By using a plurality of LEDs that emit light at different wavelengths, the amount of information to be transmitted can be increased.

  The communication unit 107 is a communication device that performs wireless communication with the image processing apparatus 300. The communication unit 107 transmits a signal that includes data measured by the sensor unit 102 and has a large capacity and is difficult to transmit by visible light communication via the light emitting unit 106. The communication unit 107 may include a Bluetooth (registered trademark) module, a wireless LAN (Local Area Network) module, and a control circuit for controlling them.

  The imaging device 200 is an imaging device that captures moving image information by imaging a measurement target. The moving image information includes the visible light signal 400 having the measurement information of the measurement object measured by the measurement apparatus 100 and transmitted by the measurement apparatus 100, and the image of the measurement object. The imaging device 200 includes a control unit 201, an imaging unit 202, a storage unit 203, and a communication unit 204. The imaging device 200 can include a video camera and a camera-equipped mobile phone.

  The control unit 201 is a processing unit that controls the photographing apparatus 200 by executing a program. The control unit 201 can include a CPU.

  The imaging unit 202 is an imaging device that captures moving image information by imaging a measurement target. The imaging unit 202 can include a lens, a complementary metal oxide semiconductor (CMOS) image sensor, a charge-coupled device (CCD) image sensor, and an A / D (Analog / Digital) converter. In this specification, as an example, it is assumed that the number of frames per second (fps) of moving image information captured by the imaging unit 202 is 30 fps. The imaging unit 202 functions as an obtaining unit or an imaging unit.

  The storage unit 203 is a storage device that stores data including moving image information acquired by the imaging unit 202. The storage unit 203 also functions as a storage area when the control unit 201 executes processing. The storage unit 203 can include a RAM, a flash memory, and a magnetic disk.

  The communication unit 204 is a communication device that performs wireless communication with the image processing apparatus 300. The communication unit 204 transmits data including moving image information acquired by the imaging unit 202 and stored in the storage unit 203. The communication unit 204 may include a Bluetooth (registered trademark) module, a wireless LAN module, and a control circuit for controlling them.

  The image processing apparatus 300 is an apparatus that displays moving image information obtained by photographing a measurement target and measured data in time synchronization. The image processing apparatus 300 includes a control unit 301, a communication unit 302, an acquisition unit 303, a storage unit 304, a display unit 305, and a reception unit 306. The image processing apparatus 300 can include a server, a personal computer, and a smartphone.

  The control unit 301 is a processing device that controls the image processing device 300 by executing a program. The control unit 301 can include a CPU.

  The communication unit 302 is a communication device that performs wireless communication with the measurement device 100 and the imaging device 200. The communication unit 302 receives a signal transmitted by the communication unit 107 of the measurement apparatus 100 or the communication unit 204 of the imaging apparatus 200. The communication unit 302 may include a Bluetooth (registered trademark) module, a wireless LAN module, and a control circuit for controlling them. The communication unit 302 functions as an obtaining unit or a receiving unit.

  The acquisition unit 303 decodes (decodes) a visible light communication signal recorded in the moving image information, and acquires information. The acquisition unit 303 performs time synchronization. The acquisition unit 303 can include a CPU. The acquisition unit 303 identifies the light emission pattern of the light emitting unit 106 of the measurement device 100 from the moving image information captured by the image capturing device 200 and transmitted to the image processing device 300, and acquires the data by decoding the light emission pattern. Furthermore, the acquisition unit 303 performs time synchronization by a method described later. The acquisition unit 303 functions as an acquisition unit.

  A method in which the acquisition unit 303 specifies a signal transmitted from the light emitting unit 106 from the moving image information will be described. FIG. 3A and FIG. 3B are diagrams for explaining a method by which the acquisition unit 303 specifies a signal. As illustrated in FIG. 3A, the acquisition unit 303 searches (scans) for a color corresponding to the light emission of the light emitting unit 106 from the entire image of one frame having moving image information. The light emission wavelength of the light emitting unit 106, the white balance and sensitivity of the photographing apparatus 200 are known, and the color corresponding to the light emission of the light emitting unit 106 is uniquely determined from these information.

  When a color corresponding to the light emission of the light emitting unit 106 is found, the acquisition unit 303 specifies that the position where the color is found is the position of the measuring apparatus 100 as shown in FIG. The position of the measuring device 100 is tracked. Thereafter, the presence or absence of light emission at the position of each measuring device 100 is acquired for each frame of the moving image information, and the acquired light emission pattern is specified as a signal transmitted by the light emitting unit 106 of the measuring device 100.

  As described above, since the presence / absence of light emission of the light emitting unit 106 is acquired for each frame of moving image information, ideally, one frame of moving image information corresponds to 1 bit of a signal. Therefore, the communication speed (bps) of visible light communication performed by the image processing system 1 ideally matches the number of frames per second (fps) of moving image information captured by the image capturing apparatus 200. In the example of this specification, since the moving image information is 30 fps, the communication speed of visible light communication is 30 bps.

  The storage unit 304 is a storage device that stores the moving image information and data received by the communication unit 302 and the data decoded by the acquisition unit 303. The storage unit 304 also functions as a storage area when the control unit 301 executes processing. The storage unit 304 can include a RAM, a flash memory, and a memory card.

  The display unit 305 is a display device that displays moving image information and data to the user. The display unit 305 displays the moving image information that is recorded by the communication unit 302 and that records the exercise to be measured. The display unit 305 displays data received by the communication unit 302 or decrypted by the acquisition unit 303. Furthermore, the display unit 305 can perform highlighting of a portion where the specific measurement apparatus 100 is located on the screen. The display unit 305 can include a liquid crystal display and a cathode ray tube display. Screen display performed by the display unit 305 will be described later. The display unit 305 functions as a display unit.

  The accepting unit 306 is a accepting device that functions as a user interface and accepts a user instruction. Via the reception unit 306, the user inputs an instruction to select a specific measurement device 100 on the screen. The accepting unit 306 can include a mouse, a touch pad, and a touch panel. The reception unit 306 functions as a reception unit.

  The visible light signal 400 encoded by the encoding unit 104 and transmitted from the light emitting unit 106 will be described with reference to the drawings.

  FIG. 4 is a diagram illustrating a configuration of the visible light signal 400. As shown in FIG. 4, the visible light signal 400 includes a head flag 401, a measurement time 402, a sensor ID 403, analysis data 404, and an alert flag 405.

  The head flag 401 is a 32-bit signal indicating the start of the visible light signal 400. The head flag 401 is included at the head of the visible light signal 400, and the acquisition unit 303 recognizes that the visible light signal 400 is started by the presence of the head flag 401.

  The measurement time 402 is a 32-bit signal (measurement signal) indicating an elapsed time after the measurement apparatus 100 that has transmitted the visible light signal 400 starts measurement. More specifically, the measurement time 402 indicates an elapsed time from when the measurement apparatus 100 starts measurement until the first light emission of the head flag 401.

  FIG. 5 is a diagram for explaining the synchronization performed by the acquisition unit 303. The acquisition unit 303 performs time synchronization between the data measured by the measurement apparatus 100 and the moving picture information photographed by the photographing apparatus 200 from the frame of the moving picture information in which light emission is recorded and the time indicated by the measurement time 402.

  The acquisition unit 303 acquires the measurement time 402. In the example of FIG. 5, the elapsed time indicated by the measurement time 402 is T seconds. A time (T2) that goes back by the elapsed time T seconds indicated by the measurement time 402 from the shooting time (T1) of the frame of the moving image information in which the first light emission of the head flag 401 of the visible light signal 400 including the measurement time 402 is recorded. Is identified. The time is the time when the measurement apparatus 100 starts measurement, and time synchronization is performed by specifying the frame shot at that time.

  Returning to FIG. 4, the sensor ID 403 is a 16-bit signal unique to the measurement apparatus 100 that has transmitted the visible light signal 400, and is a specific signal that identifies the measurement apparatus 100. Even when a plurality of measurement devices 100 are respectively attached to the measurement target, the acquisition unit 303 identifies the measurement device 100 that has transmitted the visible light signal 400 from the sensor ID 403.

  The analysis data 404 is a 16-bit signal (evaluation signal) indicating an evaluation value obtained by the analysis unit 103 of the measurement apparatus 100 analyzing the data. A plurality of pieces of analysis data 404 are included in the visible light signal 400. In the example of FIG. 4, three pieces of analysis data 404A, 404B, and 404C are included.

  The alert flag 405 is a 32-bit signal (alert signal) indicating a flag output when an evaluation value obtained by analysis by the analysis unit 103 of the measurement apparatus 100 exceeds a threshold value. The alert flag 405 includes information indicating which of the plurality of evaluation values has exceeded the threshold value.

  The visible light signal 400 is a 160-bit signal as a whole, and the communication is completed in 5.3 seconds at the communication speed of 30 bps of this embodiment.

  FIG. 6 is a flowchart showing synchronization processing as image processing. The synchronization process executed by the image processing system 1 will be described with reference to the flowchart of FIG.

  The operation of the measuring apparatus 100 will be described. When the measurement start is instructed by the control unit 101, the sensor unit 102 starts measuring the measurement target (step S101). The measured data is stored in the storage unit 105.

  When data measurement is started, the analysis unit 103 analyzes the data and calculates an evaluation value (step S102). The calculated evaluation value is stored in the storage unit 105.

  When the evaluation value is calculated, the encoding unit 104 encodes information and starts generating the visible light signal 400 (step S103). The generated visible light signal 400 is stored in the storage unit 105.

  When the visible light signal 400 is generated, the visible light signal 400 is transmitted by the light emitting unit 106 blinking (step S104).

  When the visible light signal 400 is transmitted, the communication unit 107 transmits the data measured by the sensor unit 102 to the image processing apparatus 300 (step S105). The position at which step S105 is executed is arbitrary, and may be performed before step S101, or may be performed in parallel with step S101 to step S104.

  When step S101 to step S105 are completed, the measuring apparatus 100 ends the operation.

  The operation of the photographing apparatus 200 will be described. When the start of shooting is instructed by the control unit 201, the shooting unit 202 captures the measurement target and acquires moving image information (step S201). The visible light signal 400 transmitted from the light emitting unit 106 of the measuring apparatus 100 is also photographed at the same time. The acquired moving image information is stored in the storage unit 203. Note that step S201 does not have to be performed after step S104 in which the light emitting unit 106 transmits the visible light signal 400, and the visible light signal 400 is transmitted (step S201 is being executed) (ie, during execution of step S201). S104) may be started.

  When the moving image information is acquired, the communication unit 204 transmits the moving image information acquired by the photographing unit 202 to the image processing apparatus 300 (step S202).

  When step S201 to step S202 are completed, the imaging device 200 ends the operation.

  The operation of the image processing apparatus 300 will be described. When the data is transmitted in step S105, the communication unit 302 receives the transmitted data (step S301). The received data is stored in the storage unit 304.

  When the moving image is transmitted in step S202, the communication unit 302 receives the transmitted moving image information (step S302). The received moving image is stored in the storage unit 304.

  When the moving image is received, the acquisition unit 303 identifies the visible light signal 400 recorded in the moving image and performs decoding (step S303). Information obtained by decoding is stored in the storage unit 304. Note that step S302 and step S303 may be performed prior to step S301, or may be performed in parallel with step S301.

  When the data and moving image information are received and the visible light signal 400 is decoded to obtain information, the acquisition unit 303 performs time synchronization between the moving image information and the data or the evaluation value (step S304). Time synchronization is performed from the measurement time 402 and a frame of moving image information in which a signal indicating the measurement time 402 is recorded. The synchronized moving image information and the data or the evaluation value are stored in the storage unit 304 in association with each other.

  When step S301 to step S304 are completed, the image processing apparatus 300 ends the operation.

  When the measurement apparatus 100, the imaging apparatus 200, and the image processing apparatus 300 end the operations, the image processing system 1 ends the synchronization process.

  The screen display performed by the display unit 305 will be described in detail.

  7 to 9 are diagrams showing screens displayed on the display unit 305. FIG. As shown in FIG. 7, when the alert flag 405 is included in the visible light signal 400 transmitted by the measuring device 100, the display unit 305 highlights a portion where the measuring device 100 is located on the screen. . The highlighting is performed by surrounding the part where the measuring apparatus 100 is located with an ellipse. Emphasis display using an ellipse is an example, and can be performed by various methods such as changing the lightness indicated by an arrow.

  The evaluation value is the threshold when the movement information of the measurement target exceeds the appropriate range, the biological information of the measurement target exceeds the proper range, or the environmental information around the measurement target exceeds the proper range. If the threshold value is set to exceed, the movement information of the measurement target is inappropriate, the biological information of the measurement target is inappropriate, or the environmental information around the measurement target is inappropriate. In this case, the measuring device 100 to be mounted on the measurement target is highlighted. In this way, it is possible to clearly indicate the measurement target to be noticed from among the plurality of measurement targets, and to perform efficient guidance.

  As shown in FIGS. 8A and 8B, the display unit 305 displays the evaluation value included in the analysis data 404 transmitted by the measuring apparatus 100 and the moving image information transmitted by the imaging apparatus 200 on the screen. Display superimposed or side-by-side at the same time. As illustrated in FIG. 8A, when the user selects the measurement device 100 on the screen or the measurement target on which the measurement device 100 is mounted via the reception unit 306, the acquisition unit 303 displays the measurement device 100. Elapsed time information is acquired from the measurement time 402 transmitted by 100, and an evaluation value is acquired from the analysis data 404. The control unit 301 controls the display unit 305 after performing the synchronization process, and simultaneously displays the evaluation value and the moving image information superimposed on each other or side by side as shown in FIG. 8B.

  For example, the evaluation value of exercise information is an index that makes it easier for the user to understand the effect of exercise than the data itself. By calculating the evaluation value, the user can be efficiently instructed. In addition, since the analysis data 404 can be acquired from the measurement device 100 on the screen, it is easy to associate the measurement device 100 with the evaluation value. Further, the evaluation value can be displayed only by visible light communication, and wireless communication via the communication unit 107 and the communication unit 302 is not necessary, so that data management is facilitated.

  As shown in FIGS. 9A and 9B, the display unit 305 includes data transmitted from the measurement apparatus 100 via the communication unit 107, and a moving image transmitted from the imaging apparatus 200 via the communication unit 204. , Are superimposed on the screen or displayed side by side at the same time. As illustrated in FIG. 9A, when the user selects the measurement device 100 on the screen or the measurement target on which the measurement device 100 is mounted via the reception unit 306, the acquisition unit 303 displays the measurement device 100. Elapsed time information is acquired from the measurement time 402 transmitted by 100, and information for specifying the measurement apparatus 100 is acquired from the sensor ID 403. The control unit 301 acquires data transmitted from the measurement apparatus 100 via the communication unit 107 from the storage unit 304, performs synchronization processing, and controls the display unit 305, as shown in FIG. 9B. The data and moving picture information are superimposed on the screen or displayed side by side at the same time. The time on the data corresponding to the time on the displayed moving image information is clearly indicated by a line L.

  By performing wireless communication via the communication unit 107 and the communication unit 302, it is possible to perform a more detailed evaluation of the movement of the measurement target using a large amount of data that is difficult to receive or transmit by visible light communication. it can.

  As described above, according to the image processing system 1 according to the present embodiment, the image processing apparatus 300 receives the moving image information photographed by the photographing apparatus 200, thereby measuring information based on the data measured by the measuring apparatus 100 ( Exercise information, biological information or environmental information). Since the measurement information can be acquired by receiving the moving image information, data can be received or transmitted without using a wireless transceiver.

  According to the image processing system 1, it is possible to time-synchronize the data measured by the measuring apparatus 100 and the moving image information photographed by the photographing apparatus 200 in units of one frame. By performing the time synchronization in units of one frame, it is possible to make the user accurately grasp the relationship between the movement to be measured and the data (measurement information). An example of a module that enables accurate time synchronization is GPS (Global Positioning System). In addition to the problems of increasing the size of the device and increasing the cost, GPS cannot acquire the current time indoors, so time synchronization There is a problem that cannot be performed. According to the image processing system 1, accurate time synchronization can be performed even indoors.

  According to the image processing system 1, a moving image and data or an evaluation value can be simultaneously superimposed on the screen or arranged and displayed simultaneously after time synchronization. By displaying the video information and the data or the evaluation value superimposed on each other at the same time or displayed side by side at the same time, the user can simultaneously show the body movement of the measurement target based on the video information and the measurement data obtained from the measurement target. Can facilitate the evaluation and guidance of the exercise of the measurement object.

  According to the image processing system 1, even when a plurality of measuring devices 100 are used, each measuring device 100 can be specified by the sensor ID 403 included in the visible light signal 400. Thereby, for example, even when a plurality of people wear the measuring apparatus 100 and exercise, it is possible to easily specify from which measurement object the data is acquired. It is possible to easily instruct the exercise.

(Embodiment 2)
Hereinafter, an image processing system 1 according to Embodiment 2 of the present invention will be described with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals.

  The sensor unit 102 of the measuring apparatus 100 includes a geomagnetic sensor. The sensor unit 102 measures geomagnetism, and the measured data is stored in the storage unit 105.

  FIG. 10A and FIG. 10B are diagrams for explaining the operation of the image processing system 1 according to the second embodiment. As shown in FIG. 10A, when the sensor unit 102 starts measurement, the analysis unit 103 analyzes the measured geomagnetism and the geomagnetism at the start of measurement, and calculates the angular displacement of the measurement target. The control unit 101 determines whether or not the calculated angular displacement has exceeded a threshold value. In the present embodiment, the threshold for angular displacement is ± 60 degrees.

  When it is determined that the angular displacement of the measurement target does not exceed the threshold value, transmission of the visible light signal 400 by the light emitting unit 106 is continued. When it is determined that the threshold value is exceeded as in the measurement target X in FIG. 10B, the control unit 101 controls the light emitting unit 106 to stop the transmission of the visible light signal 400. If it is determined again that the angular displacement does not exceed the threshold value after the determination that the threshold value is exceeded, the control unit 101 controls the light emitting unit 106 to stop the visible light that has been stopped. The transmission of the optical signal 400 is resumed.

  When the angular displacement of the measuring device 100 increases, the light emitted from the light emitting unit 106 is difficult to be photographed by the photographing device 200. In such a case, the light emission of the light emitting unit 106 is wasted, leading to wasted energy and shortened life. Further, the visible light signal 400 transmitted from the measuring device 100 is not photographed by the photographing device 200 and data is lost.

  The image processing system 1 according to the present embodiment can prevent wasteful light emission and prevent waste of energy and shortening of life by stopping transmission when the angular displacement of the measuring apparatus 100 exceeds a threshold value. . In addition, it is possible to prevent the visible light signal 400 from being lost without being captured by the photographing apparatus 200.

(Modification)
Although several embodiments of the present invention have been described above, the first and second embodiments are examples, and the scope of application of the present invention is not limited thereto. That is, the embodiments of the present invention can be applied in various ways, and all the embodiments are included in the scope of the present invention.

  Although the image processing system 1 includes the measurement device 100, the imaging device 200, and the image processing device 300, the present invention is not limited to this. FIG. 11 is a schematic diagram illustrating a configuration of a modified image processing system. As illustrated in FIG. 11, the image processing system 1 may not include the imaging device 200, the image processing device 300 may include the imaging unit 202, and the image processing device 300 may include the functions of the imaging device 200. Further, the image processing system 1 may not include the image processing apparatus 300, and the photographing apparatus 200 may include the function of the image processing apparatus 300.

  The light emitting unit 106 is a visible light emitting device, but is not limited thereto. Any electromagnetic wave may be used as long as the generated electromagnetic wave can be photographed by the photographing apparatus 200, and for example, an infrared ray generating apparatus may be used. Similarly, what is described as visible light communication or visible light signal in this specification is not limited to that using visible light.

  The communication unit 107 is a communication device that performs wireless communication, but is not limited thereto. For example, a communication device that includes a USB (Universal Serial Bus) module and performs wired communication may be used. Alternatively, data may be stored in a removable storage device and read by another device to exchange data. The same applies to the communication unit 204 and the communication unit 302. In this case, the moving image information is obtained from a removable storage device.

  In addition, the measuring apparatus 100 may not include the communication unit 107. Even if the communication unit 107 is not provided, the measurement information is transmitted via the light emitting unit 106, and the image capturing unit 202 of the image capturing apparatus 200 captures the image, whereby the moving image information and the measurement information can be displayed.

  In visible light communication, one frame of moving image information corresponds to one bit of a signal, but the present invention is not limited to this. For example, the blinking time may be set so that two frames of moving image information become one bit of the signal. With this setting, although the communication speed is halved, it is possible to prevent the flashing from being lost without being recorded in the frame of the moving image information.

  The visible light signal 400 includes the head flag 401, the measurement time 402, the sensor ID 403, the analysis data 404, and the alert flag 405, but is not limited thereto. For example, the visible light signal 400 may include only the head flag 401 and the measurement time 402. Further, the measurement time 402 is not included, and at least one of the sensor ID 403, the analysis data 404, and the alert flag 405 may be included. If the measurement time 402 is included in at least one of the visible light signals 400 transmitted a plurality of times, the image processing system 1 can perform time synchronization between the moving image information and the measurement data.

  The analysis data 404 indicates an evaluation value obtained by analyzing the data by the analysis unit 103 of the measurement apparatus 100, but is not limited thereto. For example, the analysis data 404 may include data itself measured by the sensor unit 102 of the measurement apparatus 100 as measurement information.

  The visible light signal 400 may include an end flag at the end of the signal. The end flag is a 32-bit signal indicating the end of the visible light signal 400. The acquisition unit 303 recognizes that the visible light signal 400 has ended due to the presence of the end flag.

  Although the image processing system 1 performs the synchronization processing as the image processing, it is not limited to this. For example, the visible light signal 400 may not include the measurement time 402 and image processing that does not perform time synchronization may be performed. If analysis data 404 is included in visible light signal 400, measurement information and moving image information can be displayed by capturing visible light signal 400 and receiving moving image information.

  In the image processing system 1 according to the second embodiment, when the visible light signal 400 includes an end flag, the light emitting unit 106 resumes transmission of the stopped visible light signal 400 from the head flag 401. By restarting transmission from the top flag 401, the visible light signal 400 can be reliably recognized by the acquisition unit 303. If the visible light signal 400 is not received for a predetermined time (for example, 1 second) even though the end flag is not received, the acquisition unit 303 determines that the transmission is interrupted, and retransmits the visible light signal 400. Prepare.

  In addition, it is possible to provide an image processing system having a configuration for realizing the functions according to the present invention in advance, and to make an existing information processing apparatus or the like function as the image processing system according to the present invention by applying a program. You can also. That is, by applying a program for realizing each functional configuration by the image processing system 1 exemplified in the first embodiment and the second embodiment so that a CPU or the like that controls an existing information processing apparatus or the like can be executed. The image processing system according to the present invention can function. The image processing method according to the present invention can be implemented using an image processing system.

  Moreover, the application method of such a program is arbitrary. The program can be applied by being stored in a computer-readable storage medium such as a flexible disk, a CD (Compact Disc) -ROM, a DVD (Digital Versatile Disc) -ROM, or a memory card. Furthermore, the program can be superimposed on a carrier wave and applied via a communication medium such as the Internet. For example, the program may be posted on a bulletin board (BBS: Bulletin Board System) on a communication network and distributed. The program may be started and executed in the same manner as other application programs under the control of an OS (Operating System) so that the above-described processing can be executed.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the specific embodiments, and the present invention includes the invention described in the claims and the equivalent scope thereof. included. Hereinafter, the invention described in the scope of claims of the present application will be appended.

(Appendix 1)
An obtaining means for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and including a visible light signal transmitted by the measurement apparatus and an image of the measurement object; ,
Obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means;
Comprising
An image processing apparatus.

(Appendix 2)
The obtaining means is photographing means for photographing the moving picture information.
The image processing apparatus according to appendix 1, wherein:

(Appendix 3)
The obtaining means is a receiving means for receiving the moving picture information from a photographing device that has photographed the moving picture information.
The image processing apparatus according to appendix 1, wherein:

(Appendix 4)
Further comprising display means for simultaneously displaying the moving image information obtained by the obtaining means and the measurement information obtained by the obtaining means,
The image processing apparatus according to any one of appendices 1 to 3, characterized in that:

(Appendix 5)
The obtaining means obtains the moving image information including the visible light signal further having a measurement time when the measurement device measures the measurement information,
The acquisition unit acquires the measurement time from the visible light signal included in the moving image information obtained by the obtaining unit, and temporally combines the measurement information and the moving image information based on the acquired measurement time. Sync to
The image processing apparatus according to any one of appendices 1 to 4, wherein

(Appendix 6)
The obtaining means obtains the moving image information including the visible light signal further including identification information for identifying the measuring device,
The acquisition unit acquires the identification information from the visible light signal included in the moving image information acquired by the acquisition unit.
The image processing apparatus according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
The obtaining means obtains the moving image information including the visible light signal further having an evaluation value calculated by analyzing the measurement information by the measuring device.
The image processing apparatus according to any one of supplementary notes 1 to 6, wherein:

(Appendix 8)
The obtaining means obtains the moving image information including the visible light signal further including an alert flag indicating that the evaluation value calculated by analyzing the measurement information by the measuring device exceeds a threshold value,
The acquisition unit acquires the alert flag from the visible light signal included in the moving image information acquired by the acquisition unit,
The display unit highlights a portion including the measurement device on the moving image information when the acquisition unit acquires the alert flag.
The image processing apparatus according to appendix 4, wherein

(Appendix 9)
A receiving means for receiving an instruction to select a specific measuring device among the plurality of measuring devices;
The display means simultaneously displays the measurement information and the moving image information of the measurement target to which the selected measurement device is attached.
The image processing apparatus according to appendix 4, wherein

(Appendix 10)
The measurement information includes exercise information acquired from the measurement object in which the measurement device is exercising,
The image processing apparatus according to any one of appendices 1 to 9, characterized in that:

(Appendix 11)
The measurement information includes biological information acquired by the measurement device from the measurement target.
The image processing apparatus according to any one of appendices 1 to 10, characterized in that:

(Appendix 12)
The measurement information includes environmental information acquired by the measurement apparatus from the environment around the measurement target.
12. The image processing apparatus according to any one of appendices 1 to 11, wherein

(Appendix 13)
The visible light signal includes a blinking pattern of visible light;
13. The image processing apparatus according to any one of appendices 1 to 12, characterized in that:

(Appendix 14)
The visible light signal includes a plurality of visible lights having different wavelengths.
14. The image processing apparatus according to any one of appendices 1 to 13, characterized in that:

(Appendix 15)
Measuring means attached to the measuring object and measuring the measurement information of the measuring object;
A light emitting means for transmitting a visible light signal having the measurement information measured by the measuring means;
With
The measuring means measures the geomagnetism to obtain the angular displacement of the measurement object,
The light emitting means emits the visible light signal when the angular displacement does not exceed a threshold, and does not emit the visible light signal when the angular displacement exceeds the threshold.
A measuring device.

(Appendix 16)
A measurement device attached to a measurement object to measure measurement information of the measurement object, and transmitting a visible light signal having the measured measurement information;
Obtaining means for obtaining moving image information including the visible light signal and the measurement target image; obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means; An image processing apparatus having
Comprising
An image processing system characterized by that.

(Appendix 17)
Obtaining moving image information including the visible light signal transmitted by the measurement device having the measurement information of the measurement target measured by the measurement device attached to the measurement target, and the video of the measurement target,
Obtaining the measurement information from the visible light signal included in the obtained moving image information;
Including that,
An image processing method.

(Appendix 18)
On the computer,
An acquisition process for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and a visible light signal transmitted by the measurement apparatus, and the image of the measurement object;
An acquisition process for acquiring the measurement information from the visible light signal included in the obtained moving image information;
To execute,
A program characterized by that.

  In addition, the invention described in the scope of claims at the beginning of the filing of the basic application is appended.

(Appendix 101)
Measuring means attached to a moving measuring object and measuring measurement information of the measuring object, and light emission for transmitting a movement signal including movement information based on the measurement information measured by the measuring means by a blinking pattern of visible light A communication means for receiving the moving image from a photographing device having a photographing means for photographing the flashing pattern of the visible light and the measurement object,
Obtaining means for obtaining the exercise information from the blinking pattern of the visible light of the moving image received by the communication means;
Display means for displaying the exercise information acquired by the acquisition means and the moving image received by the communication means in a superimposed manner;
Comprising
An image processing apparatus.

(Appendix 102)
The light emitting means transmits, in addition to the motion signal, a measurement signal including information indicating a measurement time when the measurement means performs measurement of the measurement information by the blinking pattern of the visible light,
The acquisition unit acquires the measurement time from the blinking pattern of the visible light of the moving image received by the communication unit, and temporally synchronizes the exercise information and the moving image based on the acquired measurement time. And
The display means superimposes and displays the motion information synchronized with the time and the moving image.
102. An image processing apparatus according to appendix 101, wherein

(Appendix 103)
The light emitting means emits an identification signal including identification information for identifying the measurement device in addition to the motion signal by the blinking pattern of the visible light,
The acquisition unit acquires the identification information from the blinking pattern of the visible light of the moving image received by the communication unit.
101. The image processing apparatus according to appendix 101 or 102, wherein

(Appendix 104)
The communication means receives the measurement information from the measurement device;
The display means superimposes and displays the measurement information and the moving image;
104. The image processing apparatus according to any one of appendices 101 to 103, wherein

(Appendix 105)
The measurement device further includes an analysis unit that analyzes the measurement information and calculates an evaluation value,
The exercise information includes the evaluation value,
105. The image processing apparatus according to any one of appendices 101 to 104, wherein

(Appendix 106)
The light emitting means transmits an alert signal including information indicating an alert flag when the evaluation value exceeds a threshold value by a blinking pattern of the visible light,
The acquisition unit acquires the alert flag from the blinking pattern of the visible light of the moving image received by the communication unit,
The display means highlights a portion including the measurement device on the moving image when the acquisition means acquires the alert flag.
106. The image processing apparatus according to appendix 105, wherein

(Appendix 107)
A plurality of the measuring devices are respectively attached to the plurality of measuring objects,
Further comprising a receiving means for receiving an instruction to select the measuring device;
The display means superimposes and displays the motion information of the measurement target to which the selected measurement device is attached and the moving image.
107. The image processing apparatus according to any one of appendices 101 to 106, wherein

(Appendix 108)
A measuring means attached to a moving measuring object and measuring measurement information of the measuring object;
A light emitting means for transmitting a motion signal including motion information based on the measurement information measured by the measurement means by a blinking pattern of visible light;
With
The measuring means measures the geomagnetism to obtain the angular displacement of the measurement object,
The light emitting means, when the angular displacement does not exceed a threshold value, performs transmission by the flashing pattern of visible light, and when the angular displacement exceeds a threshold value, does not perform transmission by the visible light flashing pattern,
A measuring device.

(Appendix 109)
Measuring means attached to a moving measuring object and measuring measurement information of the measuring object, and light emission for transmitting a movement signal including movement information based on the measurement information measured by the measuring means by a blinking pattern of visible light A measuring device comprising:
A photographing device having photographing means for photographing the blinking pattern of the visible light transmitted by the measuring device and the measurement object to obtain a moving image;
Communication means for receiving the moving image from the photographing device, acquisition means for acquiring the exercise information from the blinking pattern of the visible light of the moving image received by the communication means, and the exercise information acquired by the acquisition means And a display means for displaying the moving image received by the communication means in a superimposed manner,
Comprising
An image processing system characterized by that.

(Appendix 110)
The light emitting means transmits, in addition to the motion signal, a measurement signal including information indicating a measurement time when the measurement means performs measurement of the measurement information by the blinking pattern of the visible light,
The acquisition unit acquires the measurement time from the blinking pattern of the visible light of the moving image received by the communication unit, and temporally synchronizes the exercise information and the moving image based on the acquired measurement time. And
The display means superimposes and displays the motion information synchronized with the time and the moving image.
Item 110. The image processing system according to item 109.

(Appendix 111)
Measure the measurement information of the measuring object in motion,
A motion signal including motion information based on the measurement information is transmitted by a blinking pattern of visible light,
Shooting the flashing pattern of the visible light and the measurement object to obtain a video,
Obtaining the exercise information from the visible light flashing pattern of the video,
Displaying the measured motion information and the acquired moving image superimposed,
Including that,
An image processing method.

(Appendix 112)
In addition to the movement signal, a measurement signal including information indicating a measurement time when the measurement information was measured is transmitted by the blinking pattern of the visible light,
The measurement time is acquired from the blinking pattern of the visible light of the captured moving image, and the exercise information and the moving image are temporally synchronized based on the acquired measurement time,
Displaying the motion information synchronized with the time and the video in a superimposed manner;
Further including
112. An image processing method according to appendix 111, wherein

(Appendix 113)
On the computer,
Measurement processing to measure the measurement information of the measuring object in motion,
Light emission processing for transmitting a motion signal including motion information based on the measurement information by a blinking pattern of visible light,
Shooting processing for capturing a moving image by shooting the flashing pattern of the visible light and the measurement object by the light emission processing,
An acquisition process for acquiring the exercise information from the blinking pattern of the visible light of the moving image;
Display processing for displaying the measured exercise information and the acquired moving image in a superimposed manner;
To execute,
A program characterized by that.

(Appendix 114)
In the light emission processing, in addition to the motion signal, a measurement signal including information indicating a measurement time when the measurement information is measured is transmitted by the blinking pattern of the visible light,
Acquiring the measurement time from the blinking pattern of the visible light of the moving image shot by the shooting process, and based on the acquired measurement time, the synchronization processing for temporally synchronizing the motion information and the moving image Let the computer run further,
In the display process, the motion information synchronized with the time and the moving image are superimposed and displayed.
114. The program according to appendix 113, wherein

DESCRIPTION OF SYMBOLS 1 ... Image processing system, 100 ... Measuring apparatus, 101 ... Control part, 102 ... Sensor part, 103 ... Analysis part, 104 ... Coding part, 105 ... Memory | storage part, 106 ... Light emission part, 107 ... Communication part, 200 ... Imaging | photography Device 201... Control unit 202 202 Shooting unit 203 Storage unit 204 Communication unit 300 Image processing unit 301 Control unit 302 Communication unit 303 Acquisition unit 304 Storage unit 305 Display unit, 306... Accepting unit, 400... Visible light signal, 401 ... head flag, 402 ... measurement time, 403 ... sensor ID, 404, 404A, 404B, 404C ... analysis data, 405 ... alert flag

Claims (18)

An obtaining means for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and including a visible light signal transmitted by the measurement apparatus and an image of the measurement object; ,
Obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means;
Comprising
An image processing apparatus. The obtaining means is photographing means for photographing the moving picture information.
The image processing apparatus according to claim 1. The obtaining means is a receiving means for receiving the moving picture information from a photographing device that has photographed the moving picture information.
The image processing apparatus according to claim 1. Further comprising display means for simultaneously displaying the moving image information obtained by the obtaining means and the measurement information obtained by the obtaining means,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The obtaining means obtains the moving image information including the visible light signal further having a measurement time when the measurement device measures the measurement information,
The acquisition unit acquires the measurement time from the visible light signal included in the moving image information obtained by the obtaining unit, and temporally combines the measurement information and the moving image information based on the acquired measurement time. Sync to
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The obtaining means obtains the moving image information including the visible light signal further including identification information for identifying the measuring device,
The acquisition unit acquires the identification information from the visible light signal included in the moving image information acquired by the acquisition unit.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The obtaining means obtains the moving image information including the visible light signal further having an evaluation value calculated by analyzing the measurement information by the measuring device.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The obtaining means obtains the moving image information including the visible light signal further including an alert flag indicating that the evaluation value calculated by analyzing the measurement information by the measuring device exceeds a threshold value,
The acquisition unit acquires the alert flag from the visible light signal included in the moving image information acquired by the acquisition unit,
The display unit highlights a portion including the measurement device on the moving image information when the acquisition unit acquires the alert flag.
The image processing apparatus according to claim 4. A receiving means for receiving an instruction to select a specific measuring device among the plurality of measuring devices;
The display means simultaneously displays the measurement information and the moving image information of the measurement target to which the selected measurement device is attached.
The image processing apparatus according to claim 4. The measurement information includes exercise information acquired from the measurement object in which the measurement device is exercising,
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The measurement information includes biological information acquired by the measurement device from the measurement target.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The measurement information includes environmental information acquired by the measurement apparatus from the environment around the measurement target.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The visible light signal includes a blinking pattern of visible light;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. The visible light signal includes a plurality of visible lights having different wavelengths.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Measuring means attached to the measuring object and measuring the measurement information of the measuring object;
A light emitting means for transmitting a visible light signal having the measurement information measured by the measuring means;
With
The measuring means measures the geomagnetism to obtain the angular displacement of the measurement object,
The light emitting means emits the visible light signal when the angular displacement does not exceed a threshold, and does not emit the visible light signal when the angular displacement exceeds the threshold.
A measuring device. A measurement device attached to a measurement object to measure measurement information of the measurement object, and transmitting a visible light signal having the measured measurement information;
Obtaining means for obtaining moving image information including the visible light signal and the measurement target image; obtaining means for obtaining the measurement information from the visible light signal included in the moving image information obtained by the obtaining means; An image processing apparatus having
Comprising
An image processing system characterized by that. Obtaining moving image information including the visible light signal transmitted by the measurement device having the measurement information of the measurement target measured by the measurement device attached to the measurement target, and the video of the measurement target,
Obtaining the measurement information from the visible light signal included in the obtained moving image information;
Including that,
An image processing method. On the computer,
An acquisition process for obtaining moving image information including measurement information of the measurement object measured by a measurement apparatus attached to the measurement object and a visible light signal transmitted by the measurement apparatus, and the image of the measurement object;
An acquisition process for acquiring the measurement information from the visible light signal included in the obtained moving image information;
To execute,
A program characterized by that.

Images