JP2012186535A - Electronic device and grouping method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily group a plurality of electronic devices.SOLUTION: An electronic device 1 comprises: an acquisition unit 10 for acquiring rhythm information representing a temporal or spatial change pattern of an object in a captured image; a calculation unit 20 for calculating the degree of similarity between pieces of rhythm information of respective objects acquired by the acquisition unit 10 from respective captured images captured by different electronic devices; and a grouping unit 30 for generating grouping information for grouping the electronic devices which have captured the respective captured images on the basis of the degree of similarity between the pieces of rhythm information of the respective objects calculated by the calculation unit 20.

Description

  The present invention relates to an electronic device and a grouping method.

  Conventionally, an imaging apparatus that groups input images by evaluating the similarity between the input images has been disclosed (see, for example, Patent Document 1).

JP 2011-010275 A

However, an electronic device that groups images, such as the imaging device disclosed in Patent Document 1, does not group electronic devices. That is, even in the electronic device that performs grouping, the grouping target is an image, and the electronic device itself is not a grouping target. Accordingly, various application processes that can be realized when the electronic devices themselves are grouped (for example, a plurality of electronic devices grouped in the same group output sound in the same manner, emit light in the same manner, or vibrate in the same manner). There is a problem that it is not possible to realize the setting).
The present invention has been made in view of the above problems, and provides a technique for easily grouping a plurality of electronic devices.

  In order to solve the above problem, an electronic device according to one embodiment of the present invention captures an image using an acquisition unit that acquires rhythm information representing a temporal or spatial change pattern of an object in a captured image and a different electronic device. Based on the similarity of the rhythm information of each object calculated by the calculation unit, a calculation unit that calculates the similarity of the rhythm information of each object acquired by the acquisition unit from each captured image, And a grouping unit that generates grouping information for grouping electronic devices that have captured the respective captured images.

  In the electronic device, the acquisition unit includes a reception unit that receives the rhythm information of an object in a captured image captured by another electronic device, and the calculation unit receives the first received by the reception unit. The rhythm information of the object in the first other captured image captured by another electronic device and the second other captured image captured by the second other electronic device received by the receiving unit The degree of similarity of the object in the rhythm information is calculated, and the grouping unit calculates the rhythm information of the object in the first other captured image calculated by the calculator and the second other The first other electronic device and the second other electronic device are grouped into the same group when the similarity of the object in the captured image with the rhythm information is equal to or greater than a predetermined threshold value. It may be.

  In the electronic device, the acquisition unit includes an imaging unit and a reception unit that receives the rhythm information of an object in a captured image captured by another electronic device, and the calculation unit is configured by the imaging unit. Calculating the degree of similarity between the rhythm information of the object in the captured self-captured image and the rhythm information of the object in another captured image captured by another electronic device received by the receiving unit; The grouping unit is configured when the similarity between the rhythm information of the object in the self-captured image calculated by the calculation unit and the rhythm information of the object in the other captured image is equal to or greater than a predetermined threshold. The electronic device and the other electronic device may be grouped into the same group.

  The electronic device may further include a display unit that displays connection information for communicating with other electronic devices grouped into the same group by the grouping unit.

  The electronic device controls the operation of another electronic device so that a plurality of electronic devices grouped in the same group by the grouping unit outputs sound in the same manner, emits light in the same manner, or vibrates in the same manner. You may make it further provide the transmission part which transmits information.

  In order to solve the above problem, a grouping method according to another aspect of the present invention is a grouping method in an electronic device for grouping electronic devices, in which the acquisition unit of the electronic device is an object in a captured image. The rhythm information representing the temporal or spatial change pattern of the object is acquired, and the calculation means of the electronic device acquires the rhythm of each object acquired by the acquisition means from the respective captured images captured by different electronic devices. The similarity of information is calculated, and the electronic device grouping unit groups the electronic devices that captured the respective captured images based on the similarity of the rhythm information of each object calculated by the calculation unit. The grouping information to be generated is generated.

  According to the present invention, it is possible to easily group a plurality of electronic devices.

It is a schematic diagram for demonstrating the outline | summary of the process of the electronic device 1 by one Embodiment of this invention. 1 is a configuration diagram illustrating an example of an electronic device 1. It is an example of the information which the electronic device 1 acquires or memorize | stores. FIG. 11 is a configuration diagram illustrating an example of another electronic device 100. It is explanatory drawing for demonstrating the rhythm information showing the temporal change pattern of an object, the production | generation of the said rhythm information, and the calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the temporal change pattern of an object, the production | generation of the said rhythm information, and the calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the temporal change pattern of an object, the production | generation of the said rhythm information, and the calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the pattern of the spatial change of an object, the production | generation of the said rhythm information, and calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the pattern of the spatial change of an object, the production | generation of the said rhythm information, and calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the pattern of the spatial change of an object, the production | generation of the said rhythm information, and calculation of the similarity in the said rhythm information. It is explanatory drawing for demonstrating the rhythm information showing the pattern of the spatial change of an object, the production | generation of the said rhythm information, and calculation of the similarity in the said rhythm information. 3 is a flowchart illustrating an example of an operation of the electronic device 1. It is a schematic diagram for demonstrating the result of grouping by the electronic device.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram for explaining an outline of processing of the electronic apparatus 1 according to an embodiment of the present invention.

The electronic device 1 (for example, portable electronic device) groups a plurality of electronic devices (for example, portable electronic devices).
For example, the electronic device 1 groups a plurality of other electronic devices 100 existing around the electronic device 1 as shown in FIG. That is, the electronic device 1 sets a plurality of electronic devices 100 excluding its own electronic device 1 as a grouping target as illustrated by a broken line.
Further, for example, the electronic device 1 groups the electronic device 1 and a plurality of other electronic devices 100 existing around the electronic device 1 as shown in FIG. In other words, the electronic device 1 targets a plurality of electronic devices (the electronic device 1 and the electronic device 100) including the own electronic device 1 as shown in the broken line.

  Note that each of the electronic device 1 and the plurality of other electronic devices 100 automatically exchange mutual connection information (described later) by communication when approaching within a predetermined distance, for example.

  In the following, the mode shown in FIG. 1A (a mode in which a plurality of electronic devices 100 excluding the electronic device 1 are targets of grouping) will be described. The mode shown in FIG. 1B (a mode in which a plurality of electronic devices (the electronic device 1 and the electronic device 100) including the own electronic device 1 are grouped) will be described later.

  FIG. 2 is a configuration diagram illustrating an example of the electronic apparatus 1. FIG. 3 is an example of information acquired or stored by the electronic device 1. As shown in FIG. 2A, the electronic device 1 includes an acquisition unit 10, a calculation unit 20, a grouping unit 30, a display unit 40, a transmission unit 50, an operation control unit 60, a rhythm information storage unit 90, and a connection information storage. Unit 92, grouping information storage unit 94, and operation control information storage unit 96.

  The acquisition unit 10 includes a reception unit 12 as illustrated in FIG. The acquisition unit 10 may include a rhythm information generation unit 16 in addition to the reception unit 12 as illustrated in FIG. Further, as illustrated in FIG. 2D, the acquisition unit 10 may include an imaging unit 14 and a rhythm information generation unit 16 in addition to the reception unit 12. However, in the case of the mode shown in FIG. 1A, the imaging unit 14 and the rhythm information generation unit 16 are not essential, and thus the description will be made assuming that the imaging unit 14 and the rhythm information generation unit 16 are not provided.

  The acquisition unit 10 acquires rhythm information representing a temporal or spatial change pattern of an object in a captured image captured by another electronic device 100. Specifically, as shown in FIG. 3A, the receiving unit 12 in the acquisition unit 10 receives the other electronic device 100 together with the device ID and connection information of the other electronic device 100, as shown in FIG. The rhythm information of the object in the captured image captured by the electronic device 100 is received. The device ID is identification information for identifying each electronic device 100. The connection information is information for communicating with each of the electronic devices 100, and is, for example, information related to a protocol, address information, or the like. The rhythm information will be described in detail with reference to FIGS.

  In the case where one other electronic device 100 does not transmit connection information together with the rhythm information to the electronic device 1 (for example, before the transmission of the rhythm information, the other electronic device 100 has already been transmitted together with the device ID). When the connection information is transmitted (exchanged) to the electronic device 1), the receiving unit 12 receives the rhythm information of the other electronic device 100 together with the device ID of the other electronic device 100. That is, in the above case, the receiving unit 12 does not receive connection information together with rhythm information.

  The acquisition unit 10 (reception unit 12) that acquired the rhythm information stores the acquired (received) rhythm information in the rhythm information storage unit 90. For example, as illustrated in FIG. 3B, the acquisition unit 10 stores rhythm information in association with the device ID.

  The acquisition unit 10 that acquired the rhythm information stores the connection information acquired together with the rhythm information in the connection information storage unit 92. For example, as illustrated in FIG. 3C, the acquisition unit 10 stores connection information in association with the device ID. In addition, before acquiring the rhythm information from one other electronic device 100, the acquisition unit 10 has already acquired (exchanged) the connection information of the other electronic device 100 together with the device ID. At the time of acquisition (exchange), connection information is stored in association with the device ID.

  The calculation unit 20 calculates the similarity of a plurality of rhythm information stored in the rhythm information storage unit 90. That is, the calculation unit 20 includes the rhythm information of the object in the first other captured image captured by the first other electronic device 100 received by the reception unit 12 and the second rhythm information received by the reception unit 12. The similarity with the rhythm information of the object in the second other captured image captured by the other electronic device 100 is calculated. The calculation of the similarity will be described in detail with reference to FIGS.

  The timing at which the calculation unit 20 starts calculating the similarity is not particularly limited. For example, the calculation unit 20 calculates the similarity when an operation reception unit (not shown) receives a predetermined user operation. May be.

  As described above, the calculation unit 20 that has calculated the similarity outputs the calculation result of the similarity to the grouping unit 30.

  The grouping unit 30 groups other electronic devices 100 based on the rhythm information similarity (similarity calculation result) calculated by the calculation unit 20 (that is, grouping information that is a grouping result). (Described below)). More specifically, the grouping unit 30 calculates the similarity between the rhythm information of the object in the first other captured image calculated by the calculation unit 20 and the rhythm information of the object in the second other captured image. Is equal to or greater than a predetermined threshold, the first other electronic device 100 and the second other electronic device 100 are grouped into the same group.

For example, for the electronic devices A, B, and C, the grouping unit 30 includes the rhythm information of the objects in the captured image captured by the electronic device A and the rhythm information of the objects in the captured image captured by the electronic device B. The similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device C is greater than or equal to a predetermined threshold. Is equal to or greater than a predetermined threshold, and the degree of similarity between the rhythm information of the object in the captured image captured by the electronic device B and the rhythm information of the object in the captured image captured by the electronic device C is predetermined. When it is equal to or greater than the threshold, the electronic devices A, B, and C are grouped into the same group.
That is, the grouping unit 30 has a similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device B is equal to or greater than a predetermined threshold. And the degree of similarity between the rhythm information of the object in the captured image captured by the electronic device B and the rhythm information of the object in the captured image captured by the electronic device C is equal to or greater than a predetermined threshold. If the degree of similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device C is not greater than or equal to a predetermined threshold, the electronic device A , B and C are not grouped into the same group.

  Note that the grouping unit 30 has a similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device B being equal to or greater than a predetermined threshold. And the similarity between the rhythm information of the object in the captured image captured by the electronic device B and the rhythm information of the object in the captured image captured by the electronic device C is equal to or greater than a predetermined threshold, If the degree of similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device C is not equal to or greater than a predetermined threshold, the electronic device Rhythm information of the object in the captured image captured by A and the object in the captured image captured by the electronic device B The similarity between the rhythm information of the object and the rhythm information of the object in the captured image captured by the electronic device B and the similarity of the rhythm information of the object within the captured image captured by the electronic device C; Either one of the electronic device A or the electronic device C and the electronic device B may be grouped into the same group.

  That is, the grouping unit 30 uses the electronic device B to determine the similarity between the rhythm information of the object in the captured image captured by the electronic device A and the rhythm information of the object in the captured image captured by the electronic device B. If the rhythm information of the object in the captured image is higher than the similarity between the rhythm information of the object in the captured image captured by the electronic device C, the electronic devices A and B are grouped into the same group. The similarity between the rhythm information of the object in the captured image captured by the electronic device B and the rhythm information of the object in the captured image captured by the electronic device C is the same as that in the captured image captured by the electronic device A. Similarity between the rhythm information of the object and the rhythm information of the object in the captured image captured by the electronic device B Remote is higher, the electronic device B, may be grouped into the same group of C.

  Note that the timing at which the grouping unit 30 starts grouping the plurality of other electronic devices 100 is not particularly limited. For example, when the grouping unit 30 acquires the similarity calculation result from the calculation unit 20, You may group.

  As described above, the grouping unit 30 that groups a plurality of other electronic devices 100 stores the grouping information in the grouping information storage unit 94. For example, as illustrated in FIG. 3D, the grouping unit 30 stores the group ID and the belonging device information in association with each other as grouping information in the grouping information storage unit 94. The group ID is identification information for identifying each group. The belonging device information is information including a device ID for identifying the electronic devices 100 belonging to the group (that is, the electronic devices 100 grouped in the group).

  The display unit 40 displays various information. For example, the display unit 40 displays connection information for communicating with other electronic devices 100 grouped into the same group by the grouping unit 30. Specifically, the display unit 40 refers to the connection information storage unit 92 and the grouping information storage unit 94, and, for each group, the device ID of the other electronic device 100 grouped in the group and the other Connection information for communicating with the electronic device 100 is displayed. The display unit 40 may display only the device ID for each group without displaying the connection information.

  The timing at which the display unit 40 starts the above-described display is not particularly limited. For example, the display unit 40 may display when the grouping information is stored in the grouping information storage unit 94. The display unit 40 may display when an operation receiving unit (not shown) receives a predetermined user operation.

  Further, the display unit 40 displays the operation control information stored in the operation control information storage unit 96. FIG. 3E is an example of operation control information stored in the operation control information storage unit 96.

  The operation control information is control information for controlling the operations of the electronic devices 1 and 100. For example, as shown in FIG. 3E, the audio output control information for controlling the audio output and the light emission are controlled. One or more control information of the light emission control information to perform and the vibration control information for controlling a vibration are included. Each control information (audio output control information, light emission control information, vibration control information) has a plurality of settable values (or settable ranges). Note that the operation control information illustrated in FIG. 3E is an example, and may include other control information for controlling other operations of the electronic devices 1 and 100.

  The timing at which the display unit 40 starts displaying the operation control information is not particularly limited. For example, the display unit 40 displays the operation control information when an operation receiving unit (not shown) receives a predetermined user operation. It may be displayed. For example, before the transmission unit 50 transmits the operation control information, the display unit 40 displays a button (corresponding to the operation reception unit) for confirming a value (setting value) to be transmitted, and by pressing the button, You may display the value to transmit before transmission.

  The transmission unit 50 transmits various information. For example, the transmission unit 50 transmits the grouping information to the other electronic device 100. Specifically, the transmission unit 50 refers to the grouping information storage unit 94 and, for each group of other electronic devices 100 grouped in the group, the other electronic devices 100 belonging to the group. The device ID is transmitted.

  In addition, the transmission unit 50 may be configured so that the plurality of electronic devices 100 grouped into the same group by the grouping unit 30 outputs sound in the same manner, emits light in the same manner, or vibrates in the same manner. The operation control information is transmitted to 100. Specifically, the transmission unit 50 refers to the connection information storage unit 92, the grouping information storage unit 94, and the operation control information storage unit 96, and the other electronic devices 100 grouped into the group are grouped by group. The operation control information is transmitted to each of the other electronic devices 100 grouped in each group so that each outputs sound similarly, similarly emits light, or similarly vibrates.

Groups For example, the transmitting unit 50 to each of the plurality of electronic devices 100 that are grouped into groups G _1, the same set value (e.g., set value a) transmits the audio output control information, the group G ₂ The audio output control information having the same setting value (for example, setting value b) is transmitted to each of the plurality of electronic devices 100 that have been converted into the same. Thus, the grouped respective electronic device 100 to the group G _1, and outputs sound in accordance with control by the set value a, grouped respective electronic device 100 to the group G _2, under the control of the set value b Sound will be output. The same applies when transmitting light emission control information or vibration control information.

  The timing at which the transmission unit 50 starts transmitting the operation control information is not particularly limited. For example, the transmission unit 50 receives the operation control information when an operation reception unit (not shown) receives a predetermined user operation. You may send it.

  The operation control unit 60 controls various operations based on the operation control information stored in the operation control information storage unit 96. For example, control is performed so that sound is output according to the setting value set by the setting unit (not shown) among the settable values of the audio output control information, and the setting unit (not shown) among the settable values of the light emission control information. Control is performed so that light is emitted according to the set value, and vibration is controlled according to the set value set by a setting unit (not shown) among the settable values of the vibration control information.

  The setting unit (not shown) stores the setting value in the operation control information storage unit 96 when the operation receiving unit (not shown) receives a user operation for specifying a setting value.

  Next, another electronic device 100 that is a target of grouping by the electronic device 1 will be described. FIG. 4 is a configuration diagram illustrating an example of another electronic device 100. As shown in FIG. 4A, the other electronic device 100 includes an imaging unit 114, a rhythm information generation unit 116, a display unit 140, a transmission / reception unit 150, an operation control unit 160, and an operation control information storage unit 196. . FIG. 4B shows another configuration of the electronic device 100 (described later).

  The imaging unit 114 captures still images and moving images. The rhythm information generation unit 116 extracts an object from the moving image captured by the imaging unit 114, and generates rhythm information of the object extracted from the moving image. The rhythm information generation unit 116 that has generated the rhythm information outputs the generated rhythm information to the transmission / reception unit 150. The generation of rhythm information by the rhythm information generation unit 116 (the same applies to the rhythm information generation unit 16 (described later)) will be described in detail with reference to FIGS.

  The transmission / reception unit 150 transmits / receives various information to / from other electronic devices (electronic device 1, other electronic device 100). For example, the transmission / reception unit 150 transmits the rhythm information acquired from the rhythm information generation unit 116 together with the device ID and connection information of the electronic device 100 to the electronic device 1.

  Note that if the transmission / reception unit 150 has already transmitted (exchanged) the connection information together with the device ID to the electronic device 1 before transmitting the rhythm information to the electronic device 1, the transmission / reception unit 150 transmits the connection information together with the rhythm information. You do not have to send it.

  For example, the transmission / reception unit 150 receives the grouping information from the electronic device 1. When receiving the grouping information, the transmission / reception unit 150 outputs the received grouping information to the display unit 140.

  For example, the transmission / reception unit 150 receives operation control information including a setting value from the electronic device 1. When receiving the operation control information, the transmission / reception unit 150 stores (updates) the setting value of the operation control information in the operation control information storage unit 196 via a setting unit (not shown). The configuration of the operation control information storage unit 196 is the same as that of the operation control information storage unit 96 of the electronic device 1 (see FIG. 3E).

  The display unit 140 displays various information. For example, grouping information and operation control information received from the electronic device 1 are displayed.

  The operation control unit 160 controls various operations based on the operation control information stored in the operation control information storage unit 196. For example, control is performed so that sound is output according to the setting value set by the setting unit (not shown) among the settable values of the audio output control information, and the setting unit (not shown) among the settable values of the light emission control information. Control is performed so that light is emitted according to the set value, and vibration is controlled according to the set value set by a setting unit (not shown) among the settable values of the vibration control information.

  That is, the other electronic device 100 that has received the operation control information (setting value) transmitted from the electronic device 1 operates according to the operation control information (setting value). Therefore, if audio output control information having the same setting value is transmitted to a plurality of electronic devices 100 grouped in the same group, the plurality of electronic devices output sound in the same manner, and light emission control having the same setting value is performed. If the information is transmitted, the plurality of electronic devices similarly emit light, and if the vibration control information having the same set value is transmitted, the plurality of electronic devices vibrate in the same manner.

  When the acquisition unit 10 of the electronic device 1 includes the rhythm information generation unit 16 (in the case of the configuration shown in FIG. 2C or FIG. 2D), the rhythm information can be generated by the electronic device 1. The other electronic device 100 may not include the rhythm information generation unit 116 as illustrated in FIG.

  That is, when the acquisition unit 10 of the electronic device 1 has the rhythm information generation unit 16 and the other electronic device 100 has the rhythm information generation unit 116, the transmission / reception unit 150 of the other electronic device 100 A moving image captured by the imaging unit 114 is transmitted to the electronic device 1 together with the device ID and connection information. In the electronic device 1, the receiving unit 12 receives a moving image from the other electronic device 100, and the rhythm information generating unit 16 extracts an object from the moving image received by the receiving unit 12, and uses the extracted rhythm information of the object. Generated and stored in the rhythm information storage unit 90 in association with the device ID.

  Hereinafter, with reference to FIG. 5 to FIG. 11, rhythm information that represents a temporal or spatial change pattern of an object, generation of rhythm information that represents a temporal or spatial change pattern of an object, The calculation of the similarity in the rhythm information representing the spatial change pattern will be described in detail. Specifically, first, using FIG. 5 to FIG. 7, rhythm information that represents a temporal change pattern of an object, generation of rhythm information that represents a temporal change pattern of the object, and temporal change of the object The calculation of the similarity in the rhythm information representing the pattern of the object will be described in detail, and then the rhythm information representing the pattern of the spatial change of the object and the pattern of the spatial change of the object will be described with reference to FIGS. The generation of the rhythm information representing the rhythm information and the calculation of the similarity in the rhythm information representing the spatial change pattern of the object will be described in detail.

  5 to 7 are explanatory diagrams for explaining rhythm information representing a temporal change pattern of an object, generation of the rhythm information, and calculation of similarity in the rhythm information.

  The rhythm information generation unit 116 of the other electronic device 100 (the same applies to the rhythm information generation unit 16 of the electronic device 1) extracts the object from the moving image, the amount of change in the area of the circumscribed figure circumscribing the extracted object, and the long side Or use the amount of change in the length of the short side, the amount of change in the aspect ratio, the period of change in area, the period of change in length, or the period of change in aspect ratio to change the pattern of temporal change of the object. Generate rhythm information to represent.

For example, the rhythm information generation unit 116 uses one or more parameter values of parameters 1 to 12 (hereinafter referred to as prm1 to prm12) exemplified below to represent rhythm information representing a temporal change pattern of the object. Is generated. In addition, the predetermined time in prm1 to prm12 is, for example, a time (for example, one period) based on the period of change of the circumscribed rectangle. In addition, the long side and the short side in prm7-1 to prm9-2 are determined based on the length of a certain reference time (for example, the beginning of one cycle). For convenience, the Y-axis direction (or X-axis direction) may be determined as the long side.
(Parameters that make up rhythm information)
prm1: difference between the maximum area and the minimum area of the circumscribed rectangle within a predetermined time prm2: area ratio between the maximum area and the minimum area of the circumscribed rectangle within a predetermined time prm3-1: the average area and the maximum area of the circumscribed rectangle within a predetermined time Difference prm3-2: Difference between average area and minimum area of circumscribed rectangle within a predetermined time prm4-1: Area ratio between average area and maximum area of circumscribed rectangle within a predetermined time prm4-2: Average of circumscribed rectangle within a predetermined time Area ratio of area to minimum area prm5: Distribution of circumscribed rectangle area within a predetermined time (example: standard deviation)
prm6: Period of change in area of circumscribed rectangle within a predetermined time prm7-1: Maximum change amount of long side of circumscribed rectangle within a predetermined time prm7-2: Maximum change amount of short side of circumscribed rectangle within a predetermined time prm8− 1: Longitudinal distribution of circumscribed rectangles within a specified time (example: standard deviation)
prm8-2: Distribution of short sides of circumscribed rectangle within a predetermined time (example: standard deviation)
prm9-1: period of change of the long side of the circumscribed rectangle within a predetermined time prm9-2: period of change of the short side of the circumscribed rectangle within a predetermined time prm10: maximum change amount prm11 of the aspect ratio of the circumscribed rectangle within the predetermined time : Circumference rectangle aspect ratio distribution within a specified time (eg, standard deviation)
prm12: Period of change in aspect ratio of circumscribed rectangle within a predetermined time

Hereinafter, generation of rhythm information by the rhythm information generation unit 116 will be described using a specific example. P ₁ shown in FIG. 5A is one frame constituting a moving image, and is captured at the moment when a person (object O ₁ ) shakes both hands and feet. P ₃ shown in FIG. 5C is one frame constituting a moving image, and is captured at the moment when a person (object O ₁ ) swings down both hands and feet. P ₂ shown in FIG. 5B is one frame between P ₁ and P ₃ .

Figure 6 _{E 1} shown in (a) is a circumscribed rectangle circumscribing the object _{O 1} in the _{P 1} shown in Figure 5 (a). E ₂ shown in FIG. 6B is a circumscribed rectangle circumscribing the object O ₁ in P ₂ shown in FIG. 5B. E ₃ shown in FIG. 6C is a circumscribed rectangle circumscribing the object O ₁ in P ₃ shown in FIG. 5C. Incidentally, FIG. 6 (d) is obtained by comparing the size of the circumscribed rectangle _{E 1, E 2, E 3} . As shown in FIG. 6D, the shape of the circumscribed rectangle that circumscribes the object changes according to the movement of the object.

The calculating unit 20 sequentially extracts the object O ₁ from the moving images (P ₁ , P ₂ , P ₃ ,...), And sequentially extracts the circumscribed figure circumscribing the object O ₁ as shown in FIG. Then, as shown in FIG. 7B, the size of the circumscribed rectangle to be sequentially extracted is calculated. Note that the “period” shown in FIG. 7A indicates the period of change in the shape of the circumscribed rectangle. That is, time t ₁ to time t ₄ (time t ₅ to time t ₈ , time t ₉ to time t ₁₃ ,...) Is one cycle.

The rhythm information generation unit 116 calculates a predetermined one or more parameters using the size of the circumscribed rectangle, and sets a numerical value group including the calculated parameter values as elements as a temporal change pattern of the object O _1. Rhythm information to represent. For example, the rhythm information generating unit 116, as shown in FIG. 7 (c), the area ratio of the maximum area and the minimum area of circumscribed rectangle of the object _{O 1} (Prm2), the period of change in area (PRM6), the long sides of A pattern of temporal changes of the object O ₁ is represented by a numerical group having values of the maximum change amount (prm7-1), the short side maximum change amount (prm7-1), and the aspect ratio maximum change amount (prm10). Is represented as rhythm information R ₁ (prm2, prm6, prm7-1, prm7-2, prm10).

  The rhythm information may be obtained by appropriately rounding the value of each parameter for easy comparison later, or by replacing the value of each parameter with another value (score May be used).

  As described above, when the rhythm information generation unit 116 generates rhythm information including a numerical value group having the values of the parameters as elements, the calculation unit 20 of the electronic device 1 uses the first other electronic device 100. The degree of similarity between the rhythm information of the object in the first other captured image captured and the rhythm information of the object in the second other captured image captured by the second other electronic device 100 is calculated. . Specifically, the calculation unit 20 calculates the similarity between the same parameters constituting the rhythm information, and based on the similarity between the plurality of the same parameters, the objects in the first other captured image The degree of similarity between the rhythm information and the rhythm information of the object in the second other captured image is calculated.

  For example, when the rhythm information is composed of prm2, prm6, prm7-1, prm7-2, and prm10, the calculation unit 20 determines the value of prm2 constituting the rhythm information of the object in the first other captured image. And the value of prm2 constituting the rhythm information of the object in the second other captured image are compared, and the similarity between the values of prm2 is calculated. Similarly, the calculation unit 20 calculates the similarity between the values of prm6, the similarity between the values of prm7-1, the similarity between the values of prm7-2, and the similarity between the values of prm10.

  Various calculation methods are conceivable as a method for calculating the degree of similarity between the values of prm2 (also for prm6, prm7-1, prm7-2, and prm10). As an example, the calculation unit 20 may include the first other imaging. A value obtained by comparing the value of prm2 constituting the rhythm information of the object in the image with the value of prm2 constituting the rhythm information of the object in the first other captured image, and dividing one of the smaller values by the other larger May be the degree of similarity between the values of prm2. In the case of the calculation method, the similarity between the parameter values is in the range of 0 to 1, and the closer the value is, the closer the value is to 1.

  Also, a calculation for calculating the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image based on the similarity between the values of the parameters. Although various calculation methods can be considered as the method, for example, the calculation unit 20 calculates the average value of the similarity between the parameters, the rhythm information of the object in the first other captured image, and the second other It is good also as similarity with the rhythm information of the object in the picked-up image. That is, in the above example, the calculation unit 20 calculates the similarity between the values of prm2, the similarity between the values of prm6, the similarity between the values of prm7-1, the similarity between the values of prm7-2, The average value of similarities between values may be the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image. In the case of the calculation method, the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image is in the range of 0 to 1 and is similar. The value is closer to 1.

  When calculating the above average value, the average value may be calculated after weighting the degree of similarity between predetermined parameters. The average value is not limited to an arithmetic average, and a geometric average may be used. Also, instead of the average value, the median value of the similarity between parameters, the maximum value of the similarity between parameters, and the minimum value of the similarity between parameters are used as the rhythm information of the object in the first other captured image. The similarity with the rhythm information of the object in the second other captured image may be used.

  The case where the rhythm information represents the temporal change pattern of the object has been described above. Next, the case where the rhythm information represents the spatial change pattern of the object will be described. 8 to 11 are explanatory diagrams for explaining rhythm information representing a spatial change pattern of an object, generation of the rhythm information, and calculation of similarity in the rhythm information.

FIG. 8 schematically shows a traffic light object (O ₂ ) extracted from moving images (P ₁₁ , P ₁₂ , P ₁₃ ). 8 (a) is the object O ₂ when traffic is _blue, FIG. 8 (b) object O ₂ when traffic is _yellow, the object O ₂ when FIG. 8 (c) traffic light is red Is shown. 8A to 8C, r1 is an imaging area of the traffic signal body, and r2 is an imaging area of a support portion that supports the traffic signal body. r1-1 is an area within r1, which is an imaging area of a holding section that holds a blue lamp, r1-2 is an area within r1, and is an imaging area of a holding section that holds a yellow lamp, and r1-3 is within r1 This is an imaging region of a holding unit that holds a red lamp. r1-1-1 is an area within r1-1 and is an imaging area of a blue lamp, r1-2-1 is an area within r1-2 and is an imaging area of a yellow lamp, and r1-3-1 is r1- 3 is a red lamp area.

For convenience of explanation, it is assumed that when the traffic light is blue, the color of the blue lamp being turned on is blue-green, and the colors of the yellow lamp and red lamp being turned off are black. That is, in FIG. 8A, the color of the blue lamp region r1-1-1 is blue-green, the color of the yellow lamp region r1-2-1 is black, and the color of the red lamp region r1-3-1 is It shall be black. When the traffic light is yellow, the color of the yellow lamp that is lit is yellow, and the color of the blue and red lamps that are not lit is black. That is, in FIG. 8B, the color of the blue lamp region r1-1-1 is black, the color of the yellow lamp region r1-2-1 is yellow, and the color of the red lamp region r1-3-1 is black. Suppose that When the traffic light is red, the color of the red lamp being turned on is red, and the color of the blue lamp and the yellow lamp being turned off is black. That is, in FIG. 8C, the color of the blue lamp region r1-1-1 is black, the color of the yellow lamp region r1-2-1 is black, and the color of the red lamp region r1-3-1 is red. Suppose that
In addition, when the traffic light is any of blue, yellow, and red, all the areas other than the lamp are gray.

9 (a) it is intended to the pixel group (unit area) that constitute the object O ₂ schematically illustrates. A pixel group is composed of a predetermined number of adjacent pixels.

FIG. 9 (b), as shown in FIG. 9 (a), in the case of dividing the object O ₂ into a plurality of pixel groups, a rhythm information representing the pattern of the spatial variation of the object O _2. Pixel group IDs (a-4, a-5,...) Shown in FIG. 9B are identification information for identifying pixel groups constituting the object O ₂ (that is, the pixel groups shown in FIG. 9A). is there. For example, the pixel group ID “a-4” shown in FIG. 9B is defined by the pixel group of the symbol G shown in FIG. 9A (the horizontal index “4” and the vertical index “a”). Pixel group).

Each time (t1, t2,...) Shown in FIG. 9B is the imaging timing of the traffic light shown in FIG. t1 to t3 are imaging timings when the signal is blue as shown in FIG. t4 is the imaging timing when the signal is yellow as shown in FIG. t5 to t7 are imaging timings when the signal is red as shown in FIG. That, t1 to t7 is one period of the change in the color of the object _{O 2.} Note that the time shown in FIG. 9B is a time for convenience of explanation (in the case of an actual traffic light, the blue (and red) time is usually longer than the yellow time.

Each value (D1 to D7) shown in FIG. 9B is a value constituting rhythm information, and is an average pixel of each pixel group constituting the object O ₂ at each imaging timing (t1, t2,...). Value. D1 is a pixel value indicating gray, D2 is a pixel value indicating blue-green, D3 is a pixel value indicating black, D4 is a pixel value indicating black, D5 is a pixel value indicating yellow, and D6 is a pixel value indicating black , D7 are pixel values representing red.

That is, the rhythm information shown in FIG. 9B is a color change pattern for each pixel group constituting the object O ₂ , and shows, for example, the following features 1 to 10.

Feature 1: Of the main part of the object O ₂ (region r1 shown in FIG. 8), the region (region r1 shown in FIG. 8) located on the left side of the central region (region r1-2-1 shown in FIG. 8). -1-1), the color periodically changes between blue-green (D2) and black (D3).
Feature 2: The color of the central region of the main portion of the object O ₂ periodically changes between black (D4) and yellow (D5).
Feature 3: Of the main part of the object O ₂ , the region located on the right side of the central region (region r1-3-1 shown in FIG. 8) is black (D6) and red (D7). Changes periodically.
Feature 4: Out of the main area of the object O ₂ , an area excluding the central area, the area located on the left side of the central area, and the area located on the right side of the central area (from the area r1 shown in FIG. 8 to the area) The area excluding r1-1-1, the area r1-2-1, and the area r1-3-1) is always gray (D1) and has no color change.
Feature 5: An area other than the main part of the object O ₂ (area r2 shown in FIG. 8) is always gray (D1) and has no color change.
Feature 6: After the region located on the left side of the central region (region r1-1-1) changes from blue-green (D2) to black (D3), the central region (region r1-2-1) is black ( The color changes from D4) to yellow (D5).
Feature 7: After the central region (region r1-2-1) has changed from yellow (D5) to black (D4), the region (region r1-3-1) located on the right side of the central region is black (D6 ) To red (D7).
Feature 8: After the area located on the right side of the central area (area r1-3-1) changes from red (D7) to black (D6), the area located on the left side of the central area (area r1-1-) 1) changes from black (D3) to blue-green (D2).
Feature 9: a region (region r1-1-1) located on the left side of the central region that changes to blue-green (D2), a central region (region r1-2-1) that changes to yellow (D5), The region (region r1-3-1) located on the right side of the central region that changes to red (D7) has substantially the same size.
Feature 10: The time when the region (region r1-1-1) located on the left side of the central region is blue-green (D2), and the region located on the right side of the central region (region r1-3-1) is red The time for (D7) is equal and is approximately three times the time for the center (region r1-2-1) region to be yellow (D5).

The rhythm information generation unit 116 of the other electronic device 100 (the same applies to the rhythm information generation unit 16 of the electronic device 1) extracts an object from the moving image, and generates rhythm information representing a pattern of spatial change of the object. . For example, when the rhythm information generation unit 116 extracts the object O ₂ , as shown in FIG. 9B, the numerical value having each value indicated by the color change for each pixel group constituting the object O ₂ as an element. Rhythm information composed of

  When the rhythm information generation unit 116 generates rhythm information of each object as described above, the calculation unit 20 of the electronic device 1 uses the first other captured image captured by the first other electronic device 100. The similarity between the rhythm information of the object and the rhythm information of the object in the second other captured image captured by the second other electronic device 100 is related to the one or more features described above. Calculate based on

  For example, the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image is calculated using the similarity related to the feature 2 described above. May be. In this case, the calculation unit 20 may, for example, periodically change the color of the central region of the main part of the object indicated by the rhythm information of the object in the first other captured image, and in the second other captured image. The similarity with the periodic change of the color of the central region of the main part of the object indicated by the rhythm information of the object is calculated, the rhythm information of the object in the first other captured image, and the second other It is good also as similarity with the rhythm information of the object in a captured image.

  Various calculation methods are conceivable as the calculation method of “similarity of periodic change in color of the central region of the main part”. As an example, the calculation unit 20 may be configured to “change the color change of the central region of the main part. Based on the “similarity of the period” and the “similarity of the color appearing in the central area of the main part”, the “similarity of the periodic change in the color of the central area of the main part” may be calculated. Note that the calculation unit 20 uses the “rhythm change period of the color of the central region of the main part” based on the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image. Compared with “the color change period of the central area of the main part”, the value obtained by dividing the shorter period by the longer period is also used as the “similarity of the color change period of the central area of the main part” Good.

  For example, “the color change period of the central region of the main part” according to the rhythm information of the object in the first other captured image is 4 seconds, and “the main part according to the rhythm information of the object in the second other captured image” When the period of the color change in the central area of 3 seconds is 3 seconds, the value “0.75” obtained by dividing the period “3 (seconds) by the period“ 4 (seconds) ”is set as“ the color of the central area of the main part ”. The similarity of the period of the change. In the case of the calculation method, the “similarity of the period of color change in the central region of the main part” is in the range of 0 to 1, and the closer to the value, the closer to 1.

  Further, the calculation unit 20 uses the rhythm information of the object in the first other captured image, “color appearing in the central region of the main part (color with a reduced number of gradations for comparison, and so on)”, The value obtained by comparing the “number of colors appearing in the central area of the main part” with the rhythm information of the object in the other two captured images and dividing the larger number of colors by the number of colors appearing by either It is good also as "the similarity of the color which appears in the center area | region of a main part."

  For example, the “color appearing in the central area of the main part” based on the rhythm information of the object in the first other captured image has three colors, and the “central area of the main part” based on the rhythm information of the object in the second other captured image When the number of colors appearing in “5” is five colors, the value “0.625” obtained by dividing the number of colors “5 (color)” by the number of colors “8 (color)” is “similarity of colors appearing in the central region of the main part” " In the case of this calculation method, the “similarity of the color appearing in the central region of the main part” is in the range of 0 to 1, and becomes closer to 1 as the similarity is increased.

  In addition, based on “similarity of period of color change in the central area of the main part” and “similarity of color appearing in the central area of the main part”, As a calculation method for calculating the “similarity”, various calculation methods are conceivable. As an example, the calculation unit 20 may calculate the “similarity of the period of color change in the central region of the main part” and “the center of the main part”. The average value of “similarity of colors appearing in the area” may be set as “similarity of periodic change in color of the central area of the main part”.

  For example, when “similarity of the color change period in the central region of the main part” is 0.75 and “similarity of the color appearing in the central region of the main part” is 0.625, the average value “0.6875 "Is the similarity of the periodic change in the color of the central region of the main part". In the case of this calculation method, the “similarity of the periodic change in the color of the central region of the main part” is in the range of 0 to 1, and the closer to the value, the closer to 1.

  At the time of calculating the above average value, the average value is calculated after weighting the “similarity of the color change period in the central area of the main part” or the “similarity of the color appearing in the central area of the main part”. May be. The average value is not limited to an arithmetic average, and a geometric average may be used. Further, instead of the average value, the median value, the maximum value, or the minimum value may be used as the “similarity of the periodic change in the color of the central region of the main part”.

  Further, for example, using the similarity related to the above-described feature 5, the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image is calculated. May be. In this case, the calculation unit 20, for example, does not change the color of the region other than the main part of the object indicated by the rhythm information of the object in the first other captured image, and the object in the second other captured image. The similarity with the color which does not change in the area other than the main part of the object indicated by the rhythm information is calculated, and the rhythm information of the object in the first other captured image and the object in the second other captured image are calculated. It is good also as similarity with rhythm information.

  Various calculation methods can be considered as a calculation method of “similarity of colors that do not change in regions other than the main part”. As an example, the calculation unit 20 preliminarily compares the similarity between one color and another color. Similar color information indicating the degree (information that takes a range from 0 to 1 and takes a value closer to 1 as the colors are similar to each other), and based on the similar color information, the first other imaging The similarity between the color of the area other than the main part based on the rhythm information of the object in the image and the color of the area other than the main part based on the rhythm information of the object in the second other captured image is calculated. It is good also as "the similarity of the color which does not change the area | regions other than a main part". However, when the color of the region other than the main part based on the rhythm information of the object in the first other captured image changes periodically, or other than the main part based on the rhythm information of the object in the second other captured image In the case where the color of the region of the color corresponds to any of the cases where the color changes periodically, the “similarity of the color not changing in the region other than the main part” is set to 0.

  In addition, the calculation unit 20 uses the similarity related to two or more features to calculate the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image. When calculating the similarity, the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image are based on the similarity related to each feature. The degree of similarity may be calculated. For example, when using the degree of similarity related to the above-described feature 2 and feature 5, the degree of similarity related to feature 2 and the degree of similarity related to feature 5 are calculated, respectively, and the degree of similarity related to feature 2 and feature 5 are calculated. The average value of the similarities related to the above may be the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image.

  For example, “similarity related to feature 2 (for example, similarity of periodic change in color of the central region)” is 0.875, “similarity related to feature 5 (for example, other than the main part described above) In the case where the similarity of the color of the non-changing area) is 0.825, the average value “0.85” is set to the rhythm information of the object in the first other captured image and the second other captured image. It is good also as similarity with the rhythm information of the object.

  When calculating the average value, the average value may be calculated after weighting the “similarity related to feature 2” or “similarity related to feature 5”. The average value is not limited to an arithmetic average, and a geometric average may be used. Further, instead of the average value, the median value, the maximum value, or the minimum value is expressed as “the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image”. It is good also as "similarity of".

  Note that the similarity between the rhythm information of the object in the first other captured image and the rhythm information of the object in the second other captured image is calculated based on the similarity related to feature 2 and feature 5. However, the rhythm information of the object in the first other captured image and the object in the second other captured image are applied by applying the respective calculation methods according to the other characteristics. The similarity with the rhythm information may be calculated.

  In addition, although the example which uses the average pixel value for every pixel group as each value (D1-D7) shown in FIG.9 (b) was demonstrated, it replaced with an average pixel value, for example, the largest pixel value (inside a pixel group) Using the maximum pixel value of a plurality of pixels), minimum pixel value (minimum value of pixel values of a plurality of pixels in a pixel group), and median value (median value of pixel values of a plurality of pixels in a pixel group) Also good.

  Further, an example has been described in which a predetermined number of adjacent pixels are used as a pixel group, and each value (each value (D1 to D7) illustrated in FIG. 9B) that calculates rhythm information is calculated. Each value constituting the rhythm information may be calculated by setting adjacent pixels equal to or less than the value as a pixel group.

FIG. 10A schematically shows adjacent images having a pixel value difference equal to or less than a predetermined number as a pixel group in the pixels constituting the traffic light object O ₂ shown in FIG. In FIG. 10A, Ga1 to Ga4 are pixel groups that are adjacent images in which the difference in pixel value is equal to or less than a predetermined number at any of the imaging timings (t1 to t7) (see FIG. 9B). Specifically, Ga1 represents a blue lamp region r1-1-1, Ga2 represents a yellow lamp region r1-2-1, Ga3 represents a red lamp region r1-3-1, and Ga4 represents a region other than the lamp. (See FIG. 8).

FIG. 10 (b), in case of dividing the object O ₂ into a plurality of pixel groups as in FIG. 10 (a), a rhythm information representing the pattern of the spatial variation of the object O _2. Each value (D1 to D7) shown in FIG. 10B is a value constituting rhythm information, and is an average pixel of each pixel group constituting the object O ₂ at each imaging timing (t1, t2,...). Value. However, as described above, the maximum pixel value, the minimum pixel value, and the median value may be used instead of the average value.

FIG. 11A schematically shows adjacent images having a pixel value difference equal to or less than a predetermined number in the pixels constituting the traffic light object O ₂ shown in FIG. 8 as a pixel group. In FIG. 11A, Gb1 and Gb4 are pixel groups that are adjacent images whose pixel value difference is equal to or less than a predetermined number at the imaging timing (t1 to t3) when the signal is blue (see FIG. 9B). ). Specifically, Gb1 represents a blue region, and Gb4 represents a black and gray region (see FIG. 8). That is, it is assumed that the difference between the pixel values (values indicating black) in the areas of the yellow lamp and the red lamp being turned off (values indicating black) and the pixel values (values indicating gray) in the areas other than the lamps is equal to or less than a predetermined value.

  In FIG. 11B, Gb2 and Gb4 are pixel groups of adjacent images whose pixel value difference is equal to or smaller than a predetermined number at the imaging timing (t4) when the signal is yellow (see FIG. 9B). Specifically, Gb2 represents a yellow region and Gb4 represents a black and gray region (see FIG. 8). That is, it is assumed that the difference between the pixel values (values indicating black) in the areas of the blue lamp and the red lamp being turned off and the pixel values (values indicating gray) in the areas other than the lamps is equal to or less than a predetermined value.

  In FIG. 11C, Gb3 and Gb4 are pixel groups of adjacent images whose pixel value difference is equal to or less than a predetermined number at the imaging timing (t5 to t7) when the signal is red (see FIG. 9B). ). Specifically, Gb3 represents a red region, and Gb4 represents a black and gray region (see FIG. 8). That is, the difference between the pixel values (values indicating black) in the areas of the blue lamp and the yellow lamp being turned off and the pixel values (values indicating gray) in the areas other than the lamps is not more than a predetermined value.

FIG. 11 (d) in case of dividing the object O ₂ into a plurality of pixel groups as shown in FIG. 11 (a) ~ (c) , a rhythm information representing the pattern of the spatial variation of the object O _2. Each value (S1 to S7) shown in FIG. 11D is a value constituting rhythm information and is a distribution (region shape) of each pixel group at each imaging timing. Specifically, S1 is a blue lamp area r1-1-1, S2 is a yellow lamp area r1-2-1, S3 is a red lamp area r1-3-1, S4 is a non-blue lamp area, S5. Represents a region other than the yellow lamp, and S6 represents a distribution of regions other than the red lamp.

  That is, the rhythm information is not expressed by using a predetermined number of adjacent pixels as a pixel group as shown in FIG. 9B, but the difference in pixel values is predetermined as shown in FIG. 10B or FIG. Rhythm information may be expressed by using adjacent pixels that are less than or equal to a pixel group as a pixel group, and adjacent pixels that have a pixel value difference that is less than or equal to a predetermined value as shown in FIG. When the rhythm information is expressed, the similarity of the rhythm information is calculated as in the case where the rhythm information is expressed with a predetermined number of adjacent pixels as a pixel group as shown in FIG.

  As described above, the rhythm information is expressed as the temporal change pattern of the object and the similarity is calculated using FIGS. 5 to 11 and the spatial change pattern of the object. The case where the similarity is calculated individually (FIGS. 8 to 11) has been described individually, but the rhythm information is expressed as a temporal change pattern and a spatial change pattern, and the similarity is calculated. May be. That is, if the format of the rhythm information for which the similarity is calculated is the same, the similarity between the rhythm information can be calculated. Therefore, even if the temporal change pattern of the object is used as the rhythm information, the space of the object A pattern of a typical change may be used as rhythm information, and both patterns may be used as rhythm information.

  FIG. 12 is a flowchart illustrating an example of the operation of the electronic device 1. At the start of the flowchart shown in FIG. 12, the rhythm information storage unit 90 stores rhythm information of a plurality of other electronic devices 100, and the connection information storage unit 92 stores connection information of the plurality of other electronic devices 100. It shall be remembered.

  In FIG. 12, the calculation unit 20 calculates the similarity of rhythm information of a plurality of other electronic devices 100 stored in the rhythm information storage unit 90 (step S10). The calculation unit 20 outputs the similarity calculation result to the grouping unit 30.

  The grouping unit 30 groups the plurality of other electronic devices 100 based on the similarity calculation result by the calculation unit 20, and generates grouping information (step S12). The grouping unit 30 stores the grouping information in the grouping information storage unit 94. The grouping unit 30 notifies the display unit 40 that the grouping information has been stored in the grouping information storage unit 94.

  The display unit 40 displays grouping information and connection information (step S14). Specifically, the display unit 40 refers to the connection information storage unit 92 and the grouping information storage unit 94, and, for each group, the device ID of the other electronic device 100 grouped in the group and the other Connection information for communicating with the electronic device 100 is displayed.

  The transmission unit 50 refers to the connection information storage unit 92, the grouping information storage unit 94, and the operation control information storage unit 96, and each of the other electronic devices 100 grouped in the group is grouped in the same manner. The operation control information is transmitted to each of the other electronic devices 100 grouped in each group so as to output sound and emit light or vibrate similarly (step S16). Then, this flowchart ends. The transmission unit 50 may transmit the operation control information when an operation reception unit (not shown) receives a predetermined user operation.

  Next, the mode illustrated in FIG. 1B (a mode in which a plurality of electronic devices (the electronic device 1 and the electronic device 100) including the own electronic device 1 are grouped) will be described.

  As shown in FIG. 2D, the imaging unit 14 is different from the configuration of the electronic device 1 in the aspect shown in FIG. 1A described above in the configuration of the electronic apparatus 1 in the aspect shown in FIG. It is only the point which has. Hereinafter, a description will be given focusing on the difference (imaging unit 14).

  The acquisition unit 10 of the electronic device 1 includes rhythm information representing a temporal or spatial change pattern of an object in a captured image captured by another electronic device 100 and a captured image captured by the electronic device 1. Rhythm information representing a temporal or spatial change pattern of the object inside is acquired. Specifically, as shown in FIG. 3A, the receiving unit 12 receives each of the other electronic devices 100 together with the device ID and connection information of the other electronic devices 100 from the other electronic devices 100. The rhythm information is received, the imaging unit 14 acquires the captured image by the electronic device 1, the rhythm information generation unit 16 extracts the object from the moving image captured by the imaging unit 14, and the object extracted from the moving image Generate rhythm information. Note that the rhythm information generation unit 16 generates rhythm information in the same manner as the rhythm information generation unit 116 included in the other electronic device 100.

  The acquisition unit 10 (reception unit 12, imaging unit 14, rhythm information generation unit 16) that acquired the rhythm information stores the acquired rhythm information in the rhythm information storage unit 90.

  The calculation unit 20 calculates the similarity of rhythm information of a plurality of electronic devices (the own electronic device 1 and other electronic devices 100) acquired by the acquisition unit 10. That is, the calculation unit 20 is captured by the rhythm information of the object in the captured image captured by the electronic device 1 generated by the rhythm information generation unit 16 and the other electronic device 100 received by the reception unit 12. The similarity with the rhythm information of the object in the captured image is calculated. Note that the calculation unit 20 may calculate the rhythm information similarity between the other electronic devices 100 received by the reception unit 12.

  When the similarity between the rhythm information of the electronic device 1 calculated by the calculation unit 20 and the rhythm information of the other electronic device 100 is equal to or greater than a predetermined threshold, the grouping unit 30 and the electronic device 1 Electronic devices 100 are grouped into the same group. The grouping unit 30 may group the other electronic devices 100 into the same group even when the similarity of the rhythm information between the other electronic devices 100 is equal to or greater than a predetermined threshold.

  When the own electronic device 1 is grouped into any group by the grouping unit 30, the own electronic device 1 grouped into the group and one or more other electronic devices 100 similarly output sound. Then, it is processed so that it emits light similarly or vibrates in the same way. For example, the transmission unit 50 may transmit the same value as the setting value (the setting value of the electronic device 1) stored in the operation control information storage unit 96 to another electronic device 100, or the setting unit ( A value (not shown) transmitted by the transmission unit 50 to the other electronic device 100 (a setting value for the other electronic device 100) may be stored in the operation control information storage unit 96 as a setting value of the own electronic device 1. .

FIG. 13 is a schematic diagram for explaining the result of grouping by the electronic device 1.
FIG. 13A shows the result of grouping in the mode shown in FIG. 1A, and the electronic device 1 includes three of the five other electronic devices 100 (electronic devices A to E). the electronic device 100 (electronic device a, C, D) and the group _{G 1,} the two electronic devices (electronic devices B, E) as a group _{G 2,} further electronic device (electronic grouped into _{G 1} equipment a, C, D) in the same manner as to output sound, grouped electronic equipment (electronic device B to G _2, and represents a state in which similarly to output audio to E).

  That is, according to the electronic device 1, a plurality of other electronic devices 100 are simply grouped, and a plurality of other electronic devices 100 in the same group are similarly output with sound, similarly emitted, or similarly It becomes possible to vibrate. Therefore, for example, in the scene where the audience uses the electronic device 100 to capture the same subject (for example, a parade at an amusement park, a concert artist, a circus performer), the same subject is imaged. The behavior of two or more other electronic devices 100 can be controlled in the same manner (behavior can be synchronized).

FIG.13 (b) is the result of the grouping in the aspect shown in FIG.1 (b), Comprising: The electronic device 1 is the own electronic device 1 and five electronic devices (electronic device AE), four electronic devices and (the primary electronic device 1, the electronic device a, C, D) and the group _{G 3,} further grouped electronics _{G 3} (the primary electronic device 1, the electronic device a, C, D) Similarly, a state in which sound is output is shown.

  That is, according to the electronic device 1, a plurality of other electronic devices 100 that are in the same group as the electronic device 1 are simply grouped, and a plurality of electronic devices in the group are similarly output with sound and similarly emitted. Alternatively, it can be vibrated in the same manner. Therefore, the behavior of one or more electronic devices 100 that are imaging the same subject as the electronic device 1 can be controlled in the same way.

FIG.13 (c) is the result of the grouping in the aspect shown in FIG.1 (b), Comprising: The electronic device 1 is the self-electronic device 1 and five electronic devices (electronic device AE), four electronic equipment (the primary electronic device 1, the electronic device a, C, D) and the group _{G 3,} the two electronic devices (electronic devices B, E) as a group _{G 2,} further grouped into _{G 3} electronic devices that are (the primary electronic device 1, the electronic device a, C, D) to output the audio similarly, likewise to output audio to grouped electronic equipment (electronic device B, E) to G ₂ It shows how it is.

  That is, according to the electronic device 1, a plurality of other electronic devices 100 that are in the same group as the own electronic device 1 and a plurality of other electronic devices 100 that are not in the same group as the own electronic device 1 are simply grouped. A plurality of electronic devices in the same group can be similarly output as audio, similarly emitted, or similarly vibrated. Therefore, the behaviors of two or more other electronic devices 100 that capture the same subject are controlled in the same manner, and the behaviors of two or more electronic devices 100 that capture the same subject are controlled in the same manner. can do.

  As described above, according to the electronic device 1, a plurality of electronic devices can be easily grouped. Moreover, according to the electronic device 1, it becomes possible to make a plurality of electronic devices grouped in the same group output sound in the same manner, similarly emit light, or vibrate similarly.

  Note that a program for executing each process of the electronic apparatus 1 according to an embodiment of the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Thus, each process of the electronic device 1 may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Electronic device, 10 ... Acquisition part, 12 ... Reception part, 14 ... Imaging part, 16 ... Rhythm information generation part, 20 ... Calculation part, 30 ... Grouping part, 40 ... Display part, 50 ... Transmission part, 60 ... Operation control unit, 90 ... Rhythm information storage unit, 92 ... Connection information storage unit, 94 ... Grouping information storage unit, 96 ... Operation control information storage unit, 100 ... Electronic device (other electronic devices), 114 ... Imaging unit, 116: Rhythm information generation unit, 140: Display unit, 150: Transmission / reception unit, 160: Operation control unit, 196: Operation control information storage unit

Claims (6)

An acquisition unit for acquiring rhythm information representing a temporal or spatial change pattern of an object in the captured image;
A calculation unit for calculating the similarity of the rhythm information of each object acquired by the acquisition unit from respective captured images captured by different electronic devices;
A grouping unit that generates grouping information for grouping electronic devices that have captured the respective captured images based on the similarity of the rhythm information of each object calculated by the calculation unit. Electronic equipment. The electronic device according to claim 1,
The acquisition unit
A receiver that receives the rhythm information of an object in a captured image captured by another electronic device;
The calculation unit includes:
Imaged by the second other electronic device received by the receiver and the rhythm information of the object in the first other captured image captured by the first other electronic device received by the receiver Calculating the similarity with the rhythm information of the object in the second other captured image,
The grouping unit
The similarity between the rhythm information of the object in the first other captured image calculated by the calculation unit and the rhythm information of the object in the second other captured image is greater than or equal to a predetermined threshold. In this case, the first other electronic device and the second other electronic device are grouped into the same group. The electronic device according to claim 1,
The acquisition unit
An imaging unit;
A receiving unit that receives the rhythm information of an object in a captured image captured by another electronic device;
The calculation unit includes:
The similarity between the rhythm information of the object in the self-captured image captured by the imaging unit and the rhythm information of the object in another captured image captured by another electronic device received by the reception unit Calculate
The grouping unit
When the degree of similarity between the rhythm information of the object in the self-captured image calculated by the calculation unit and the rhythm information of the object in the other captured image is equal to or greater than a predetermined threshold, An electronic apparatus characterized in that the other electronic apparatuses are grouped into the same group. The electronic device according to any one of claims 1 to 3,
An electronic device further comprising: a display unit that displays connection information for communicating with other electronic devices grouped into the same group by the grouping unit. The electronic device according to any one of claims 1 to 4,
Transmission for transmitting operation control information to other electronic devices so that a plurality of electronic devices grouped in the same group by the grouping unit similarly outputs sound, emits light in the same manner, or vibrates in the same manner An electronic device, further comprising a unit. An electronic device grouping method for grouping electronic devices,
The acquisition unit of the electronic device acquires rhythm information representing a temporal or spatial change pattern of an object in a captured image,
The electronic device calculation means calculates the similarity of the rhythm information of each object acquired by the acquisition means from each captured image captured by different electronic devices,
The electronic device grouping unit generates grouping information for grouping the electronic devices that captured the respective captured images based on the similarity of the rhythm information of each object calculated by the calculation unit. A grouping method characterized by

Images