JP2020113920A - Imaging apparatus - Google Patents

Abstract

To properly generate a video of a group that includes multiple target people.SOLUTION: A camera 10 includes an acquisition unit 12 that acquires a captured image, a detection unit 13 that detects information on a plurality of target persons from the captured image, identifies the plurality of target persons shown in the captured image, and divides the plurality of target persons into a first group including two or more target persons and a second group including a target person different from the target person included in the first group, a determining unit 14 that determines which of the first group and the second group is to be preferentially imaged on the basis of predetermined priority order, and a control unit 15 that controls at least one of the angle of view and the direction of the camera 10 on the basis of the determination of the determination unit 14.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image pickup device.

For example, a service is known in which a plurality of fixed-point cameras are installed on a site in a theme park or the like, and scenes captured by the fixed-point cameras are connected to generate a moving image. In such a service, for example, it is conceivable to generate a moving image in which scenes in which the person is shown are connected for each visitor. When a moving image for each visitor is generated, a person is identified by a face authentication technique described in Patent Document 1, for example.

JP, 2007-272896, A

The moving image as described above is preferably an image focused on the target person (entrance). Here, in a moving image, for example, there is a case where it is desired to image a plurality of target persons who act together as a set (to create an image focused by the set). However, for example, when a plurality of sets (groups) of target persons are included in the imaging range, it is not possible to determine which group should be focused and imaged, and a moving image of a group including a plurality of target persons can be displayed. It is possible that it cannot be generated properly.

The present invention has been made in view of the above circumstances, and an object thereof is to appropriately generate a moving image of a group including a plurality of target persons.

An imaging apparatus according to one aspect of the present invention detects a captured image acquisition unit that acquires a captured image, detects information about a plurality of target persons from the captured image, and specifies the plurality of target persons shown in the captured image. A detection unit that divides a plurality of target persons into a first group including two or more target persons and a second group including target persons different from the target persons included in the first group, and a predetermined priority order. A determination unit that determines which of the first group and the second group is to be preferentially imaged, and a control unit that controls at least one of the angle of view and the direction of the imaging device based on the determination of the determination unit. ..

In the imaging device according to one aspect of the present invention, the plurality of target persons shown in the captured image are divided into a first group and a second group, and which group is preferentially imaged is determined based on a predetermined priority order, The angle of view and the orientation of the imaging device are controlled according to the determination result. In this way, by determining which group is to be preferentially imaged based on a predetermined priority order, even when a plurality of groups are included in the captured image, the video of the group can be appropriately displayed based on a predetermined criterion. Can be imaged. In addition, by controlling the angle of view of the image capturing apparatus so as to capture the group for which priority image capturing is performed, it is possible to efficiently capture the moving image of the group while suppressing the number of image capturing apparatuses.

In the above-mentioned imaging device, the determination unit may determine which of the first group and the second group is to be preferentially imaged based on the priority order in which the target person having a smaller number of imaged scenes has a higher priority. As a result, the angle of view of the image capturing apparatus is controlled so that the group including the target person with few captured scenes is preferentially captured, so that it is possible to suppress the occurrence of a group with few captured scenes, and The video of each group can be captured.

In the above-mentioned imaging device, the determining unit may determine which of the first group and the second group is to be preferentially imaged, based on a predetermined priority order of group units. By determining the group to be preferentially imaged based on the priority order of the group person instead of the individual person, it is possible to more appropriately image the moving image of the group unit.

In the above-mentioned imaging device, when the detection unit determines, for each of two or more target persons included in the first group, a combination of possible target persons as a subgroup, and the determination unit determines to preferentially image the first group. In, the sub-group may be determined to be preferentially imaged based on the priority order in which the sub-group having a smaller number of already-captured scenes has a higher priority. As described above, for example, by controlling so that a group having a smaller number of imaged scenes is preferentially imaged, it is possible to prevent a group having a smaller number of imaged scenes from occurring, and to ensure fairness between the groups (the number of captured images). And the imaging intervals can be equalized). In addition, regarding two or more target persons included in the same group, each combination is set as a subgroup, and subgroups having a smaller number of image capturing scenes as a combination are included in the same group by preferentially capturing images. Various (various combinations) moving images of the target person can be captured, and a more memorable scene for the user can be captured.

In the imaging device, the detection unit may divide the two or more target persons included in the first group into subgroups according to positions in the captured image. Thereby, a plurality of target persons who act at a close distance to each other can be included in the same subgroup, and the subgroup can be appropriately divided.

In the above-mentioned imaging device, the captured image acquisition unit acquires captured images that are continuous in time series, and the control unit, based on the captured images that are continuous in time series, the target included in the group determined to be preferentially captured by the determination unit. The moving direction of the person may be specified, and at least one of the angle of view and the direction of the imaging device may be controlled according to the moving direction. As described above, by specifying the moving direction of the target person and controlling the direction of the image capturing device in consideration of the moving direction, it is possible to more appropriately capture the moving image of the group including the target person.

In the above-described image pickup apparatus, the detection unit further detects a predetermined object from the picked-up image, and the control unit determines that the target person of the group determined to be preferentially imaged by the determination unit and the object detected by the detection unit. At least one of the angle of view and the direction of the imaging device may be controlled so that both images are captured. Thus, for example, by setting a landmark (famous building or the like) as an object to be detected in advance, it is possible to capture a moving image of the landmark and the target person.

According to the present invention, it is possible to appropriately generate a moving image of a group including a plurality of target persons.

It is a figure explaining the basic operation of the imaging system concerning this embodiment. It is a block diagram which shows the functional structure of the camera contained in the imaging system which concerns on this embodiment. It is a flow chart which shows processing which a camera included in an imaging system concerning this embodiment performs. It is a figure which shows the hardware constitutions of the camera contained in an imaging system.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference symbols, without redundant description.

FIG. 1 is a diagram illustrating a basic operation of the imaging system 1 according to this embodiment. The imaging system 1 shown in FIG. 1 includes a plurality of cameras 10 (imaging devices) and a server 50. The plurality of cameras 10 are fixed-point cameras fixedly installed at different points on a specific site. The camera 10 is configured to be able to upload the captured moving image to the server 50 via the network (Internet). The imaging system 1 is a system that extracts a scene in which a predesignated target person is captured, using inputs from a plurality of cameras 10. For example, as shown in FIG. 1, when the person A is the target person, when the plurality of cameras 10 of the imaging system 1 can detect (identify) the person A in the captured moving image, The moving image is uploaded to the server 50, and when the person A cannot be detected in the captured moving image, the moving image is not uploaded to the server 50. Then, the server 50 connects the moving images (the moving images including the scenes in which the person A is captured) acquired from each camera 10 to generate the moving image of the person A in the site where the plurality of cameras 10 are installed. The imaging system 1 is used, for example, for a service of automatically generating and providing a moving image (commemorative image) of each visitor in a site (wide space) such as a theme park or a ski resort.

In the imaging system 1, the detection result of the target person is shared among the plurality of cameras 10. The detection result of the target person includes not only the face information of the target person but also related information (details will be described later) related to the target person. By sharing such a detection result between the cameras 10, for example, even in the camera 10 that cannot image the target person from the front and cannot detect the face information of the target person, the face information of the target person and the related information can be obtained. Since the information is shared in advance, it becomes possible to detect the related information and extract the scene of the target person related to the related information. As described above, the imaging system 1 shares the detection result of the target person among the plurality of cameras 10 to extract the scene in which the target person is captured with higher accuracy and without omission.

Further, in the imaging system 1, a plurality of target persons detected from the captured image are grouped, and scenes of the plurality of target persons are extracted for each group. At this time, when a plurality of groups are included in the captured image, the imaging system 1 determines the group to be preferentially imaged based on a predetermined priority order, and the camera 10 is set so that the group to be preferentially imaged is focused (zoomed). The angle of view and the direction of are controlled. With this, even when the captured image includes a plurality of groups, it is possible to appropriately capture a moving image of a desired group based on a predetermined standard.

FIG. 2 is a block diagram showing the functional configuration of the camera 10 included in the imaging system 1 according to this embodiment. As shown in FIG. 2, each of the plurality of cameras 10 includes a storage unit 11, an acquisition unit 12 (captured image acquisition unit), a detection unit 13, a determination unit 14, a control unit 15, and an output unit 16. And are equipped with.

The storage unit 11 is a database that stores person records, group records, and landmark records. The information stored in the storage unit 11 is also stored in the server 50. That is, common information is stored between the plurality of cameras 10 and the server 50.

The person record is information set for each target person who is an imaging target, and is information in which a person ID, a face image, related information, and a priority order are associated with each other. The person ID is identification information that uniquely identifies a person. The face image is a face image of the person (specifically, feature data relating to the face image). The related information is information other than the face of the target person and related to the target person. The related information is, for example, information carried by the target person, and is an image of clothes, a bag, food, sunglasses, a mask, a hat, a ring, a hairstyle, and the like (specifically, feature data relating to the image). Such a face image and related information are used by the detection unit 13 to identify the target person. The priority order is the order in which priority imaging is easily performed for each person. The priority imaging is imaging focused by adjusting the angle of view and orientation of the camera 10. The priority order is set, for example, such that the target person having a smaller number of imaged scenes has a higher priority. In this case, as the priority order, the order itself may not be defined, and for example, the number of captured scenes may be defined. Further, the priority order may be set, for example, in consideration of the imaging time. That is, the priority order may be set to be higher, for example, as the target person has a longer elapsed time from the last image capturing time.

The information in the person record is rewritten as the time series progresses. Of the information of the person record, the person ID and the face image are set (stored) in the server 50 in advance and shared with each camera 10, and the initial state is not waited for before the camera 10 detects the target person. This is the information stored from the state. Among the information of the person record, the related information is associated with the person ID and the face image based on the information from at least one camera 10 that has succeeded in detecting both the face information and the related information of the target person. This is the stored information. Among the respective pieces of information of the person record, the priority order is, for example, information set based on information from the server 50 that generated the moving image such that the target person having a smaller number of captured scenes has a higher priority.

The group record is information set for each group including a plurality of target persons, and is information in which a group ID, a target person list, and a priority order are associated with each other. The group ID is identification information that uniquely identifies the group. The target person list is a list of person IDs of a plurality of target persons included in the group. It should be noted that the person ID in the target person list is associated with the feature data relating to the face image of each corresponding person. The priority order is a priority order of priority imaging in units of groups. For example, the priority order is set to be higher in a group having a smaller number of captured scenes in a combination of two or more target persons included in the group. In this case, the priority itself may not be defined as the priority order, and for example, the number of imaged scenes for each group may be defined. Further, the priority order may be set, for example, in consideration of the imaging time. That is, the priority order may be set to be higher, for example, in a group having a longer elapsed time from the last image capturing time.

The information in the group record is rewritten as the time series progresses. The target person list among the respective pieces of information of the group record is stored in association with the group ID based on information from at least one or more cameras 10 that simultaneously detect the face information of the respective target persons included in the target person list. This is the information to be provided. The target person list may be associated with the group ID in advance by setting. That is, the plurality of target persons forming the group may be determined based on the detection result of the camera 10 or may be preset. Regarding each item of the information of the group record, the priority order is based on, for example, the information from the server 50 that generated the moving image, a group with a small number of scenes that have already been captured (imaging with a combination of two or more target persons included in the group has been performed). The information is set such that the higher the number of scenes, the higher the number.

The landmark record is information that defines an object that is adjusted so as to be included in the imaging range, and is information in which a landmark ID and a landmark image are associated with each other. The landmark ID is identification information that uniquely identifies an object (landmark). The landmark image is an image of the object (specifically, feature data relating to the image of the object). As the object (landmark), for example, a historic building, playground equipment, trees, or other feature that can serve as a mark on the site is selected.

The acquisition unit 12 acquires a captured image captured by the image sensor. The acquisition unit 12 reads a captured image for each frame from the image sensor. That is, the acquisition unit 12 acquires captured images that are continuous in time series. The acquisition unit 12 outputs the read captured image for each frame to the detection unit 13 and the control unit 15.

The detection unit 13 detects information on a plurality of target persons from the captured image, identifies the plurality of target persons shown in the captured image, and for the plurality of target persons, a first group including two or more target persons. And a second group including target persons different from the target persons included in the first group. Note that, in reality, the captured image may include three or more groups, but for convenience of description, the captured image includes two groups (first group and second group) below. As described below.

The detection unit 13 refers to the person record in the storage unit 11 and determines whether or not the face image associated with each person ID can be detected from the captured image. The face image is detected by using, for example, a well-known face authentication technique. Further, the detection unit 13 refers to the person record in the storage unit 11 and determines whether or not the related information associated with each person ID can be detected from the captured image. The detection of the related information by the detection unit 13 may be performed by image pattern matching or object detection AI. The detection unit 13 specifies a plurality of target persons shown in the captured image by detecting a plurality of face images or related information from the captured image.

The detection unit 13 refers to the target person list of the group record of the storage unit 11, and includes, for the plurality of target persons shown in the captured image, a first group including two or more target persons, and a first group included in the first group. It is divided into a second group including at least one target person different from the target person. That is, when the plurality of target persons shown in the captured image include two or more target persons included in the target person list of the group indicated by a certain group ID, the detection unit 13 detects the two target persons. The above group of target persons is referred to as a first group. Further, when the plurality of target persons shown in the captured image include the target persons included in the target person list of the group other than the first group, the detection unit 13 The group of target persons is the second group. The detection unit 13 may divide the plurality of target persons into a first group and a second group according to the positions of the plurality of target persons in the captured image. That is, the detection unit 13 may perform grouping so that a plurality of target persons that are close to each other in the captured image form the same group. In this case, the detection unit 13 may or may not refer to the group record in the storage unit 11.

The detection unit 13 may further detect a predetermined object (landmark) from the captured image. The detection unit 13 refers to the landmark record in the storage unit 11 and determines whether or not the landmark image associated with each landmark ID can be detected from the captured image. The detection of the landmark image by the detection unit 13 may be performed by image pattern matching or object detection AI.

The determination unit 14 determines whether to preferentially image either the first group or the second group based on a predetermined priority order. The determining unit 14 determines, for example, the first group and the second group based on the priority order (the higher the priority level of the target person having a smaller number of imaged scenes) defined in the person record (information about the individual person of the target person) of the storage unit 11. It is determined which of the groups is to be preferentially imaged. That is, for example, the priorities of the persons A and B included in the first group identified by the detection unit 13 are first and second, respectively, and the priorities of the persons C and D included in the second group are In the case of the third place and the fourth place, respectively, the determining unit 14 determines to preferentially image the first group having a higher priority order of the included target persons. Note that the determination unit 14 may, for example, simply set a group including a target person having the highest priority as a target for priority imaging, or a total priority value (average, etc.) of all target persons included in the group is high. A group may be a target of priority imaging.

The determination unit 14 may determine which of the first group and the second group is to be preferentially imaged, based on a predetermined priority order in group units. That is, the determination unit 14 is based on the priority order defined in the group record of the storage unit 11 (the higher the priority order of a group having a smaller number of captured scenes in a combination of two or more target persons included in the group), It may be determined which of the first group and the second group is to be preferentially imaged. For example, when the priority order of the first group identified by the detection unit 13 is 1st and the priority order of the second group is 2nd, the determining unit 14 preferentially images the first group having a high priority order. decide. Note that the determining unit 14 determines a group to be preferentially imaged in consideration of both the priority order of the person record (priority order regarding the individual person) and the priority order of the group record (priority order for each group). May be.

The control unit 15 controls at least one of the angle of view and the direction of the camera 10 based on the determination of the determination unit 14. The angle of view is an angle indicating the imaging range of the camera 10. The narrower the angle of view, the more zoomed the imaging target can be. The direction is the direction in which the camera 10 takes an image. For example, when the determination unit 14 determines to preferentially image the first group (including the persons A and B), the control unit 15 causes the persons A and B to be focused on the persons A and B included in the first group. The camera 10 is directed in the direction of B, and the angle of view is adjusted so that the persons A and B are zoomed. More specifically, the control unit 15 adjusts the orientation of the camera 10 so that the average coordinates of the persons A and B (intermediate coordinates between the persons A and B) are in the center of the imaging range in the horizontal direction. Further, when the detection unit 13 detects a predetermined landmark together with the target person in the group for preferential imaging, the control unit 15 determines the target person in the group for preferential imaging (a full-body rectangle or bust-up of the target person). The angle of view and the direction of the camera 10 are controlled so that both the image and the landmark detected by the detection unit 13 are captured. For example, the control unit 15 adjusts the angle of view so that the target person and the landmark of the group to be preferentially imaged are zoomed to the maximum.

The control unit 15 estimates the moving directions of the plurality of target persons of the group for which priority image capturing is performed, and controls the angle of view and the direction of the camera 10 to appropriately image the moving target persons. The control unit 15 acquires the captured images (the captured images for each frame) that are continuous in time series from the acquisition unit 12, and detects the change in the coordinates of the target person between frames at adjacent times, for example, to thereby detect the target person. The moving direction and the moving amount per unit time are specified. Then, the control unit 15 estimates the position where the target person is likely to exist after a predetermined time in consideration of the moving direction and the moving amount (for example, assuming that the target person makes a uniform linear motion), and The angle of view and the direction of the camera 10 are controlled so that the estimated position is zoomed. The predetermined time here is a constant determined in consideration of the time required for control of the camera 10 (adjustment of the angle of view and orientation), for example. In the control of the orientation of the camera 10, how much the orientation of the camera 10 is moved differs depending on whether the imaging target area is flat enough to be regarded as a flat surface, or whether there are undulations, slobes, etc. and cannot be regarded as a flat surface. For example, when the area to be imaged can be regarded as a plane, an appropriate amount of change in the orientation of the camera 10 can be calculated by elementary mathematics. On the other hand, when the imaging target area cannot be regarded as a plane, for example, for which mesh and how much the camera 10 is changed with respect to the moving direction on the image for each mesh obtained by dividing the imaging target area with appropriate granularity (how much to move). ) May be calibrated in advance, and the change amount of the orientation of the camera 10 may be derived by referring to the conversion table. Instead of using the conversion table, the conversion parameter may be autonomously obtained from the actual data by machine learning, for example.

The output unit 16 extracts a scene including the target person detected by the detection unit 13, performs encoding processing to generate moving image data, and transmits the moving image data to the server 50. The server 50 receives the moving image data and performs a decoding process. Further, the server 50 repeats the video highlight scene extraction for each frame to keep the time width of the video within a predetermined length, for example, to generate a highlight video including the target person.

The output unit 16 also transmits the detection result (feature data and related information of the face image of the target person) by the detection unit 13 to the server 50. Further, the server 50 stores the detection result of the camera 10 and transmits the detection result to each camera 10. As a result, the detection result of each camera 10 is shared between the plurality of cameras 10 and the server 50.

Next, processing performed by the camera 10 included in the imaging system 1 will be described with reference to FIG. FIG. 3 is a flowchart showing a process performed by the camera 10 included in the imaging system 1 according to this embodiment.

As shown in FIG. 3, the camera 10 first reads a captured image for each frame from the image sensor (step S1). Subsequently, the camera 10 detects a person from the captured image (step S2). Specifically, the camera 10 refers to the person record in the storage unit 11 and determines whether or not the face image associated with each person ID can be detected from the captured image. The camera 10 also refers to the person record in the storage unit 11 and determines whether or not the related information associated with each person ID can be detected from the captured image. The detection unit 13 specifies a plurality of target persons shown in the captured image by detecting a plurality of face images or related information from the captured image.

Subsequently, the camera 10 identifies the group to which the identified plurality of target persons belong (step S3). Specifically, the camera 10 refers to the target person list of the group record of the storage unit 11, and with respect to the plurality of target persons shown in the captured image, a first group including two or more target persons and the first group. The target person included in the group is divided into a second group including a different target person. In other words, when the plurality of target persons shown in the captured image include two or more target persons included in the target person list of the group indicated by a certain group ID, the camera 10 is more than the two target persons. The group of target persons of is the first group. In addition, when the plurality of target persons shown in the captured image include the target persons included in the target person list of the group other than the first group, the camera 10 selects the target persons. The group of persons is the second group.

Subsequently, the camera 10 determines a group for preferential imaging from a plurality of groups based on a predetermined priority order (step S4). Specifically, the camera 10 sets the first group and the first group based on the priority order defined in the person record (information about the individual person of the target person) in the storage unit 11 (the higher the priority order of the target person having a smaller number of imaged scenes). It is determined which of the second groups is to be preferentially imaged. Alternatively, the camera 10 determines the first priority based on the priority defined in the group record of the storage unit 11 (the higher the priority of a group having a smaller number of captured scenes in a combination of two or more target persons included in the group). It is determined which of the first group and the second group is to be preferentially imaged.

Subsequently, the camera 10 controls the angle of view and the direction of the camera 10 based on the determination of the determination unit 14 so that the target person of the group for which priority imaging is performed is zoomed (step S5). By the above processing, the moving image of the target person of the group for preferential imaging is appropriately captured.

Next, the function and effect of this embodiment will be described.

The camera 10 according to the present embodiment detects an information about a plurality of target persons from the captured image and an acquisition unit 12 that acquires a captured image, identifies a plurality of target persons shown in the captured image, and determines the plurality of target persons. Regarding a person, a detection unit 13 that divides the person into a first group including two or more target persons and a second group including a target person different from the target persons included in the first group, and based on a predetermined priority order, A determination unit 14 that determines which of the first group and the second group is to be preferentially imaged, and a control unit 15 that controls at least one of the angle of view and the direction of the camera 10 based on the determination of the determination unit 14. Prepare

In such a camera 10, the plurality of target persons shown in the captured image are divided into the first group and the second group, which group is preferentially imaged based on a predetermined priority order, and the target person is determined according to the determination result. The angle of view and orientation of the camera 10 are controlled. In this way, by determining which group is to be preferentially imaged based on a predetermined priority order, even when a plurality of groups are included in the captured image, the video of the group can be appropriately displayed based on a predetermined criterion. Can be imaged. Further, by controlling the angle of view of the camera 10 so as to capture the group for which priority imaging is performed, it is possible to efficiently capture the moving image of the group while suppressing the number of cameras 10 installed. That is, according to the camera 10, the number of cameras 10 can cover a wide image capturing area even in a wide space, and a moving image with a zooming effect can be captured in a group including the target person.

In the camera 10, the determination unit 14 determines which of the first group and the second group is to be preferentially imaged, based on the higher priority order (priority order of person records) in which the target person having a smaller number of imaged scenes has higher priority. As a result, the angle of view of the camera 10 is controlled so that a group including a target person having a small number of imaged scenes is preferentially imaged, so that it is possible to suppress the occurrence of a group having a small number of imaged scenes. The video of each group can be captured.

In the camera 10, the determining unit 14 determines which of the first group and the second group is to be preferentially imaged based on a predetermined priority order (priority order of group records) in units of groups. By determining the group to be preferentially imaged based on the priority order of the group person instead of the individual person, it is possible to more appropriately image the moving image of the group unit.

In the camera 10, the determination unit 14 determines the first group and the first group based on the priority order (the priority order of the group record) that becomes higher in a group having a smaller number of captured scenes in a combination of two or more target persons included in the group. It is determined which of the two groups is to be preferentially imaged. As a result, the angle of view of the camera 10 and the like are controlled so that a group having a small number of scenes already captured in units of groups (combinations of target persons) is preferentially captured. In addition, it is possible to more reliably suppress the occurrence of a group having a small number of already-captured scenes, and it is possible to properly capture a moving image of each group.

In the camera 10, the detection unit 13 divides the plurality of target persons into the first group and the second group according to the positions of the plurality of target persons in the captured image. As a result, it is possible to appropriately perform grouping so that a plurality of target persons who act together are in the same group.

In the camera 10, the acquisition unit 12 acquires captured images that are continuous in time series, and the control unit 15 is a target included in the group that the determination unit 14 has determined to perform priority imaging based on the captured images that are continuous in time series. The moving direction of the person is specified, and the angle of view and the direction of the camera 10 are controlled according to the moving direction. In this way, by specifying the moving direction of the target person and controlling the direction of the camera 10 and the like in consideration of the moving direction, it is possible to more appropriately capture the moving image of the group including the target person.

In the camera 10, the detection unit 13 further detects a predetermined object (landmark) from the captured image, and the control unit 15 detects the target person of the group determined to be preferentially imaged by the determination unit 14 and the detection unit 13. The angle of view and the direction of the camera 10 are controlled so that both the object (landmark) detected by the image are captured. Thus, for example, by setting a landmark (famous building or the like) as an object to be detected in advance, it is possible to capture a moving image of the landmark and the target person.

Finally, the hardware configuration of the camera 10 included in the imaging system 1 will be described with reference to FIG. The camera 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the camera 10 may be configured to include one or a plurality of each device illustrated in the figure, or may be configured not to include some devices.

Each function in the camera 10 causes a predetermined software (program) to be loaded on hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs an arithmetic operation, communication by the communication device 1004, and the memory 1002 and the storage 1003. It is realized by controlling reading and/or writing of data.

The processor 1001 operates an operating system to control the entire computer, for example. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, a calculation device, a register, and the like. For example, the control function of the acquisition unit 12 and the like of the camera 10 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module, and data from the storage 1003 and/or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least part of the operations described in the above-described embodiments is used. For example, the control function of the acquisition unit 12 and the like of the camera 10 may be implemented by a control program stored in the memory 1002 and operated by the processor 1001, and may be implemented similarly for other functional blocks. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The program may be transmitted from the network via an electric communication line.

The memory 1002 is a computer-readable recording medium, and is configured by, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. May be done. The memory 1002 may be called a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code) that can be executed to implement the wireless communication method according to the embodiment of the present invention, a software module, and the like.

The storage 1003 is a computer-readable recording medium, for example, an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (for example, a compact disc, a digital versatile disc, a Blu-ray disc). (Registered trademark) disk, smart card, flash memory (for example, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be called an auxiliary storage device. The storage medium described above may be, for example, a database including the memory 1002 and/or the storage 1003, a server, or another appropriate medium.

The communication device 1004 is hardware (transmission/reception device) for performing communication between computers via a wired and/or wireless network, and is also called, for example, a network device, a network controller, a network card, a communication module, or the like.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may be integrated (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured with a single bus or different buses among devices.

The camera 10 is configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). Alternatively, part or all of each functional block may be realized by the hardware. For example, processor 1001 may be implemented with at least one of these hardware.

Although the present embodiment has been described in detail above, it is obvious to those skilled in the art that the present embodiment is not limited to the embodiment described in this specification. The present embodiment can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description of the present specification is for the purpose of exemplifying explanation, and does not have any restrictive meaning to the present embodiment.

For example, the imaging apparatus divides a plurality of target persons (persons A, B, and C) in the same group into more detailed subgroups, and determines which subgroup of the target persons is to be preferentially imaged. May be. That is, the detection unit of the image capturing device sets, for each of two or more target persons (persons A, B, and C) included in a certain group (first group), a combination of possible target persons as a subgroup. The combinations of all possible target persons include subgroup 1 (including only person A), subgroup 2 (including only person B), subgroup 3 (including only person C), and subgroup 4 (person). A, B is included), subgroup 5 (including persons A and C), subgroup 6 (including persons B and C), and subgroup 7 (including persons A, B, and C) are conceivable. As described above, the same person does not belong to different groups in grouping, but the same person may belong to different subgroups in subgrouping. Here, the detection unit may divide the sub-groups of two or more target persons included in the first group according to the positions in the captured image. That is, for example, with respect to the persons A, B, and C, the persons B and C are located at a distance close to each other (a distance at which they can be imaged at the same time), and the person A is separated from the persons B and C (persons B and C). When it is located in a place where cannot be imaged at the same time), the detection unit divides the group into the above-described subgroup 1 (including only person A) and subgroup 6 (including persons B and C).

Now, it is assumed that the above-described first group (persons A, B, and C) and second group (person D) are simultaneously present in a range that can be imaged by the camera shake adjustment. Specifically, it is assumed that there are subgroup 1 (person A) and subgroup 6 (persons B and C) of the first group, and second group (person D). The determining unit of the image capturing apparatus first compares the groups (first group and second group) to determine the group to be preferentially imaged. That is, for example, it is assumed that the entire first group (the total of the subgroup 1 to the subgroup 6) has been imaged 10 times in the past. In addition, it is assumed that the second group (person D) has been imaged 12 times in the past. In this case, the determining unit determines to preferentially image the first group having a small number of imaged scenes. Furthermore, when it is determined that the first group is to be preferentially imaged, the subgroup determines which subgroup is to be preferentially imaged based on the priority order in which the subgroups having a smaller number of imaging scenes in combination have a higher priority. For example, it is assumed that the subgroup 1 (person A) has been imaged three times and the subgroup 6 (persons B and C) has been imaged once in the past. In this case, the determination unit determines to preferentially image the subgroup 6 having a small number of image capturing scenes as a combination. As described above, for example, by controlling so that a group having less captured scenes is preferentially captured, it is possible to prevent a group having less captured scenes from occurring and to ensure fairness between groups (the number of captured images). And the imaging intervals can be equalized). In addition to this, for two or more target persons included in the same group, each combination is set as a subgroup, and subgroups having a smaller number of image capturing scenes as a combination are included in the same group by preferentially capturing images. It is possible to capture various (various combinations) moving images of the target person, and capture a scene that is more memorable for the user.

Each aspect/embodiment described in this specification is LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-Wide). Band), Bluetooth (registered trademark), or any other system using an appropriate system and/or a next-generation system extended based on the system.

As long as there is no contradiction, the order of the processing procedures, flowcharts, etc. of each aspect/embodiment described in this specification may be changed. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

The input/output information and the like may be stored in a specific place (for example, a memory) or may be managed by a management table. Information that is input/output may be overwritten, updated, or added. The output information and the like may be deleted. The input information and the like may be transmitted to another device.

The determination may be performed by a value represented by 1 bit (whether 0 or 1), may be performed by a Boolean value (Boolean: true or false), and may be performed by comparing numerical values (for example, a predetermined value). (Comparison with the value).

The aspects/embodiments described in the present specification may be used alone, in combination, or may be switched according to execution. Further, the notification of the predetermined information (for example, the notification of “being X”) is not limited to the explicit notification, and is performed implicitly (for example, the notification of the predetermined information is not performed). Good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or any other name, instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules , Application, software application, software package, routine, subroutine, object, executable, thread of execution, procedure, function, etc. should be construed broadly.

Moreover, software, instructions, etc. may be transmitted and received via a transmission medium. For example, the software may use a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technology such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wireline and/or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description include voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any of these. May be represented by a combination of

Note that the terms described in the present specification and/or terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

In addition, the information, parameters, and the like described in this specification may be represented by absolute values, relative values from predetermined values, or may be represented by other corresponding information. ..

User terminals are defined by those skilled in the art as mobile communication terminals, subscriber stations, mobile units, subscriber units, wireless units, remote units, mobile devices, wireless devices, wireless communication devices, remote devices, mobile subscriber stations, access terminals, It may also be referred to as a mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable term.

The terms "determining" and "determining" as used herein may encompass a wide variety of actions. “Judgment” and “decision” are, for example, calculating, computing, processing, deriving, investigating, looking up (eg, table, database or another). It may include, for example, a search in a data structure), and the fact that ascertaining is regarded as “judgment” or “decision”. In addition, "decision" and "decision" include receiving (eg, receiving information), transmitting (eg, transmitting information), input (input), output (output), access (accessing) (for example, accessing data in a memory) can be regarded as “judging” and “deciding”. In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when things such as resolving, selecting, choosing, establishing, establishing, and comparing are done. May be included. That is, the “judgment” and “decision” may include considering some action as “judgment” and “decision”.

As used herein, the phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

When the designations "first," "second," etc. are used herein, any reference to that element does not generally limit the amount or order of those elements. These designations may be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not imply that only two elements may be employed therein, or that the first element must precede the second element in any way.

As long as the terms “include”, “including”, and variations thereof are used in the present specification or claims, these terms are the same as the term “comprising”. It is intended to be comprehensive. Furthermore, the term "or" as used in the specification or claims is not intended to be an exclusive OR.

In this specification, a plurality of devices are also included unless the context or technology clearly indicates that only one device exists.

Throughout this disclosure, the plural includes the plural unless the context clearly indicates the singular.

10... Camera, 12... Acquisition part (captured image acquisition part), 13... Detection part, 14... Determination part, 15... Control part.

Claims (7)

A captured image acquisition unit that acquires a captured image;
Information about a plurality of target persons is detected from the captured image, a plurality of target persons shown in the captured image are identified, and a first group including two or more target persons for the plurality of target persons, A detection unit that divides the target person included in the first group into a second group including a target person different from the target person;
A determination unit that determines which of the first group and the second group is to be preferentially imaged based on a predetermined priority order;
An imaging device, comprising: a control unit that controls at least one of an angle of view and an orientation of the imaging device based on the determination of the determination unit. The image capturing apparatus according to claim 1, wherein the determining unit determines which of the first group and the second group is to be preferentially imaged based on the priority order in which the target person having a smaller number of imaged scenes has a higher priority. The image capturing apparatus according to claim 1, wherein the determination unit determines which of the first group and the second group is to be preferentially imaged, based on the predetermined priority order for each group. The detection unit sets, for each of two or more target persons included in the first group, a combination of possible target persons as a subgroup,
When the determining unit determines to preferentially image the first group, the subgroup determines which subgroup is preferentially imaged based on the priority order in which a subgroup having a smaller number of captured scenes in a combination has a higher priority. Item 1. The image pickup device according to any one of items 1 to 3. The imaging device according to claim 4, wherein the detection unit divides two or more target persons included in the first group into the subgroups according to positions in the captured image. The captured image acquisition unit acquires the captured images that are continuous in time series,
The control unit specifies a moving direction of a target person included in a group determined to be preferentially imaged by the determining unit based on the captured images that are continuous in time series, and an image of the imaging device is determined according to the moving direction. The imaging device according to claim 1, wherein at least one of a corner and a direction is controlled. The detection unit further detects a predetermined object from the captured image,
The control unit sets at least one of the angle of view and the orientation of the imaging device so that both the target person of the group determined to be preferentially imaged by the determination unit and the object detected by the detection unit are imaged. The imaging device according to claim 1, which is controlled.

Images