JP7319521B2 - Information processing device, information processing method, program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

By releasing the shutter when the jumping subject reaches the top of the trajectory, it is popular to take a picture that makes the subject look as if it is floating and post it on SNS.

In order to take such a picture, it is necessary for a third person to release the shutter with good timing, or to set a self-timer and jump in time with the shutter being released.

Further, Japanese Patent Laid-Open No. 2002-200001 discloses a technique for automatically tracking a subject and photographing it at a predetermined timing.

JP 2003-069884 A

As described above, Patent Literature 1 describes a technique for automatically tracking a subject and capturing an image at a predetermined timing. However, the subject is not limited to one person, and there are cases where it is desired to take a photograph in which multiple people appear to be floating at the same time. In such a case, it is not possible to take a desired photograph unless each subject is treated separately, but a plurality of subjects included in one image are treated as related.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a mechanism capable of recording photographs at desired timing even when there are a plurality of subjects.

The information processing apparatus of the present invention includes acquisition means for acquiring a photographed image, movement amount calculation means for calculating a movement amount of a subject from the image acquired by the acquisition means, and specifying means for specifying an image whose movement amount satisfies a predetermined condition.

Further, the information processing method of the present invention includes an acquisition step in which acquisition means of an information processing device acquires a photographed image; and a specifying step of specifying an image in which the moving amount calculated by the moving amount calculating step satisfies a predetermined condition. Characterized by

Further, the program of the present invention comprises an information processing apparatus configured to: acquire means for acquiring a photographed image; move amount calculation means for calculating the amount of movement of a subject from the image acquired by the acquisition means; A program for functioning as specifying means for specifying an image whose movement amount calculated by means satisfies a predetermined condition.

According to the present invention, it is possible to record photographs at desired timing even when there are a plurality of subjects.

1 is a diagram showing a configuration example of an information processing system according to the present invention; FIG. Block diagram showing an example of hardware configuration Flowchart showing processing contents of the present invention A diagram showing the processing of steps S301 to S304 Diagram showing processing of steps S307 to S311 Graph showing time-series changes in the amount of vertical movement of the subject

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an example of the configuration of an information processing system according to the present invention.

As shown in FIG. 1, in the information processing system according to the present invention, an imaging unit and an information processing apparatus 101 are communicably connected and shown as separate housings. A case of . For example, smartphones and digital cameras are suitable examples.

FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing apparatus of the present invention.

As shown in FIG. 2, the information processing apparatus includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an input controller 205, a video controller 206, A memory controller 207 and a communication I/F controller 208 are connected.

The CPU 201 comprehensively controls each device and controller connected to the system bus 204 .

ROM 202 or external memory 211 stores BIOS (Basic Input/Output System) and OS (Operating System), which are control programs executed by CPU 201, computer-readable executable programs for realizing this information processing method, and necessary various types of programs. Holds data (including data tables).

A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . The CPU 201 loads programs and the like necessary for executing processing from the ROM 202 or the external memory 211 to the RAM 203, and executes the loaded programs to realize various operations.

The input controller 205 controls inputs from input devices such as a keyboard 209 and a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) an icon, cursor, or button displayed on the touch panel.

Also, the touch panel may be a touch panel, such as a multi-touch screen, capable of detecting positions touched by multiple fingers.

Video controller 206 controls display on an external output device such as display 210 . The display shall include the display of a notebook computer integrated with the main body. Note that the external output device is not limited to a display, and may be a projector, for example. In addition, an input device is also provided for the device capable of receiving the above-described touch operation.

The video controller 206 can control a video memory (VRAM) for performing display control, and can use part of the RAM 203 as a video memory area, or provide a separate dedicated video memory. It is possible.

A memory controller 207 controls access to the external memory 211 . External memory can be connected to an external storage device (hard disk), flexible disk (FD), or PCMCIA card slot for storing boot programs, various applications, font data, user files, edit files, and various data via an adapter. A compact flash (registered trademark) memory or the like can be used.

A communication I/F controller 209 connects and communicates with an external device via a network, and executes communication control processing in the network. For example, communication using TCP/IP, telephone lines such as ISDN, and communication using 3G lines for mobile phones are possible.

The CPU 201 enables display on the display 210 by, for example, rasterizing an outline font to a display information area in the RAM 203 . The CPU 201 also allows the user to issue instructions using a mouse cursor (not shown) on the display 210 .

Next, the contents of the processing of the present invention will be described with reference to the flow chart of FIG.

The processing shown in the flowchart of FIG. 3 is processing that the CPU 201 of the information processing apparatus 101 reads and executes a predetermined control program.

In addition, the present invention processes captured images (streaming data) sent from the camera sensor at any time, and feels that the most floating feeling (weightless state) is obtained from the captured images within a certain period of time in the past. It implements the process of acquiring an image that can be obtained.

The processes of steps S301 to S305 identify the same person between the person appearing in the photographed image I(t) and the person appearing in the photographed image I(t−1) photographed a unit time ago, and identify the same person. This is a process of saving information (position, etc.) related to the face of the person who has been photographed.

In step S301, an image I(t) captured by the imaging unit (or the imaging unit of the information processing apparatus 101) is acquired. FIG. 4A is a diagram showing the acquired image I(t).

Note that the information processing apparatus 101 according to the present invention may be an apparatus including an imaging unit in the same housing as described with reference to FIG. 1, or an apparatus communicably connected to another apparatus including an imaging unit. may be

In step S302, a person's face is detected from the captured image I(t) acquired in step S301. If multiple people are shown, multiple faces are detected. FIG. 4B is a diagram showing a situation in which human faces (face 1(t), face 2(t) . . . face n(t)) are detected from the captured image I(t).

Here, let information (information specifying a person, position information in the image) about n faces detected from the photographed image I(t) be a face M(t) (M=1, 2, . . . n). .

In step S303, face information (here, face N(t-1)) detected from a captured image I(t-1) captured a unit time before the captured image acquired in step S301. ). (C) of FIG. 4 is a diagram showing a situation in which the face N(t-1) is detected.

In step S304, the face (face M(t)) detected in step S302 and the face (face N(t-1)) obtained in step S303 are compared to identify the face of the same person. As a specific method, a method of determining whether or not the person is the same person by face authentication, a method of determining the face closest to the face detected from the image a unit time ago as the same person, or the like is used.

Then, the face M(t) determined as the same person is transferred to the face N(t).

FIG. 4D is a diagram showing a situation in which face 1(t) and face a(t−1) are identified as the same person.

In step S305, information about each detected face (information specifying a person, position, etc.) is saved in a temporary area.

The processing of steps S306 to S310 is processing for calculating the amount of movement of the face per unit time.

In step S306, it is determined whether or not all detected faces have been processed. If all faces have been processed (step S306: YES), the process proceeds to step S311. If there is a face that has not been processed (step S306: NO), the process proceeds to step S307 to start processing the face that has not been processed.

In step S307, the position of the face (position N(t) of face N) reflected in the captured image I(t) is obtained from the temporary area (FIG. 5A).

In step S308, the position N(t-1) of the face N in the image I(t-1) captured a unit time earlier than the captured image I(t) is obtained from the temporary area (FIG. 5B). ).

In step S309, the movement amount N(Δt) of the face N per unit time is calculated from the captured image I(t). Specifically, it is calculated by position N(t)-position N(t-1) (FIG. 5(C)).

In step S310, information (person, position, movement amount) on the detected face is stored in a temporary area.

Then, the process returns to step S306, and when the process has been executed for all the faces appearing in the captured image I, the process proceeds to step S311.

In step S311, an average value Average (N(Δt)) is obtained from the moving amount N(Δt) of each face calculated in step S309 ((D) in FIG. 5).

In step S312, the amount of movement N(Δt) of the face is embedded in the captured image I(t).

In step S313, it is determined whether or not the average value of the movement amount calculated in step S311 (vertical movement amount is used here) exceeds a threshold value (Tv). set. Then, with the flag set (that is, after it is determined that the threshold value (Tv) has been exceeded), if it is determined that the negative threshold value (-Tv) has been exceeded, the flag is reset, and the process proceeds to step S314. transition to

That is, when it is detected that the subject has jumped and landed, the process proceeds to step S314 (the state in which the flag is set means the state during the jump).

In step S314, when the flag is set and the average value of the amount of movement in the vertical direction is 0 or minimum, it is regarded as a photo opportunity (P), and the photographed image I(P) is picked up.

FIG. 6 is a graph showing time series changes in the amount of vertical movement. The photographed image I(P) acquired at the timing when the amount of movement becomes 0 or minimum during the period from when the amount of movement exceeds the threshold (Tv) to when it exceeds the negative threshold (-Tv) is used by the user. is obtained as the desired photograph of

With the above processing, it is possible to acquire an image at the timing when multiple subjects jump and reach the top of the trajectory based on the average amount of movement of the subjects per unit time, and all the subjects appear to be floating. Photographs can be recorded.

In this embodiment, the processing is performed based on the face, but the amount of movement and the like may be calculated using parts other than the face.

The present invention can be embodied as, for example, a system, device, method, program, recording medium, or the like. Specifically, it may be applied to a system composed of a plurality of devices, or may be applied to an apparatus composed of a single device.

A program according to the present invention is a program that allows a computer to execute the processing method of the flowchart shown in FIG. 3, and a storage medium of the present invention stores a program that enables the computer to execute the processing method of FIG. The program in the present invention may be a program for each processing method of each device in FIG.

As described above, a recording medium recording a program for realizing the functions of the above-described embodiments is supplied to a system or device, and the computer (or CPU or MPU) of the system or device reads the program stored in the recording medium. Needless to say, the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium implements the novel functions of the present invention, and the recording medium recording the program constitutes the present invention.

Examples of recording media for supplying programs include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon A disk or the like can be used.

Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program, the OS (operating system) and the like running on the computer are actually executed. Needless to say, a case where part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing are included.

Furthermore, after the program read from the recording medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read according to the instruction of the program code. It goes without saying that a case where a CPU or the like provided in a function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Moreover, the present invention may be applied to a system composed of a plurality of devices or to an apparatus composed of a single device. Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or apparatus. In this case, by loading a recording medium storing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Furthermore, by downloading and reading out the program for achieving the present invention from a server, database, etc. on the network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that all configurations obtained by combining each of the above-described embodiments and modifications thereof are also included in the present invention.

101 information processing device

Claims (5)

an acquisition means for acquiring an image;
movement amount calculation means for calculating movement amounts of a plurality of subjects included in the acquired image based on the image acquired by the acquisition means ;
an identifying means for identifying an image in which the average value of the moving amounts of the plurality of subjects calculated by the moving amount calculating means satisfies a predetermined condition;
An information processing device comprising: 2. The information processing apparatus according to claim 1 , wherein the object for which the movement amount is calculated by said movement amount calculation means is the face of the subject. The specifying means is an image captured between when the amount of upward movement of the subject satisfies a predetermined condition and before the amount of downward movement of the subject satisfies the predetermined condition, and the movement amount calculating means is an image captured by the subject. 3. The information processing apparatus according to claim 1, wherein an image having a minimum average value of vertical movement amounts calculated by the above is specified. an acquisition step in which the acquisition means of the information processing device acquires an image;
a movement amount calculation step in which the movement amount calculation means of the information processing apparatus calculates movement amounts of each of a plurality of subjects included in the acquired image based on the image acquired in the acquisition step;
a specifying step in which the specifying means of the information processing apparatus specifies an image in which the average value of the moving amounts of the plurality of subjects calculated in the moving amount calculating step satisfies a predetermined condition;
An information processing method comprising: A program for causing a computer to function as each means according to any one of claims 1 to 3 .

Images