JP5546949B2 - Terminal device and program - Google Patents

Description

  The present invention relates to a terminal device and a program for processing a series of images.

  Generally, in a device that records a series of images (for example, a long-time television broadcast), a chapter (scene break) is inserted every time a scene changes greatly. The method of inserting chapters in a series of images according to instructions places a heavy burden on the user. Conventionally, for example, chapters are automatically inserted into a series of images by capturing scene changes as subject movements. The technique which did it was developed (refer patent document 1).

Japanese Patent Laid-Open No. 2008-4980

  However, in the above-described prior art, it is possible to greatly reduce the burden on the user by automatically inserting chapters into a series of images, but for the user, the scene changes depending on the chapter position. There is a problem that the contents of the scene cannot be confirmed unless the contents are reproduced.

  An object of the present invention is to allow a user to easily confirm why a series of images are separated.

In order to solve the above-described problem, the invention described in claim 1
A terminal device that processes a series of images,
Image acquisition means for acquiring the series of images;
Change determination means for determining whether or not a predetermined change state for dividing a series of images acquired by the image acquisition means into scenes has been reached;
When it is determined by the change determining means that a predetermined change state has been reached, delimiter means for delimiting the position where the change state has occurred as a scene delimiter;
An information acquisition unit that acquires delimiter information indicating why the image is delimited when the series of images is delimited by the delimiter;
Storage control means for inserting and storing the delimiter information acquired by the information acquisition means at a delimiter position delimited by the delimiter means in the series of images ;
It is characterized by providing.

As an invention dependent on claim 1,
The image acquisition means is an imaging means for taking a series of images,
The change determining unit determines whether or not a change in an image obtained by sequentially analyzing the images at the time of capturing a series of images by the imaging unit is in the predetermined change state.
The invention according to claim 2, which is configured as described above.

As an invention dependent on claim 1,
The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether or not a change in an image obtained by sequentially analyzing the images taken when the series of images taken by the imaging means is in the predetermined change state;
The invention according to claim 3, which is configured as described above.

As an invention dependent on claim 1,
The image acquisition means is a reception means for receiving a series of images,
The change determining means determines whether the change in the image obtained by sequentially analyzes of reproduction at the image series of images Ri received by the receiving means becomes the predetermined changing state,
The invention according to claim 4, which is configured as described above.

As an invention dependent on claim 1,
A judgment criterion for determining whether or not to divide the series of images, and a criterion storage means for storing delimitation information indicating the reason for delimiting according to the criterion;
The change determination means determines whether or not the predetermined change state has been reached, depending on whether or not the determination criterion is met,
The information acquisition means reads and acquires the separation information stored in the determination criterion storage means in association with the determination criterion when the series of images are divided based on the determination criterion;
The invention according to claim 5, which is configured as described above.

As an invention dependent on claim 1,
The change determination unit determines whether or not the brightness of the subject has changed to the predetermined change state by analyzing a series of images acquired by the image acquisition unit;
The invention according to claim 6 is characterized in that it is configured as described above.

As an invention dependent on claim 1,
It further comprises position acquisition means for acquiring the current position of the terminal device,
The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether the current position acquired by the position acquisition means during shooting by the imaging means is a shooting position, and whether or not the shooting position is in the predetermined change state.
The invention according to claim 7, characterized in that it is configured as described above.

As an invention dependent on claim 1,
The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether or not the movement of a subject included in the series of images has become the predetermined change state during photographing by the imaging means.
The invention according to claim 8, which is configured as described above.

As an invention dependent on claim 1,
It further comprises a motion detection means for detecting the motion of the terminal device,
The change determining means determines whether or not the predetermined change state has been reached in accordance with the change state of the movement of the terminal device detected by the motion detection means.
The invention according to claim 9, characterized in that it is configured as described above.

As an invention dependent on claim 1,
It further comprises a direction detecting means for detecting the direction of the terminal device,
The change determining means determines whether or not the predetermined change state has been reached in accordance with the change state of the direction of the terminal device detected by the direction detection means.
The invention according to claim 10, which is configured as described above.

As an invention dependent on claim 1,
It further comprises position acquisition means for acquiring the current position of the terminal device,
The information acquisition means includes map information or an address corresponding to the current position acquired by the position acquisition means as separation information indicating why the separation is performed when the series of images are separated by the separation means. Get information,
The invention according to claim 11, which is configured as described above.

As an invention dependent on claim 1,
It further has a time measuring means for measuring the current date and time,
The information acquisition means acquires the current date and time acquired by the time measuring means as a part of separation information indicating why the separation is performed when the series of images are separated by the separation means.
The invention according to claim 12 is characterized in that it is configured as described above.

As an invention dependent on claim 1,
The storage control unit stores the delimiter information acquired by the information acquisition unit as additional information to be added to the series of images when storing the delimiter information in association with the delimiter positions delimited by the delimiter unit.
The invention according to claim 13, characterized in that it is configured as described above.

As an invention dependent on claim 1,
The storage control unit stores other information different from the storage unit that stores the series of images when storing the delimiter information acquired by the information acquisition unit in association with the delimiter positions delimited by the delimiter. Linked to the department and memorized,
The invention according to claim 14 is characterized in that it is configured as described above.

As an invention dependent on claim 1 ,
The information acquisition means generates and acquires information related to a scene at a separation position separated by the separation means as separation information;
The invention according to claim 15 , characterized in that it is configured as described above.

As an invention dependent on claim 1 ,
The information acquisition means selectively acquires information arbitrarily selected by a user operation as the delimiter information.
The invention according to claim 16 is characterized in that it is configured as described above.

As an invention dependent on claim 1,
In a state where the delimiter information acquired by the information acquisition unit is stored in association with the delimiter positions delimited by the delimiter unit, the delimiter positions in the series of images are searched in response to a delimiter search instruction. Search means;
Output control means for outputting the break information associated with the break position searched by the search means;
Further comprising
The invention according to claim 17 is characterized in that it is configured as described above.

In order to solve the above-mentioned problem, the invention according to claim 18
Against the computer,
A function to acquire a series of images,
A function of determining whether or not a predetermined change state for dividing the acquired series of images for each scene has been reached;
A function of dividing the position where the predetermined change state has occurred as a scene break;
A function of acquiring delimiter information indicating why the image is delimited when the series of images is delimited,
A function of inserting and storing the acquired delimiter information at the delimiter position in the series of images ;
It is a program for realizing.

  According to the present invention, the user can easily confirm for what reason the series of images are separated, which is highly convenient.

The block diagram which showed the basic component of the mobile telephone with a camera function applied as a terminal device. (1)-(5) is a figure for demonstrating the address book AD, the chapter flag F1, the chapter information flag F2, the position information flag F3, and the scene change table ST which are memorize | stored in the memory | storage part 3. FIG. The figure for demonstrating notionally the case where a chapter (scene division | segmentation) is set whenever the scene (scene) of a series of images (a moving image or a continuous shooting still image) changes. The main flowchart for demonstrating the operation | movement (characteristic operation | movement of this embodiment) of the mobile telephone started execution according to power activation. The flowchart following the operation | movement of FIG. 6 is a flowchart following the operation of FIG. 7 is a flowchart for explaining in detail chapter information setting processing (step A19 in FIG. 5 and step A35 in FIG. 6). (1)-(8) is the figure which showed the example of a display of the chapter information displayed at the time of reproduction | regeneration of a series of the image | photographed image | video (a moving image / continuous shooting still image). (1) to (7) are diagrams showing display examples of chapter information when, for example, a series of images (moving images / continuous shot still images) received by downloading or acquired from an external recording medium are reproduced. It is a figure for demonstrating the modification of embodiment, The state which stored the chapter information matched with the chapter position in the memory | storage part different from the memory | storage part which memorize | stores a series of images (a moving image / continuous shooting still image). The figure shown conceptually. The figure for demonstrating the other modification of embodiment, and the figure which showed notionally the state which inserted and memorize | stored chapter information in a series of images (moving image / continuous-shot still image).

An embodiment of the present invention will be described with reference to FIGS.
This embodiment exemplifies a case where the present invention is applied to a mobile phone with a camera function as a terminal device, and FIG. 1 is a block diagram showing basic components of the mobile phone with a camera function.
The mobile phone is, for example, a foldable mobile phone in which two cases (an operation unit case and a display unit case) are foldably attached, and includes a voice call function, an e-mail function, and an Internet connection function (Web access function). In addition, it has a camera function that captures a series of images (moving images, continuous shots).

  Although not shown in the figure, when the cellular phone is connected to the wireless communication network from the nearest base station, it can enter a call state with another cellular phone (not shown) via the wireless communication network. Further, when the mobile phone is connected to the Internet via a wireless communication network, the mobile phone can access and browse a Web site, and a series of images (moving images, still images, televisions) from a server device (image distribution device). Video), map information, address information, etc. can be downloaded and received via the Internet and a wireless communication network. In addition to receiving and acquiring a series of images from the image distribution apparatus in this way, it is possible to acquire a series of images captured by the above-described camera function or to acquire a copy of a series of images from an external recording medium. It is like that.

  The control unit 1 operates by supplying power from the power supply unit 2 including a secondary battery, and controls the overall operation of the mobile phone according to various programs in the storage unit 3. Are provided with a CPU (Central Processing Unit) and a memory (not shown). The storage unit 3 is an internal memory such as a ROM or a RAM, and has a program area and a data area (not shown). The program area in the storage unit 3 corresponds to an operation procedure shown in FIGS. A program for realizing the present embodiment is stored. The data area in the storage unit 3 stores various flag information and various information necessary for the operation of the mobile phone. The storage unit 3 may include a removable portable memory (recording medium) such as an SD card or an IC card, or a configuration having the storage unit 3 on a predetermined external server (not shown). It may be.

  The operation unit 4 performs dial input, character input, command input, and the like. Although not illustrated, the operation unit 4 includes a key for turning on / off the camera function in addition to an off-hook key and an on-hook key for calls. A shutter key or the like is provided. The control unit 1 performs various processes such as a transmission process, an e-mail reception process, and a camera process as processes according to the operation signal from the operation unit 4. The display unit 5 uses, for example, a high-definition liquid crystal or organic EL, and displays, for example, standby images, icons, date and time information, text data, mails, web pages, etc., and when using the camera function. The viewfinder screen displays the live view image.

  The wireless communication unit 6 includes a wireless unit, a baseband unit, a demultiplexing unit, and the like (not shown). For example, the wireless communication unit 6 transmits and receives data to and from the nearest base station when the voice call function, the electronic mail function, and the Internet connection function are operated. When the call function is in operation, a signal is taken from the receiving side of the baseband unit, demodulated into a received baseband signal, and given to the control unit 1, and the control unit 1 is connected to the call speaker SP via the telephone unit 7. The voice is output, and the input voice data from the call microphone MC is captured from the telephone unit 7 and encoded into a transmission baseband signal, which is then given to the transmission side of the baseband unit and transmitted from the antenna AT1.

  The imaging unit 8 is a component that constitutes a camera function, and can capture moving images as well as still images. Although not shown, a camera lens unit, an imaging element (for example, CCD or CMOS), an image signal processing unit , Analog processing unit, compression / expansion unit, various sensor units (ranging sensor, light amount sensor, etc.), etc., adjust optical zoom, drive control during autofocus, shutter drive control, exposure, white balance, etc. And the shutter speed (exposure time) is measured. The moving image reproduction unit 9 reproduces a moving image captured by the camera function as described above, reproduces a moving image downloaded from a server device (image distribution device), or a moving image obtained by copying from an external recording medium. It performs operations such as playing back images, and has a fast-forward function and jump function.

  The position acquisition unit 10 constitutes a GPS receiving unit (Global Positioning System) having the antenna AT2, and receives and acquires the current position (longitude / latitude information) based on the GPS signal. The moving speed detection unit 11 detects whether the mobile phone is moving or stationary due to a user's walking or the like, and when the mobile phone is moving, how fast the mobile phone is moving is displayed on the mobile phone. This is detected based on the applied vibration, and has, for example, a three-axis type acceleration sensor. The holding angle detection unit 12 detects the holding angle (tilt angle) of the mobile phone with respect to the direction of gravity when the mobile phone is held by the user, and has, for example, a three-axis type acceleration sensor and is orthogonal to each other. The holding angle (tilt angle) indicating how much the mobile phone is tilted with respect to the direction of gravity is detected based on the acceleration components in the three-axis directions (X, Y, and Z directions). The clock unit 13 counts the current date and time.

FIG. 2 is a diagram for explaining the address book AD, chapter flag F1, chapter information flag F2, position information flag F3, and scene change table ST stored in the storage unit 3.
As shown in FIG. 2A, the address book AD stores information related to a communication partner, and stores “name”, “mail address”, “phone number”, “image”,. Yes. The “image” is a partner image obtained by capturing the face of the communication partner. The chapter flag F1 indicates whether a chapter (scene delimiter) is set every time a scene (scene) of a series of images (moving images or continuous shot still images) changes as shown in FIG. When the value of the flag is “0”, it indicates that “a scene change is determined and a chapter is not set”, and when it is “1”, “a scene change is determined and a chapter is set” is indicated, The contents are arbitrarily set by a user operation.

  The chapter information flag F2 is set as shown in FIG. 2 (3) when the chapter flag F1 is “1”, that is, when chapter information (separation information) is set every time the scene changes. This flag indicates “information according to the chapter setting reason” when the value is “0”, and “user selected image” when the value is “1”, and is arbitrarily set by a user operation. is there. “Information according to the reason for setting a chapter” is chapter information indicating what reason is used for division, and is one of the reasons defined in the scene change table ST described later. The “user selected image” is, for example, an image arbitrarily selected as a chapter information from various images prepared in advance by a user operation.

  As shown in FIG. 2 (4), the scene change table ST has a plurality of types of determination criteria for determining whether or not to divide a series of images and reasons for setting the chapter information. In this configuration, five types of determination criteria (setting reasons) are stored in correspondence with the identification numbers “0 to 4”. In the illustrated example, the case where the determination criterion and the setting reason are stored in the same manner is shown. However, the present invention is not limited to this. For example, the detailed reason is set as the reason for setting or the simplified content is set as the reason for setting Alternatively, the determination criterion and the setting reason may be stored separately.

  In the illustrated example, in the scene change table ST, as a determination criterion (setting reason) corresponding to the identification number “0”, a change of a predetermined value or more is detected in the brightness of the subject in the captured image / reproduction target image. Is done "is memorized. In other words, the reason for this setting is that when a series of images (captured images or images to be reproduced) are analyzed, the brightness of the subject included in the images changes rapidly (the change state per unit time is predetermined). If the value is greater than or equal to the value, it indicates that the chapter information is set to the same).

  As a determination criterion (setting reason) corresponding to the identification number “1”, “a change greater than or equal to a predetermined value is detected in the shooting position by the position acquisition unit” is stored. That is, this setting reason is set when the current position (shooting position) detected by the position acquisition unit 10 during shooting by the imaging unit 8 changes abruptly (when the change state per unit time is a predetermined value or more). It is shown that it is chapter information. As a determination criterion (setting reason) corresponding to the identification number “2”, “a change greater than or equal to a predetermined value is detected in the amount of movement of a specific subject in the captured image / reproduction target image” is stored. That is, the reason for this setting is that, as a result of analyzing a series of images (captured images or reproduction target images), a specific subject (for example, an arbitrary subject selected by a user operation) included in the image is abruptly changed. This indicates that the chapter information is set when moving greatly (when the change state per unit time is equal to or greater than a predetermined value).

  As a determination criterion (setting reason) corresponding to the identification number “3”, “a change of a predetermined value or more in the movement of the terminal device detected by the moving speed detection unit” is stored. That is, this setting reason indicates that the chapter information is set when the moving speed of the mobile phone detected by the moving speed detecting unit 11 changes rapidly (when the change state is a predetermined value or more). Yes. As a criterion (setting reason) corresponding to the identification number “4”, “a change of a predetermined value or more in the orientation of the terminal device detected by the holding angle detection unit” is stored. That is, this setting reason indicates that the chapter information is set when the holding angle of the mobile phone detected by the holding angle detection unit 12 changes abruptly (when the change state is a predetermined value or more). Yes.

  The position information flag F3 acquires map information corresponding to the current position (shooting position) as shown in FIG. 2 (5) when the reason defined in the scene change table ST is a sudden change in shooting position. In response to “0”, “Get and set map information of current position” is stored in correspondence with “0”, and “1” is stored. "Acquire and set address information of current position" is stored. The position information flag F3 is arbitrarily set by a user operation.

FIG. 3 is a diagram for conceptually explaining a case where chapter information is set every time a scene of a series of images (moving images or continuous still images) changes. In the figure, the upper series of images indicate moving images, and the lower series of images indicate continuous shot still images.
The control unit 1 determines whether or not a predetermined change state for dividing a series of images for each scene has been reached, and when the predetermined change state is reached, a chapter for dividing the series of images for each scene. The position (separation position) is detected and the chapter information (separation information) is set in association with the division position (chapter position). At this time, the chapter information is associated with each detected chapter position. The additional information of a series of images is collectively stored in the header portion. Further, in this header portion, chapter information is stored in association with the chapter position, and a chapter setting flag F4 indicating that the chapter information has been set in a series of images is stored.

  Next, the operation concept of the mobile phone in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. In addition, the operation according to the above-described program code transmitted via a transmission medium such as a network can be sequentially executed. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium.

4 to 6 are main flowcharts for explaining the operation (characteristic operation of the present embodiment) of the mobile phone that is started when the power is turned on.
First, after executing an initialization process for clearing a memory or the like in response to a power-on operation, the control unit 1 enters a standby state such as displaying a standby image (step A1 in FIG. 4). Now, when an incoming call is detected (YES in step A2), a response process (off-hook operation) in response to this incoming call (YES in step A3) starts a call process for enabling a call (step A4). By subsequent on-hook operation (YES in step A5), after terminating the line connection and terminating the call (step A6), the process returns to step A2 for checking whether or not there is an incoming call.

  In the standby state, it is checked whether a moving image / continuous shot still image shooting operation has been performed (step A9 in FIG. 5), or whether a moving image / continuous shot still image playback operation has been performed (FIG. 6). In step A24), it is checked whether other operations have been performed (step A7 in FIG. 4). Now, when other operations are performed (YES in step A7 in FIG. 4), for example, after performing a transmission process, a mail transmission process, etc. as a process corresponding to the operation (step A8), the above-described incoming / outgoing presence / absence is checked. Return to step A2 to check.

  When a shooting operation instructing shooting of moving images / continuous shot still images is performed (YES in step A9 in FIG. 5), the imaging unit 8 is activated to start shooting of moving images or continuous shot still images. Thereafter (step A10), a series of images photographed by the imaging unit 8 are captured and displayed as a live view image on the finder screen, and the captured image is sequentially stored (step A11). Then, referring to the chapter flag F1, it is checked whether or not the value is “1”, that is, “setting a chapter by determining scene change” is specified (step A12).

  If the chapter flag F1 is “0” (NO in step A12), it is checked whether an instruction to end the shooting of the moving image / continuous shooting still image is given (step A20). This termination instruction is a termination instruction by a user operation or a forced termination instruction due to a lack of storage capacity, a battery shortage, or the like. If there is no termination instruction for shooting (NO in step A20), step A11 is performed to continue the shooting process. Return to. If the end of shooting is instructed (YES in step A20), after shooting of moving images or continuous shooting still images is ended (step A21), it is checked whether the chapter flag F1 is “1”. (Step A22) Since it is “0” (NO in Step A22), the process returns to Step A2 in FIG.

  If the value of the chapter flag F1 is “1” (YES in step A12), it is checked whether or not a subject selection operation has been performed (step A13). Here, when a desired subject (person) is captured in the live view image displayed on the viewfinder screen, the photographer performs an operation for selecting the subject. This subject selecting operation is performed by identifying the identification number “2” of the above-described determination criterion (setting reason), that is, “a change of a predetermined subject or more is detected in the amount of movement of a specific subject in the captured image / reproduction target image”. This is the operation performed in the case. If such a subject selection operation is not performed (NO in step A13), the process proceeds to step A17 described later.

  Now, when a subject selection operation is performed (YES in step A13), the selected subject (person) is extracted from the live view image, and each “image” in the address book AD is referenced based on this person image ( In step A14), it is checked whether similar images (same person) that are characteristically similar are registered (step A15). If similar images (same person) are registered (YES in step A15), the selected subject is selected. “Name” and “image” are read and acquired from the address book AD (step A16). Thereafter, the process proceeds to step A17. The “name” and “image” are used when generating chapter information indicating that the identification number of the above-described determination criterion (setting reason) is “2”.

  In step A17 described above, a scene determination process is performed to determine whether or not a predetermined change state for dividing a series of images for each scene is made with reference to the scene change table ST. That is, the control unit 1 analyzes a series of images to detect the change state of the brightness of the subject, to detect the change state of the movement amount of the specific subject, or to obtain the current position (from the position acquisition unit 10) The change state of the shooting position is detected based on the shooting position), the change state is detected based on the movement of the mobile phone (terminal device) detected by the moving speed detection unit 11, or the holding angle detection unit 12 The change state is detected based on the detected orientation of the mobile phone (terminal device). Then, the control unit 1 checks whether or not the detected change state corresponds to any criterion in the scene change table ST, that is, whether or not the detected change state is a predetermined change state (Step S1). A18) If it is a predetermined change state corresponding to one of the determination criteria (YES in step A18), the process proceeds to a chapter information setting process (step A19).

FIG. 7 is a flowchart for explaining the chapter information setting process (step A19 in FIG. 6 and the like) in detail.
First, when the control unit 1 is in a predetermined change state for dividing a series of images for each scene (when any of the determination criteria in the scene change table ST is satisfied), the control unit 1 stores the position (image storage). Position) is detected and acquired as a chapter position (step B1). The chapter position is not limited to the image storage position when the predetermined change state is reached, and may be, for example, an elapsed time since the start of photographing. Then, it is checked whether the value is “1” with reference to the chapter information flag F2 (step B2).

  If the value of the chapter information flag F2 is “1” indicating “user selected image” (YES in step B2), a process of selecting an arbitrary image as chapter information by a user operation is performed (step B3). In this case, for example, an arbitrary image is selected from various images prepared in advance, but an image selected in advance by a user operation may be read out. Then, the chapter information (optionally selected chapter image) is stored in the header portion of a series of images in association with the chapter position (step B4).

  If the value of the chapter information flag F2 is “0” indicating “information according to the chapter setting reason” (NO in step B2), the chapter setting reason corresponding to the corresponding determination criterion is read from the scene change table ST. After the acquisition (step B5), the identification number of the determination criterion (setting reason) is “2”, that is, “a change greater than a predetermined value is detected in the amount of movement of a specific subject in the image to be captured / reproduced”. It is checked whether it is a case (step B6). Here, if the identification number of the criterion (setting reason) is “2” (YES in step B6), as described above, the “name” and “image” of the selected subject from the address book AD in step A16 of FIG. Whether the “name” and “image” of the selected subject can be specified (step B7). If the selected subject can be identified (YES in step B7), an image including the “name” and “image” of the selected subject is generated as chapter information (chapter image) (step B8).

  Then, it is checked whether or not shooting is in progress (step B9). Since shooting is now being performed (YES in step B9), the current date and time is acquired from the clock unit 13 (step B10), and then the criterion (setting reason) Is identified as “1”, that is, whether “a change of a predetermined value or more is detected at the photographing position by the position acquisition unit” is checked (step B11). Here, if the identification number of the criterion (setting reason) is not “1” (NO in Step B11), the chapter information is stored in the header portion of a series of images in association with the chapter position together with the current date and time information (Step S11). B12).

  In this case, if the identification number of the criterion (setting reason) is “0”, the reason (fixed message) indicating that the separation is due to the brightness change of the subject is stored as the chapter information. If the identification number of the determination criterion (setting reason) is “2”, the generated image including the subject “name” and “image” in the reason (message) indicating that the separation is due to a change in the movement of the specific subject. Is stored as chapter information. If the identification number of the determination criterion (setting reason) is “3”, the reason (fixed message) indicating that the terminal device (mobile phone) is delimited by a change in the movement (moving speed of the mobile phone) is a chapter. Stored as information. Also, if the identification number of the identification number of the determination criterion (setting reason) is “4”, the reason (fixed message) indicating that it is a break due to the change in orientation of the terminal device is stored as chapter information.

  If the identification number of the determination criterion (setting reason) is “1” (YES in step B11), after acquiring the current position from the position acquisition unit 10 (step B13), the position information flag F3 is referred to, and the value is Whether it is “1”, that is, “acquire and set address information of current position” is checked (step B14). If the position information flag F3 is “1” (YES in step B14), the address information corresponding to the current position is downloaded from the server device (image distribution device) via the Internet, or stored in the storage unit 3. If the address database is stored, the address information is read out from the storage unit 3 and acquired (step B15), and then the chapter information (address information) together with the current date and time information is associated with the chapter position and the header of a series of images. (Step B16).

  If the position information flag F3 is “0” (NO in step B14), map information corresponding to the current position is downloaded and acquired from the server device (image distribution device) via the Internet, or the map database is stored in the storage unit 3. Is stored, the map information is read out from the storage unit 3 and acquired, and an image including the map information is generated as chapter information (step B17). Then, the chapter information (the chapter image of the generated map) is stored in the header portion of the series of images together with the current date and time information in association with the chapter position (step B18).

  When the chapter information setting process is completed, the process returns to step A11 in FIG. 5 to continue the shooting process, and the above operation is repeated on condition that the value of the chapter flag F1 is “1”. Here, as a result of executing Step A17 described above, if it is not a predetermined change state that does not correspond to any of the determination criteria (NO in Step A18), the process proceeds to Step A20 for checking the end of photographing. If the end of shooting is not instructed (NO in step A20), the process returns to step A11 of FIG. 5 to continue the shooting process. However, when the end of shooting is instructed (YES in step A20), The shooting or continuous shooting still image shooting is terminated (step A21). Then, on condition that the chapter flag F1 is “1” (YES in step A22), the chapter set flag F4 is set to “1” (step A23), and then the above-described incoming / outgoing presence check is performed. Return to A2.

  On the other hand, when a reproduction operation for instructing reproduction is performed on a series of images (moving images / continuous shot still images) taken by the imaging unit 8 (YES in step A24 in FIG. 6), the reproduction target is set by a user operation. A series of captured images (moving image / continuous shot still image) selected as is read out (step A25), and a reproduction operation for sequentially outputting from the display unit 5 is started (step A26). In this case, if the reproduction target is a moving image, the moving image reproduction unit 9 is activated. Then, with reference to the chapter setting flag F4 added to the header portion of the series of images selected as the reproduction target, it is checked whether the value is “1”, that is, the chapter has been set (step A27). . If the chapter set flag F4 is “1”, that is, if the chapter has been set (YES in step A27), the presence / absence of a chapter search operation is checked (step A36).

  This chapter search operation is an instruction to sequentially search chapter positions and start playback from that chapter position. When the chapter search operation is performed (YES in step A36), the header portion of a series of images is searched. (Step A37), the chapter information stored in association with the searched chapter position is read out and displayed on the display unit 5 (Step A38), and the jump is made to the searched chapter position and reproduction is started from the chapter position. (Step A39). Then, returning to the above-described step A26, each time a chapter search operation is performed (YES in step A36), the chapter information is displayed while sequentially searching for the chapter position, and the jump to the searched chapter position is performed. The operation of starting playback from the chapter position is repeated.

FIG. 8 is a diagram illustrating a display example of chapter information displayed when a captured moving image / continuous shot still image is reproduced.
8 (1) to (7) show a case where the chapter information flag F2 is “0”. Further, FIG. 8 (1) shows a case where the setting reason is “0”, and a break due to a change in the brightness of the subject. The reason (fixed message) indicating that it is is displayed together with the current date and time. In FIG. 8B, when the setting reason is “1” and the position information flag F3 is “0”, a generated image including map information corresponding to the current position is displayed together with the current date and time. FIG. 8 (3) shows the case where the identification number of the setting reason is “1” and the position information flag F3 is “1”, and the address information corresponding to the current position is displayed together with the current date and time.

  FIG. 8 (4) shows the reason (fixed message) indicating that the setting subject identification number is “2” and the “name” and “image” of the selected subject can be specified, and that the separation is due to the change in the movement of the specific subject. ) Displays a generated image including the subject “name” and “image”. FIG. 8 (5) shows the reason (fixed) indicating that the identification name of the setting reason is “2” and the “name” and “image” of the selected subject cannot be specified, and that the separation is due to a change in the movement of the specific subject. Message) is displayed. FIG. 8 (6) displays the reason (fixed message) indicating that the setting reason identification number is “3” and that it is a break due to a change in the movement of the terminal. FIG. 8 (7) shows the reason (fixed message) indicating that the setting reason identification number is “4” and that it is a break due to a change in the orientation of the terminal. FIG. 8 (8) shows a case where the chapter information flag F2 is “1” and the user selection screen is displayed.

  If no chapter search operation is performed (NO in step A36), it is checked whether or not an instruction to end reproduction of a moving image / continuous shot still image is given (step A40). The termination instruction in this case is a termination instruction by a user operation or a forced termination instruction due to a lack of storage capacity, a battery shortage, or the like. If the reproduction termination is not instructed (NO in step A40), the reproduction process is continued. Return to Step A26. When the end of playback is instructed (YES in step A40), after shooting of moving images or continuous playback of still images is ended (step A41), it is checked whether the chapter flag F1 is “1”. (Step A42), if the chapter flag F1 is "0" (NO in Step A42), the process returns to Step A2 in FIG.

On the other hand, a series of images that have been captured by the imaging unit 8 and have not been set with chapters , or playback operations for images that have not been set with other chapters , for example, downloaded from the server device (image distribution device) via the Internet, or externally recorded When a reproduction operation is performed on a series of images (moving images / continuous shot still images) obtained by copying from the medium, it is determined that the chapter set flag F4 is “0”, that is, the chapter has not been set. Therefore (NO in step A27), the chapter flag F1 is referred to and it is checked whether or not the value is “1”, that is, “when a scene change is determined and a chapter is set” (step A28). If the chapter flag F1 is “0” (NO in step A28), the process proceeds to step A36 where the above-described chapter search operation is checked, but if the chapter flag F1 is “1” (step S36). Hereinafter, the same operation as that at the time of shooting described above is performed.

  That is, when a subject selection operation is performed (YES in step A29), the address book AD is referred to (step A30), and if similar similar images (the same person) are registered (YES in step A31). ) The “name” and “image” of the selected subject are read out and acquired from the address book AD (step A32). Then, with reference to the scene change table ST, a process for determining whether or not a predetermined change state for dividing a series of images into scenes is performed (step A33), and a predetermined condition corresponding to any one of the determination criteria is performed. It is checked whether or not a change state has been reached (step A34). If a predetermined change state has been reached (YES in step A34), the process proceeds to a chapter information setting process (step A35).

  The chapter information setting process in this case is also executed according to the flowchart of FIG. That is, after detecting and acquiring the chapter position (step B1 in FIG. 7), the control unit 1 refers to the chapter information flag F2 and checks whether the value is “1” (step B2). If the chapter information flag F2 is “1” (YES in step B2), an arbitrary image is selected as the chapter information by a user operation (step B3), and then the chapter information (optionally selected) is associated with the chapter position. (Chapter images) are stored in the header of a series of images (step B4).

  If the chapter information flag F2 is “0” (NO in step B2), after reading and acquiring the chapter setting reason corresponding to the determination criterion corresponding to the scene change table ST (step B5), the determination criterion (setting) It is checked whether the identification number of “reason” is “2” (step B6), and if it is 2 (YES in step B6), the “name” and “image” of the selected subject are specified from the address book AD. On the condition that it has been completed (YES in step B7), an image including “name” and “image” is generated as chapter information (step B8). Whether or not shooting is in progress is checked (step B9), but since shooting is not in progress (NO in step B9), chapter information is stored in the header portion of a series of images in association with chapter positions (step B19). ).

  When the chapter information setting process is completed, the process returns to step A26 in FIG. 6 to continue the reproduction process. Hereinafter, the above-described reproduction operation is repeated on condition that the value of the chapter flag F1 is “1”. . If the result of executing step A33 is not the predetermined change state (NO in step A34), the process proceeds to step A40 for checking the end of reproduction. If the end of playback is not instructed (NO in step A40), the process returns to step A26 to continue the playback process. However, if the end of playback is instructed (YES in step A40), a moving image is captured or continuously. The reproduction of the still image is terminated (step A41). Then, on condition that the chapter flag F1 is “1” (YES in step A42), the chapter set flag F4 is set to “1” (step A43), and then the above-described presence / absence of the incoming call is checked. Return to A2.

  After a chapter is set for a series of images (moving images / continuous shot still images) obtained by download reception or copied from an external recording medium in this way, if playback of the images is instructed (step of FIG. 6) A24), the reproduction process is executed (steps A25 and A26). In this case, since the chapter setting flag F4 of the image is “1” (YES in step A27), the chapter search operation is performed thereafter. Each time (YES in step A36), the chapter information is displayed while sequentially searching for the chapter position, and the operation of jumping to the searched chapter position and starting reproduction from the chapter position is performed (steps A37 to A39). ).

  FIG. 9 is a diagram showing a display example of chapter information at the time of reproduction of a series of images (moving images / continuous shot still images) received by download reception or copied from an external recording medium, for example. FIGS. 9 (1) to (6) show the case where the chapter information flag F2 is “0”, and FIG. 9 (1) shows the brightness of the subject when the setting reason identification number is “0”. The reason (fixed message) indicating that the change is due to a change is displayed. FIG. 9B shows a reason (fixed message) indicating that the setting reason identification number is “1” and that it is a break due to a change in the current position (shooting position). Hereinafter, FIGS. 9 (3) to 9 (6) are the same as FIGS. 8 (4) to 8 (7) except that the current date and time are not displayed. FIG. 9 (7) shows a case where the chapter information flag F2 is “1”, which is the same as the case of FIG. 8 (8).

  As described above, in the present embodiment, the control unit 1 sets the delimiter information (chapter information) indicating the reason for delimiting when a predetermined change state for delimiting a series of images for each scene is reached. Since it is stored in association with the break position (chapter position), the user can freely read the break information (chapter information) at any time, and a series of images is separated for any reason. It can be easily confirmed whether it is convenient or not.

  Since the image section 8 is divided when a series of images are captured when the imaging section 8 is in a predetermined change state, it is possible to set chapters in appropriate scenes during the capture.

  Since a series of images captured by the imaging unit 8 is reproduced when the images are in a predetermined change state, the chapters are set in appropriate scenes during the reproduction of the captured images. Can do.

  Since a series of images are separated when the images are in a predetermined change state at the time of reception, chapter setting can be performed for the images at appropriate scenes during image reception.

  The chapter information corresponding to the determination criterion is read out from the scene change table ST on the assumption that the predetermined change state has been reached when the determination criterion stored in the scene change table ST is met. Chapter information of various contents can be obtained.

  By analyzing a series of images, it is divided when the brightness of the subject is in a predetermined change state, so it is possible to catch a sudden change in brightness of the subject as a change in the scene and Settings can be made.

  Since the current position (shooting position) acquired from the position acquisition unit 10 during shooting by the imaging unit 8 is changed to a predetermined change state, a sudden change in shooting position can be regarded as a change in scene. It is possible to set chapters in appropriate scenes.

  Since the movement of the subject included in the image during the shooting by the imaging unit 8 is changed into a predetermined change state during the shooting, the sudden movement of the subject can be captured as a change in the scene. It is possible to set chapters in appropriate scenes.

  Since the movement of the terminal device detected by the movement speed detection unit 11 (the movement speed of the mobile phone) is changed to a predetermined change state, the movement of the terminal device is captured as a change in scene. You can set chapters in appropriate scenes.

  Since the direction of the terminal device detected by the holding angle detection unit 12 is divided when the state changes to a predetermined state, the direction of the terminal device can be regarded as a change in scene, and in an appropriate scene Chapter settings can be made.

  When the chapter position is detected, the map information or the address information corresponding to the current position acquired by the position acquisition unit 10 is acquired as the chapter information indicating the reason for dividing the chapter position. Easy-to-understand chapter information can be obtained.

  When the chapter position is detected, the current date and time from the clock unit 13 is acquired as a part of the chapter information indicating the reason why the chapter position is divided. Can be obtained.

  When the chapter information is stored in association with the chapter position, it is stored as additional information (header part information) of the series of images. Management becomes easy.

  Since the information related to the scene at the chapter position is generated as the chapter information, appropriate chapter information related to the scene can be obtained.

  Since the information arbitrarily selected by the user operation as the chapter information is selected and acquired, it is possible to obtain appropriate chapter information that suits the user's preference.

  In a state where the chapter information is stored in association with the chapter position, the chapter position in the series of images is searched in response to the chapter search instruction, and the chapter information corresponding to the chapter position is displayed. In this case, chapter information can be freely obtained at any time for each chapter position.

  In the above-described embodiment, when chapter information is stored in association with chapter positions, it is stored in the header portion of a series of images (moving images / continuous shot still images). You may make it memorize | store and link with the other memory | storage part different from the memory | storage part to memorize | store. The storage unit is not limited to another storage medium, and may be a storage area, file, or folder in the same storage medium.

  FIG. 10 is a diagram conceptually illustrating a state in which chapter information is stored in association with chapter positions in a storage unit different from the storage unit that stores a series of images (moving images / continuous shot still images). . In the figure, the upper series of images indicate moving images, and the lower series of images indicate continuous shot still images. In the illustrated example, a chapter information file CP is shown as another storage unit. In this case, the chapter position is stored as the link information by storing the chapter position in the header portion of the series of images and storing the chapter information in the chapter information file CP in association with the chapter position.

  In this way, if chapter information is stored in association with chapter positions in another storage unit that is different from the storage unit that stores a series of images, even if the storage unit capacity is insufficient, the chapter information is distributed and stored. It can be managed and becomes flexible.

  In the above-described embodiment, when chapter information is stored in association with chapter positions, it is stored in the header portion of a series of images (moving images / continuous shot still images). The chapter information (chapter still image) corresponding to each chapter position may be inserted and stored.

  FIG. 11 is a diagram conceptually showing a state in which chapter information is inserted and stored in a series of images (moving images / continuous shot still images). In the figure, the upper series of images indicate moving images, and the lower series of images indicate continuous shot still images. In the illustrated example, “CI” indicates chapter information (chapter still image) inserted at a chapter position. Also in this case, the chapter position is the link information by storing the chapter position in the header portion of the series of images and storing the chapter information in the chapter information file CP in association with the chapter position. Note that even if the chapter information is composed only of a character string such as a fixed message sentence, an address, and a date and time, it is a chapter still image obtained by imaging the character string.

  In this way, if chapter information (chapter still image) is inserted and stored at a chapter position in a series of images, chapter information (chapter still image) is displayed during playback of the series of images. Thus, it becomes possible to clearly know the chapter position and the reason.

  In the above-described embodiment, as a determination criterion (setting reason) for setting a chapter, when the brightness of the subject changes abruptly, when the shooting position changes abruptly, a specific subject moves abruptly and greatly. In this case, when the moving speed of the mobile phone changes abruptly, each case where the holding angle of the mobile phone changes abruptly is used as a reference, but a combination of two or more of them may be used as a determination reference (setting reason).

  Further, in the above-described embodiment, the chapter position in the series of images is searched and the chapter information corresponding to the chapter position is displayed. However, the display method is arbitrary, and is displayed on the entire playback screen. In addition to being displayed or partially displayed, a voice message may be generated and output.

  In the above-described embodiment, the mobile phone is provided with a function of searching for chapter positions in a series of images and displaying chapter information. However, search and playback are performed by different devices (such as personal computers and PDAs). You may make it carry out. That is, chapter setting may be performed on the mobile phone side, and a series of images set with chapters may be reproduced by another device.

  In the above-described embodiments, the case where the present invention is applied to a mobile phone as a terminal device has been shown. However, a personal computer with a camera function, a PDA, a digital camera, a music player, or the like may be used. May be.

  Further, the “device” and “machine” shown in the above-described embodiments may be separated into a plurality of cases according to function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

DESCRIPTION OF SYMBOLS 1 Control part 3 Memory | storage part 4 Operation part 5 Display part 6 Wireless communication part 8 Imaging part 9 Moving image reproduction part 10 Position acquisition part 11 Movement speed detection part 12 Holding angle detection part 13 Clock part AD Address book CP Chapter information file F1 Chapter Flag F2 Chapter information flag F3 Position information flag F4 Chapter set flag ST Scene change table

Claims (18)

A terminal device that processes a series of images,
Image acquisition means for acquiring the series of images;
Change determination means for determining whether or not a predetermined change state for dividing a series of images acquired by the image acquisition means into scenes has been reached;
When it is determined by the change determining means that a predetermined change state has been reached, delimiter means for delimiting the position where the change state has occurred as a scene delimiter;
An information acquisition unit that acquires delimiter information indicating why the image is delimited when the series of images is delimited by the delimiter;
Storage control means for inserting and storing the delimiter information acquired by the information acquisition means at a delimiter position delimited by the delimiter means in the series of images ;
A terminal device comprising: The image acquisition means is an imaging means for taking a series of images,
The change determining unit determines whether or not a change in an image obtained by sequentially analyzing the images at the time of capturing a series of images by the imaging unit is in the predetermined change state.
The terminal device according to claim 1, which is configured as described above. The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether or not a change in an image obtained by sequentially analyzing the images taken when the series of images taken by the imaging means is in the predetermined change state;
The terminal device according to claim 1, which is configured as described above. The image acquisition means is a reception means for receiving a series of images,
The change determining means determines whether the change in the image obtained by sequentially analyzes of reproduction at the image series of images Ri received by the receiving means becomes the predetermined changing state,
The terminal device according to claim 1, which is configured as described above. A judgment criterion for determining whether or not to divide the series of images, and a criterion storage means for storing delimitation information indicating the reason for delimiting according to the criterion;
The change determination means determines whether or not the predetermined change state has been reached, depending on whether or not the determination criterion is met,
The information acquisition means reads and acquires the separation information stored in the determination criterion storage means in association with the determination criterion when the series of images are divided based on the determination criterion;
The terminal device according to claim 1, which is configured as described above. The change determination unit determines whether or not the brightness of the subject has changed to the predetermined change state by analyzing a series of images acquired by the image acquisition unit;
The terminal device according to claim 1, which is configured as described above. It further comprises position acquisition means for acquiring the current position of the terminal device,
The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether the current position acquired by the position acquisition means during shooting by the imaging means is a shooting position, and whether or not the shooting position is in the predetermined change state.
The terminal device according to claim 1, which is configured as described above. The image acquisition means is an imaging means for taking a series of images,
The change determining means determines whether or not the movement of a subject included in the series of images has become the predetermined change state during photographing by the imaging means.
The terminal device according to claim 1, which is configured as described above. It further comprises a motion detection means for detecting the motion of the terminal device,
The change determining means determines whether or not the predetermined change state has been reached in accordance with the change state of the movement of the terminal device detected by the motion detection means.
The terminal device according to claim 1, which is configured as described above. It further comprises a direction detecting means for detecting the direction of the terminal device,
The change determining means determines whether or not the predetermined change state has been reached in accordance with the change state of the direction of the terminal device detected by the direction detection means.
The terminal device according to claim 1, which is configured as described above. It further comprises position acquisition means for acquiring the current position of the terminal device,
The information acquisition means includes map information or an address corresponding to the current position acquired by the position acquisition means as separation information indicating why the separation is performed when the series of images are separated by the separation means. Get information,
The terminal device according to claim 1, which is configured as described above. It further has a time measuring means for measuring the current date and time,
The information acquisition means acquires the current date and time acquired by the time measuring means as a part of separation information indicating why the separation is performed when the series of images are separated by the separation means.
The terminal device according to claim 1, which is configured as described above. The storage control unit stores the delimiter information acquired by the information acquisition unit as additional information to be added to the series of images when storing the delimiter information in association with the delimiter positions delimited by the delimiter unit.
The terminal device according to claim 1, which is configured as described above. The storage control unit stores other information different from the storage unit that stores the series of images when storing the delimiter information acquired by the information acquisition unit in association with the delimiter positions delimited by the delimiter. Linked to the department and memorized,
The terminal device according to claim 1, which is configured as described above. The information acquisition means generates and acquires information related to a scene at a separation position separated by the separation means as separation information;
The terminal device according to claim 1, which is configured as described above. The information acquisition means selectively acquires information arbitrarily selected by a user operation as the delimiter information.
The terminal device according to claim 1, which is configured as described above. In a state where the delimiter information acquired by the information acquisition unit is stored in association with the delimiter positions delimited by the delimiter unit, the delimiter positions in the series of images are searched in response to a delimiter search instruction. Search means;
Output control means for outputting the delimiter information associated with the delimiter positions delimited by the search means;
Further comprising
The terminal device according to claim 1, which is configured as described above. Against the computer,
A function to acquire a series of images,
A function of determining whether or not a predetermined change state for dividing the acquired series of images for each scene has been reached;
A function of dividing the position where the predetermined change state has occurred as a scene break;
A function of acquiring delimiter information indicating why the image is delimited when the series of images is delimited,
A function of inserting and storing the acquired delimiter information at the delimiter position in the series of images ;
A program to realize

Images