JP2017022606A - Encoding method and encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an encoding method capable of improving efficiency in encoding images in the case where the images are displayed adjacently so as to have spatial continuity, and an encoding program.SOLUTION: The encoding method is an encoding method in an encoding device and includes the steps of: acquiring an image subjected to encoding; determining scores in a plurality of regions determined in a row direction or a column direction in the image in accordance with a predetermined element; and rearranging the regions in accordance with whether it is a region of a relatively high score in the image when encoding the image into an intra-frame, and encoding the image in which the regions are rearranged.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an encoding method and an encoding program.

  In recent years, with the spread of broadband services, it has become easy to view moving image content via a network. It is said that more than half of the data on the network becomes moving image data by making it easier to view moving image content.

  In addition, there is an image service that gives viewers a high sense of realism by displaying an all-sky image so that the viewing angle is 360 degrees. The moving image includes a plurality of frames (still images). In the all-sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the all-sky image, it is displayed so that the end in the frame is adjacent to the end in the other frame. For example, in the all-sky image, the right end of the frame is displayed so as to be adjacent to the left end of another frame. For example, in the all-sky image, the upper end of the frame is displayed so as to be adjacent to the lower end of another frame.

  In addition, H.263, which is one of the video compression standards, employs non-restricted motion compensation technology such as out-of-screen reference (see Non-Patent Document 1).

ITU-T H.263 (ANNEX D)

  The conventional encoding apparatus has a problem that the efficiency of image encoding cannot be improved when images are displayed adjacently so as to have spatial continuity.

  In view of the above circumstances, the present invention provides an encoding method and an encoding program capable of improving the efficiency of image encoding when images are displayed adjacently so as to have spatial continuity. The purpose is to do.

  One aspect of the present invention is an encoding method in an encoding device, which is determined in a row direction or a column direction in the image according to a step of acquiring an image to be encoded and a predetermined element. Re-arranging the areas according to whether or not the score is relatively high in the image when the image is encoded into an intra frame, Encoding the image in which the regions are rearranged.

  One aspect of the present invention is the encoding method described above, in which, when the image is encoded in an interframe, the regions at the ends facing each other in the image are set as a motion search range, and the motion search is performed. And a step of encoding the image based on a result.

  One aspect of the present invention is an encoding program for causing a computer to execute the above encoding method.

  According to the present invention, when moving images are displayed adjacent to each other so as to have spatial continuity, it is possible to improve the efficiency of encoding frames of moving images.

It is a figure which shows the example of a structure of the encoding apparatus in 1st Embodiment. It is a figure which shows the example of rearrangement of the block line of the column direction in 1st Embodiment. It is a figure which shows the example of rearrangement of the block line of a row direction in 1st Embodiment. It is a flowchart which shows the example of the procedure of the pre-processing in 1st Embodiment. It is a flowchart which shows the example of the procedure of the pre-processing in 2nd Embodiment. 10 is a flowchart illustrating an example of a preprocessing procedure according to the third embodiment. It is a flowchart which shows the example of the procedure of the pre-processing in 4th Embodiment.

Embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram illustrating an example of the configuration of the encoding device 10. The encoding device 10 includes a preprocessing unit 11, a storage unit 12, an encoding unit 13, and an output unit 14. Some or all of the preprocessing unit 11, the encoding unit 13, and the output unit 14 are software that functions when, for example, a processor such as a CPU (Central Processing Unit) executes a program stored in a memory. It is a functional part. Some or all of these functional units may be hardware functional units such as LSI (Large Scale Integration) and ASIC (Application Specific Integrated Circuit).

  The preprocessing unit 11 acquires a frame (original image) to be encoded, which is an all-sky image. The image may be a still image or a moving image. The encoding standard is not limited to a specific standard. For example, the still image may be encoded based on the JPEG (Joint Photographic Experts Group) standard. For example, a moving image may be encoded based on the MPEG (Moving Picture Experts Group) standard or the H.264 standard.

  Below, the pre-processing part 11 acquires a moving image as an example. The moving image includes a plurality of frames (still images). In the all-sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the all-sky image, the frame is displayed so that the edge in the frame is adjacent to the edge in the other frame. The preprocessing unit 11 performs preprocessing on a moving image including a frame to be encoded. The preprocessing unit 11 transfers the preprocessed moving image to the encoding unit 13.

  The preprocessing unit 11 processes a frame to be encoded into a frame that is easily encoded by the encoding unit 13. An easily encoded frame is, for example, a frame with high image redundancy. The encoding unit 13 can improve the encoding efficiency by performing an encoding process using image redundancy on a frame (original image).

  Hereinafter, a unit area of a frame to be encoded is referred to as a “block”. Hereinafter, a plurality of blocks arranged in one direction in the frame are referred to as “block lines”.

  The preprocessing unit 11 determines a plurality of block lines in the row direction or the column direction in the frame. The preprocessing unit 11 may acquire a frame in which a plurality of block lines are determined in advance in the row direction or the column direction. The preprocessing unit 11 determines a score based on an image element in the block line in the block line. The image element is, for example, an image activity. The activity is, for example, the complexity or flatness of the texture as a pixel value. For example, the pre-processing unit 11 determines a higher score for the block line as the texture in the block line is more complicated. That is, the pre-processing unit 11 determines a lower score for the block line as the texture on the block line is flatter.

  Further, the elements of the image are expressed by, for example, whether or not a gradation exists in the image. For example, the preprocessing unit 11 determines a higher score for a block line where gradation exists than a block line where gradation does not exist. The elements of the image are expressed by luminance as pixel values, for example. For example, the preprocessing unit 11 determines a higher score for a block line in which a high luminance area exists than a block line in which a high luminance area does not exist.

  When the first block line, the second block line, and the third block line are arranged in this order in the frame, the pre-processing unit 11 satisfies the second condition that satisfies all of the first condition to the third condition. Detect block lines. Note that the preprocessing unit 11 may detect a second block line that satisfies at least one of the first condition to the third condition.

  The first condition is that the difference between the score of the first block line and the score of the second block line is not more than a threshold value. The second condition is that the difference between the score of the third block line and the score of the second block line is equal to or greater than a threshold value. The third condition is that the difference between the score of the first block line and the score of the third block line is the largest in the frame compared to the difference of the scores between the other block lines.

  FIG. 2 is a diagram illustrating an example of rearrangement of column lines in the column direction according to the first embodiment. In FIG. 2, the frame to be encoded is composed of block lines 1a to 6a in the column direction. In FIG. 2, as an example, the difference between the score of the block line 2a and the score of the block line 3a is equal to or less than the threshold value. The difference between the score of the block line 3a and the score of the block line 4a is equal to or greater than the threshold value. The difference between the score of the block line 2a and the score of the block line 4a is the largest in the frame compared to the difference of the scores between the other block lines.

  The pre-processing unit 11 separates the first block line having a small score difference from the detected second block line and the detected second block line. In FIG. 2, the preprocessing unit 11 separates the block line 2a and the block line 3a. The preprocessing unit 11 may separate the block line group from other block line groups. In FIG. 2, the pre-processing unit 11 separates the block lines 1a and 2a and the block lines 3a to 6a when separating the block line 2a and the block line 3a.

  The preprocessing unit 11 rearranges the block line group in the frame by joining the separated block line to another block line. The pre-processing unit 11 adjoins block lines having high scores by joining block lines having high scores. In FIG. 2, the preprocessing unit 11 arranges the block line 1 a and the block lines 4 a to 6 a so as to be adjacent to each other than the end of the frame. The encoding unit 13 can improve the encoding efficiency by performing an encoding process using image redundancy on a frame in which block lines having high scores are adjacent to each other.

  The preprocessing unit 11 rearranges the block lines having a low score at the end of the frame. The preprocessing unit 11 may rearrange a plurality of block lines having low scores at both ends of the frame. In FIG. 2, the preprocessing unit 11 arranges the block line 2a at the right end of the frame. The preprocessing unit 11 arranges the block line 3a at the left end of the frame. In the all-sky image, the leftmost block of the frame and the rightmost block of the other frame have spatial continuity, so that the redundancy of the image is high. The encoding unit 13 can improve the efficiency of encoding by performing an encoding process using image redundancy on a frame in which a plurality of block lines having low scores are arranged at both ends.

  FIG. 3 is a diagram illustrating an example of rearrangement of block lines in the row direction according to the first embodiment. In FIG. 3, the frame to be encoded is composed of block lines 1b to 6b in the row direction. In FIG. 3, as an example, the difference between the score of the block line 4b and the score of the block line 5b is equal to or less than the threshold value. The difference between the score of the block line 5b and the score of the block line 6b is equal to or greater than the threshold value. The difference between the score of the block line 4b and the score of the block line 6b is the largest in the frame compared to the difference of the scores between the other block lines.

  The pre-processing unit 11 separates the first block line having a small score difference from the detected second block line and the detected second block line. In FIG. 3, the preprocessing unit 11 separates the block line 4b and the block line 5b. The preprocessing unit 11 may separate the block line group from other block line groups. In FIG. 3, the preprocessing unit 11 separates the block lines 1b to 4b and the block lines 5b and 6b when the block line 4b and the block line 5b are separated.

  The preprocessing unit 11 rearranges the block line group in the frame by joining the separated block line to another block line. The pre-processing unit 11 adjoins block lines having high scores by joining block lines having high scores. In FIG. 3, the pre-processing unit 11 arranges the block line 6b and the block lines 1b to 3b adjacent to each other except at the end of the frame. The encoding unit 13 can improve the encoding efficiency by performing an encoding process using image redundancy on a frame in which block lines having high scores are adjacent to each other.

  The preprocessing unit 11 rearranges the block lines having a low score at the end of the frame. The preprocessing unit 11 may rearrange a plurality of block lines having low scores at both ends of the frame. In FIG. 3, the preprocessing unit 11 arranges the block line 1b at the lower end of the frame. The preprocessing unit 11 arranges the block line 5b at the upper end of the frame. In the all-sky image, the redundancy of the image is high because the block at the lower end of the frame and the block at the upper end of the other frame have spatial continuity. The encoding unit 13 improves encoding efficiency by performing encoding processing using image redundancy on a frame in which a plurality of block lines having low scores are arranged at both ends (ends facing each other). Can be made.

  The storage unit 12 includes, for example, a nonvolatile storage medium (non-temporary recording medium) such as a ROM (Read Only Memory), a flash memory, and an HDD (Hard Disk Drive). The storage unit 12 may include, for example, a volatile storage medium such as a RAM (Random Access Memory) or a register. The storage unit 12 may store a program for causing the software function unit to function, for example. The storage unit 12 may store parameters used for the encoding process, for example. The storage unit 12 may store parameters used for preprocessing, for example. The storage unit 12 may store a threshold value.

  The encoding unit 13 acquires a moving image including a plurality of frames subjected to preprocessing from the preprocessing unit 11. The encoding unit 13 encodes a plurality of frames that have been subjected to preprocessing. In the first embodiment, the encoding unit 13 encodes a frame by intra coding. For example, the encoding unit 13 encodes a frame by executing an encoding program. The encoding unit 13 transmits the encoded frame to the output unit 14. The encoding unit 13 may store the encoded frame in the storage unit 12.

  The output unit 14 outputs the encoded frame to the display device. The display device decodes the encoded frame and displays a moving image including the decoded frame as an all-sky image. That is, the output unit 14 outputs a moving image to the display device so that a plurality of frames are displayed adjacent to each other.

  FIG. 4 is a flowchart illustrating an example of a preprocessing procedure. Steps S103 to S106 shown in FIG. 4 are executed for at least one of the block line in the column direction and the block line in the row direction.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S101). The pre-processing unit 11 determines whether or not the acquired frame is a frame displayed adjacently so as to have spatial continuity. That is, the preprocessing unit 11 determines whether or not the acquired frame is an all-sky image (step S102).

  When the acquired frame is not an all-sky image (step S102: No), the preprocessing unit 11 advances the processing to step S107. When the acquired frame is an all-sky image (step S102: Yes), the preprocessing unit 11 determines a score based on the element for each block line (step S103). The preprocessing unit 11 determines whether a block line that satisfies the third condition from the first condition is detected (step S104).

  When the block line satisfying the first condition to the third condition is not detected (step S104: No), the preprocessing unit 11 advances the process to step S107. When a block line satisfying the third condition from the first condition is detected (step S104: Yes), the preprocessing unit 11 detects a block line in which a score having a small difference from the score of the detected block line is determined, and The blocked block line is separated (step S105). The pre-processing unit 11 places a block line with a score that has a small difference from the score of the detected block line at the end of the frame (step S106). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by intra encoding (step S107).

  The decoded frame may be subjected to processing for returning the block line rearranged before encoding to the original arrangement before rearrangement.

  As described above, the encoding method according to the first embodiment is an encoding method in the encoding device 10, and obtains a target image (frame) to be encoded according to a predetermined element. In the image, the step of setting a score in a plurality of regions (block lines) defined in the row direction or the column direction, and if the image is encoded into an intra frame, whether the score is a relatively high region in the image Re-arranging the areas according to whether or not, and encoding the image in which the areas are rearranged.

  As a result, the encoding apparatus 10 of the first embodiment can improve the efficiency of image encoding when images are displayed adjacently so as to have spatial continuity.

(Second Embodiment)
The second embodiment is different from the first embodiment in that the encoding unit 13 encodes a frame by inter coding. In the second embodiment, only differences from the first embodiment will be described.

  In the all-sky image, the frames are displayed adjacent to each other so as to have spatial continuity. That is, in the all-sky image, the frame is displayed so that the edge in the frame is adjacent to the edge in the other frame. The encoding unit 13 sets the end block lines facing each other in the frame as a motion search range. Also, the encoding unit 13 may use one of the end block lines facing each other in the frame as a motion search range. Also, the encoding unit 13 may set a range including one of the end block lines facing each other in the frame as a motion search range. In the second embodiment, the encoding unit 13 encodes a frame by inter coding.

  FIG. 5 is a flowchart illustrating an example of a preprocessing procedure according to the second embodiment. The preprocessing unit 11 acquires a frame (original image) to be encoded (step S201). The pre-processing unit 11 determines whether or not the acquired frame is a frame displayed adjacently so as to have spatial continuity. That is, the preprocessing unit 11 determines whether or not the acquired frame is an all-sky image (step S202).

  When the acquired frame is not an all-sky image (step S202: No), the preprocessing unit 11 advances the processing to step S204. When the acquired frame is an all-round image (step S202: Yes), the encoding unit 13 sets the end block lines facing each other in the frame as a motion search range (step S203). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by inter encoding (step S204).

  As described above, the encoding method according to the second embodiment is an encoding method in the encoding device 10, and when an image is encoded into an inter frame, regions at the ends facing each other in the image (frame). (Block line) is set as a range of motion search, and an image is encoded based on the result of motion search. The encoding unit 13 may set one of end regions facing each other in the image as a motion search range. Further, the encoding unit 13 may set a range including one of end regions facing each other in the image as a motion search range.

  Thereby, the encoding apparatus 10 of the second embodiment can further improve the efficiency of image encoding when images are displayed adjacently so as to have spatial continuity.

(Third embodiment)
The third embodiment is different from the first embodiment in that the preprocessing unit 11 performs preprocessing when the target to be encoded is not an inter frame (predicted frame). In the third embodiment, only differences from the first embodiment will be described.

  The preprocessing unit 11 performs preprocessing on an intra frame and an inter frame subsequent to the intra frame. When the moving image stream has a Closed GOP (Group Of Pictures) structure, the preprocessing unit 11 performs preprocessing on a plurality of frames in units of GOP. When the moving image stream has an Open GOP (Group Of Pictures) structure, the preprocessing unit 11 performs preprocessing on a plurality of frames in units of scenes.

  The encoding unit 13 encodes the intra frame of the moving image that has been pre-processed by the pre-processing unit 11 by intra encoding. The encoding unit 13 encodes an inter frame of a moving image that has not been pre-processed by the pre-processing unit 11 by inter-coding.

  FIG. 6 is a flowchart illustrating an example of a preprocessing procedure according to the third embodiment. Steps S303 to S306 shown in FIG. 6 are executed for at least one of the block line in the column direction and the block line in the row direction.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S301). The preprocessing unit 11 determines whether an object to be encoded is an inter frame or an intra frame (step S302). When the object to be encoded is encoded into an intra frame (step S302: No), the preprocessing unit 11 determines a score based on the element for each block line (step S303). The preprocessing unit 11 determines whether a block line satisfying the third condition from the first condition is detected (step S304).

  When the block line satisfying the first condition to the third condition is not detected (step S304: No), the preprocessing unit 11 advances the process to step S308. When a block line satisfying the first condition to the third condition is detected (step S304: Yes), the preprocessing unit 11 detects a block line in which a score having a small difference from the score of the detected block line is determined, and The blocked block line is separated (step S305). The pre-processing unit 11 places a block line with a score that has a small difference from the score of the detected block line at the end of the frame (step S306).

  When the target to be encoded is encoded into an inter frame (step S302: Yes), the encoding unit 13 sets the end block lines facing each other in the frame as a motion search range (step S307). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by intra encoding or inter encoding (step S308).

  As described above, the encoding method according to the third embodiment is an encoding method in the encoding apparatus 10, and includes a step of acquiring an image (frame) to be encoded and a predetermined element. In the image, the step of setting a score in a plurality of regions (block lines) defined in the row direction or the column direction, and if the image is encoded into an intra frame, whether the score is a relatively high region in the image Re-arranging the areas according to whether or not, and encoding the image in which the areas are rearranged. The encoding method according to the third embodiment is an encoding method in the encoding device 10, and is an encoding method in the encoding device 10, where an image (frame) is encoded when the image is encoded into an interframe. , The regions (block lines) facing each other are set as motion search ranges, and an image is encoded based on the result of motion search. The encoding unit 13 may set one of end regions facing each other in the image as a motion search range. Further, the encoding unit 13 may set a range including one of end regions facing each other in the image as a motion search range.

  Thereby, the encoding apparatus 10 of the third embodiment can further improve the efficiency of image encoding when images are displayed adjacently so as to have spatial continuity.

(Fourth embodiment)
The fourth embodiment is different from the third embodiment in that the encoding unit 13 encodes an inter frame by intra encoding or inter encoding. In the fourth embodiment, only differences from the third embodiment will be described.

  FIG. 7 is a flowchart illustrating an example of a preprocessing procedure according to the fourth embodiment. Steps S403 to S406 shown in FIG. 7 are executed for at least one of the block line in the column direction and the block line in the row direction. In the third embodiment, the process of encoding an inter frame is subdivided.

  The preprocessing unit 11 acquires a frame (original image) to be encoded (step S401). The preprocessing unit 11 determines whether an object to be encoded is an inter frame or an intra frame (step S402). When the target to be encoded is encoded into an intra frame (step S402: No), the preprocessing unit 11 determines a score based on the element for each block line (step S403). The preprocessing unit 11 determines whether a block line that satisfies the third condition from the first condition is detected (step S404).

  When a block line that satisfies the third condition from the first condition is not detected (step S404: No), the preprocessing unit 11 advances the process to step S409. When a block line that satisfies the first condition to the third condition is detected (step S404: Yes), the preprocessing unit 11 detects a block line in which a score having a small difference from the score of the detected block line is determined, and detection The blocked block line is separated (step S405). The pre-processing unit 11 places a block line with a score that has a small difference from the detected score of the block line at the end of the frame (step S406).

  When the target to be encoded is encoded into an inter frame (step S402: Yes), the encoding unit 13 determines whether the target block to be encoded is a target block to be inter-encoded (step S402). S407). If the block to be encoded is a block to be intra-encoded (step S407: No), the encoding unit 13 advances the process to step S409.

  If the block to be encoded is a block to be inter-encoded (step S407: Yes), the encoding unit 13 sets the block lines at the ends facing each other in the frame as a motion search range (step S408). ). The encoding unit 13 encodes the frame of the moving image transmitted from the preprocessing unit 11 by intra encoding or inter encoding (step S409).

  As described above, the encoding method according to the fourth embodiment is an encoding method in the encoding device 10, and when the block to be encoded is the block to be inter-encoded, the frames are mutually correct. There is a step of setting the motion search range between the block lines at the opposite ends.

  Thereby, the encoding apparatus 10 of the fourth embodiment can further improve the efficiency of image encoding when images are displayed adjacently so as to have spatial continuity.

  You may make it implement | achieve at least one part of the encoding apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. You may implement | achieve using programmable logic devices, such as FPGA (Field Programmable Gate Array).

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1a ... Block line, 2a ... Block line, 3a ... Block line, 4a ... Block line, 5a ... Block line, 1b ... Block line, 2b ... Block line, 3b ... Block line, 4b ... Block line, 5b ... Block line, DESCRIPTION OF SYMBOLS 10 ... Encoding apparatus, 11 ... Pre-processing part, 12 ... Memory | storage part, 13 ... Encoding part, 14 ... Output part

Claims (3)

An encoding method in an encoding device, comprising:
Obtaining an image to be encoded;
According to a predetermined element, the step of determining a score in a plurality of regions defined in a row direction or a column direction in the image;
When the image is encoded into an intra frame, the region is rearranged according to whether the score is a relatively high region in the image, and the image in which the region is rearranged is encoded. Steps to
An encoding method comprising: When the image is encoded into an inter frame, the regions at the ends facing each other in the image are set as a range of motion search, and the image is encoded based on the result of the motion search;
The encoding method according to claim 1, further comprising:   The encoding program for making a computer perform the encoding method as described in any one of Claim 1 or Claim 2.

Images