JP5397372B2 - Telop movement vector calculation method, apparatus and program - Google Patents

Description

  The present invention relates to a telop movement vector calculation method, apparatus, and recording medium for storing a program, and more particularly, in detection / recognition of a telop (movement telop) displayed with motion superimposed on video content. The present invention relates to a method, an apparatus, and a recording medium for storing a program for calculating a movement vector.

  Conventionally, in detection / recognition of a telop displayed with motion superimposed on video content, a method based on a telop movement vector calculated using a feature amount extracted from the periphery of the telop of each frame image Many have been proposed. For example, in Patent Document 1, a telop movement vector is calculated by assuming a telop movement direction as one direction and estimating a telop movement amount with respect to that direction. This is based on the premise that the telop movement direction is known, and cannot be applied as it is when the telop movement direction is not known.

On the other hand, in Patent Document 2, when the telop movement direction is not known, a telop detection process is performed for all assumed telop movement directions, and a method that adopts the result only when the detection is successful is proposed. Yes. In this method, search is performed in all the directions assumed for the frame image, and the optimum direction is determined among them.
Japanese Patent No. 3439105 JP 2007-6216 A

  When the moving direction of the telop superimposed on the video content is not known, the conventional method simultaneously determines the telop moving direction and estimates the telop moving amount. This method has a problem that it takes a calculation processing time because a search is performed in all directions when calculating a movement vector of a movement telop.

  The present invention has been made against the background of such circumstances, and an object of the present invention is to provide a telop movement vector calculation method capable of speeding up the movement vector calculation process of movement telops superimposed in video content, An object of the present invention is to provide a recording medium for storing an apparatus and a program.

  In the telop movement vector calculation method according to one aspect of the present invention, the telop movement direction is determined for an input video, and the telop movement amount estimation processing is performed for the direction specified by the determination. The movement vector of the telop is calculated.

  A telop movement vector calculation device according to an aspect of the present invention includes a movement direction determination unit that determines a telop movement direction for an input video, and a telop of the telop with respect to the direction specified by the movement direction determination unit. A movement vector calculating means for calculating a movement vector of the telop by performing a movement amount estimation process is provided.

  A recording medium storing a telop movement vector calculation program according to an aspect of the present invention determines a telop movement direction for an input video, and performs a telop movement amount estimation process for the direction specified by the determination. By performing the above, the computer is caused to execute processing including the step of calculating the movement vector of the telop.

  ADVANTAGE OF THE INVENTION According to this invention, the recording medium which stores the telop movement vector calculation method, apparatus, and program which can speed-up the movement vector calculation process of the movement telop superimposed on the video content can be provided.

It is a block diagram which shows schematic structure of the telop movement vector calculation apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structural example of a telop movement direction determination means. It is a flowchart for demonstrating the telop movement vector calculation method which concerns on 1st Embodiment. It is a block diagram which shows an example of a frame telop candidate edge image generation means. It is a block diagram which shows an example of a frame telop candidate edge image generation means. It is a block diagram which shows an example of a field telop candidate edge image generation means. It is a block diagram which shows an example of a field telop candidate edge image generation means. It is a block diagram which shows an example of a field telop candidate edge image generation means. It is a flowchart which shows an example of the telop movement direction determination process in 1st Embodiment. It is a flowchart which shows an example of the telop movement direction determination process in 1st Embodiment. It is a flowchart which shows an example of the telop movement direction determination process in 1st Embodiment. It is a block diagram which shows the structural example of the telop movement direction determination means in 2nd Embodiment. It is a flowchart for demonstrating the telop movement vector calculation method which concerns on 2nd Embodiment. It is a block diagram which shows an example of a binarization process means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows an example of a binarization process means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows an example of a binarization process means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows an example of a binarized pixel appearance pattern analysis means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows an example of a binarized pixel appearance pattern analysis means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows an example of a binarized pixel appearance pattern analysis means. It is a flowchart which shows an example of the telop movement direction determination process in 2nd Embodiment. It is a block diagram which shows the structural example of the telop movement direction determination means in 3rd Embodiment. It is a flowchart for demonstrating the telop movement vector calculation method which concerns on 3rd Embodiment. It is a block diagram which shows the structural example of the telop movement direction determination means in 4th Embodiment. It is a flowchart for demonstrating the telop movement vector calculation method which concerns on 4th Embodiment. It is a block diagram which shows an example of an image value analysis means. It is a flowchart which shows an example of the telop movement direction determination process in 4th Embodiment. It is a block diagram which shows the structural example of the telop movement direction determination means in 5th Embodiment. It is a flowchart for demonstrating the telop movement vector calculation method which concerns on 5th Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 Image | video input means 12 Telop movement direction determination means 13 Telop movement vector calculation means 14 Telop movement vector output means 101 Frame image generation means 102 Field image generation means 103 Frame telop candidate edge image generation means 104 Field telop candidate edge image generation means 105 Edge Pixel number comparison means 106 Edge pixel number comparison result threshold judgment means 1001 Frame edge image generation means 1002 Frame edge image storage means 1003 Still object frame edge pixel removal means 1101 Field edge image generation means 1102 Field edge image storage means 1103 Still object field edge Pixel removal means 1201 Field image vertical direction interpolation means 1202 Vertical direction interpolation field image edge detection means 201 Binary processing Stage 202 Binary pixel appearance pattern analysis means 2001 Frame image storage means 2002 Inter-frame difference image generation means 2003 Inter-frame difference image threshold processing means 2101 Luminance hue saturation conversion means 2102 Luminance hue saturation threshold processing means 2201 Binary image Projection means 2202 Binary image distribution analysis means 2301 Binary pixel centroid calculation means 2302 centroid calculation result storage means 2303 centroid position motion analysis means 2401 Vertical binary alternate appearance region search means 2402 Horizontal appearance value analysis means 301 Filtering processing Means 302 Filtering output value pattern analysis means 401 Inter-frame difference image storage means 402 Inter-frame cumulative difference image generation means 403 Image value analysis means 3001 Inter-frame cumulative difference image sum projection means 3002 Inter-frame cumulative difference image distribution solution Analysis means 501 Header information extraction means 502 Header information storage means 503 Header information analysis means

First embodiment.
A telop movement vector calculation apparatus according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of a telop movement vector calculation apparatus according to the present embodiment. As shown in FIG. 1, the telop movement vector calculation apparatus according to the present invention includes a video input unit 11 for inputting video data to be processed, and a telop movement of a telop superimposed on the video data input from the video input unit 11. Using the telop movement direction determining means 12 for determining the direction, the telop movement vector calculating means 13 for calculating the telop movement vector using the information of the telop movement direction determined by the telop movement direction determining means 12, and the telop movement vector calculating means 13 There is provided a telop movement vector output means 14 for outputting the telop movement vector.

  In the video input means 11, compressed video or video data obtained by decoding the video is input as video data. When compressed video data is input, the compression format may be MPEG. H.264 is also acceptable.

  The telop movement direction determination unit 12 determines the telop movement direction of the telop superimposed on the video data input from the video input unit 11 in units of frame images, and moves the telop in units of the frame images. Output direction information. In determining the telop movement direction, if it is difficult to determine the telop movement direction with high accuracy, the telop movement direction in the frame image may be “determination impossible”. In this case, information notifying that determination is impossible is output.

  In addition, since the moving direction of the telop superimposed in the video is often limited to a specific direction, as an example, the determination may be limited to the vertical or horizontal two directions. In this case, information identifying two vertical and horizontal directions is output. Alternatively, it may be information for identifying three types including vertical, horizontal, and non-determinable including non-determinable. If the direction of telop movement can be assumed in advance, the information may be output. That is, it may be information that identifies five items, including determination impossible, upper, lower, left, right, and determination impossible. This means is realized by, for example, a CPU equipped with a program that operates according to a predetermined rule. Details of the telop movement direction determination means 12 will be described later.

  The telop movement vector calculation means 13 uses the information on the telop movement direction in units of frame images output from the telop movement direction determination means 12 for the video data input from the video input means 11 and uses the information about the telop movement direction. Only the position of the same telop is searched, and the processing for estimating the movement amount of the telop is performed, so that the telop movement vector is calculated in units of frames. As an example, when information indicating the vertical direction is input, the search is performed only in the vertical direction. Further, the direction of movement of the telop can be assumed in advance, and when information indicating only one direction is input, a search is performed only in that direction. For example, when the upward direction is designated, only the upward direction may be searched without considering the downward movement. When the telop movement direction information output from the telop movement direction determination means 12 is “determination impossible”, the telop is estimated by performing a process of estimating the movement amount not only in a specific direction but in all directions. A movement vector is calculated. This means is realized by, for example, a CPU equipped with a program that operates according to a predetermined rule.

  The telop movement vector output means 14 outputs information on the direction (telop movement direction) and the size (telop movement amount) of the telop movement vector in frame image units calculated by the telop movement vector calculation means 13. For example, when the present invention is implemented as a program and information is notified to a program that performs subsequent processing via the memory, the telop movement vector output means 14 outputs information about the movement vector to the memory.

  Next, the telop movement direction determination means 12 will be described in detail with reference to FIG. FIG. 2 is a block diagram illustrating a configuration example of the telop movement direction determination unit 12 in the present embodiment. As shown in FIG. 2, the telop movement direction determination means 12 includes a frame image generation means 101, a field image generation means 102, a frame telop candidate edge image generation means 103, a field telop candidate edge image generation means 104, an edge pixel number comparison means. 105, edge pixel number comparison result threshold determination means 106 is provided.

  The frame image generation unit 101 generates a frame image from the video data input from the video input unit 11. When the video data is a compressed video, the compressed video is decoded and a frame image is constructed. If the video data is an uncompressed video that has already been decoded, a frame image is constructed by extraction. The field image generation unit 102 receives the frame image from the frame image generation unit 101, and generates an image (field image) composed of only odd lines and even lines of the frame image.

  The frame telop candidate edge image generation unit 103 receives the frame image from the frame image generation unit 101, calculates an edge component that becomes a telop candidate portion for the frame image, and generates a frame telop candidate edge image. A more detailed description of the frame telop candidate edge image generation unit 103 will be described later. The field telop candidate edge image generation unit 104 receives a field image from the field image generation unit 102, calculates an edge component that becomes a telop candidate portion for the field image, and generates a field telop candidate edge image. A more detailed description of the field telop candidate edge image generation unit 104 will be described later.

  The edge pixel number comparison unit 105 includes the number of edge pixels of the two images of the frame telop candidate edge image received from the frame telop candidate edge image generation unit 103 and the field telop candidate edge image received from the field telop candidate edge image generation unit 104. Compare The edge pixel number comparison means 105 can perform comparison based on the ratio of the number of edge pixels of the field telop candidate edge image to the number of edge pixels of the frame telop candidate edge image, for example. The frame telop candidate edge image generation unit 103 and the field telop candidate edge image generation unit 104 calculate only the vertical edge component that is a telop candidate portion for the frame image and the field image, and the frame telop candidate edge image and the field It is also possible to generate a telop candidate edge image. In that case, the edge pixel number comparing means 105 compares the number of pixels for which the edge component in the vertical direction in the frame image and the field image is calculated.

  The edge pixel number comparison result threshold determination means 106 performs threshold processing on the comparison result obtained by the edge pixel number comparison means 105, and determines the telop movement direction according to the result. For example, when the telop movement direction is only two directions, vertical or horizontal, and threshold processing is performed on the ratio of the number of edge pixels of the field telop candidate edge image to the number of edge pixels of the frame telop candidate edge image, the threshold value may be greater than or equal to the threshold value. For example, the telop movement direction is determined to be horizontal, and if it is lower than the threshold value, the telop movement direction is determined to be vertical. This is because when the telop moving direction is horizontal, the number of pixels in which the vertical edge component in the frame telop candidate edge image is detected is reduced.

  Here, the telop movement vector calculation method according to the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart for explaining the telop movement vector calculation method according to the present embodiment. 3, the telop movement vector according to the first embodiment including the video input means 11, the telop movement direction determination means 12, the telop movement vector calculation means 13, and the telop movement vector output means 14 shown in FIG. The processing flow of the whole calculation apparatus is shown.

  As shown in FIG. 3, first, video data is input in S11 (S101), and video data corresponding to the total number NEND is acquired (S102). At this time, the number of processed frames is N = 0 (S103). Thereafter, the telop movement direction of the superimposed telop is determined based on the input video data (S12). Specifically, a frame image is generated (S104), and a field image is generated (S105). Then, a frame telop candidate edge image is generated, and the number of edge pixels N1 is calculated (S106). Further, a field telop candidate edge image is generated, and the number of edge pixels N2 is calculated (S107).

  Next, the number of edge pixels N1 of the frame telop candidate edge image is compared with the number of edge pixels N2 of the field telop candidate edge image (S108). For example, the value of N2 / N1 is calculated. Note that an increment N ++ is performed to perform the same processing for the next frame. Then, the value of N2 / N1 is compared with a preset threshold value (S109). Thereby, the telop movement direction is determined.

  Then, the telop position is searched only for the determined telop movement direction, and the telop movement vector is calculated (S13). Specifically, when the value of N2 / N1 is equal to or greater than the threshold value in S109 (S109 YES), the telop movement direction is determined to be horizontal, and the same telop position is searched for in the horizontal direction (S110). If the value of N2 / N1 is lower than the threshold (NO in S109), the telop movement direction is determined to be vertical, and the position of the same telop is searched for in the vertical direction (S111).

  Then, the processing for estimating the amount of movement of the telop is performed using the information of the telop movement direction, thereby calculating the telop movement vector in units of frames (S112). Thereafter, the calculated telop movement vector is output (S113). If the number of processed frames is less than or equal to the total number of frames NEND, the process returns to S12 again and the same process is performed for all frames (S114). As described above, in the present embodiment, the moving direction of the telop is determined, and the moving amount estimation process for the telop is performed only in the direction specified by the determination. For this reason, it is possible to reduce the processing amount when calculating the movement vector of the movement telop superimposed on the video content, and to speed up the processing.

  Here, a configuration example of the frame telop candidate edge image generation unit 103 in the present embodiment will be described with reference to FIGS. 4 and 5. 4 and 5 are block diagrams showing configuration examples of the frame telop candidate edge image generating unit 103, respectively. As shown in FIG. 4, the frame telop candidate edge image generation unit 103 can be configured by only the frame edge image generation unit 1001. The frame edge image generation unit 1001 uses an edge detection filter such as a two-dimensional Laplacian filter or a Canny filter for the frame image received from the frame image generation unit 101. When calculating only the edge component in the vertical direction, a one-dimensional edge detection filter may be used. The value after using the edge detection filter is subjected to threshold processing, and becomes a binarized image. Since many edge components are calculated from the location where the telop exists, the frame edge image generated here is an image indicating the telop existence candidate region.

  As shown in FIG. 5, the frame telop candidate edge image generation unit 103 can be configured by a frame edge image generation unit 1001, a frame edge image storage unit 1002, and a stationary object frame edge pixel removal unit 1003. The frame edge image generation means 1001 is the same as that shown in FIG. The frame edge image storage unit 1002 stores and accumulates the frame edge image generated by the frame edge image generation unit 1001 on the memory. The stationary object frame edge pixel removing unit 1003 reads out the frame edge image created from each of the processing target frame image and the temporally adjacent frame images from the frame edge image storage unit 1002 and creates the processing target frame image from the processing target frame image. The overlapped edge pixels in the two frame edge images are removed from the obtained frame edge image to create a stationary object frame edge removed image. Thereby, it is possible to remove an edge detected from a stationary object in the frame image.

  In addition, a configuration example of the field telop candidate edge image generation unit 104 in the present embodiment will be described with reference to FIG. 6, FIG. 7, and FIG. 6, 7, and 8 are block diagrams illustrating configuration examples of the field telop candidate edge image generation unit 104. As shown in FIG. 6, the field telop candidate edge image generation unit 104 can be configured by only the field edge image generation unit 1101. The field edge image generation unit 1101 has the same operation as the frame edge image generation unit 1001 except that the input is a field image received from the field image generation unit 102 and the output is a field edge image.

  As shown in FIG. 7, the field telop candidate edge image generation unit 104 can be configured by a field edge image generation unit 1101, a field edge image storage unit 1102, and a stationary object field edge pixel removal unit 1103. The field edge image generation means 1101 is the same as that shown in FIG. The field edge image storage unit 1102 stores and accumulates the field edge image generated by the field edge image generation unit 1101 on the memory. The stationary object field edge pixel removing unit 1103 reads out the field edge image created from each of the processing target field image and the field image generated from the temporally adjacent frame images from the field edge image storage unit 1102, and performs processing. An overlapping edge pixel in two field edge images is removed from a field edge image created from the target field image to create a stationary object field edge removed image. Thereby, it is possible to remove an edge detected from a stationary object in the field image.

  Further, as shown in FIG. 8, the field telop candidate edge image generation unit 104 can be configured by a field image vertical direction interpolation unit 1201 and a vertical direction interpolation field image edge detection unit 1202. The field image vertical direction interpolation unit 1201 interpolates at least one field image received from the field image generation unit 102 in the vertical direction to obtain an image having a frame image size. Here, when the field image is interpolated in the vertical direction, the pixel values in the same column of adjacent rows of the field image are used, for example, the pixel value between the rows is estimated by obtaining an average value, and the value is interpolated between the rows. . For the end row, the pixel values in that row are interpolated to the upper or lower row as they are. The vertical direction interpolation field image edge detection unit 1202 uses an edge detection filter such as a two-dimensional Laplacian filter or a Canny filter for the vertical direction interpolation field image received from the field image vertical direction interpolation unit 1201. When calculating only the edge component in the vertical direction, a one-dimensional edge detection filter may be used. The value after using the edge detection filter is subjected to threshold processing, and becomes a binarized image. Since many edge components are calculated from the location where the telop exists, the vertical direction interpolation field edge image generated here is an image indicating the telop existence candidate region.

  The field telop candidate edge image generation unit 104 shown in FIG. 8 can obtain the same output result with the configuration of the field telop candidate edge image generation unit 104 shown in FIG. That is, instead of performing edge detection on the output result obtained by the field image vertical direction interpolation unit 1201, an edge detection filter that produces the same result as the above processing is created, and the edge detection filter is used as the field image. Apply directly to.

  According to the present embodiment, when calculating the telop movement vector, the telop movement amount only needs to be obtained in the determined movement direction, and the search range can be narrowed down when the telop movement amount is estimated. As described above, in the present invention, a method other than collation using the feature amount extracted from the periphery of the telop can be applied to the direction extraction, so that the telop movement vector calculation process can be speeded up. In addition, since it is possible to suppress an event in which it is determined that a telop is moving in a direction different from an actual telop due to an accidental vector match, high accuracy can be achieved.

  FIG. 9 shows an example of the telop movement direction determination processing flow (S12) when the frame telop candidate edge image generation unit 103 shown in FIG. 4 and the field telop candidate edge image generation unit 104 shown in FIG. 6 are used. In the example shown in FIG. 9, the frame edge image generating unit 1001 shown in FIG. 4 generates a frame edge image indicating a telop existence candidate area (S1001). Further, a field edge image is generated by the field edge image generation means 1101 shown in FIG. 6 (S1002). Other processes are the same as those shown in FIG.

  FIG. 10 shows an example of a telop movement direction determination processing flow when the frame telop candidate edge image generating unit 103 shown in FIG. 5 and the field telop candidate edge image generating unit 104 shown in FIG. 7 are used. In the example shown in FIG. 10, a frame edge image is generated (S1001), a field edge image is generated (S1002), and these are stored (S1003, S1004).

  Then, by using the frame edge image created from each of the frame images to be processed and the temporally adjacent frame images, the frame edge images created from the processing target frame images are duplicated in the two frame edge images. A still object frame edge-removed image from which edge pixels have been removed is created (S1005). In addition, by using a field edge image created from each of the field image to be processed and each of the field images that are temporally close to each other, the field edge image created from the field image to be processed overlaps in two field edge images. A still object field edge-removed image from which edge pixels have been removed is created (S1006). Thereafter, the number of edge pixels N1 ′ of the stationary object frame edge-removed image is compared with the number of edge pixels N2 ′ of the stationary object field edge-removed image (S108). For example, the value of N2 ′ / N1 ′ is calculated, and this value is compared with a preset threshold value (S109). The subsequent processing is the same as the processing shown in FIG.

  FIG. 11 shows an example of the telop movement direction determination processing flow (S12) when the frame telop candidate edge image generation unit 103 shown in FIG. 4 and the field telop candidate edge image generation unit 104 shown in FIG. 8 are used. In the example shown in FIG. 11, a frame edge image is generated (S1001). Further, the field image is interpolated in the vertical direction to generate a vertical direction interpolated field image (S1007). Then, an edge detection filter is used for the vertical direction interpolation field image to generate a vertical direction interpolation field edge image. Similarly, the number of edge pixels is compared, and the same processing as that shown in FIG. 3 is performed thereafter.

Second embodiment.
A telop movement vector calculation apparatus according to a second embodiment of the present invention will be described with reference to FIG. FIG. 12 is a block diagram illustrating a configuration example of the telop movement direction determination unit 12 according to the present embodiment. The schematic configuration of the telop movement vector calculation device according to the second embodiment is different from that of the first embodiment in that the telop movement direction determination means 12 has the configuration as shown in FIG. Note that the components other than the telop movement direction determination unit 12 are the same as those in the first embodiment, and a detailed description thereof will be omitted.

  As shown in FIG. 12, the telop movement direction determination unit 12 according to this embodiment includes a frame image generation unit 101, a binarization processing unit 201, and a binarized pixel appearance pattern analysis unit 202. The frame image generation unit 101 is the same as that in the first embodiment, and detailed description thereof is omitted. The binarization processing unit 201 receives a frame image from the frame image generation unit 101, performs binarization processing on the frame image, and generates a binarized image. A more detailed description of the binarization processing unit 201 will be described later. The binarized pixel appearance pattern analysis means 202 receives the binarized image from the binarization processing means 201, determines the telop movement direction by analyzing the appearance pattern of the pixels of the binarized image, and Outputs information about the telop movement direction. A more detailed description of the binarized pixel appearance pattern analysis means 202 will be described later.

  Here, the telop movement vector calculation method according to the present embodiment will be described with reference to FIG. FIG. 13 is a flowchart for explaining an example of the telop movement vector calculation method according to the present embodiment. Since the operations of the video input unit 11, the telop movement vector calculation unit 13, and the telop movement vector output unit 14 are the same as those in the first embodiment, only the processing flow of the telop movement direction determination unit 12 is shown in FIG.

  As shown in FIG. 13, a frame image is generated (S104), and binarization processing is performed on the generated frame image (S201). Then, by analyzing the appearance pattern of the pixels of the binarized image (S202), information on the telop movement direction is output. Then, from the information on the telop movement direction, the telop movement direction is determined by comparing with the predetermined threshold value as described above.

  Here, a configuration example of the binarization processing unit 201 in the present embodiment will be described with reference to FIG. 14, FIG. 16, and FIG. 14, 16, and 18 are block diagrams illustrating configuration examples of the binarization processing unit 201. As shown in FIG. 14, the binarization processing unit 201 can be configured by only the frame edge image generation unit 1001. The operation of the frame edge image generation unit 1001 in this case is the same as that of the frame telop candidate edge image generation unit 103 of FIG. 4 in the first embodiment, and detailed description thereof is omitted.

  As shown in FIG. 16, the binarization processing unit 201 can also be composed of a frame image storage unit 2001, an inter-frame difference image generation unit 2002, and an inter-frame difference image threshold processing unit 2003. The frame image storage unit 2001 stores and accumulates the frame image generated by the frame image generation unit 101 on the memory. The inter-frame difference image generation unit 2002 reads out the processing target frame image and the temporally adjacent frame image from the frame image storage unit 2001, obtains a pixel value difference between the two frame images for each pixel, A difference image is generated. The inter-frame difference image threshold processing unit 2003 is a process for binarizing the absolute value of the pixel value of the inter-frame difference image generated by the inter-frame difference image generation unit 2002 according to the threshold value. Since the telop used in the video is often used with a large contrast with the background, the absolute value of the pixel value of the inter-frame difference image becomes large at the location where the moving telop exists. That is, the binarized image output from the inter-frame difference image threshold processing unit 2003 is an image indicating a telop existence candidate region.

  As shown in FIG. 18, the binarization processing unit 201 can also be composed of a luminance hue saturation conversion unit 2101 and a luminance hue saturation threshold processing unit 2102. The luminance hue saturation conversion unit 2101 receives the frame image from the frame image generation unit 101 and generates a luminance image, a hue image, and a saturation image. The luminance hue saturation threshold processing unit 2102 obtains a pixel value of any one of the luminance image, hue image, and saturation image generated by the luminance hue saturation conversion unit 2101, or a combination thereof, and uses the value as a threshold. Then, a single binarized image is generated by combining the threshold processing results. Since telops used in video often have luminance, hue, and saturation that increase the contrast with the background, the threshold value processing results for luminance, hue, and saturation images are often used. By combining either one or a combination as necessary, a binarized image indicating a telop candidate area is generated.

  An example of the telop movement direction determination processing flow (S12) when the binarization processing unit 201 has the configuration shown in FIG. 14 is shown in FIG. 15, and the telop movement direction determination processing flow (S12) when the configuration shown in FIG. FIG. 17 shows a telop movement direction determination processing flow (S12) in the case of the configuration shown in FIG. 18, respectively. In the example shown in FIG. 15, the frame edge image generation unit 1001 generates a frame edge image indicating a telop existence candidate region (S1001). Then, binarized pixel appearance pattern analysis is performed on the generated frame edge image (S202), and the moving direction is determined.

  In the example shown in FIG. 17, a frame image is generated (S104) and stored (S2001). Then, a frame image that is temporally close to the processing target frame image is read out, a pixel value difference between the two frame images is obtained for each pixel, and an inter-frame difference image is generated (S2002). Thereafter, the absolute value of the pixel value of the generated inter-frame difference image is binarized according to a threshold value (S2003). Then, binarized pixel appearance pattern analysis is performed on the binarized inter-frame difference image (S202), and the moving direction is determined.

  In the example shown in FIG. 19, a luminance image, a hue image, and a saturation image are generated from a frame image (S2004), and the generated luminance image, hue image, saturation image, or a combination value thereof is generated. And thresholding the value to generate a binarized image (S2005). Then, binarized pixel appearance pattern analysis is performed on the binarized inter-frame difference image (S202), and the moving direction is determined.

  Next, a configuration example of the binarized pixel appearance pattern analysis unit 202 in the present embodiment will be described with reference to FIGS. 20, 22, and 24. 20, 22, and 24 are block diagrams illustrating configuration examples of the binarized pixel appearance pattern analysis unit 202. As shown in FIG. 20, the binarized pixel appearance pattern analysis unit 202 can be configured by a binarized image projection unit 2201 and a binarized image distribution analysis unit 2202. The binarized image projection unit 2201 projects the pixel values of the binarized image received from the binarization processing unit 201 onto the vertical axis and the horizontal axis. The binarized image distribution analysis unit 2202 analyzes the distribution of the projection results of the pixel values of the binarized image in the binarized image projection unit 2201 and determines the telop movement direction. For example, when the telop is moving in the horizontal direction, the projection result on the horizontal axis shows a uniform distribution as a whole at the stage when a sufficient amount of time has passed since the moving telop started to be displayed, whereas the vertical axis The result of projecting to is a distribution showing a high value only in the row where the telop is displayed.

  Therefore, for example, if threshold processing is performed on the projection result, only a part of the vertical axis is extracted, and the horizontal axis is not extracted. Using the difference, it is possible to determine that the telop movement direction is horizontal. Or if the dispersion | distribution of the projection result with respect to each axis | shaft is calculated | required, only a vertical axis | shaft will become a big value and a horizontal axis will become a small value. By using this difference, it is possible to determine that the telop moving direction is horizontal. In the binarized image projecting means 2201, the vertical axis and the horizontal axis may be projected only within a certain width at the upper and lower ends or the left and right ends in the binarized image. In that case, the binarized image distribution analysis unit 2202 determines the telop movement direction based on the difference between the projection results on each axis output by the binarized image projection unit 2201.

  For example, when the telop moves in the horizontal direction, the telop has a sufficient spread in the horizontal direction at a stage when a sufficient time has elapsed after the moving telop starts to be displayed. In general, the telop is often displayed at a certain distance from the top and bottom and left and right edges of the image. Therefore, the binarized image projection result in this case is much larger in the projection result on the horizontal axis than the projection result on the vertical axis. It is also possible to determine that the telop movement direction is horizontal by using the difference in the projection results on each axis.

  As shown in FIG. 22, the binarized pixel appearance pattern analysis unit 202 can also be configured by a binarized pixel centroid calculation unit 2301, a centroid calculation result storage unit 2302, and a centroid position motion analysis unit 2303. . The binarized pixel centroid calculation unit 2301 receives the binarized image from the binarization processing unit 201, and derives position information of the binarized pixel centroid in the binarized image for each frame image. The centroid calculation result storage unit 2302 stores / accumulates the centroid position information of the binarized image calculated by the binarized pixel centroid calculation unit 2301 on the memory. The barycentric position motion analysis unit 2303 uses the barycentric position information of the binarized pixels for each frame image received from the barycentric calculation result storage unit 2302 to analyze the movement and determine the telop movement direction. For example, when the telop moves in the horizontal direction, considering the barycentric position information for a plurality of frame images received from the barycentric calculation result storage unit 2302, the movement of the barycenter is considered to be little in the vertical direction compared to the horizontal direction. . Therefore, it is possible to determine the telop movement direction by using the relative difference in the amount of movement with respect to each direction.

  Also, as shown in FIG. 24, the binarized pixel appearance pattern analysis unit 202 can be configured by a vertical direction binary alternate appearance region search unit 2401 and a horizontal direction appearance value analysis unit 2402. The vertical direction binary alternating appearance area searching unit 2401 searches for an area where the binary appears alternately in the vertical direction pixel by pixel. The searched area where the binary appears alternately one pixel at a time is best to be the same size as the vertical size of the telop used in the video, but this need not be the case. The horizontal direction appearance value analysis means 2402 is a means for determining the telop movement direction by performing a horizontal appearance pattern analysis of the binarized pixel value for the area searched by the vertical binary alternate appearance area search means 2401. It is.

  When the telop moves in the horizontal direction, the character stroke extending in the vertical direction is separated in the horizontal direction in the interlace method. For this reason, two regions where the binary appears alternately one pixel at a time are generated, but the binary appearance patterns in the two regions are reversed. On the other hand, when the telop moves in the vertical direction, the character stroke extending in the vertical direction is separated in the vertical direction, and a region where the binary appears alternately one pixel at a time appears on the column where the character stroke exists. Therefore, first, when there is a region where binary values appear alternately one pixel at a time in a certain column, the value in the same row (value in the horizontal direction) is searched with reference to that location. If there is an area where binary values appear alternately on the line one pixel at a time, if the binary appearance pattern is different from the binary appearance pattern of the reference location, the telop movement direction is horizontal. Judge that there is. If it is the same as the binary appearance pattern of the reference location, it is determined that the telop movement direction is vertical.

  As described above, also in the present embodiment, similarly to the first embodiment, it is only necessary to determine the amount of movement of the telop only in the determined movement direction, and the telop movement vector calculation process can be speeded up. In addition, since it is possible to suppress an event in which it is determined that a telop is moving in a direction different from an actual telop due to an accidental vector match, high accuracy can be achieved.

  An example of the telop movement determination direction processing flow (S12) when the binarized pixel appearance pattern analysis means 202 has the configuration shown in FIG. 20 is shown in FIG. 21, and the telop movement direction determination processing in the case of the configuration shown in FIG. FIG. 23 shows the flow (S12), and FIG. 25 shows the telop movement direction determination processing flow (S12) when the configuration shown in FIG. 24 is used. In the example shown in FIG. 21, the pixel values of the binarized image are projected on the vertical axis and the horizontal axis (S2101). Then, the distribution of the projection results of the pixel values of the binarized image is analyzed (S2102), and the telop movement direction is determined.

  In the example shown in FIG. 23, the position information of the center of gravity of the binarized pixel in the binarized image is derived for each frame image (S2103). Then, the calculated centroid position information of the binarized image is stored / accumulated in the memory (S2104). Thereafter, using the stored barycentric position information of the binarized pixels for each frame image, the movement is analyzed (S2105), and the telop movement direction is determined.

  In the example shown in FIG. 25, a search is performed for a region where the binary appears alternately one pixel at a time in the vertical direction (S2107). If there is no appearance area, the column number increment W ++ is performed, and the same processing is performed for the next column. If there is an appearance area, the appearance pattern analysis in the horizontal direction of the binarized pixel value is performed on the searched area (S2108). In the horizontal direction, when there is no vertical binary alternate appearance region, the column number increment W ++ is performed, and the same processing is performed on the next column. If there is a vertical binary appearance area, the vertical binary alternate appearance pattern is analyzed (S2109). Thereby, the telop movement direction is determined.

Third embodiment.
A telop movement vector calculation device according to a third embodiment of the present invention will be described with reference to FIG. FIG. 26 is a block diagram illustrating a configuration example of the telop movement direction determination unit 12 according to the present embodiment. The schematic configuration of the telop movement vector calculation device according to the third embodiment is different from that of the first embodiment in that the telop movement direction determination means 12 has a configuration as shown in FIG. Note that the components other than the telop movement direction determination unit 12 are the same as those in the first embodiment, and a detailed description thereof will be omitted.

  As shown in FIG. 26, the telop movement direction determination unit 12 according to the present embodiment includes a frame image generation unit 101, a filtering processing unit 301, and a filtering processing output value pattern analysis unit 302. The frame image generation unit 101 is the same as that in the first embodiment, and detailed description thereof is omitted. The filtering processing unit 301 receives a frame image from the frame image generating unit 101, and performs a filtering process that emphasizes pixel values every other pixel in the vertical direction of the frame image. As a filter for performing this filtering process, for example, the filtering process is performed using a filter having a pattern with a cycle of two pixels. The sum of the coefficients of the filter having this rectangular wave pattern is assumed to be zero. The size of the filter is best set to the same size as the vertical size of the telop used in the video, but this need not be the case.

  This filtering process can be realized by applying the above-described filter while shifting it in the vertical direction of the frame image. At this time, it does not matter whether or not there are overlapping regions of the filters in the column on which the filtering process is performed. However, the amount by which the filter is shifted in the vertical direction must match in all columns. The filtering processing output result generated here is obtained by extracting only the region where the pixel value is repeated one by one in the vertical direction. In particular, telops used in images often have a large contrast with the background, and telop characters generally have uniform brightness and color, so that each pixel has a uniform brightness and color. The absolute value of the filtering processing result becomes large at a pattern portion where a part of the telop character appears.

  The filtering process output value pattern analysis unit 302 determines the telop movement direction by analyzing the horizontal appearance pattern of the filtering process output value with respect to the filtering result for the frame image received from the filtering process unit 301. The information of the moving direction of the telop is output. When the telop moves in the horizontal direction, the character stroke extending in the vertical direction is separated in the horizontal direction, and two regions are generated in which the pixel value is repeated in the vertical direction one pixel at a time. The high and low pattern of the pixel value in the region is reversed.

  Further, when the telop moves in the vertical direction, the character stroke extending in the vertical direction is separated in the vertical direction, and an area in which the pixel value is repeated one pixel at a time appears on the column where the character stroke exists. Therefore, first, when there is a location where the absolute value of the filtering processing output value is large in a certain column, the value (horizontal value) in the same row is searched with reference to the location where the output value appears. When a filtering process output value having a large absolute value but having a sign different from that of the filtering process output value at the reference location appears on the line, the telop movement direction is determined to be horizontal. Further, when a filtering process output value having a large absolute value and the same sign as the filtering process output value at the reference location appears on the line, the telop moving direction is determined to be vertical. Similar to the first embodiment, in this embodiment, it is possible to increase the speed of the telop movement vector calculation process and to increase the accuracy.

  Here, with reference to FIG. 27, a telop movement vector calculation method according to the present embodiment will be described. FIG. 27 is a flowchart for explaining an example of the telop movement vector calculation method according to the present embodiment. Since the operations of the video input unit 11, the telop movement vector calculation unit 13, and the telop movement vector output unit 14 are the same as those in the first embodiment, only the processing flow of the telop movement direction determination unit 12 is shown in FIG.

  As shown in FIG. 27, a frame image is generated (S104). At this time, the number of filtered columns is W = 0 (S301). Then, when the number of filtered columns is equal to or less than the horizontal width of the frame image, vertical filtering is performed (S302). Then, it is determined whether or not there is a region where the pixel value repeats increasing and decreasing in the vertical direction by one pixel. If there is no region in which the pixel value repeats increasing and decreasing by one pixel in the vertical direction, increment W ++ is performed, and the same processing is performed on the column next to the column on which the filtering processing has been performed. On the other hand, if there is a region where the pixel value repeats high and low in the vertical direction one pixel at a time, the filtering processing output value in the horizontal direction is analyzed for the filtering result for the frame image (S303).

  Then, as a result of analyzing the appearance pattern in the horizontal direction, if there is no region in which the pixel value repeats high and low in the vertical direction, incrementing is performed, and vertical filtering is performed again (S302). On the other hand, as a result of analyzing the appearance pattern in the horizontal direction, if there is a region where the pixel value repeats one pixel at a time in the vertical direction, the sign of the filtering process value is confirmed (S304). Thereby, the telop movement direction can be determined.

Fourth embodiment.
A telop movement vector calculation apparatus according to the fourth embodiment of the present invention will be described with reference to FIG. FIG. 28 is a block diagram illustrating a configuration example of the telop movement direction determination unit 12 according to the present embodiment. The schematic configuration of the telop movement vector calculation device according to the fourth embodiment is different from that of the first embodiment in that the telop movement direction determination means 12 is configured as shown in FIG. Note that the components other than the telop movement direction determination unit 12 are the same as those in the first embodiment, and a detailed description thereof will be omitted.

  As shown in FIG. 28, the telop movement direction determination unit 12 according to the present embodiment includes a frame image generation unit 101, a frame image storage unit 2001, an inter-frame difference image generation unit 2002, an inter-frame difference image storage unit 401, a frame It is comprised from the interim accumulation difference image generation means 402 and the image value analysis means 403. The frame image generation unit 101 is the same as that in the first embodiment, and detailed description thereof is omitted. The frame image storage unit 2001 and the inter-frame difference image generation unit 2002 are the same as the operations in the binarization processing unit 201 in the second embodiment, and detailed description thereof is omitted.

  The inter-frame difference image storage unit 401 stores and accumulates the inter-frame difference image generated by the inter-frame difference image generation unit 2002 on the memory. The inter-frame cumulative difference image generation unit 402 reads a plurality of inter-frame difference images from the inter-frame difference image storage unit 401, and generates an inter-frame cumulative difference image in which the sum of absolute values of the pixel values is calculated for each pixel. To do. Since the telop used in the video is often used with a large contrast with the background, the absolute value of the pixel value of the inter-frame difference image becomes large at the location where the moving telop exists. Accordingly, in the inter-frame cumulative difference image generated by the inter-frame cumulative difference image generation unit 402, large pixel values are distributed only in the telop movement direction as a whole in the area where the moving telop exists. The image value analysis unit 403 analyzes the image value of the inter-frame cumulative difference image generated by the inter-frame cumulative difference image generation unit 402 and determines the telop movement direction. Detailed description of the image value analysis means 403 will be described later.

  Here, the telop movement vector calculation method according to the present embodiment will be described with reference to FIG. FIG. 29 is a flowchart for explaining an example of a telop movement vector calculation method according to the present embodiment. Since the operations of the video input unit 11, the telop movement vector calculation unit 13, and the telop movement vector output unit 14 are the same as those in the first embodiment, only the processing flow of the telop movement direction determination unit 12 is shown in FIG.

  As shown in FIG. 29, a frame image is generated (S104), and the frame image is stored (S2001). When the number of frames is 1 or more, an inter-frame difference image is generated (S2002), and the inter-frame difference image is stored (S401). Thereafter, an inter-frame cumulative difference image in which the sum of absolute values of the pixel values is calculated for each pixel is generated using an arbitrary plurality of frame images NT (S402). Thereafter, the image value of the inter-frame cumulative difference image is analyzed (S403), and the telop movement direction is determined.

  FIG. 30 is a configuration example of the image value analysis unit 403. As shown in FIG. 30, the image value analysis means 403 can be composed of an inter-frame cumulative difference image projection means 3001 and an inter-frame cumulative difference image distribution analysis means 3002. The inter-frame cumulative difference image projection unit 3001 projects the pixel values of the inter-frame cumulative difference image generated by the inter-frame cumulative difference image generation unit 402 on the vertical axis and the horizontal axis. The inter-frame cumulative difference image distribution analysis unit 3002 analyzes the projection result distribution of the pixel values of the inter-frame cumulative difference image projection unit 3001 to determine the telop movement direction.

  For example, when the telop is moving in the horizontal direction, the projection result on the horizontal axis has a substantially uniform distribution as a whole, and the projection result on the vertical axis has a distribution showing a high value only in a partial region. Therefore, for example, if threshold processing is performed on the projection result, only a part of the vertical axis is extracted, and the horizontal axis is not extracted. Using the difference, it is possible to determine that the telop movement direction is horizontal. Or if the dispersion | distribution of the projection result with respect to each axis | shaft is calculated | required, only a vertical axis | shaft will become a big value and a horizontal axis will become a small value. By using this difference, it is possible to determine that the telop moving direction is horizontal.

  As in the first embodiment, for the direction extraction in this embodiment, a method other than collation using feature amounts extracted from the periphery of the telop can be applied, so the telop movement vector calculation process can be speeded up. It becomes. In addition, since it is possible to suppress an event in which it is determined that the telop is moving in a direction different from the actual telop due to an accidental vector match, high accuracy can be achieved. In particular, in the present embodiment, since a plurality of inter-frame difference images are accumulated, in the case of a video having a complicated background in which a moving telop is not extracted from one inter-frame difference image. In addition, the telop movement vector can be calculated with high accuracy.

  An example of the telop movement direction determination processing flow (S12) when the image value analysis means 403 has the configuration shown in FIG. 30 is shown in FIG. In the example shown in FIG. 31, as shown in FIG. 29, after generating an inter-frame cumulative difference image (S402), the pixel values of the inter-frame cumulative difference image are projected onto the vertical and horizontal axes (S4001). Then, the distribution of the projection results of the pixel values of the inter-frame cumulative difference image is analyzed (S4002), and the telop movement direction is determined.

Fifth embodiment.
A telop movement vector calculation device according to a fifth embodiment of the present invention will be described with reference to FIG. FIG. 32 is a block diagram showing a configuration example of the telop movement direction determination unit 12 according to the present embodiment. The schematic configuration of the telop movement vector calculation device according to the fifth embodiment is different from that of the first embodiment in that the telop movement direction determination means 12 has a configuration as shown in FIG. Note that the components other than the telop movement direction determination unit 12 are the same as those in the first embodiment, and a detailed description thereof will be omitted. As shown in FIG. 32, the telop movement direction determination unit 12 in the present embodiment includes a header information extraction unit 501, a header information storage unit 502, and a header information analysis unit 503.

  The header information extraction unit 501 extracts header information in the compressed video input from the video input unit 11. For example, when video compressed in MPEG format is input, the header information stores information on the DCT mode (frame DCT or field DCT) used in units of macroblocks. Obtained by the header information extraction means 501. The header information storage unit 502 stores and accumulates the header information acquired by the header information extraction unit 501 in the memory.

  The header information analysis unit 503 analyzes the header information read from the header information storage unit 502, and determines the telop movement direction according to the result. For example, in the case of a video compressed in the MPEG format, all information regarding the DCT mode used in the macroblock existing in the frame image is obtained from the header information storage unit 502 for the frame image for which the moving direction is determined. The frequency of use of each mode of DCT in the entire frame image is calculated. For the DCT of macroblock unit used in MPEG, a mode in which the data amount when transform coding is reduced is selected. However, when moving telop is superimposed on the video, the pixel value is set by one pixel in the vertical direction. Since there are many regions where the height is repeated around the moving telop, there is a high possibility that the field DCT is selected instead of the frame DCT. Therefore, when the macroblocks with the field DCT selected are widely present in the horizontal direction, it is determined that the telop moving direction is horizontal, and conversely, when the macroblocks with the field DCT selected are widely present in the vertical direction, It is determined that the moving direction is vertical.

  As in the first embodiment, for the direction extraction in this embodiment, a method other than collation using feature amounts extracted from the periphery of the telop can be applied, so the telop movement vector calculation process can be speeded up. It becomes. In addition, since it is possible to suppress an event in which it is determined that the telop is moving in a direction different from the actual telop due to an accidental vector match, high accuracy can be achieved.

  Here, a telop movement vector calculation method according to the present embodiment will be described with reference to FIG. FIG. 33 is a flowchart for explaining an example of the telop movement vector calculation method according to the present embodiment. Since the operations of the video input unit 11, the telop movement vector calculation unit 13, and the telop movement vector output unit 14 are the same as those in the first embodiment, only the processing flow of the telop movement direction determination unit 12 is shown in FIG.

  As shown in FIG. 33, at the start of processing, the number of information extracted macrobooks is B = 0. First, header information in the input compressed video is extracted (S501), and the acquired header information is stored in the memory (S502). If the number B of information-extracted macrobooks is equal to or less than the number of macrobooks in the frame image, increment B ++ is performed. On the other hand, if the number of extracted macrobooks is not less than or equal to the number of macrobooks in the frame image, the stored header information is read and analyzed (S503), and the telop movement direction is determined according to the result.

  As described above, according to the present invention, by implementing telop detection and telop recognition based on the telop movement vector calculation of the telop superimposed in the video, only the scene related to the word used in the telop is obtained. The present invention can be applied to a system that enables browsing or a system that extracts right information from a telop that flows at the beginning or end of a broadcast program.

  This application claims the priority on the basis of Japanese application Japanese Patent Application No. 2008-111583 for which it applied on April 22, 2008, and takes in those the indications of all here.

  The present invention can be applied to detection / recognition of a telop (moving telop) displayed with motion superimposed on video content.

Claims (35)

For the input video, independently of the calculation of the movement amount component, the telop movement direction is determined from any of the four directions of up / down / left / right,
By calculating the amount of movement of the telop in the direction specified by the determination, the movement vector of the telop is calculated,
The determination of the telop movement direction is as follows:
Binarizing the frame image to derive a binarized image;
Analyzing the appearance pattern of the binarized image and determining the telop movement direction . By edge detection the frame image, the telop motion vector calculation method according to claim 1, wherein the deriving the binarized image. 2. The telop movement vector calculation method according to claim 1, wherein a difference between pixel values of the frame image and a frame image that is temporally adjacent is obtained, and the binarized image is derived by performing threshold processing. 2. The telop movement vector calculation according to claim 1, wherein the binary image is derived by obtaining a value of any one or a combination of luminance, hue, and saturation of the frame image and performing threshold processing. Method. Projecting the pixel values of the binarized image in the vertical axis and the horizontal axis, one of claims 1 to 3, characterized in that the analysis of appearance pattern of the binarized image by analyzing the distribution 2. The telop movement vector calculation method according to item 1. Projecting the pixels of the binarized image to the vertical and horizontal axes only within a certain width of the upper and lower ends or the left and right ends of the binarized image, and performing an appearance pattern analysis by analyzing the distribution The telop movement vector calculation method according to claim 5 , wherein: For the binarized image corresponding to the frame image and the binarized image corresponding to a plurality of adjacent frame images that are temporally different, the position information of the center of gravity of the binarized pixels is obtained for each binarized image. 5. The telop movement vector calculation method according to claim 1 , wherein appearance pattern analysis is performed by deriving and analyzing the movement vector. Obtains an area value in the binarized image appear alternately by one pixel in the vertical direction, according to claim 4, characterized in that the appearance pattern analysis binarization pixel value of the region laterally Telop movement vector calculation method. For the input video, independently of the calculation of the movement amount component, the telop movement direction is determined from any of the four directions of up / down / left / right,
By calculating the amount of movement of the telop in the direction specified by the determination, the movement vector of the telop is calculated,
The determination of the telop movement direction is as follows:
Generating a frame image from the video;
Generating a field image composed of only odd lines and even lines from the frame image;
For the frame image, calculating an edge component to be a telop candidate portion to generate a frame telop candidate edge image;
A step of calculating an edge component that is a telop candidate portion for the field image to generate a field telop candidate edge image;
Comparing the number of edge pixels of the frame telop candidate edge image with the number of edge pixels of the field telop candidate edge image;
Telop motion vector calculation method comprising the step of determining the moving direction based on the edge pixel number comparison results. A frame telop candidate edge image is generated by calculating only the vertical edge component for the frame image,
10. The telop movement vector calculation method according to claim 9 , wherein a field telop candidate edge image is generated by calculating only a vertical edge component for the field image. Using edge pixels detected from each of the frame images and temporally adjacent frame images, the edge pixels of the frame image are removed to generate the frame telop candidate edge images,
Using the edge pixels detected from the field image and each of the field images generated from the temporally adjacent frame images, the edge pixels of the field image are removed to generate the field telop candidate edge image. The telop movement vector calculation method according to claim 9 or 10 , wherein: 11. The telop movement vector calculation method according to claim 9 or 10 , wherein edge detection is performed on an image having at least one frame image size obtained by interpolating a field image in the vertical direction to generate a field telop candidate edge image. . For the input video, independently of the calculation of the movement amount component, the telop movement direction is determined from any of the four directions of up / down / left / right,
By calculating the amount of movement of the telop in the direction specified by the determination, the movement vector of the telop is calculated,
The determination of the telop movement direction is as follows:
Using a filter having a rectangular wave pattern with a period of 2 pixels in the vertical direction of the frame image, extracting a region in which the pixel value repeats high and low in the vertical direction one by one, and outputting a filtering processing result;
A telop movement vector calculation method including a step of performing an appearance pattern analysis in a horizontal direction on the filtering processing result and determining a telop movement direction. For the input video, independently of the calculation of the movement amount component, the telop movement direction is determined from any of the four directions of up / down / left / right,
By calculating the amount of movement of the telop in the direction specified by the determination, the movement vector of the telop is calculated,
The determination of the telop movement direction is as follows:
And the frame image, out of the plurality of temporally different proximity frame images, sequentially calculates a difference by two frame image pixel values, and deriving a difference image between a plurality of frames,
Calculating a sum of absolute values of pixel values for each of the plurality of inter-frame difference images, deriving an inter-frame cumulative difference image, analyzing an image value of the inter-frame cumulative difference image, and moving a telop A telop movement vector calculation method including a step of determining a direction. 15. The telop movement vector calculation method according to claim 14 , wherein image value analysis is performed by projecting pixel values of the inter-frame cumulative difference image onto the vertical axis and the horizontal axis, and analyzing the distribution. For the input video, independently of the calculation of the movement amount component, the telop movement direction is determined from any of the four directions of up / down / left / right,
By calculating the amount of movement of the telop in the direction specified by the determination, the movement vector of the telop is calculated,
The input video is compressed video,
The determination of the telop movement direction is as follows:
Extracting header information contained in the compressed video data;
Analyzing the extracted header information;
A telop movement vector calculation method including a step of determining a movement direction based on an analysis result of the header information. The telop movement vector is calculated by performing a movement amount estimation process for a predetermined direction when the telop movement direction is not specified in one or more directions in the determination of the telop movement direction. The telop movement vector calculation method according to any one of -16 . A moving direction determination means that determines a moving direction of the telop from any of the four directions of up, down, left and right, independently of the calculation of the moving amount component, for the input video;
Movement vector calculation means for calculating a movement vector of the telop by performing a movement amount estimation process of the telop with respect to the direction specified by the movement direction determination means;
Bei to give a,
The moving direction determining means includes
Binarization processing means for binarizing the frame image to derive a binarized image;
A telop movement vector calculation device having binarized pixel appearance pattern analysis means for analyzing an appearance pattern of the binarized image and determining a telop movement direction . 19. The telop movement vector calculation device according to claim 18 , wherein the binarization processing unit includes a frame edge image generation unit that derives the binarized image by detecting an edge of the frame image. The binarization processing means includes:
An inter-frame difference image generation means for obtaining a difference between pixel values of the frame image and a temporally adjacent frame image;
19. The telop movement vector calculation device according to claim 18 , further comprising an inter-frame difference image threshold processing unit that derives the binarized image by performing threshold processing on the inter-frame difference image generated by the inter-frame difference image generation unit. . The binarization processing means includes:
Luminance hue saturation conversion means for obtaining a value of any one or a combination of luminance, hue, saturation of the frame image;
Luminance hue saturation threshold processing means for deriving the binarized image by thresholding any one or a combination of luminance, hue, and saturation generated by the luminance hue saturation conversion means. The telop movement vector calculation device according to claim 18 . Binarized image projecting means for projecting the pixel values of the binarized image on the vertical and horizontal axes;
Claim 19-21 and a binarized image distribution analyzing means for analyzing appearance pattern of the binarized image by analyzing the distribution of the projected pixel values by the binarized image projecting means The telop movement vector calculation device according to item 1. Projecting the pixels of the binarized image to the vertical and horizontal axes only within a certain width of the upper and lower ends or the left and right ends of the binarized image, and performing an appearance pattern analysis by analyzing the distribution 23. The telop movement vector calculation apparatus according to claim 22 , wherein For the binarized image corresponding to the frame image and the binarized image corresponding to a plurality of adjacent frame images that are temporally different from each other, the binarized pixel centroid position information is obtained for each binarized image. A binarized pixel centroid calculating means for deriving;
Claim 18-21 and a center-of-gravity position movement analyzing means for performing appearance pattern analysis by analyzing the centroid moving vector of the pixel binarized derived by the binarized pixel centroid calculator 1 The telop movement vector calculation device according to the item. Vertical direction binary alternating appearance area search means for obtaining an area where the values in the binarized image alternately appear one pixel at a time in the vertical direction;
The telop movement vector calculation according to claim 21 , further comprising horizontal appearance value transfer means for horizontally performing an appearance pattern analysis of the binarized pixel values of the area obtained by the vertical direction binary alternating appearance area search means. apparatus. A moving direction determination means that determines a moving direction of the telop from any of the four directions of up, down, left and right, independently of the calculation of the moving amount component, for the input video;
Movement vector calculation means for calculating a movement vector of the telop by performing a movement amount estimation process of the telop with respect to the direction specified by the movement direction determination means;
With
The moving direction determining means includes
Frame image generation means for generating a frame image from the video;
Field image generation means for generating a field image composed of only odd lines and even lines from the frame image;
Frame telop candidate edge image generation means for generating an edge component to be a telop candidate portion and generating a frame telop candidate edge image for the frame image;
Field telop candidate edge image generation means for generating an edge component that is a telop candidate portion and generating a field telop candidate edge image for the field image;
Edge pixel number comparing means for comparing the number of edge pixels of the frame telop candidate edge image with the number of edge pixels of the field telop candidate edge image;
A telop movement vector calculation device including determination means for determining a movement direction based on the comparison result of the number of edge pixels. A frame telop candidate edge image is generated by calculating only the vertical edge component for the frame image,
27. The telop movement vector calculation device according to claim 26 , wherein a field telop candidate edge image is generated by calculating only a vertical edge component for the field image. The frame telop candidate edge image generating means includes:
A stationary object frame edge image pixel that generates the frame telop candidate edge image by removing edge pixels of the frame image using edge pixels detected from each of the frame images and temporally adjacent frame images Having removal means,
The field telop candidate edge image generating means includes:
Using the edge pixels detected from the field image and each of the field images generated from the temporally adjacent frame images, the edge pixels of the field image are removed to generate the field telop candidate edge image. 28. The telop movement vector calculation device according to claim 26, further comprising a stationary object field edge image pixel removing unit. The field telop candidate edge image generating means includes:
Field image vertical direction interpolation means for interpolating the field image in the vertical direction;
28. The method according to claim 26, further comprising vertical interpolation field image edge detection means for performing edge detection on an image of at least one frame image size interpolated by the field image vertical interpolation means and generating a field telop candidate edge image. Telop movement vector calculation device. A moving direction determination means that determines a moving direction of the telop from any of the four directions of up, down, left and right, independently of the calculation of the moving amount component, for the input video;
Movement vector calculation means for calculating a movement vector of the telop by performing a movement amount estimation process of the telop with respect to the direction specified by the movement direction determination means;
With
The moving direction determining means includes
Filtering processing means for extracting a region where pixel values repeat high and low by one pixel in the vertical direction using a filter having a rectangular wave pattern with a period of 2 pixels with respect to the vertical direction of the frame image, and outputting a filtering processing result; ,
A telop movement vector calculation device having filtering processing output value pattern analysis means for performing an appearance pattern analysis in a horizontal direction on the filtering processing result and determining a telop movement direction. A moving direction determination means that determines a moving direction of the telop from any of the four directions of up, down, left and right, independently of the calculation of the moving amount component, for the input video;
Movement vector calculation means for calculating a movement vector of the telop by performing a movement amount estimation process of the telop with respect to the direction specified by the movement direction determination means;
The moving direction determining means includes
An inter-frame difference image calculation means for sequentially calculating a difference between pixel values of two frame images of a plurality of adjacent frame images that are temporally different from the frame image, and deriving a plurality of inter-frame difference images;
For the plurality of inter-frame difference images, an inter-frame cumulative difference image calculating means for calculating a sum of absolute values of pixel values for each pixel and deriving an inter-frame cumulative difference image;
A telop movement vector calculation device comprising: determination means for analyzing an image value of the inter-frame cumulative difference image and determining a telop movement direction. The determination means includes
An inter-frame cumulative difference image projection means for projecting pixel values of the inter-frame cumulative difference image on the vertical axis and the horizontal axis;
32. The telop movement vector calculation device according to claim 31 , further comprising an inter-frame cumulative difference image distribution analysis unit that performs image value analysis by analyzing a distribution of pixel values projected by the inter-frame cumulative difference image projection unit. A moving direction determination means that determines a moving direction of the telop from any of the four directions of up, down, left and right, independently of the calculation of the moving amount component, for the input video;
Movement vector calculation means for calculating a movement vector of the telop by performing a movement amount estimation process of the telop in the direction specified by the movement direction determination means,
The input video is a compressed video,
The moving direction determining means includes
Header information extraction means for extracting header information included in the compressed video data;
Header information analysis means for analyzing the header information obtained by the header information extraction means;
A telop movement vector calculation device comprising determination means for determining a movement direction based on an analysis result of the header information analysis means. 19. When the telop movement direction is determined and the direction is not specified as one or more directions, a movement vector estimation process is performed on a predetermined direction to calculate a telop movement vector. 34. The telop movement vector calculation device according to any one of -33. For the input video, the telop movement direction is determined independently of the calculation of the movement amount component,
By performing a movement amount estimation process of the telop in the direction specified by the determination, the computer is caused to execute a process including a step of calculating a movement vector of the telop,
In the determination of the telop movement direction,
Binarizing the frame image to derive a binarized image;
A program for causing a computer to execute a process including a step of analyzing an appearance pattern of the binarized image and determining a telop moving direction .

Images