JP2864015B2 - Intelligent coding of image signals - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an encoding method for efficiently transmitting and storing an image signal, and particularly to a moving image signal such as a television signal. The present invention relates to a method of performing encoding based on means of analyzing contents and characteristics, a simplified expression of analysis results using parameters, and the like, and further combining moving images based on parameters. (Prior art and its problems) As an encoding method for transmitting or storing an image signal, especially a moving image signal such as a television signal, information representing density and color change is sequentially scanned on a screen. 2. Description of the Related Art Waveform coding has been conventionally used in which waveform information is extracted and the waveform is encoded as faithfully as possible. Specifically, predictive coding, orthogonal transform coding, coding using vector quantization, and the like are well known. In waveform coding, basically, it is necessary to code a change in shading or color of each pixel constituting a screen, and for coding a moving image signal, the current technology requires several tens of Kbits / s. The above information amount is required. On the other hand, as image coding completely different from waveform coding,
The concept of a new coding method called intelligent coding, analysis / synthesis coding, etc. has been shown. FIG. 1 shows a general configuration conceptual diagram of conventional intelligent coding. In the figure, A indicates an analysis unit, B indicates a transmission system of analysis results, and C indicates a synthesis unit. Consider a digitized image as an input image. In A, an analysis is first performed on an input image to extract features for expressing the image. In B, the analysis result obtained in A is transmitted to C. In C, an image corresponding to the input image is synthesized using the transmitted analysis result. By using the above-described method, it is possible to encode image change information in a moving image signal as a change in the value of a corresponding feature expression parameter. For this reason, it is possible to express moving image information with a smaller amount of information, for example, 1/10 or less than that of conventional waveform coding. Further, by synthesizing a moving image by arbitrarily giving a feature expression parameter, it is possible to generate a moving image corresponding to an arbitrary movement. Regarding intelligent coding, although the basic concept as described above has been shown, as for the specific implementation method,
The situation is not yet known. In particular, in order to express a moving image, it is necessary to provide a specific method for extracting what information and in what method. In intelligent coding, although it can be conceptually applied to a wide range of objects, in order to perform analysis and synthesis efficiently and make it realistic, the basic structure, shape and motion of the object It is assumed that knowledge has been obtained in advance. Therefore, in concrete implementation,
The target needs to be limited to some extent. Here, in consideration of application to an image transmission service such as a videophone, a person's shoulder image will be considered. (Objects and Features of the Invention) An object of the present invention is to provide a specific method for realizing intelligent coding of an image signal. It is a main object of the present invention to provide an intelligent encoding method of an image signal to be extracted and encoded. A first feature of the present invention is to create a feature image expressing features of an image with respect to an input image signal, and extract information on the position and shape of each element constituting the image based on the feature image. , And to simply express the extracted information with a small number of parameters. A second feature is that, in successive frames of a moving image signal, consistency (continuity) is maintained with respect to changes in the position and shape of each element. (Configuration and Function of the Invention) The present invention will be described below in detail with reference to the drawings. FIG. 2 is a block diagram for explaining the basic operation of the present invention in which the configuration of the analysis unit A of FIG. 1 which has not been established in the prior art is clarified. As an input image signal, a digital image signal obtained by sampling and analog / digital converting a television signal obtained by a television camera or the like is provided. In FIG. 2, 1 is a feature image creating unit, 2 is a position information detecting unit, 3 is a position information expressing unit, 4 is a shape information detecting unit, and 5 is a shape information expressing unit.
900,920,920 for inputting 900 digital image signals
Are terminals for outputting position information and shape information obtained from the position information expression section 3 and the shape information expression section 5, respectively. These information are sent out to a transmission line or stored in a storage device. Next, the operation of each unit will be described. Feature image creation unit 1
Then, for example, a binarization process using the threshold Th is performed. That is, for a pixel f (i, j) constituting a digital image, when f (i, j) ≧ Th g (i, j) = 1 When f (i, j) <Th g (i, j) ) = 0 to obtain a binary image G = {g (i, j)}. FIG. 3 shows an example of a binary image obtained by performing a binarization process on an input image. For example, the upper part of the person's shoulder shown in FIG.
A binary image as shown in FIG. This binary image shows the characteristics of the target (the mouth in this case). The feature of the eye portion can be similarly extracted by the binarization process. When creating a feature image, a method of detecting a portion where the shading changes sharply, that is, an edge portion is also applicable. Next, in the position information detecting unit 2 and the shape information detecting unit 4,
Based on the feature image obtained from the feature image creating unit 1, information on the position and shape of each element constituting the image is extracted.
In the shape information detection unit 4, first, using the feature image,
For example, for a mouth part, feature points necessary for expressing the shape of the mouth are extracted. The feature point, to select the point as shown by P ₁ to P ₄ in FIG. 4 in (a). FIG.
In this example, four points, ie, the left and right ends of the mouth, and the uppermost and lowermost points are selected. Basically, these four points can be used to represent the shape, but if more points are selected, a more detailed representation is possible. The feature point P _i is basically represented by a set of spatial coordinates (x _i , y _i ) as shown in FIG. However, if the set of coordinates is used, information indicating the position and size of the element and information indicating a change in the shape of the element are mixed and difficult to handle. Therefore, the shape information detection unit 4 separates the set of coordinates into position information and shape information as shown in FIG. 4 (c). For the sake of simplicity, the z-coordinate indicating the depth is omitted in FIG. In FIG. 3C, the coordinates (x ₁ , y ₁ ) of the feature point P ₁ ,
P _1, the length l between P _2, the inclination δ of the horizontal line P ₁ P _2, the position and orientation in a screen of the mouth portion is information representing the size, be referred to these as position information To On the other hand, the parameter group Is information expressing the degree of opening of the mouth and the degree of distortion, and is referred to as shape information. Whereas the position information is information unique to each element, that is, information whose value changes when the person is different or the size of the person on the screen changes, the shape information is the relative shape of the shape. It indicates the degree of change, and is information that can be used for general purposes even if the person is different or the size of the person on the screen changes. However, a different parameter group is required for each element type such as a mouth portion and an eye portion. The position information detector 2 obtains position information of each element from the shape information detector 4. Prior to this, based on the feature image obtained by the feature image creating unit 1, the position of the face portion on the screen is detected, and further, the approximate position of the element such as the mouth and the eye is detected. Do. The above-described processing is performed in the shape information detecting unit 4 based on these approximate positions. The position information detector 3 expresses the position information obtained by the position information detector 2 as numerical data. The position information includes numerical values such as a coordinate value, a size, and an angle. The position information expressing unit 3 performs an operation of expressing these numerical values as they are, or expressing them more simply by using quantization and variable length coding. The shape information expressing section 5 expresses the shape information obtained by the shape detecting section 4 as numerical data. As shape information,
It is conceivable to use the shape itself such as a contour line,
When the target is limited to the mouth, eyes, and the like, as described above, a feature point expressing a shape can be extracted, and a parameter group obtained from the feature point can be used. The shape information expressing unit 5 performs an operation of expressing these parameters as numerical values as they are, or expressing them more simply by using quantization and variable length coding. FIG. 5 shows an embodiment of the present invention. When the present invention is applied to moving image information such as a television signal, it is necessary to maintain consistency (continuity) in changes in the position and shape of each element in continuous frames. In other words, in each frame, the position and shape of each element may be erroneously detected due to noise or the like. At this time, even if the individual frame is not unnatural, when compared with the previous and next frames, Changes in position or shape may be discontinuous, and the naturalness as a moving image may be lost. Fifth
In the figure, in order to prevent such unnaturalness as a moving image, a configuration in which some elements are added to FIG. 2 is adopted. In FIG. 5, 6 is a detection position adjustment unit,
7 is a detected shape adjusting unit, 8 and 9 are memories, 21 is a position information detecting unit, and 41 is a shape information detecting unit. Others are the same as FIG. The position and shape information of each element detected in each frame is stored in the memories 8 and 9 for at least one frame period. In the current frame, the position and shape information of each element detected by the position information detecting unit 21 and the shape information detecting unit 41 are stored in the detected position adjusting unit 6 and the detected shape adjusting unit 7 in the memory.
The position and shape information of the corresponding element of the previous frame or up to the previous frame stored in 8, 9 is compared. There are several ways to evaluate the continuity of change.
In the simplest case, for example, the position (x _i , y _i ,
z _i ), the position (X _i , Y _i , Z _i ) in the current frame is x _i −Δx <X _i <x _i + Δx y _i −ΔY <Y _i <y _i + Δy z _i −Δz There is a method of evaluating whether or not <Z _i <z _i + Δz is satisfied. Where Δx, Δ
y and Δz represent allowable variation amounts given in advance. When the detection position adjustment unit 6 or the detection shape adjustment unit 7 determines that the continuity of the change is not maintained, the position and shape information up to the previous frame stored in the memories 8 and 9 is referred to. Then, the position information detecting section 21 or the shape information detecting section 41 detects the position and the shape of the corresponding element again. In this manner, intelligent coding can be performed on a moving image while maintaining consistency with respect to the position and shape change of an element. The analysis method for the input image signal has been described above. In addition, when expressing a moving image, it has been described that each component is handled separately in position information and shape information. Here, a method of synthesis, that is, a method of reproducing a moving image signal based on position and shape information that is an analysis result will be described. In the intelligent coding, it is assumed that information on a three-dimensional shape of a target (in this case, a person's head) is prepared in advance on a transmitting / receiving side as a three-dimensional shape model represented by a wire frame or the like. Becomes It is also assumed that information on the shade and color given to the three-dimensional shape model has been sent to the receiving side in advance. By changing the position and shape of the corresponding part of the three-dimensional shape model according to the change information of the position and shape of each component sent from the transmitting side, and further adding shades and colors to the deformed model The image is reproduced (combined). (Effects of the Invention) With regard to intelligent coding of image signals, only a general framework has been conventionally shown. According to the present invention, in the intelligent coding, it is clarified that the position information and the shape information of the components should be handled separately, and a method of extracting the information from the input image signal becomes clear. It became possible to realize. In particular, for moving image signals,
Expression and transmission with a small amount of information can be performed while maintaining the consistency of the position and shape change of the component, and the effect is extremely large.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a general framework of intelligent coding of an image signal, FIG. 2 is a block diagram corresponding to a basic operation of the present invention, and FIG. FIG. 4 is a schematic diagram for explaining the operation of the feature image creating unit 1 in the present invention, FIG. 4 is a schematic diagram for explaining the operation of the shape information detecting unit 3 in the present invention, and FIG. It is a block diagram corresponding to operation of the present invention. 1 ... Character image creation unit, 2,21 ... Position information detection unit, 3 ...
Position information expression section, 4, 41 ... shape information detection section, 5 ... shape information expression section, 6 ... detection position adjustment section, 7 ... detection shape adjustment section, 8, 9 ... memory.

Claims (1)

(57) [Claims] A feature image for extracting a feature image showing the features of the individual elements necessary to parameterize and express the shape change and movement of the individual elements constituting the image from a moving image signal obtained by digitizing a television signal or the like A creating unit, a position information detecting unit that determines the position of each of the elements on the screen as position information from the feature image obtained from the output of the characteristic image creating unit, and obtains a shape of the individual element as shape information A shape information detection unit, a position information expression unit for numerically expressing position information as position numerical data, a shape information expression unit for numerically expressing shape information as shape numerical data, and a position information expression unit. A first memory for storing the obtained position numerical data for at least one frame period, and at least one frame for storing the shape numerical data obtained from the shape information expressing unit; A second memory that stores more than the first frame, and a position variation amount between the position numerical data obtained in the current frame and the position numerical data up to the previous frame obtained from the first memory is a predetermined first position. When the allowable variation amount is exceeded, the position numerical data up to the previous frame obtained from the first memory is given again to the position information detection unit, and the position variation amount falls within the first allowable variation amount. A detection position adjustment unit for extracting the position information obtained in the current frame as detection position information,
When the shape variation between the shape numeric data obtained in the current frame and the shape numeric data up to the previous frame obtained from the second memory exceeds a predetermined second allowable variation, the second The shape numerical value data up to the previous frame obtained from the second memory is again given to the shape information detection unit, and the shape change amount obtained in the current frame when the shape change amount falls within the second allowable change amount is obtained. A detection shape adjustment unit for extracting shape information as detection shape information, wherein the position numerical data expressed numerically by the position information expression unit of the detection position information extracted from the detection position adjustment unit; An image signal configured so that the detected shape information extracted from the adjustment unit and the shape numerical data numerically expressed by the shape information expressing unit are output as intelligently encoded output of the image signal. Intellectual encoding scheme.