JP4881336B2 - Image display control device, image display control program, and semiconductor device - Google Patents

Description

  The present invention relates to an image display control apparatus and an image display control program for controlling movement of a moving body image based on image parameters when moving a virtual moving body image on a displayed image. .

  Conventionally, a main character image is synthesized with a background image, the character image is moved, or the moving body image generated from the character image is reflected on the edge of the background image and bounced or passed. There are various computer graphic (CG) games that can be transformed. In order to realize this game, techniques for extracting parameters such as edges when combining images, edge extraction algorithms, and image processing (projection) are used (see Non-Patent Document 1 as an example).

Further, Japanese Patent Laid-Open No. 2004-26883 is a related art in which a real image and a CG image are combined and affect each other. This patent document 2 is a game in which a player image taken by a camera and a computer image are overlapped, and the computer image is moved in accordance with an area occupied by the player image.
Special issue on transistor technology “Creating Robot Eyes” CQ Publisher JP 2006-014874 A

  However, in the above prior art, both the background image and the movement target image must be subjected to graphic calculation processing, and the closer the background image is to a real image, the larger the amount of processing. For this reason, a control device capable of high-precision arithmetic processing is required.

  In addition, the technique (game) that combines a live-action image and a CG image simply shows a movement corresponding to the real-shot image, and follows a fine ridgeline or boundary line of the live-action image, or a change in color or density. I can't do it.

  In consideration of the above facts, the present invention combines an image (mainly a live-action image) with a moving body image generated separately from this image, and performs a simple calculation process according to various changes in the image. It is an object to obtain an image display control device, an image display control program, and a semiconductor device that can change movement.

According to a first aspect of the present invention, there are provided image display control means for controlling display of an image on a display screen, and image parameters including edges, luminance, and hue of the image in synchronization with display of the image on the display screen. Leveled image information generating means for generating level-divided image information having a predetermined bit group or boundary line of the image parameter for the image as compared with a preset threshold, and the image Separately, a moving body image generating means for generating a moving body image, and the moving body image is displayed based on initial value information including at least initial position information, initial speed information, acceleration information, and initial moving direction information. And moving body image display control means for controlling the display of the moving body based on the initial value information of the moving body image and starting the display superimposed on the displayed display screen. A moving body image, and recognizes the relative position of the level divided image information, when said moving object image is facing the predetermined bit group or the boundary line of the level divided image information, the moving object image in the region Correction means for correcting a moving direction of the moving body image so that the moving body image bounces off at the predetermined bit group or the boundary line when the number of pixels of the predetermined bit group or the boundary line is greater than or equal to a predetermined value; ,have.

According to the first aspect, the image parameter is reflected in the movement of the moving body image in real time, and the arithmetic processing is also simplified.
According to the first invention, when the number of pixels of the predetermined bit group or the boundary line in the area of the moving body image is greater than or equal to a predetermined value, the moving body image is the predetermined bit group or the boundary. The moving direction of the moving body image is corrected so as to bounce off the line.

  Note that the level-divided image information is binarized image information, which is the simplest information, for example, image information converted into white or black. One or more halftones (gray) between white and black may be added to the binarized information to obtain ternary or quaternary information or more.

  The interaction is a reaction when the moving body image faces the bit group or the boundary line, and includes, for example, rebound, passage, absorption extinction, increase and decrease in the number and area, and the like.

Furthermore, the image information that the moving object image confronts can be expressed in dot units, such as bitmap data, depending on the form of the image information (bit group), the direction and position from one point to another point by vector and coordinates May be expressed by information (boundary line), and any image information can be applied in the first invention.
In addition, by setting the overlapping state of the moving object image and the predetermined bit group or boundary line by the number of pixels of the predetermined bit group or boundary line, the overlapping degree can be freely set.

  In the first invention, the image parameter is at least one of an edge, hue, RGB, saturation, brightness, luminance, speed, moving direction, curvature, and curvature of the image.

  As an image parameter, at least one change amount of edge, hue, RGB, saturation, brightness, luminance, speed, moving direction, curvature, and bending degree can be applied.

  In the first aspect of the invention, the level-divided image information is updated as needed so as to follow changes in image parameters of the image.

  For example, when the image is a moving image, the level-divided image information is updated as needed. Basically, it is preferable to update in real time, but the update time may be intentionally provided with a delay time.

  Furthermore, in the first invention, the image displayed on the display screen is a through image that displays an image captured by the imaging unit in real time, or a recorded image that is previously captured by the imaging unit and recorded on the recording medium. It is characterized by being.

  The captured image may be a so-called currently captured image (through image), or may be an image (reproduced image) that has been once recorded on a recording medium or the like.

  In the first invention, the fluctuation of the image parameter of the image is flickering of a light source during imaging by the imaging unit.

  The power supplied from a Japanese power company is 50 Hz in the Kanto region and 60 Hz in the Kansai region, and the light source of the captured image may flicker depending on this frequency. Originally, this flickering causes a reduction in image quality, but here, the fact that the level-divided image information fluctuates may be used positively.

  Furthermore, in the first invention, when the image parameter variation of the image is changed, an additional image is newly written or an existing image that has been written is erased with respect to the information writing image that is fixedly captured. In other words, the level-divided images are updated in real time when an additional image is written or an existing image is erased.

  For example, the movement of the moving body image can be changed by using the whiteboard as a captured image, moving the moving body image within the range, and performing writing or erasing of the writing by the user or the like.

Further, in the first invention, the periphery of the display screen, the predetermined bit group or is characterized by being a part of the boundary line.

  By using the periphery of the captured image as the boundary line, the moving body image can be prevented from deviating from the angle of view.

Further, in the first invention, the is adding additional information to an initial value information, the correction means, based on the additional information, the predetermined bit group or the recoil of the moving object image in the boundary line It is characterized by expression.

  For example, by adding to the initial value information as physical property information specifying the surface hardness or the like as additional information, it is possible to express the deformation of the moving object image when the interaction changes according to the initial value information.

Further, in the first invention, the correction means, in addition to rebound before Symbol moving object image, deformation during rebound, shape change, to correct at least one, or two or more combinations of pattern, and color change It is characterized by.

  The rebound state refers to the amount of repulsion and the direction of the moving body image. The deformation at the time of rebound means elastic deformation, plastic deformation, and reversible deformation when facing a predetermined bit group or boundary line. A shape change refers to a change in the outer shape of the moving object image. The change in pattern and color refers to a change in pattern and color within a range surrounded by the outline of the moving object image.

  The first aspect of the invention further includes perspective information calculation means for calculating perspective information depending on the depth of the image on the display screen based on the leveled image information, and the perspective information calculation means. This calculation result is reflected in the expansion / contraction of the outer dimension with respect to the movement of the moving body image in the depth direction.

  In the three-dimensional virtual space, it is possible to give a sense of perspective of the moving object image.

  Furthermore, in the first aspect of the invention, there is further provided image recognition means for recognizing a specific image including a predetermined bit group or boundary line opposed to the moving body image, and the moving body image is set to the predetermined bit group or boundary line. It is characterized in that the form of interaction is determined based on a specific image recognized when facing each other.

  By recognizing a specific image from a predetermined bit group or boundary line facing each other, it is possible to realize a variety of interaction forms.

According to a second aspect of the present invention, an initial image including at least initial position information, initial speed information, acceleration information, and initial movement direction information is reproduced and displayed on a display screen directly or once after a captured image captured by an imaging unit is recorded. A moving body image in which value information is set is displayed superimposed on the captured image on the display screen, and is moved based on the initial value information, and the edge, hue, and RGB of the captured image displayed on the display screen are displayed. , saturation, lightness, brightness, speed, moving direction, curvature, while following the at least one degree of flexion, as compared to the threshold value set in advance, a predetermined bit group of images parameters against the captured image Alternatively, leveled image information having a boundary line is generated, a relative position between the moving body image and the leveled image information is recognized, and the moving body image is the predetermined bit group or the leveled image information. When facing a boundary line, if the number of pixels of the predetermined bit group or the boundary line in the area of the moving body image is greater than or equal to a predetermined value, the moving body image rebounds at the predetermined bit group or the boundary line. As described above, the computer is caused to execute correction of the moving direction of the moving body image.

  In the second aspect of the invention, perspective information that depends on the depth of the image on the display screen is calculated based on the level-divided image information, and the outside dimension is expanded or reduced with respect to the movement of the moving body image in the depth direction. It is characterized by reflecting.

A third invention is characterized in that the image display system Gopu program is a semiconductor device having a storage memory element.

  As described above, according to the present invention, an image (mainly a live-action image) and a moving body image generated separately from this image are synthesized, and movement of the moving body image is performed in accordance with various changes in the image by simple arithmetic processing. It has an excellent effect that can be changed.

  FIG. 1 shows an image display control apparatus 10 according to the present embodiment. The image display control apparatus 10 includes an apparatus main body 12, a camera 14 connected to the input terminal 12A of the apparatus main body 12, and a display unit 16 (in this embodiment, a projector connected to the output terminal 12B of the apparatus main body 12). 16P and screen 16S).

  The camera 14 has a structure in which a CCD (solid-state imaging device) (not shown) is accommodated in a casing 14A, and a lens unit 14B is attached to a part of the casing 14A. When the power is supplied from the apparatus main body 12, the camera 14 is always imaged by the CCD, and sends the captured image (captured image data) to the apparatus main body 12 (so-called “through image”). ).

  Further, the projector 16P adjusts the amount of light of the three primary color (RGB) light sources (not shown) and irradiates the screen 16S. At this time, the lens 16A is focused on the screen 16S.

When an image signal is input from the apparatus main body 12 to the projector 16P, the light amounts of the three primary color light sources are controlled based on the image signal (for example, gradation of 8 bits (2 ⁸ = 256)). An image is projected on the screen 16S.

  The display unit is not limited to the projector 16P, and may be a television screen (a display method is CRT, LCD, plasma, etc.) in which the light source and the display surface are integrated.

  As shown in FIG. 1, the apparatus main body 12 has a drive unit 12D for taking in data from an external recording medium (for example, a CD or a DVD) on the front surface of the casing 12C, and issues an instruction to the apparatus main body 12 as necessary. A plurality of mounting portions 12E to which a user interface 62 (see FIG. 2) and the like are mounted are provided. The user interface 62 may be a game controller or the like.

  FIG. 2 shows an outline of the hardware configuration of the image display control apparatus 10.

  The apparatus main body 12 includes a microcomputer 50. The microcomputer 50 includes a CPU 52, a RAM 54, a ROM 56, an I / O (input / output port) 58, and a bus 60 such as a data bus or a control bus for connecting them.

  The user interface 62 is connected to the I / O 60. A commercially available personal computer may be applied as the configuration of the microcomputer 50 and the user interface 62.

  In addition, a video control board 64, an object control board 66, and a display control board 68 are connected to the I / O 58.

  The video control board 64 has a role of processing captured image data input from the camera 14.

  When the captured image 69 (see FIG. 5) is projected onto the screen 16S, the object control board 66 generates a moving body image (object image 70 (see FIG. 5)) displayed so as to be superimposed on the captured image 69. And has the role of controlling the movement.

  In the present embodiment, the object image 70 has a square shape, but this shape is not particularly limited. Depending on various specifications, a two-dimensional shape such as a circle, an ellipse, a line, a polygon, a sphere, It may be a three-dimensional shape such as a cylinder. Also, the material of the surface and the inside can be specified, and it can be set to be soft like rubber or sponge, or hard like metal or ore.

  The display control board 68 has a function of synthesizing the captured image data and the object image data in synchronism with the input of the captured image data and the object image data.

  A display driver 72 is connected to the display control board 68. The display driver 72 is connected to the display unit 16.

  Here, in the image display control apparatus 10 according to the present embodiment, the image captured by the camera 14 is displayed on the screen 16S, and the captured image 69 displayed on the screen 16S is displayed on the object control board 66. Control is performed to display the generated object image 70 in an overlapping manner.

  In addition, the image display control device 10 gives information on the position and movement to the object image 70 itself, and changes (corrects) the movement of the object image 70 under a predetermined condition based on the captured image 69. ing.

  In other words, the captured image 69 is a natural scene (actual image) equivalent to human vision, and the object is a virtual CG (computer graphic) image on a computer, and is usually not involved at all (no correlation). ) Realization of correlation between images in real time.

  FIG. 3 is a block diagram functionally showing each control centering on the video control board 64, the object control board 66, and the display control board 68 in the image display control apparatus 10. As shown in FIG. Note that the blocks shown in FIG. 3 are shown by function only and do not limit the hardware configuration.

  The microcomputer 50 shown in FIG. 2 receives parameters (image parameters) for binarizing the captured image data and information about the object image 70 using the user interface 62.

  Image parameters include “Hue”, “Saturation”, “Value”, “Luminance”, “Edge”, “Speed”, “Moving direction”, “Bending”, “ RGB ”,“ curvature ”and the like. Among these parameters, “edge”, “speed”, “moving direction”, “curvature”, “flexion” are arbitrary filter operations for a two-dimensional array in a certain area with values such as “lightness”. Is a parameter obtained by executing. In the present embodiment, binarized data is generated from captured image data using “edge” as an image parameter (details will be described later).

  In addition, as information regarding the object image 70, there is the following information.

(Object information 1) Generation data for generating the object image 70 such as the outer shape, size, quantity, etc. (Object information 2) Initial position information, initial speed information, acceleration information, initial movement direction on the screen (screen 16S) Initial value data including information (object information 3) Instruction data for starting or resetting display of the object image 70 In the present embodiment, the shape of the object image 70 is a VGA (640 pixels × 480 pixels) image. Under the size, it is a square of 50 pixels × 50 pixels (size of about 0.8%).

  Further, as shown in FIG. 5, in this embodiment, the object image 70 starts to be displayed from the vicinity of the upper left corner toward the screen 16S (at the time of reset / start), and has a predetermined speed with a downward slope. Is set to move in.

(Video control board 64)
As shown in FIG. 3, the camera 14 is connected to the frame memory 100 of the video control board 64, and stores captured images for each frame.

  The frame memory 100 is connected to the correction processing unit 102. Image data is sent from the frame memory 100 to the correction processing unit 102 in units of frames (one screen unit), and general image correction processing such as gradation correction, γ correction, and luminance correction is executed. Is sent to. The data output unit 104 is connected to the I / O 58, and the captured image data is temporarily stored in a cache memory or the like generated in the RAM 54 or the ROM 56 of the microcomputer 50 shown in FIG.

(Object control board 66)
As shown in FIG. 3, the image parameter data is output from the I / O 58 to the recognition unit number specifying unit 106 of the object control board 66.

  The recognition unit number specifying unit 106 has a function of setting the number of recognition units 108 to be operated based on the number of image parameters set by the microcomputer 50. That is, the recognition unit specifying unit 106 is connected to the image parameter data distribution unit 110, and the recognition unit 108 (three recognition units in FIG. 3) prepared by the number of image parameter data by the image parameter data distribution unit 110 is prepared. Each image parameter data is distributed to 108). In the present embodiment, as described above, since only “edge” is selected as the image parameter, only the uppermost recognition unit 108 in FIG. 3 operates.

  Captured image data temporarily stored in the microcomputer 50 is input to the recognition unit 108.

  Further, the I / O 58 is connected to an initial value data memory 112, an object generation unit 114, and an object motion control unit 116 that constitute a part of the object control board 66, respectively.

  Information relating to the “object information 1” is sent to the initial value data memory 112.

  Information relating to the “object information 2” is sent to the object generation unit 114.

  Information relating to the “object information 3” is sent to the object motion control unit 116.

  As a result, when the object generation unit 114 is instructed to generate the object image 70 from the I / O 58, the object generation unit 114 generates the object image 70, fetches the initial value data from the initial value data memory 112, and generates the generated object image 70. Is associated with the initial value data, sent to the object motion control unit 116, and waits for a start instruction.

  Note that the initial value data memory 112, the object generation unit 114, and the object motion control unit 116 are provided corresponding to the number of object images 70, respectively. In FIG. Although the operation is controlled (moves on the screen 16S), the number of the object images 70 is not limited.

  Here, the object motion control unit 116 is connected to each of the recognition units 108 described above. Therefore, in addition to the movement control (normal movement control) based on the initial value data, the object motion control unit 116 detects the position information when corresponding to the image parameter (here, “edge”) recognized by the recognition unit 108. The movement information including the movement direction and the movement speed is corrected (corrected movement control).

  The object motion control unit 116 feeds back the position information and movement information of the object image 70 set by the normal movement control and the correction movement control to the I / O 58.

(Details of the recognition unit 108)
FIG. 4 shows detailed control function blocks of the recognition unit 108.

  The image parameter (“edge”) is input to the image parameter input unit 118. The captured image data is captured by the captured image data capturing unit 120.

  The image parameter input unit 118 is connected to the captured image data capturing unit 120, and sends out captured image data to the binarized data generation unit 122 when an output instruction is issued from the image parameter input unit 118. The image parameter input unit 118 is connected to the binarized data generation unit 122 and sends out image parameter information.

  As a result, the binarized data generation unit 122 sets the input captured image data (in the case of a moving image, input in frame units (one image unit)) to “0 (white) with a predetermined threshold value (density value). "Or" 1 (black) ". As a result, the captured image data becomes a line drawing having no gradation (density), and the outline becomes a boundary line. The “edge” detection process is a well-known technique and will not be described in detail.

  The binarized data generated by the binarized data generating unit 122 is sent to the object motion calculating unit 124. The object motion calculation unit 124 receives object image data (current position) from the object motion control unit 116 (see FIG. 3).

  Thereby, based on the relative positional relationship between the boundary line of the binarized data and the object image 70, it is calculated whether or not the number of pixels of the boundary line in the region of the object image 70 is greater than or equal to a predetermined value.

  The object motion calculation unit 124 is connected to the position / movement information correction unit 126. When the number of pixels on the boundary line in the region of the object image 70 is equal to or larger than a predetermined value, the movement of the object image data at the current position is defined as a boundary. The line is rebounded (with correction), and if it is less than the predetermined value, the boundary is passed (without correction).

  The position / movement information correction unit 126 is connected to the object motion control unit 116 (see FIG. 3), and sends out the corrected object image data (“No correction” is also corrected).

  FIG. 6 shows an example in which the object image 70 enters the “edge” (boundary line S) of the binarized image set based on the density from an oblique direction.

  In FIG. 6A, the pixels of the boundary line S are included in one line in the object image 70, and the number of pixels is equal to or greater than the predetermined number. Therefore, in the state of FIG. 6A, the object image 70 rebounds at the boundary line S as shown by the chain line in FIG. 6A (hereinafter referred to as “pattern A”).

  On the other hand, the three circled areas 80, 82, and 84 in FIG. 6B show another pattern in the circled area 86 in FIG. Hereinafter, pattern B, pattern C, and pattern D are sequentially set from the left side of FIG.

  The pattern B is an example in which the line width of the boundary line S is thin and the number of pixels is insufficient, and as a result, the object image 70 passes through the boundary line S.

  The pattern C is an example in which the boundary line S is interrupted and the number of pixels is insufficient, and as a result, the object image 70 passes through the boundary line S.

  The pattern D is an example in which the shape of the boundary line S is non-linear, so the number of pixels is insufficient, and as a result, the object image 70 passes through the boundary line S.

  The object image 70 includes various boundary lines S of the binarized image generated from the captured image 69, and the boundary line S that changes every moment for a moving image, or when the light source at the time of imaging is an AC power source. It is sequentially compared with the boundary line S that changes due to the flickering, and the operation of any of the patterns (A, B, C, D) of the rebound in FIG. 6A and the passage in FIG. 6B is repeated. It has become.

  In the above description, the AC power source causes flickering. However, flickering may be intentionally generated using a pulse waveform or the like. Further, flicker may be generated by changing a threshold value when generating a binarized image.

(Display control board 68)
As shown in FIG. 3, the display control board 68 is provided with a captured image data input unit 128 and an object image data input unit 130.

  Captured image data is transmitted from the I / O 58 to the captured image data input unit 128, and object image data is transmitted from the I / O 58 to the object image data input unit 130.

  The captured image data input unit 128 and the object image data input unit 130 are respectively connected to the image composition unit 132. In the image composition unit 132, the captured image data and the object image data are combined, and the image data output unit 134. To the display driver 72.

  The display driver 72 controls the display unit 16 to display the synthesized captured image 69 and object image.

  Hereinafter, the operation of the present embodiment will be described with reference to the flowchart of FIG. In this flowchart, in addition to the processing in the CPU 52 of the microcomputer 50, processing in each control board (video control board 64, object control board 66, display control board 68) based on a program executed by the CPU 52 is included. .

  When the image display control apparatus 10 according to the present embodiment is activated, first, in step 200, an instruction to input the number of objects is notified by the user interface 62, and whether or not the user inputs the number of objects by this notification. Judgment is made (step 202).

  When an affirmative determination is made in step 202, that is, when the number of objects is input, the process proceeds to step 204 to execute a setting process for each object image 70. In this setting process, the user interface 62 is applied, the object information 1 (generated data for generating the object image 70 such as the external shape, size, quantity, etc.), the object information 2 (initial data on the screen (screen 16S)). (Initial value data including position information, initial speed information, acceleration information, initial movement direction information, etc.).

  When the setting in step 204 is completed, the process proceeds to step 206, where each object image 70 is generated, and the process proceeds to step 208.

  In step 208, an instruction to select an image parameter is notified, and it is determined whether or not the user has selected an image parameter by this notification (step 210).

  These image parameters are parameters for binarizing the captured image data, and are “Hue”, “Saturation”, “Value”, “Luminance”, “Edge”, “ There are “speed”, “movement direction”, “degree of bending”, and the like. In the present embodiment, an image parameter is selected for the “edge”.

  If an affirmative determination is made in step 210, that is, if it is determined that an image parameter has been selected, the process proceeds to step 212, the selected image parameter is stored, and the process proceeds to step 214.

  In step 214, it is determined whether or not captured image data has been input. If an affirmative determination is made, the process proceeds to step 216 to generate a binarized image based on the captured image data and image parameters. At this time, the preparation for projection on the screen 16S is completed, and projection of only the captured image data is started first (step 218).

  In the next step 220, it is determined whether or not there has been a start instruction. If an affirmative determination is made, the process proceeds to step 222 to display the object image at the initial position set by the object information 2, and then the process proceeds to step 224. Thus, the captured image data and the object image data are output in synchronization. Here, “synchronization” refers to making a correlation with the “edge” of the captured image 69 currently displayed (projected).

  In other words, in the display (projection) on the screen 16S, the captured image 69 is an image having a gradation, but separately from the displayed (projected) captured image 69, a binary image is reproduced simultaneously. The object image is processed, and the object image is synchronized with the binarized image. Accordingly, the position of the object image is recognized in the next step 226, and then, in step 228, the “edge” of the binarized image in the region of the object image 70 (in this embodiment, a square of 50 pixels × 50 pixels). It is determined whether or not the number of pixels on the boundary line S is greater than or equal to a predetermined number.

  At this time, on the display (projection) of the screen 16S, the object image corresponds to a certain ridgeline of the captured image 69, the contour of the object image, or the like.

  If an affirmative determination is made in step 228, it is determined that the movement of the object image 70 is blocked by the boundary line S, the process proceeds to step 230, the movement direction is corrected based on the incident angle to the boundary line S, and The process proceeds to step 230 (corresponding to pattern A in FIG. 6A).

  If a negative determination is made in step 228, it is determined that the object image 70 is not blocked by the boundary line S, the movement continues on the extension that has been moved, and the process proceeds to step 230 ( (Corresponding to patterns B, C, and D in FIG. 6B).

  In step 230, the moving direction is corrected based on the incident angle on the boundary line S. However, the moving direction may be reversed (180 ° direction changed) regardless of the incident angle.

  In the next step 232, it is determined whether or not there has been an operation end instruction. If an affirmative determination is made, this routine ends.

  If a negative determination is made in step 232, the process proceeds to step 234 to determine whether or not a reset / start instruction has been issued.

  If an affirmative determination is made in step 234 (with a reset / start instruction), the process returns to step 222 to restart the object image from the initial position. If a negative determination (no reset / start instruction) is made in step 234, the process returns to step 224, and the captured image 69, the object image, and the synchronized display are continued.

  As described above, in the present embodiment, a real image (captured image 69) having a gradation equivalent to human vision captured by the camera 14 and an object image 70 generated by the object control board 66 are displayed on the screen. Separately from the captured image 69 displayed (projected) superimposed on 16S, the captured image data is converted into a predetermined image parameter (here, “edge”) by control internal processing separately from the captured image 69 being displayed (projected). Based on the binarization process, a line image of “white” or “black” is generated, and the relative position between the boundary line S such as the ridgeline or outline of the object in the binarized image data and the object image 70 Based on the amount of overlap (the number of overlapping pixels), it is determined whether the object image 70 should be bounced or passed and reflected in the object image 70 on the screen 16S. It was. Thereby, it is not necessary to program in advance, and an unexpected motion depending on the captured image 69 displayed (projected) on the screen 16S can be given to the object image 70.

  Note that the captured image 69 may be a still image or a moving image. In the case of a moving image, the boundary line changes in real time. By binarizing this at any time, the moving image changes. The movement of the object image can be reflected and reflected.

  The captured image 69 requires a light source. If this light source is, for example, a fluorescent lamp that emits light using an AC power source, flicker may occur. Such a light source is unsuitable for general imaging. However, in the image display control apparatus 10 according to the present embodiment, by using a flickering light source, the result of the binarization process differs, and the stationary Even if the boundary line S is at the same position in the image, the object image may bounce back or pass through, and further unexpected movement is possible.

  Furthermore, in the present embodiment, the captured image 69 is recognized as a planar image (two-dimensional image), but may be recognized as a three-dimensional image having a depth. In this case, when the object image 70 moves in the depth direction, the size (basic size is 50 images × 50 pixels) is preferably changed.

  For example, when the captured image 69 is an indoor space image 69A as shown in FIG. 8, the size of the object image 70 is changed to the inclination angle of the boundary between the wall surface and the floor surface or the ceiling surface. Based on this, the size of the object image 70 may be adjusted.

  That is, depth information is defined by the angle of view of the camera 14 (see FIG. 8A). Next, a beam or the like in the indoor space image 69A is recognized by edge extraction (see FIG. 8B). The position of the wall in the depth direction is defined by the depth information and the beam recognition (see FIG. 8C).

  In this embodiment, binarized data is generated from captured image data as an image parameter. However, the present invention is not limited to this binarization and may be ternary, quaternary, or higher data. Good. In this case, in consideration of digital processing, conversion in bit units is preferable. For example, if the basic image is image data generated with a density of 8-bit gradation (256 values), it may be equal to or less than 7-bit gradation (128 values) image data.

  Examples using the image display control apparatus 10 according to the present embodiment will be described below.

(Example 1: Synthesis with still image)
FIG. 9 shows a captured image 69 obtained by capturing an image in which the vehicle image 69 </ b> B is stopped in the indoor space image 69 </ b> A with the camera 14 and combining it with the object image 70. At this time, as shown in the upper right of FIG. 9, the still image has a binarized image 71 with “edge” as an image parameter, and based on the generated binarized image 71, Under the composite image (captured image 69 and object image 70) shown in the center of FIG. 9, the object image 70 rebounds or passes by the ridgeline of the captured image 69 or the like.

  At this time, when the object image 70 is in the closed space of the vehicle image 69B, the rebound is repeated in the closed space. Note that when the still state is captured as a moving image, the object image 70 may escape from the closed space due to a change in the binarized image 71 due to flickering of the light source or the like.

(Example 2: Combining with still image, adding gravity element to object image)
FIG. 10 shows a captured image 69 obtained by capturing an image in which the vehicle image 69 </ b> B is stopped in the indoor space image 69 </ b> A with the camera 14 and combining it with the object image 70. At this time, as shown in the upper right of FIG. 10, the still image has a binarized image 71 having “edge” as an image parameter, and based on the generated binarized image 71, Under the composite image (captured image 69 and object image 70) shown in the center of FIG. 10, the object image 70 rebounds or passes by the ridgeline or the like of the captured image 69.

  At this time, when the object image 70 is in the closed space of the vehicle image 69B, the rebound is repeated in the closed space. Note that when the still state is captured as a moving image, the object image 70 may escape from the closed space due to a change in the binarized image 71 due to flickering of the light source or the like.

  Moreover, in Example 2, since the gravity element is added to the object image 70, a movement locus becomes a parabolic motion.

(Example 3: Composition with moving image)
FIG. 11 shows a captured image 69 in which an image in which the vehicle image 69B is stopped in the indoor space image 69A is captured by the camera 14 and synthesized with the object image 70 (hereinafter, the captured image 69 and the object image). 70 composite images are referred to as composite images 73). Reference numerals 1 to 4 in FIG. 11 denote composite images 73A, 73B, 73C, and 73D obtained by frame-moving moving images, and binary images 71A, 71B, 71C, and 71D are generated in the respective frames.

  That is, the binarized images 71A, 71B, 71C, and 71D shown in the upper part are generated so as to be linked with the composite images 73A, 73B, 73C, and 73D shown in the lower part of FIG.

  As a result, the object image 70 rebounds or passes through corresponding to the edge of the moving vehicle image 69B.

  Also in this moving image, a weight element may be added to the object image 70.

(Example 4: Composition using a whiteboard as a live-action image)
FIG. 12 shows a case where a whiteboard image 69W is applied as a real image captured by the camera 14.

  The whiteboard image 69W is originally a white background, and an object is to freely draw a line drawing or a character with the black marker M on the whiteboard image 69W. For this reason, there is no significant change between the captured image 69 captured by the camera 14 and the binarized image 71 binarized based on the captured image 69 using “edge” as an image parameter.

  When the object image 70 is combined with such a whiteboard image 69W, the object image 70 moves in the whiteboard image 69W and is drawn on the white background of the wide board image 69W as shown in FIG. Rebound to the drawn line drawing (or character).

  In this way, when taking the wide board image 69W and presenting something, the white board image 69W can be used as a moving image.

  Furthermore, if the imaging with the camera 14 is continued, the movement of the object image 70 can be changed by adding or erasing line drawings or characters.

(Example 5: Composition using a whiteboard as a live-action image, adding a gravity element to an object image)
FIG. 14 shows a case where the whiteboard image 69 </ b> W is applied as a real image captured by the camera 14.

  When the object image 70 is combined with the whiteboard image 69W, the object image 70 moves in the whiteboard image 69W as shown in FIG. Rebound to (or character).

  Thus, when the whiteboard image 69W is imaged and something is presented, the whiteboard image 69W can be used as a moving image.

  Furthermore, if the imaging with the camera 14 is continued, the movement of the object image 70 can be changed by adding or erasing line drawings or characters.

  Moreover, in Example 5, since the gravity element is added to the object image 70, a movement locus becomes a parabolic motion.

(Example 6: Composition using a whiteboard as a live-action image, insertion of a hand image)
As shown in FIG. 15, the object image 70 moves in the whiteboard image 69W, and rebounds to the line drawing (or character) drawn on the white background of the wide board image 69W.

  At this time, when the human hand image 69H is inserted into the imaging range of the camera 14 (within the wide-board image 69W region), the human hand image 69H is also binarized using the edge as an image parameter. The hand image 69H can be a target to be rebounded from the object image 70.

(Embodiment 7: Composition using a whiteboard as a live-action image, insertion of a hand image, addition of a gravity element to an object image)
As shown in FIG. 16, the object image 70 moves in the whiteboard image 69W, and rebounds to the line drawing (or character) drawn on the white background of the wide board image 69W.

  At this time, when the human hand image 69H is inserted into the imaging range of the camera 14 (within the wide-board image 69W region), the human hand image 69H is also binarized using the edge as an image parameter. The hand image 69H can be a target to be rebounded from the object image 70.

  Moreover, in Example 7, since the gravity element is added to the object image 70, a movement locus becomes a parabolic motion.

(Embodiment 8: Composition using a whiteboard as a live-action image, insertion of a hand image, and a composite of “edge” and “Hue” as an image parameter)
As shown in FIG. 17, the object image 70 moves in the whiteboard image 69W, and rebounds to a line drawing (or character) drawn on the white background of the wide board image 69W.

  At this time, the wide board image 69W is binarized using “edge” as an image parameter.

  Here, when the human hand image 69H is inserted into the imaging range of the camera 14 (in the wide-board image 69W region), binarization processing using “hue” as an image parameter is performed on the hand image 69H. Executed.

  For this reason, when the human hand image 69H is inserted into the imaging range of the camera 14 (in the region of the wide board image 69W), the object image 70 once captured in the region of the human hand image 69H becomes the hand image. Repeats bouncing in the 69H area.

(Application to various fields)
The application range of the image display control apparatus 10 according to the present embodiment is that the binarized image with various image parameters is combined by the recognition unit 108 under the synthesis of the real image and the CG (computer graphic) image. Highly accurate composite video can be realized.

  For example, by photographing a real golf course and combining the golf course captured image 69 with a player as a CG image and a ball (object image), there is a reality that could not be realized in a conventional golf game. Play is possible. Note that application to game processing is useful not only for golf games. In addition, it is possible to save the trouble of creating a CG close to a real image.

  In addition, when presenting experimental results in research presentations, projecting slide images (still images, moving images) in advance is currently performed as usual. By adopting general laws (gravity, buoyancy, lift, specific gravity, etc.) and physical properties (elasticity, plasticity, reversibility, etc.), experiments can be performed in real time under a live-action image.

  In addition, the location where the wind hits and the location where the wind does not hit the imaged vehicle is set in advance as an image parameter, while an object image as a wind grain is generated and synthesized, for example in a wind tunnel experiment of the vehicle. It is possible to obtain experimental results without actually using a vehicle.

1 is a system configuration diagram of an image display control apparatus according to the present embodiment. It is a hardware block diagram of the control system of the image display control apparatus which concerns on this Embodiment. FIG. 3 is a block diagram functionally showing respective controls centering on a video control board, an object control board, and a display control board in the image display control apparatus. It is a functional block which shows the detail of control of the recognition unit of FIG. It is a front view of a screen on which a captured image and an object image are combined and displayed. The movement at the time of the correspondence between the object image and the boundary line when “edge” is selected as the image parameter is shown. (A) shows a state in which the object image bounces off at the boundary line (pattern A), and (B) shows the object image at the boundary. It is a front view of the state (pattern B, C, D) which passes a line. It is a flowchart which shows the flow of control including the process in each control board based on the program performed with CPU. It is a three-dimensional model construction figure which shows the principle which changes the magnitude | size of an object image when a captured image is a three-dimensional space. It is a front view which shows the screen display state under image control using the image display control apparatus of this Embodiment (Example 1). (Example 2) which is a front view which shows the screen display state under image control using the image display control apparatus of this Embodiment. (Example 3) which is a front view which shows the screen display state under image control using the image display control apparatus of this Embodiment. It is the schematic which shows the synthetic | combination procedure with the object image in the case of making a whiteboard image into a captured image under image control using the image display control apparatus of this Embodiment. FIG. 13 is a front view showing a screen display state by image composition in FIG. 12 (Example 4). FIG. 13 is a front view showing a screen display state by image composition in FIG. 12 (Example 5). FIG. 13 is a front view showing a screen display state by image composition in FIG. 12 (Example 6). FIG. 13 is a front view showing a screen display state by image composition in FIG. 12 (Example 7). FIG. 13 is a front view showing a screen display state by image composition in FIG. 12 when “edge” is applied to a whiteboard image and “Hue (hue)” is applied to a hand image as image parameters (Example 8).

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image display control apparatus 12 Apparatus main body 12A Input terminal 12B Output terminal 12C Casing 12D Drive unit 14 Camera (imaging means)
14A Casing 14B Lens Unit 16 Display Unit 16A Lens 16P Projector 16S Screen 50 Microcomputer 52 CPU
54 RAM
56 ROM
58 I / O
60 bus 62 user interface 64 video control board 66 object control board 68 display control board 69 captured image 70 object image 72 display driver 80, 82, 84, 86 area 100 frame memory 102 correction processing unit 104 data output unit 106 identification unit number identification Unit 108 Recognition unit 110 Image parameter data distribution unit 112 Initial value data memory 114 Object generation unit (moving body image generation means)
116 Object Operation Control Unit 118 Image Parameter Input Unit 120 Captured Image Data Capture Unit 122 Binarized Data Generation Unit (Leveled Image Information Generation Unit)
124 Object motion calculation unit 126 Position / movement information correction unit (correction means)
128 Captured image data input unit (image display control means)
130 Object image data input unit (moving object image display control means)
132 Image composition unit (image display control means, moving object image display control means)
134 Image data output section S Boundary line

Claims (14)

Image display control means for controlling display of the image on the display screen;
In synchronization with the display of the image on the display screen, the image parameter including the edge, brightness, and hue of the image is compared with a preset threshold value, and a predetermined bit group of the image parameter for the image Or leveled image information generating means for generating leveled image information having a boundary line;
Separately from the image, a moving body image generating means for generating a moving body image;
Based on initial value information including at least initial position information, initial speed information, acceleration information, and initial movement direction information, the moving body image is overlapped on a display screen on which the image is displayed, and display is started. Mobile body image display control means for controlling display of the mobile body based on the initial value information of the mobile body image;
Recognizing the relative position between the moving body image on the display screen and the leveled image information, and when the moving body image faces the predetermined bit group or the boundary line of the leveled image information, When the number of pixels of the predetermined bit group or the boundary line in the area of the moving body image is greater than or equal to a predetermined value, the movement of the moving body image is performed so that the mobile body image rebounds at the predetermined bit group or the boundary line. Correction means for correcting the direction;
An image display control device.   The image display control apparatus according to claim 1, wherein the image parameter is at least one of an edge, hue, RGB, saturation, brightness, brightness, speed, moving direction, curvature, and curvature of the image. .   3. The image display control apparatus according to claim 1, wherein the level-divided image information is updated at any time so as to follow a change in an image parameter of the image.   The image displayed on the display screen is a through image that displays an image captured by the imaging unit in real time, or a recorded image that is captured in advance by the imaging unit and recorded on a recording medium. The image display control apparatus according to any one of claims 1 to 3.   The image display control apparatus according to claim 4, wherein the fluctuation of the image parameter of the image is flickering of a light source during imaging by the imaging unit.   The change in the image parameter of the image is that an additional image is newly written or an existing image that has been written is deleted with respect to the information writing image that is fixedly captured, and the additional image is written. 5. The image display control apparatus according to claim 1, wherein the level-divided images are updated in real time with respect to erasure of existing images. Edge of the display screen, wherein the or a predetermined bit group image display control device according to any one of claims 1 to 6, characterized in that the part of the boundary line. Wherein said is adding additional information to an initial value information, the correction means, based on the additional information, the predetermined bit group or, characterized in that to represent the recoil of the moving object image in the boundary line The image display control device according to any one of claims 1 to 7. Wherein the correction means, in addition to rebound before Symbol moving object image, deformation during rebound, shape change, at least one, or claim 8, wherein the correcting two or more combinations of pattern, and color change Image display control device. Perspective information calculation means for calculating perspective information depending on the depth of the image on the display screen based on the leveled image information,
The image display according to any one of claims 1 to 9, wherein a calculation result of the perspective information calculation means is reflected in expansion / contraction of an outer dimension with respect to movement of the moving body image in a depth direction. Control device. Image recognition means for recognizing a specific image including a predetermined bit group or boundary line facing the moving object image;
Determining whether the mobile image rebounds at the predetermined bit group or the boundary line based on a specific image recognized when the mobile object image faces a predetermined bit group or the boundary line; The image display control device according to claim 1, wherein the image display control device is a feature. The captured image captured by the imaging means is directly or once recorded and then reproduced and displayed on the display screen.
A moving body image in which initial value information including at least initial position information, initial speed information, acceleration information, and initial moving direction information is set in advance is superimposed on the captured image on the display screen, and the initial value information is displayed. Move based on
While comparing at least one of the edge, hue, RGB, saturation, brightness, brightness, speed, moving direction, curvature and curvature of the captured image displayed on the display screen, it is compared with a preset threshold value. Te generates a sorted level image information having a predetermined bit group or the boundary line of images parameters against the captured image,
Recognizing the relative position of the moving body image and the leveled image information, and when the moving body image faces the predetermined bit group or boundary line of the leveled image information, When the number of pixels of the predetermined bit group or the boundary line is greater than or equal to a predetermined value, correcting the moving direction of the moving body image so that the moving body image rebounds at the predetermined bit group or the boundary line,
An image display control program to be executed by a computer. Perspective information that depends on the depth of the image on the display screen is calculated based on the leveled image information,
12. The image display system Gopu program of wherein to reflect the scaling of the outer dimensions relative movement in the depth direction of the moving object image. The semiconductor device according to claim 12 or the image display system Gopu program according to claim 13 comprising a storage memory element.