JP4787681B2 - Image composition apparatus and image composition program - Google Patents

Description

  The present invention generates an image corresponding to a coordinate value of an input image coordinate system for inputting a predetermined position of a camera image, and combines the generated image and a camera image indicating an image captured by a camera. The present invention relates to an apparatus and an image composition program.

  In general, when displaying characters and figures for learning and explanation, for example, a blackboard (or a whiteboard or the like) and a chalk (or a magnet or the like) are used. For example, in sports learning and commentary, a coach or the like draws a blackboard on a blackboard and draws a figure or the like for explaining a tactic, or analyzes a tactic. In addition to such a simple blackboard, a video is displayed on the screen, and on the displayed video, a predetermined icon, an arbitrary curve manually input (manually input image), and the like are combined and displayed. Screen chalk board devices are known. Currently, the synthesized video synthesized by such an apparatus is actually used in a broadcast program (for example, a sports program).

A sports coaching system including a display with a built-in computer with a touch panel, a video capture, a hard disk, and a VTR is also known (see Patent Document 1). In this sports coaching system, when a still image such as a person is displayed on the display, a figure (manual input image) drawn on the touch panel with a pen can be displayed superimposed on the still image, The displayed display data is stored in the hard disk as necessary.
Japanese Patent Laying-Open No. 2003-180895 (paragraphs 0010-0015, FIG. 1)

However, in the conventional sports coaching system, the locus drawn on the touch panel with a pen can be synthesized and displayed only in the case of a still image.
Also, in the conventional video screen chalk board device, when the camera parameter of the image taken by the camera (hereinafter referred to as camera image) changes, the manually input image input before the camera parameter changes is displayed on the display as it is. Will be. Here, the camera parameters are various parameters relating to the camera including the camera posture such as pan and zoom. For example, if an arrow (manual input image) is input to an image object such as a person displayed on the left in the display screen in a frame of camera video, the camera pans to the left in another frame. Suppose the camera video changes. In this case, the image object is displayed on the right in the display screen. However, the arrow input to the image object remains displayed on the left in the display screen. Therefore, the input arrow becomes meaningless. Therefore, it is necessary to erase the remaining arrow and input it again at the position after the camera angle (camera parameter) change.

  In particular, when a composite video of a camera video and a manual input image is used for a broadcast program, if the camera angle changes while a manual input image is input by a commentator or the like, the camera before the change as described above. Since the manually input image is displayed at an angle, the viewer feels uncomfortable. Therefore, in the current broadcasting, it is necessary to use only a video with a fixed camera angle or input a manual input image with the video still.

  The present invention has been made in view of the above-described problems, and is an image composition device that synthesizes a manually input image and an image captured by a camera in conjunction with camera parameters of the camera. And an image composition program.

In order to achieve the above object, an image composition apparatus according to claim 1 of the present invention provides a camera image indicating an image of a subject photographed by a camera and input image coordinates for inputting a predetermined position of the camera image. An image synthesizing apparatus that synthesizes an image generated in accordance with a coordinate value of a system, the image being connected to a display integrated with a touch panel and being synthesized with the camera image from the input device via the touch panel Is input as a coordinate value of the input image coordinate system, and includes coordinate storage means, coordinate conversion means, image generation means, image composition means, and second image generation means .

According to such a configuration, the image composition device stores the coordinate value of the input image coordinate system in the coordinate storage means. Then, the image composition device converts the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space by the coordinate conversion means, and then converts the converted coordinate value into the camera image. using the conversion rule shown a series of transformations that converts the coordinate value of the output image coordinate system by using the camera parameters including a photographing angle of the camera when the photographing, the coordinate values stored in the coordinate storage unit the output image Convert to coordinate value of coordinate system . Here, the camera parameter includes, for example, a three-dimensional coordinate indicating the camera installation location and the camera height, an axis movement angle indicating the camera posture reflecting operations such as the camera pan axis, tilt axis, zoom amount, and focus position. And axis movement distance. The image synthesis device generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the coordinate conversion unit by the image generation unit, and the image synthesis unit The image generated by the image generation means and the camera image are synthesized.

The image synthesizing apparatus according to claim 1, at the position indicated by the stored coordinate values before Symbol coordinate storage unit, the second image generating means for generating an image corresponding to the coordinate values of the input image coordinate system The image generated by the second image generating means is presented on a display integrated with the touch panel .

According to such a configuration, the image synthesizing apparatus, since stored in the coordinate storage means the input coordinate values, by the second image generation unit generates an image at the position indicated by the input coordinates. Therefore, the image generated by the second image generation means can faithfully reproduce the shape of the image indicated by the input coordinate value.

In order to achieve the above object, an image composition apparatus according to claim 2 of the present invention is an input for inputting a camera image indicating an image of a subject photographed by a camera and a predetermined position of the camera image. An image composition device that synthesizes an image generated corresponding to a coordinate value of an image coordinate system, and is connected to a display integrated with a touch panel, and is combined with the camera image from the input device via the touch panel. Coordinates about the image are input as coordinate values of the input image coordinate system, and the first coordinate conversion means, the coordinate storage means, the second coordinate conversion means, the image generation means, the image composition means, and the inverse conversion means, And a second image generating means .

  According to this configuration, the image synthesizing apparatus converts the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space by the first coordinate conversion unit. Here, the conversion is, for example, linear conversion or non-linear conversion. The image synthesizing apparatus stores the coordinate value converted by the first coordinate conversion unit in the coordinate storage unit. Then, the image synthesizing apparatus uses the camera parameters including the shooting angle of the camera when the camera image is captured by using the second coordinate conversion unit to store the coordinate value stored in the coordinate storage unit in the image coordinate system. Convert to coordinate values. Then, the image synthesizing device generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the second coordinate converting unit by the image generating unit, and the image synthesizing unit The image generated by the image generation means and the camera image are combined.

Further, according to the configuration, the image synthesizing apparatus, the inverse conversion means, the stored coordinate values in the coordinate storing unit, and converts the coordinate value of the input image coordinate system. Here, the inverse transformation means preferably uses a vector function indicating the transformation used in the first coordinate transformation means and a vector function that is inversely transformed. The image synthesizing apparatus, the second image generation means, the position indicated by the transformed coordinates values in the inverse conversion means, generates an image corresponding to the coordinate values of the input image coordinate system, the second image The image generated by the generating means is presented on a display integrated with the touch panel . Therefore, the image generated by the second image generation means can faithfully reproduce the shape of the image indicated by the input coordinate value.

In order to achieve the object, an image composition device according to a third aspect of the present invention includes a first camera image indicating an image of a subject photographed by the first camera, and a predetermined position of the first camera image. And an image generated corresponding to the coordinate value of the input image coordinate system for inputting the image, and a second camera image indicating an image obtained by photographing the subject with the second camera and the generated image. An image composition device for composition, which is connected to a display integrated with a touch panel, and coordinates of an image to be composed with the first camera image and the second camera image from the input device via the touch panel is input as a coordinate value of the coordinate system, the first coordinate conversion means, a coordinate storage unit, and a second coordinate transformation means, and an image generating unit, and the image synthesizing means, and inverse conversion means, second image generation And stage, it was decided and a second image synthesizing means. The first camera and the second camera may be the same.

According to this configuration, the image synthesizing apparatus uses the first coordinate conversion unit to convert the first camera parameter including the coordinate value of the input image coordinate system and the imaging angle of the first camera when the first camera image is captured. was used to convert the coordinate value of the actual coordinate system corresponding to the real space, and stores the converted coordinate values in the first coordinate transformation hands stage coordinate storing unit. Then, the image composition apparatus uses the second camera parameter including the photographing angle of the second camera when the second camera image is photographed by using the coordinate value stored in the coordinate storing means by the second coordinate converting means. To convert it to a coordinate value in the image coordinate system. Then, the image synthesizing device generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the second coordinate converting unit by the image generating unit, and the image synthesizing unit The image generated by the image generating means and the second camera image are combined. In addition, the image composition device converts the coordinate value stored in the coordinate storage unit into the coordinate value of the input image coordinate system by the inverse conversion unit. Then, the image synthesis device generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the inverse conversion unit by the second image generation unit, and the second image synthesis unit By the means, the image generated by the second image generating means and the first camera image are combined , and the combined image of the second image combining means is presented on a display integrated with the touch panel .

According to a fourth aspect of the present invention, there is provided an image composition program corresponding to a camera image indicating an image of a subject photographed by a camera and a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image. In order to synthesize the image, the coordinates of the image that is connected to the display integrated with the touch panel and is to be synthesized with the camera image from the input device are input as coordinate values of the input image coordinate system. Is a program for causing a computer to function as coordinate conversion means, image generation means, image composition means , and second image generation means .

According to this configuration, the image composition program converts the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space by the coordinate conversion unit, and then converts the converted coordinate value into using the conversion rule shown a series of transformations that converts the coordinate value of the output image coordinate system by using the camera parameters including the imaging angle of the camera when capturing the camera image, the coordinate values of the input image coordinate system Convert to coordinate value of output image coordinate system . The image synthesis program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the coordinate conversion unit by the image generation unit, and the image synthesis unit The image generated by the image generation means and the camera image are synthesized. The image composition program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value stored in the coordinate storage unit by the second image generation unit, and is integrated with the touch panel. Output to a computerized display.

The image composition program according to claim 5 is generated in correspondence with a camera image indicating an image of a subject photographed by a camera and a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image. In order to synthesize the image, the coordinates of the image that is connected to the display integrated with the touch panel and is to be synthesized with the camera image from the input device are input as coordinate values of the input image coordinate system. the computer that is, the first coordinate conversion unit, second coordinate transformation means, the image generation means, image synthesizing means, inverse transformation means, a program for functioning as a second image generation means.

According to such a configuration, the image composition program converts the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space by the first coordinate conversion unit, and the second coordinate conversion unit The coordinate value converted by the first coordinate conversion means is converted into a coordinate value of an image coordinate system using a camera parameter including a shooting angle of the camera when the camera image is shot. Note that the coordinate values converted by the first coordinate conversion means are written in a storage means such as a memory and then converted by the second coordinate conversion means. The image synthesis program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the second coordinate conversion unit by the image generation unit, and the image synthesis unit The image generated by the image generation means and the camera image are combined. Then, the image composition program converts the coordinate value stored in the coordinate storage unit into the coordinate value of the input image coordinate system by the inverse conversion unit. The image composition program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the inverse conversion unit by the second image generation unit, and is integrated with the touch panel. Output to a computerized display.

According to a sixth aspect of the present invention, there is provided an image composition program comprising: a first camera image indicating an image of a subject photographed by a first camera; and coordinates of an input image coordinate system for inputting a predetermined position of the first camera image. The image generated in accordance with the value is combined with the touch panel to combine the generated image with the second camera image indicating the image obtained by photographing the subject with the second camera . A computer connected to a display and configured to input coordinates of an image to be combined with the first camera image and the second camera image from the input device via the touch panel as coordinate values of the input image coordinate system; conversion means, second coordinate transformation means, the image generation means, image synthesizing means, inverse transformation means, second image generation means, program for functioning as a second image synthesizing means It is.

According to such a configuration, the image composition program includes the first camera parameter including the coordinate value of the input image coordinate system and the imaging angle of the first camera when the first camera image is captured by the first coordinate conversion unit. was used to convert the coordinate value of the actual coordinate system corresponding to the real space, by the second coordinate transformation means, the converted coordinate values in the first coordinate transformation hands stage, when photographing the second camera image The second camera parameter including the shooting angle of the second camera is converted into a coordinate value of the image coordinate system. The image synthesis program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the second coordinate conversion unit by the image generation unit, and the image synthesis unit The image generated by the image generating means and the second camera image are combined. Moreover, the synthesis programs, by the inverse conversion means, the converted coordinate values in the first coordinate transformation hand stage is converted into the coordinate values of the input image coordinate system. Note that the coordinate values converted by the first coordinate conversion unit are written in the storage unit such as a memory and then converted by the inverse conversion unit. Then, the image synthesis program generates an image corresponding to the coordinate value of the input image coordinate system at the position indicated by the coordinate value converted by the inverse conversion unit by the second image generation unit. By the means, the image generated by the second image generation means and the first camera image are combined and output to a display integrated with the touch panel .

According to the invention described in claim 1 or 4 , the manually input image is generated at a position indicated by the coordinate value of the image coordinate system converted using the camera parameter. The obtained image and the camera image can be synthesized in conjunction with the camera parameters of the camera. As a result, when a composite image is displayed, a visual effect can be obtained in which a manually input image is directly drawn on the camera video.

According to the invention described in any one of claims 1 to 6, an image that faithfully reproduces the shape of the input image can be obtained in addition to the composite image of the manually input image and the camera image. it can. Therefore, when this image is displayed, the operator who has performed the input operation can check the input image by viewing the image.

According to the second or fifth aspect of the invention, the manually input image and the camera image can be synthesized in conjunction with the camera parameter of the camera. Also, an operation for converting the coordinate value of the input image coordinate system into a coordinate value of the real coordinate system corresponding to the real space, and an operation of converting the converted coordinate value into a coordinate value of the image coordinate system using the camera parameters. Can be executed independently. As a result, it is possible to easily formulate a vector function or a matrix indicating each conversion.

According to the invention described in claim 3 or claim 6 , the manually input image and the first camera image can be synthesized in conjunction with the first camera parameter, and the manually input image and The second camera image can be synthesized in conjunction with the second camera parameter. Therefore, it is possible to distinguish and use one composite image as a confirmation image for the operator who performed the input operation and the other composite image as a viewer image.

Hereinafter, the best mode (hereinafter referred to as “embodiment”) for carrying out an image composition apparatus according to the present invention will be described in detail for each of the first to third embodiments with reference to the drawings.
(First embodiment)
FIG. 1 is a schematic diagram showing an outline of the operation of the image composition apparatus according to the first embodiment of the present invention.
The image composition device 1 generates an image corresponding to the coordinate value of the image coordinate system of the input figure 101 input by the input device 2, the camera image 102 showing the image captured by a camera (not shown), the generated image, Is generated. The composite video 103 is output to a display device (or image storage device) (not shown). The camera video 102 is composed of camera images.

  In the present embodiment, it is assumed that the synthesized video 103 is broadcast to the viewer via a network such as the Internet or a broadcast wave. Hereinafter, the synthesized video 103 is also referred to as a viewer video. In addition, the camera will be described as an image of a sport (soccer) game.

The image composition device 1 is connected to a display 3, and a touch panel 4 is attached to the surface of the display 3.
The display 3 displays an image input from the touch panel 4 by an operation of the operator of the input device 2, and is configured by, for example, a liquid crystal display. Hereinafter, the image displayed on the display 3 is also referred to as an operator image.
The touch panel 4 corresponds to a display area of the display 3 or a window area displayed on the display 3, and inputs coordinates and an event regarding an image to be combined with the camera video 102 via the input device 2 such as a stylus pen. Is.

The event includes one or more of “press”, “release”, and “move”.
“Pressing” is an event indicating that the input device 2 has touched the touch panel 4.
“Open” is an event indicating that the input device 2 has detached from the touch panel 4.
“Move” is an event indicating that the input device 2 has moved in contact with the touch panel 4.
In FIG. 1, katakana “I” is illustrated as the input graphic 101, but the content input by the input device 2 is not limited to characters but may be a symbol, graphic, or the like.

  In the present embodiment, the input device 2 and the touch panel 4 are coordinate and event input means for the image composition device 1. Note that the present invention is not limited to this, and the operator may input coordinates and events by using a finger or the like to touch the touch panel 4 without using the input device 2. A pen tablet or the like may be used instead of the touch panel 4.

Further, for example, a mouse (or trackball, hereinafter the same) may be used as coordinates and event input means for the image composition device 1. In this case, the coordinate value currently instructed by a pointer display such as a mouse cursor becomes the input coordinate value (the coordinate value of the image coordinate system). Further, when the mouse is used, the “press”, “release”, and “move” events have the following meanings.
“Press” is an event indicating that a mouse button has been pressed.
“Open” is an event indicating that a mouse button has been released.
“Move” is an event indicating that the mouse pointer coordinate has been moved while the mouse button is pressed.

  The image composition apparatus 1 receives a camera image 102 photographed by a camera (not shown) and a camera parameter θ. Here, the camera parameter θ includes a shooting angle of a camera (not shown) when the camera image 102 is shot. The camera parameter θ is, for example, an axis movement angle or axis movement indicating a camera posture reflecting operations such as a three-dimensional coordinate indicating a camera installation location or a camera height, a camera pan axis, a tilt axis, a zoom axis, or a focus axis. The distance, the focal length of the camera lens, the pixel pitch of the imaging element of the camera, and the like. Note that the number of cameras that output the camera video 102 may be plural.

  A fixed auxiliary image 110 is displayed on the display 3 in order to visually assist the input operation by the operator. The auxiliary image 110 is, for example, an image of a line indicating a soccer court (touch line, goal line, halfway line, center circle, penalty area, goal area, penalty arc, etc.). The auxiliary image 110 is displayed by, for example, CG (Computer Graphics). The auxiliary image 110 may be printed in advance on the touch panel 4 (or on the display 3).

The image composition device 1 generates two types of CGs, that is, an input corresponding CG 111 and an output corresponding CG 121 based on the input figure 101 written on the touch panel 4.
The input corresponding CG 111 is a CG displayed on the display 3, mainly for the operator of the input device 2 to confirm on the display 3 an image corresponding to the input graphic 101 (an image having the same shape as the input graphic). Generated. The generated input corresponding CG 111 is displayed on the display 3.

  The output correspondence CG 121 is a CG synthesized with the synthesized video 103 output to a display device (or image storage device) (not shown), and the input figure 101 written on the touch panel 4 is converted corresponding to the camera video 102. Is. In the synthesized video 103, the katakana “I” character indicated by the output corresponding CG 121 is projected onto the camera video 102 in a shape that sticks to the ground of the soccer court. Therefore, by changing the camera angle (camera parameter) from moment to moment in conjunction with the camera operation, the output corresponding CG 121 is deformed. As a result, according to the image composition device 1, it is possible to obtain a video effect as if the output corresponding CG 121 exists in the actual camera video 102.

[Configuration of Image Synthesizer ]
FIG. 2 is a functional block diagram illustrating a configuration example of the image composition apparatus illustrated in FIG.
As shown in FIG. 2, the image composition apparatus 1 includes a storage unit 11, a first coordinate conversion unit 12, a second coordinate conversion unit 13, an image generation unit 14, an image synthesis unit 15, and an inverse conversion unit 16. And the second image generation means 17.

<Storage means>
The storage unit 11 includes, for example, a RAM (Random Access Memory) 111 used for arithmetic processing by a CPU (Central Processing Unit), a ROM (Read Only Memory) 112 storing a predetermined program, a coordinate storage unit 113, and the like. It has.
Although details will be described later, the coordinate storage unit 113 is a memory that stores one or more coordinate values, and can be realized by an arbitrary storage device such as a magnetic disk or a magneto-optical disk in addition to a semiconductor memory.

Each functional block other than the storage unit 11 is realized, for example, when the CPU expands and executes a predetermined program stored in the ROM 112 or the like of the storage unit 11 on the RAM 111. Each of these functional blocks is roughly divided into three as shown below.
The first coordinate conversion means 12 is a block for preprocessing (preprocessing block).
The second coordinate conversion unit 13, the image generation unit 14, and the image synthesis unit 15 are blocks (viewer image blocks) for synthesizing the viewer video.
The inverse conversion means 16 and the second image generation means 17 are blocks for generating an operator image (an operator image block).
Below, it demonstrates in order of a pre-processing block, the image block for operators, and the video block for viewers.

[Preprocessing block]
<First coordinate conversion means>
The first coordinate conversion means 12 converts the coordinate value of the image coordinate system (input image coordinate system) input to the touch panel 4 into the coordinate value of the real coordinate system corresponding to the real space. The converted coordinate value is stored in the coordinate storage means 113. In addition, the coordinate value input into the touch panel 4 and the converted coordinate value correspond one-to-one.

3A and 3B are diagrams showing an example of coordinate conversion by the first coordinate conversion means shown in FIG. 2, wherein FIG. 3A shows a display screen and FIG. 3B shows a conversion destination space.
For example, the first coordinate conversion unit 12 converts the coordinates (x, y) of the point P designated on the display screen 301 (screen of the display 3) shown in FIG. 3A to the conversion destination shown in FIG. Conversion to the coordinates (X, Y, Z) of the point Q in the space 302 is performed. The coordinates (x, y) of the point P are specified by pressing a predetermined point on the touch panel 4 (see FIG. 2) with the input device 2 (see FIG. 2). In the following description, the value of an arbitrary coordinate on the display 3 is expressed as (x, y) unless otherwise distinguished, and the names of the coordinate axes are x-axis and y-axis. Similarly, the value of an arbitrary coordinate in the conversion destination space 302 is expressed as (X, Y, Z) unless otherwise distinguished, and the names of the coordinate axes are the X axis, the Y axis, and the Z axis.

  The coordinates (x, y) of the point P shown in FIG. 3 are shown in units of pixels, for example. In addition, the conversion destination space 302 indicates a real space, and here, a soccer court is illustrated. Therefore, the coordinates (X, Y, Z) of the point Q are shown, for example, in units of meters.

  The first coordinate conversion means 12 converts the coordinate value (x, y) to the coordinate value (X, Y, Z) using the vector function g as shown in Expression (1). Formula (1) is a general form. An example of a special expression of formula (1) is shown in formula (2). A vector function g shown in Expression (2) is a function that performs linear transformation.

  Hereinafter, the coordinates (image coordinates) on the display 3 are referred to as display coordinates. As the display coordinates, for example, coordinates [(0, 0) to (639, 479)] formed by 640 × 480 pixels on the display 3 can be used. The display coordinates have an origin at the upper left, an x axis in the right direction, and a y axis in the lower direction.

  In addition, as coordinates in real space, for example, coordinates formed on a 105 × 68 meter soccer court can be used. These coordinates have the center of the soccer court as the origin, take the X axis parallel to the touch line and in the right direction, the Y axis in the depth direction on the halfway line, and the Z axis in the vertical direction.

When display coordinates and real space coordinates are assumed in this way, it is possible to specify specific numerical values of g _ll to g ₃₂ and h ₁ to h _{3 in} the vector function g shown in Equation (2). it can. In this case, equation (2) can be rewritten as shown in equation (3), for example.

In the present embodiment, the vector function g shown in the above equation (1) is not limited to a fixed one as shown in the above equation (3). For example, g _ll to g ₃₂ and h ₁ to h ₃ in the vector function g shown in Expression (2) may be used as coordinate conversion parameters, and these values may be variable. Specifically, a predetermined coordinate conversion parameter may be input to the first coordinate conversion unit 12, and the first coordinate conversion unit 12 may change the vector function g according to the input coordinate conversion parameter.

In relation to this, for example, the image composition device 1 displays a window whose size and position are variable on the display 3 and executes a program for acquiring coordinates on the window via the touch panel 4 or a mouse. You may comprise. When the image synthesizing apparatus 1 is configured in this way, the first coordinate conversion unit 12 performs g _ll in the vector function g shown in Expression (2) according to the size and position of the window acquired by the image synthesizing apparatus 1. It is also possible to vary g ₃₂ and h ₁ to h ₃ .

Below, the 1st coordinate conversion means 12 demonstrates as what converts the coordinate (x, y) on the display 3 into the coordinate (X, Y, Z) of the meter unit in real space.
4A and 4B are explanatory diagrams when the conversion example shown in FIG. 3 is applied to a plurality of coordinate values. FIG. 4A shows display coordinates, and FIG. 4B shows coordinates in the conversion destination space.
The display screen 301 shows points 400 to 406 respectively indicating a plurality (seven in FIG. 4) of coordinate values when katakana “I” is input.

  As shown in FIG. 4A, in the first screen of Katakana “I”, a “press” event is input at point 400, followed by a “move” event at points 401 and 402, and finally At point 403, an “open” event is entered. Each event is input due to the relationship between the input device 2 (see FIG. 2) and the touch panel 4 (see FIG. 2). Similarly, in the second drawing of Katakana “I”, a “press” event is input at point 404, followed by a “move” event at point 405, and finally an “open” event at point 406. .

The coordinate values of the points 400 to 406 shown in FIG. 4A are indicated by display coordinates, and are converted into coordinate values in the real space by the first coordinate conversion means 12.
The coordinate values of the points 450 to 456 shown in FIG. 4B are obtained by converting the coordinate values of the points 400 to 406 shown in FIG.

  The first coordinate conversion means 12 stores the converted coordinate values in the coordinate storage means 113 as shown in FIG. FIG. 5 is a diagram showing an example of a database stored in the coordinate storage means shown in FIG.

The database 500 includes an address 501, coordinates 502, and an event 503 as items.
An address 501 indicates an address (address) on the memory of the coordinate storage unit 113.
A coordinate 502 indicates a coordinate value (converted coordinate value) converted by the first coordinate conversion unit 12.
The event 503 indicates any one of the above-described “press”, “move”, and “release” events input to the touch panel 4 (see FIG. 2).
In the present embodiment, as an example, the first coordinate conversion unit 12 sequentially stores the converted coordinate values in the coordinate storage unit 113 in accordance with the order in which coordinates and events are input.

[Image block for operator]
<Inverse conversion means>
The inverse conversion means 16 converts the coordinate values stored in the coordinate storage means 113 into coordinate values in the input image coordinate system. The converted coordinate values are output to the second image generation means 17. This inverse conversion means 16 converts the coordinate value (X, Y, Z) into the coordinate value (x, y) using the vector function h as shown in the equation (4).

  Equation (4) is a general form. An example of a special expression of formula (4) is shown in formula (5). The vector function h shown in Expression (5) and the vector function g used in the first coordinate conversion means 12 are inversely converted from each other.

  When the vector function h is expressed by the equation (5) and the vector function g is expressed by the above equation (3), the above equation (5) is rewritten as shown in the equation (6), for example. It is done.

<Second image generating means>
The second image generation unit 17 generates an image (input correspondence CG) at the position indicated by the coordinate value converted by the inverse conversion unit 16. The generated image is output to the display 3. The second image generation unit 17 draws a figure in accordance with the event stored in the coordinate storage unit 113. For example, the second image generation unit 17 generates a CG of a broken line that connects a group of coordinates from a certain “pressing” event, through zero or more “moving” events, to the immediately following “opening” event. Generate. Further, for example, a CG of a spline curve or a Bezier curve may be generated instead of a broken line.

FIG. 6 is a diagram illustrating an example of the input corresponding CG generated by the second image generation unit illustrated in FIG. FIG. 6 shows a CG generated by the second image generation means 17 (see FIG. 2) based on the database 500 illustrated in FIG. In this case, the input screen CG 601 is displayed on the display screen 301. The input correspondence CG 601 indicates the Katakana “I”, and is composed of a CG 601 a indicating the first stroke of the Katakana “I” and a CG 601 b indicating the second stroke. The CG 601a reflects the events stored corresponding to the addresses “0” to “3” shown in FIG. 5, and the CG 601b stores the events stored corresponding to the addresses “4” to “6” shown in FIG. Is reflected. That is, the second image generating means 17 (see FIG. 2) uses the coordinate value (display coordinates) corresponding to the event “press” as the start point and the coordinate value (display coordinates) corresponding to the event “open” as the end point. Then, a broken line CG 601a and a straight line CG 601b are drawn. The input correspondence CG 601 faithfully reproduces the shape of the input figure 101 (see FIG. 1). In the present embodiment, the second image generation means 17 (see FIG. 2) synthesizes the input corresponding CG 601 and the auxiliary image 602, which is the generated CG, and outputs it to the display 3 (see FIG. 2).
Returning to FIG. 2, the description will be continued.

[Video block for viewers]
<Second coordinate conversion means>
The second coordinate conversion unit 13 converts the coordinate value stored in the coordinate storage unit 113 into a coordinate value in the image coordinate system using the camera parameter θ. The converted coordinate value is output to the image generation means 14. The second coordinate conversion means 13 can perform coordinate conversion based on, for example, a perspective projection model.

FIG. 7 is an explanatory diagram of perspective projection. In FIG. 7, in the real space 705, a Cartesian coordinate system is shown in which the X axis and the Y axis are taken in the horizontal plane and Z is taken in the height direction. In perspective shadowing, as shown in FIG. 7, a line segment 704 that connects a real coordinate 706 on the real space 705 and the projection center 701 of the imaging system is obtained, and an image of a point where the line segment 704 and the image plane 702 intersect is obtained. The coordinates 703 are obtained as (C _x , C _y ). (C _x , C _y ) are coordinate values in display coordinates. The values (C _x , C _y ) of the image coordinates 703 can be determined by the values (X, Y, Z) of the real coordinates 706 and the camera parameter θ.

Real coordinate 706 corresponds to the coordinates 502 (see FIG. 5) stored in the coordinate storage hand stage 113 (see FIG. 2).

Second coordinate transformation hands stage 13 converts the converted coordinate values stored in the coordinate storage unit 113 (see FIG. 2) is subjected to a perspective projection described with reference to FIG. 7 to the value of the display coordinate .

  The second coordinate conversion means 13 uses the vector function f and the camera parameter θ as shown in the equation (7), and uses the coordinate values (X, Y) stored in the coordinate storage means 113 (see FIG. 2). , Z) are converted into coordinate values (ξ, η) of the display coordinate system (image coordinate system). Equation (7) is a general form. Further, the vector function θ shown in Expression (7) represents various camera parameters θ collectively.

  In the present embodiment, the coordinate values (ξ, η) of the display coordinate system are coordinate values in a coordinate space different from the resolution and screen size of the display 3. In addition, this invention is not limited to this, The coordinate space of the conversion destination of the 2nd coordinate conversion means 13 may be the same as the coordinate space of the coordinate value input into the touch panel 4. FIG.

<Image generation means>
The image generation means 14 generates an image (output correspondence CG) at the position indicated by the coordinate values converted by the second coordinate conversion means 13. The generated image is output to the image composition unit 15. The image generation unit 14 draws a figure in accordance with the event stored in the coordinate storage unit 113. Since the method of drawing a figure is the same as that of the second image generating means 17, the description thereof is omitted.

FIG. 8 is a diagram illustrating an example of the output correspondence CG generated by the image generation unit illustrated in FIG.
FIG. 8 shows a CG generated by the image generation unit 14 (see FIG. 2) based on the database 500 illustrated in FIG. In this case, as shown in FIG. 8, an output corresponding CG 801 is generated. The output-corresponding CG 801 indicates Katakana “I”, and includes a CG 801 a indicating the first stroke of Katakana “I” and a CG 801 b indicating the second stroke. The CGs 801a and 801b are generated by the same method as the CGs 601a and 601b shown in FIG. Since the image generating means 14 (see FIG. 2) draws at the position of the coordinate value reflecting the camera parameter θ, the output corresponding CG 801 is the input figure 101 (see FIG. 1) in accordance with the camera video 102 (see FIG. 1). Has a deformed shape.
Returning to FIG. 2, the description will be continued.

<Image composition means>
The image synthesizing unit 15 synthesizes the image (output correspondence CG) generated by the image generating unit 14 and the camera video. If made video is output to a not-shown display device (or an image storing apparatus). Note that the image synthesis unit 15 may synthesize the CG generated by the image generation unit 14 in an opaque or translucent manner so as to cover the camera video, for example. Further, the image composition unit 15 may superimpose the CG generated by the image generation unit 14 only on a specific color area (for example, a lawn color area) of the camera video.

  The image composition apparatus 1 shown in FIG. 2 can be realized by causing a general computer to operate according to a program that functions as each of the above-described means. This program (image composition program) can be distributed via a communication line, or can be distributed by writing on a recording medium such as a CD-ROM.

[Operation of image synthesizer]
<Presentation of CG for input>
The operation of the image composition apparatus 1 shown in FIG. 2 will be described with reference to FIG. 9 (see FIG. 2 as appropriate). FIG. 9 is a flowchart showing an input-compatible CG presentation operation by the image composition apparatus shown in FIG. First, the image synthesizing apparatus 1 converts the input display coordinates into real coordinates (coordinates in the real space) by the first coordinate conversion means 12 and stores them in the coordinate storage means 113 (step S1). Then, the image synthesizing apparatus 1 inversely transforms the actual coordinates stored in the coordinate storage means 113 into display coordinates by the inverse transformation means 16 (step S2). Then, the image synthesizing apparatus 1 generates an input corresponding CG at the position indicated by the inversely transformed coordinates by the second image generating means 17 (step S3), and outputs the generated input corresponding CG to the display 3 (step S3). S4). For example, if a line is input, an input corresponding CG is generated along the position indicated by the coordinates of the input line.

<Presentation operation of CG for output>
FIG. 10 is a flowchart showing an output corresponding CG combining operation by the image combining apparatus shown in FIG. First, the image synthesizing apparatus 1 converts the input display coordinates into real coordinates by the first coordinate conversion means 12 and stores them in the coordinate storage means 113 (step S11). Then, the image synthesizing apparatus 1 converts the actual coordinates stored in the coordinate storage unit 113 by the second coordinate conversion unit 13 based on the camera parameter θ (step S12). Then, the image synthesizing apparatus 1 uses the image generation unit 14 to generate an output corresponding CG at a position indicated by the converted coordinates (step S13). Then, the image synthesizing apparatus 1 synthesizes and outputs the output corresponding CG and the camera video by the image synthesizing unit 15 (step S14). For example, if a line is input, an output-corresponding CG is generated along the position indicated by each coordinate of the line obtained by converting the input line.

  According to the first embodiment, the output corresponding CG corresponding to the input figure input to the touch panel 4 by the operator is generated at the position indicated by the coordinate value converted based on the camera parameter θ. Therefore, the manually input image and the camera video characterized by the camera parameter θ can be synthesized in conjunction with the camera parameter θ. As a result, when the camera angle of the camera image in the composite image changes, the output corresponding CG input before the change automatically moves with the image object such as a person in the camera image, or its shape changes. Or That is, when the composite video is displayed, it is possible to obtain a visual effect such that an manually input image is directly drawn on the camera video.

(Second Embodiment)
FIG. 11 is a functional block diagram showing a configuration example of an image composition device according to the second embodiment of the present invention. The image composition apparatus 1A according to the second embodiment is shown in FIG. 2 except that the first coordinate conversion means 12 and the inverse conversion means 16 shown in FIG. 2 are omitted and the function of the coordinate conversion means 13A is different. Since the configuration is the same as that of the image synthesizing apparatus 1, the same configuration is denoted by the same reference numeral, and the description thereof is omitted.

The coordinate conversion means 13A converts the coordinate value of the image coordinate system into the coordinate value of the real coordinate system corresponding to the real space, and then uses the converted coordinate value as the coordinate of the image coordinate system using the camera parameter θ. The coordinate value stored in the coordinate storage unit 113 is converted using a conversion rule indicating a series of conversions to be converted into values. That is, the coordinate conversion unit 13A has a function that combines the function of the first coordinate conversion unit 12 and the function of the second coordinate conversion unit 13 shown in FIG.

  The image composition apparatus 1A stores the coordinates input from the touch panel 4 in the coordinate storage unit 113 as they are. That is, the coordinate system of the coordinate values stored in the coordinate storage unit 113 is display coordinates. Then, the image synthesizing apparatus 1A reads out coordinate values (coordinate values input from the touch panel 4) stored in the coordinate storage unit 113 by the coordinate conversion unit 13A and the second image generation unit 17. According to the second embodiment, the configuration of the image composition device can be simplified.

(Third embodiment)
FIG. 12 is a schematic diagram showing an outline of the operation of the image composition device according to the third embodiment of the present invention. Image synthesis apparatus 1B according to the third embodiment, synthesized and the input corresponding CG1211 and camera image V _A is intended to be displayed on the display 3.

The image synthesizing apparatus 1B captures a camera image V _A captured by a camera (first camera) (not shown) and a camera parameter θ _A (first camera parameter) related to the first camera, and a camera (second camera) (not shown). The camera image V _B and the camera parameter θ _B (second camera parameter) relating to the second camera are input. Note that the first camera and the second camera may be different cameras or the same camera.

Here, the camera video V _A is displayed on the display 3 in order to visually assist the input operation by the operator. This camera video V _A is a video to be combined with the input graphic 1201 written on the touch panel 4.
The camera video V _B is synthesized with the input figure 1201 written on the touch panel 4 to become a synthesized video 1202.

The image composition device 1 </ b> B generates two types of CGs, that is, an input correspondence CG 1211 and an output correspondence CG 1221, based on the input figure 1201 written on the touch panel 4.
The input corresponding CG 1211 is a CG displayed on the display 3, mainly for confirming on the display 3 an image input by the operator of the input device 2 in an attempt to draw the input figure 1201 on the camera video V _A. Generated. The input correspondence CG 1211 is obtained by converting the input figure 1201 written on the touch panel 4 in accordance with the camera video V _A. The generated input corresponding CG 1211 is displayed on the display 3. The Katakana character “I” indicated by the input correspondence CG 1211 is projected and synthesized as if it was stuck to the ground of the soccer court. Accordingly, by changing the camera parameter θ _A every moment in conjunction with the operation of the first camera, the input corresponding CG 1211 is deformed, and the video as if the input corresponding CG 1211 exists in the camera video V _A. An effect is obtained.

The output correspondence CG 1221 is a CG synthesized with the synthesized video 1202 output to a display device (or image storage device) (not shown), and the input figure 101 written on the touch panel 4 is converted corresponding to the camera video V _B. Except for the above, since it is the same as the input-compatible CG 1211, detailed description thereof is omitted.

[Configuration of Image Synthesizer]
FIG. 13 is a functional block diagram illustrating a configuration example of the image composition device illustrated in FIG. 12.
The image composition apparatus 1B has the same configuration as the image composition apparatus 1 shown in FIG. 2 except that the functions of the first coordinate conversion means 12B and the inverse conversion means 16B are different and the second image composition means 30 is provided. The same components are denoted by the same reference numerals, and the description thereof is omitted. Note that the camera parameter θ _B is input to the second coordinate conversion unit 13, and the camera video V _B is input to the image composition unit 15.

<First coordinate conversion means>
The first coordinate conversion unit 12B converts the coordinate value of the image coordinate system (input image coordinate system) input to the touch panel 4 into the coordinate value of the real coordinate system corresponding to the real space using the camera parameter θ _A. Is. The converted coordinate value is stored in the coordinate storage means 113. The first coordinate conversion unit 12B performs non-linear conversion using a perspective projection model, and converts display coordinates to real space coordinates.

FIG. 14 is an explanatory diagram of a technique for executing the inverse transformation of the perspective projection by the first coordinate transformation means shown in FIG. 13 under a plane constraint condition.
FIG. 14 shows a Cartesian coordinate system in which the X axis and the Y axis are taken in the horizontal plane and the Z axis is taken in the height direction in the real space. The first coordinate conversion unit 12B (see FIG. 13) generates a half line 1404 that passes through the image coordinates 1403 of the image plane 1402 from the projection center 1401 of the imaging system of the first camera (not shown), and generates the half line 1404 and the conversion destination. A conversion coordinate 1406 that is an intersection with the space 1407 is obtained. That is, the two-dimensional image coordinates 1403 are back-projected as three-dimensional conversion coordinates 1406 in the real space. The conversion destination space 1407 is a curved surface or a group of curved surfaces set in the real space (here, a curved surface includes a plane), and represents, for example, the ground surface in the real space. Particularly in a sports stadium, for example, a ground surface (such as a pitch surface of a football) of the stadium can be represented. Further, for example, in the case of a stadium with ups and downs such as a golf course, the curved surface can be formed in consideration of the ups and downs.

Here, the image coordinate 1403 indicates the input coordinate (for example, pixel unit) on the display 3, and the conversion coordinate 1406 indicates the real space coordinate (for example, meter unit). Further, the value of the image coordinate 1403 is (x, y), and the value of the converted coordinate 1406 is (X, Y, Z). The converted coordinate values (X, Y, Z) can be determined from the image coordinate values (x, y) and the camera parameter θ _A.

  When the conversion destination space 1407 is a soccer court as in the present embodiment, Z = 0. Here, the converted coordinate values (X, Y, Z) correspond to the values of the coordinates 502 (see FIG. 5) stored in the coordinate storage means 113 (see FIG. 2).

As shown in Expression (8), the first coordinate conversion unit 12B uses the vector function g ′ and the camera parameter θ _A to input coordinates on the display 3, that is, image coordinate values (x, y). ) Is converted into converted coordinate values (X, Y, Z). Equation (8) is a general form.

  The conversion destination space 1407 is not limited to a plane (ground) with Z = 0. For example, a plane (wall) with X = 0 or a plane (wall) with Y = 0 may be used. Further, the conversion destination space 1407 is not limited to a plane, and may be a curved surface. Here, the curved surface also means, for example, actual bumpy ground. In this way, when the conversion destination space 1407 is a curved surface, there may be a plurality of intersections between the conversion destination space 1407 and the half line 1404. Thus, when there are a plurality of intersections, the first coordinate conversion unit 12B selects one intersection according to a predetermined rule. Here, for example, the predetermined rule may be to select an intersection closest to the projection center 1401.

<Inverse conversion means>
The inverse conversion unit 16B converts the coordinate value stored in the coordinate storage unit 113 into the coordinate value of the image coordinate system on the display 3. As shown in the equation (9), the inverse conversion means 16B uses the vector function h ′ and the camera parameter θ _A to calculate the coordinate values (X, Y, Z) of the real coordinate system corresponding to the real space. The coordinate value (x, y) on the display 3 is converted. Equation (9) is a general form. Since this coordinate transformation is derived on the same principle as the above-described equation (7), its description is omitted. The coordinate values converted by the inverse conversion unit 16B are output to the second image generation unit 17. Thereby, the second image generation means 17 can generate CG (input correspondence CG) linked to the camera parameter θ _A.

<Second image composition means>
The second image synthesizing unit 30 synthesizes the CG (input correspondence CG) generated by the second image generating unit 17 and the camera video V _A. The synthesized video is output to the display 3. Since the synthesizing method is the same as that of the image synthesizing means 15, the description is omitted.

[Operation of image synthesizer]
The operation of the image composition apparatus 1B shown in FIG. 13 will be described with reference to FIG. 15 (see FIG. 13 as appropriate). FIG. 15 is a flowchart showing an input-compatible CG combining operation by the image combining apparatus shown in FIG. First, the image synthesizing apparatus 1B converts the input display coordinates into real coordinates by the first coordinate conversion means 12B and stores them in the coordinate storage means 113 (step S21). Then, the image synthesizing apparatus 1B inversely transforms the actual coordinates stored in the coordinate storage means 113 into display coordinates by the inverse transformation means 16B (step S22). Then, the image synthesizing apparatus 1B generates an input-corresponding CG at the position indicated by the inversely transformed coordinates by the second image generating means 17 (step S23). The image synthesizing apparatus 1 then synthesizes the generated input corresponding CG and the camera video V _A by the second image synthesizing unit 30 and outputs the synthesized CG to the display 3 (step S24). For example, if a line is input, an input corresponding CG is generated along the position indicated by the coordinates of the input line. Note that the output corresponding CG combining operation in the image combining apparatus 1B is the same as that in the flowchart of FIG.

According to the third embodiment, the input corresponding CG and the output corresponding CG corresponding to the input figure input to the touch panel 4 by the operator are indicated by coordinate values converted based on the camera parameters θ _A and θ _B. Two composite images can be generated separately at the position. Therefore, it is possible to distinguish and use one composite video as an operator video and the other composite video as a viewer video.

  As mentioned above, although this invention was demonstrated based on each embodiment, this invention is not limited to these. For example, although the synthesized image synthesized by the image synthesizing unit 15 has been described as being broadcast to the viewer, for example, it is not limited to this. For example, the synthesized image synthesized by the image synthesizing unit 15 may be output to another display arranged near the display 3 so that the operator of the touch panel 4 can view it. Further, the synthesized image synthesized by the image synthesizing unit 15 may be output to the display 3 for viewing by the operator of the touch panel 4. In this case, the operator switches and uses the display screen of the display 3 between input and confirmation.

  Further, the above-described image generation means 14 and image composition means 15 may be realized by a known virtual studio CG composition apparatus. Similarly, the second image generating means 17 and the second image synthesizing means 30 of the image synthesizing apparatus 1B may be realized by a known virtual studio CG synthesizing apparatus.

  In each embodiment, it has been described that a camera image related to soccer is input to the image composition device 1 (1A, 1B). Camera images related to various sports may be used. Furthermore, not only sports, but a live-action camera image such as a news program, entertainment program, educational program, cooking program, weather forecast, etc. may be used.

It is a schematic diagram which shows the outline | summary of operation | movement of the image synthesizing | combining apparatus which concerns on 1st Embodiment of this invention. It is a functional block diagram which shows the structural example of the image synthesizing | combining apparatus shown in FIG. It is a figure which shows the example of a coordinate conversion by the 1st coordinate conversion means shown in FIG. 2, (a) has shown the display screen, (b) has each shown the conversion destination space. FIG. 4 is an explanatory diagram when the conversion example shown in FIG. 3 is applied to a plurality of coordinate values, where (a) shows display coordinates and (b) shows coordinates in a conversion destination space. It is a figure which shows an example of the database memorize | stored in the coordinate memory | storage means shown in FIG. It is a figure which shows an example of the input corresponding | compatible CG produced | generated by the 2nd image generation means shown in FIG. It is explanatory drawing of a perspective projection. It is a figure which shows an example of output corresponding | compatible CG produced | generated by the image production | generation means shown in FIG. 3 is a flowchart showing an input-compatible CG presentation operation by the image composition device shown in FIG. 2. FIG. 3 is a flowchart showing an output corresponding CG combining operation by the image combining apparatus shown in FIG. 2. FIG. It is a functional block diagram which shows the structural example of the image synthesizing | combining apparatus which concerns on 2nd Embodiment of this invention. It is a schematic diagram which shows the outline | summary of operation | movement of the image synthesizing | combining apparatus which concerns on 3rd Embodiment of this invention. It is a functional block diagram which shows the structural example of the image synthesizing | combining apparatus shown in FIG. It is explanatory drawing of the method of performing reverse transformation of perspective projection by the 1st coordinate transformation means shown in FIG. 13 under planar restraint conditions. It is a flowchart which shows the synthetic | combination operation | movement of input corresponding | compatible CG by the image synthesis apparatus shown in FIG.

Explanation of symbols

1, 1A, 1B Image composition device 2 Input device 3 Display 4 Touch panel 11 Storage means 111 RAM
112 ROM
113 Coordinate storage unit 12, 12B First coordinate conversion unit 13, 13A Second coordinate conversion unit 14 Image generation unit 15 Image combination unit 16, 16B Reverse conversion unit 17 Second image generation unit 30 Second image combination unit

Claims (6)

An image synthesizer that synthesizes a camera image showing an image of a subject photographed by a camera and an image generated corresponding to a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image. ,
Connected to a display integrated with a touch panel, the coordinates of an image to be combined with the camera image from the input device via the touch panel are input as coordinate values of the input image coordinate system,
Coordinate storage means for storing coordinate values of the input image coordinate system;
The camera parameter including the shooting angle of the camera when the camera image is captured after the coordinate value of the input image coordinate system is converted into the coordinate value of the real coordinate system corresponding to the real space. using the conversion rule shown a series of transformations that converts the coordinate value of the output image coordinate system using the coordinate conversion means for converting the coordinate values stored in the coordinate storage means to the coordinate value of the output image coordinate system ,
Image generating means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the coordinate converting means;
Image synthesizing means for synthesizing the image generated by the image generating means and the camera image;
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value stored in the coordinate storage means;
Equipped with a,
An image synthesizing apparatus that presents an image generated by the second image generating means on a display integrated with the touch panel . An image synthesizer that synthesizes a camera image showing an image of a subject photographed by a camera and an image generated corresponding to a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image. ,
Connected to a display integrated with a touch panel, the coordinates of an image to be combined with the camera image from the input device via the touch panel are input as coordinate values of the input image coordinate system,
First coordinate conversion means for converting the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space;
Coordinate storage means for storing the coordinate values converted by the first coordinate conversion means;
Second coordinate conversion means for converting a coordinate value stored in the coordinate storage means into a coordinate value of an image coordinate system using a camera parameter including a shooting angle of the camera when the camera image is shot;
Image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the second coordinate conversion means;
Image synthesizing means for synthesizing the image generated by the image generating means and the camera image;
Inverse conversion means for converting the coordinate values stored in the coordinate storage means into coordinate values of the input image coordinate system;
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the inverse conversion means;
Equipped with a,
An image synthesizing apparatus that presents an image generated by the second image generating means on a display integrated with the touch panel . A first camera image indicating an image of a subject photographed by the first camera is synthesized with an image generated in accordance with a coordinate value of an input image coordinate system for inputting a predetermined position of the first camera image. And an image composition device that synthesizes a second camera image indicating an image obtained by photographing the subject with a second camera and the generated image,
Connected to a display integrated with a touch panel, the coordinates about the image to be combined with the first camera image and the second camera image are input from the input device via the touch panel as coordinate values of the input image coordinate system,
The coordinate value of the input image coordinate system is converted into the coordinate value of the real coordinate system corresponding to the real space using the first camera parameter including the shooting angle of the first camera when the first camera image is shot. First coordinate conversion means;
Coordinate storage means for storing the converted coordinate values in the first coordinate transformation hands stage,
Second coordinates for converting a coordinate value stored in the coordinate storage means into a coordinate value of an image coordinate system using a second camera parameter including a shooting angle of the second camera when the second camera image is shot. Conversion means;
Image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the second coordinate conversion means;
Image synthesizing means for synthesizing the image generated by the image generating means and the second camera image;
Inverse conversion means for converting the coordinate values stored in the coordinate storage means into coordinate values of the input image coordinate system;
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the inverse conversion means;
Second image synthesis means for synthesizing the image generated by the second image generation means and the first camera image;
Equipped with a,
An image synthesizing apparatus, wherein the image synthesized by the second image synthesizing means is presented on a display integrated with the touch panel . In order to synthesize a camera image showing an image of a subject photographed by a camera and an image generated corresponding to a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image, an integrated touch panel A computer that is connected to an integrated display and that inputs coordinates of an image to be combined with the camera image from the input device via the touch panel as coordinate values of the input image coordinate system ;
The camera parameter including the shooting angle of the camera when the camera image is captured after the coordinate value of the input image coordinate system is converted into the coordinate value of the real coordinate system corresponding to the real space. series of transformations using the conversion rule indicating the coordinate conversion means for converting the coordinate values of the input image coordinate system to the coordinate value of the output image coordinate system into a coordinate value of an output image coordinate system using,
Image generating means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the coordinate converting means;
Image synthesizing means for synthesizing the image generated by the image generating means and the camera image;
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value stored in the coordinate storage means, and outputting the image to a display integrated with the touch panel;
Image composition program to function as In order to synthesize a camera image showing an image of a subject photographed by a camera and an image generated corresponding to a coordinate value of an input image coordinate system for inputting a predetermined position of the camera image, an integrated touch panel A computer that is connected to an integrated display and that inputs coordinates of an image to be combined with the camera image from the input device via the touch panel as coordinate values of the input image coordinate system ;
First coordinate conversion means for converting the coordinate value of the input image coordinate system into the coordinate value of the real coordinate system corresponding to the real space;
Second coordinate conversion means for converting the coordinate value converted by the first coordinate conversion means into a coordinate value of an image coordinate system using a camera parameter including a shooting angle of the camera when the camera image is shot;
Image generating means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the second coordinate converting means;
Image synthesizing means for synthesizing the image generated by the image generating means and the camera image;
Inverse conversion means for converting the coordinate values stored in the coordinate storage means into coordinate values of the input image coordinate system;
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the inverse conversion means, and outputting the image to a display integrated with the touch panel;
Image composition program to function as A first camera image indicating an image of a subject photographed by the first camera is synthesized with an image generated in accordance with a coordinate value of an input image coordinate system for inputting a predetermined position of the first camera image. In addition, in order to synthesize the second camera image indicating the image obtained by photographing the subject with the second camera and the generated image, the display is integrated with a touch panel, and is input from the input device via the touch panel. A computer in which coordinates of an image to be combined with the first camera image and the second camera image are input as coordinate values of the input image coordinate system ;
The coordinate value of the input image coordinate system is converted into the coordinate value of the real coordinate system corresponding to the real space using the first camera parameter including the shooting angle of the first camera when the first camera image is shot. First coordinate conversion means;
The converted coordinate values in the first coordinate transformation hands stage, using a second camera parameters including photographing angle of the second camera upon shooting the second camera image is converted into the coordinate values of the image coordinate system Second coordinate conversion means;
Image generating means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the second coordinate converting means;
Image synthesizing means for synthesizing the image generated by the image generating means and the second camera image;
Inverse conversion means for converting the transformed coordinates values in the first coordinate transformation hands stage, the coordinate values of the input image coordinate system,
Second image generation means for generating an image corresponding to the coordinate value of the input image coordinate system at a position indicated by the coordinate value converted by the inverse conversion means;
Second image synthesizing means for synthesizing the image generated by the second image generating means and the first camera image and outputting them to a display integrated with the touch panel ;
Image composition program to function as

Images