JP5321176B2 - Projection auxiliary image generation method and generation apparatus, and image forming apparatus - Google Patents

Description

  The present invention relates to a method and apparatus for generating a projection auxiliary image used as an auxiliary to a projection image projected on a screen from a projection apparatus, and an image forming apparatus.

  Generally, OHP (Over Head Projector) is used for presentations in business sites. That is, for example, a projection image for presentation created by PowerPoint (registered trademark) or the like is stored in advance in a personal computer. At the time of presentation, the projected image is projected on a screen by a projector to give presentations and explanations.

  In order to give a more effective presentation, the presenter sometimes makes a supplementary explanation by writing a line or a character on a screen on which a projected image is projected or pasting an image such as a photograph while giving a presentation.

  Thus, there is a demand for adding additional information added for supplementary explanation during the presentation in an electronic manner as presentation information and storing it as a projection auxiliary image. There have been some proposals for the generation of auxiliary projection images.

  For example, by capturing an image on the screen and performing image processing such as binarization and pattern extraction on the captured image, a human finger, a pointing stick, or a laser is applied to the projected image projected on the screen from the projector. It has been proposed to specify a location pointed by a pointer or the like (Patent Document 1).

  Further, an image of the annotation data including the image of the identification information is taken in a state where the image of the identification information for identifying the original document data is projected onto the projection target, and an image of the annotation data with the identification information is acquired. Then, based on the identification information image included in the acquired annotation data image with identification information, it has been proposed to synthesize the image of the corresponding original document and the annotation data image (Patent Document 2).

  In addition, it has been proposed that handwritten writing information written on a screen is scanned by a scanner unit to digitize a written image and synthesized with a projection image projected by a projector unit (Patent Document 3).

JP2000-200139A JP 2006-108813 A JP 2001-16384 A

However, when the screen is a white board or the like, dirt due to unerasing or forgetting to erase the previous writing by a sign pen or the like may adhere to the surface of the screen.

  In that case, an image obtained by photographing the screen also includes a dirt image, and even if a projection auxiliary image is generated using such an image as it is, a clear image can be obtained. Can not. Therefore, there is a problem that when the image is reprojected by the projector, displayed on the screen of the display device, or printed on paper, the image is difficult to see or the stains are not noticeable.

  With respect to this problem, Patent Document 1 discloses that image processing is performed on an image captured by an imaging apparatus, but it is disclosed that some sort of image processing is performed on a stain image on the screen surface. There is no. Also, Patent Document 2 and Patent Document 3 do not disclose a measure for solving this problem.

  In view of the problems described above, an object of the present invention is to obtain a projection auxiliary image in which the influence of dirt is reduced even when dirt such as an unwritten part is present on the surface of the screen.

A method according to an embodiment of the present invention is a method of generating a projection auxiliary image that is used as an auxiliary to a projection image projected on a screen from a projection device, the projection screen being in a state where the projection image is not projected A step of photographing a surface image with a photographing device to obtain a first image; and an additional material added for assistance to the projected image with the screen as a background and without projecting the projected image. A step of obtaining a second image by photographing at a step, a step of obtaining a difference by comparing the first image and the second image, and a step of obtaining a difference image including an additional image that is an image of the additional object When the first image and comparing the second image, determines that the additional image region is a region occupied a portion of the change in the concentration of the additional image, the portion without the change in concentration Determining a background image area that is an area occupied by a background image of an image other than the additional image, obtaining a maximum density in the background image area, and uniformly setting the entire background image area to the maximum density A step of generating a fourth image by combining a background image having a high density and the additional image included in the difference image; and a state in which the fourth image is projected from the projection device onto the screen. Capturing a fifth image and acquiring the projection auxiliary image based on the fifth image .

  By obtaining the difference between the first image obtained by photographing the surface of the screen in a state in which the projected image is not projected and the second image obtained by photographing the additional object with the same screen as the background, the image due to dirt is cancelled.

  Preferably, the method includes a step of enhancing the density of the additional image.

  According to the present invention, it is possible to obtain a projection auxiliary image in which the influence of the dirt is reduced even when there is dirt such as an unwritten part on the surface of the screen.

1 is a diagram illustrating an overall schematic configuration of a projector cooperation system to which an image forming apparatus according to a first embodiment of the present invention is applied. 2 is a block diagram illustrating a functional configuration of the image forming apparatus. FIG. It is a block diagram which shows the structure of the production | generation apparatus of the projection auxiliary image implement | achieved in an image forming apparatus. It is a figure for demonstrating the specific example of the production | generation process of a projection auxiliary image. 3 is a flowchart illustrating a flow of a method for generating a projection auxiliary image in the image forming apparatus according to the first embodiment. It is a flowchart which shows the flow of a process in the camera of 1st Embodiment. It is a block diagram which shows the structure of the production | generation apparatus of the projection auxiliary image of 2nd Embodiment. It is a figure for demonstrating the specific example of the production | generation process of a projection auxiliary image. It is a figure for demonstrating the specific example of the production | generation process of a projection auxiliary image. 10 is a flowchart illustrating a flow of a method for generating a projection auxiliary image in the image forming apparatus according to the second embodiment.

[First Embodiment]
In FIG. 1, a projector cooperation system 1 according to the first embodiment includes an image forming apparatus 11, a projector 12, a camera 13, a screen 14, a personal computer (PC) 15, and the like.

  Referring also to FIG. 2, the image forming apparatus 11 is a multifunction device called a so-called MFP (Muliti Function Peripheral) in the present embodiment. The image forming apparatus 11 has a copy function, a network printing function, a FAX function, a scanner function, a document server function, and the like. The image forming apparatus 11 controls the projector (projection apparatus) 12 and the camera (imaging apparatus) 13, projects the projection image GT by the projector 12, and photographs the surface of the screen 14 by the camera 13, thereby generating a projection auxiliary image GH. To do. Details will be described later.

  The projector 12 projects the projection image GT on the screen 14 based on the projection data (projection image data) DT sent from the image forming apparatus 11. The projector 12 is connected to the image forming apparatus 11 via a video cable 31 and receives the above-described projection data DT from the image forming apparatus 11. In addition, the image forming apparatus 11 turns on and off the projection of the projection image GT, light Projection on / off, power on / off, and the like are controlled.

  The projection image GT includes a document image GK, a composite image GP, a projection auxiliary image GH, and the like. By projecting the projection data DT input to the projector 12, a projection image GT is obtained. The projection data DT is RGB data for a color image, for example. In the present specification, the “image” that can be visually recognized and the “image data” that is the state of the data are the same in terms of content, and may be described without distinction.

  The camera 13 captures (captures) an image of the surface of the screen 11. In the present embodiment, the camera 13 captures a still image and outputs a captured image (image data) G. The first image G1 and the second image G2 are obtained according to the timing of capturing the image G, that is, according to the state of the screen 11 when the image G is captured.

  For example, a digital camera is used as the camera 13, but it may be an imaging device that can capture a moving image. The camera 13 is connected to the image forming apparatus 11 via a control cable 32, and shooting timing (shutter), flash on / off, and the like are controlled by the image forming apparatus 11, and the captured image G is sent to the image forming apparatus 11. The camera 13 is installed and fixed at a position where the entire screen 14 can be appropriately photographed from the front.

  A projection image GT from the projector 12 is projected on the screen 14. As the screen 14, various types and sizes of screens such as a reflective screen, a transmissive screen, a presentation screen, and a projection screen can be used. Further, a white board for writing characters or the like, or a white wall surface can be used as the screen.

  As will be described later, an additional FB may be added to the screen 14 by writing a line or a character or pasting an image such as a photograph or a document. It should be as easy as possible. In this regard, if it has a function like a whiteboard, it is possible to directly write handwritten characters using a sign pen or the like, and writing can be easily erased. However, a sheet may be pasted on the screen and characters may be indirectly written on the sheet. In order to attach a photograph or a document, for example, it may be stopped with a push pin, attached with an adhesive tape, or fixed with a magnet. Further, when a whiteboard or a wall surface is used as the screen, it is preferable to have a surface state in which halation does not occur as much as possible when shooting with the camera 13. In order to suppress halation, the indoor lighting and the light emission amount of the projector 12 may be adjusted.

  The personal computer 15 is a so-called personal computer, and includes a CPU, a memory, a magnetic disk, an input / output interface, and the like. The personal computer 15 stores various projection data DT for use in presentation. Such projection data DT is created inside the personal computer 15 using appropriate software such as PowerPoint (registered trademark), or inputted from the outside.

  The personal computer 15 is connected to the image forming apparatus 11 by a video cable 33 and transmits, for example, projection data DT including RGB data to the image forming apparatus 11. The personal computer 15 can also be connected to the image forming apparatus 11 via the LAN 17.

  As the video cables 31 and 33 or the control cable 32, a USB cable or a network cable may be used.

  In the projector cooperation system 1 according to the present embodiment, the personal computer 15 is not necessarily required as long as the projection data DT can be taken into the image forming apparatus 11. For example, the projection data DT may be captured by connecting a USB memory storing the projection data DT to the image forming apparatus 11. Further, the projection data DT may be taken into the image forming apparatus 11 by using various functions of the image forming apparatus 11 such as a box function (Box function) and a scan function of the image forming apparatus 11. Further, the projection data DT may be generated inside the image forming apparatus 11.

  The projection data DT stored in the storage unit of the personal computer 15 is once taken into the image forming apparatus 11 under the control of the image forming apparatus 11 and accumulated as presentation image data. The projection data DT accumulated in the image forming apparatus 11 is transmitted to the projector 12 in response to a user command, and is projected on the screen 14 as a projection image GT. Under the control of the image forming apparatus 11, an image G on the surface of the screen 14 is taken by the camera 13. The image G photographed by the camera 13 is sent to the image forming apparatus 11, subjected to image processing inside the image forming apparatus 11, stored in a memory, and output to the outside as necessary.

  2, the image forming apparatus 11 includes an operation control unit 21, a memory 22, an HDD 23, a projection data creation unit 24, an image calculation unit 25, a user IF 26, a projection data output unit 27, a camera control unit 28, and a projection data input unit 29. The network IF 30, the document reading unit 34, the printing unit 35, and the like.

  The operation control unit 21 controls the projection data creation unit 24, the image calculation unit 25, and the like to perform necessary processing, and also controls the document reading unit 34, the printing unit 35, and the like to perform the entire operation of the image forming apparatus 11. To control. The operation control unit 21 also controls the projector 12 and the camera 13 as described above in the projector mode.

  The memory 22 temporarily stores generated or input projection data DT, an image (image data) G taken by the camera 13, other images or data, and the like.

  The HDD (hard disk device) 23 stores projection data DT, image G, other images, data, programs, and the like.

  The projection data creation unit 24 creates projection data DT. In particular, in the projector mode, projection data DT, that is, a projection auxiliary image GH and a composite image GP, which are a kind of presentation image, are created. That is, the projection auxiliary image GH is created based on the first image G1 and the second image G2 taken by the camera 13, and the composite image GP is created by synthesizing the projection auxiliary image GH and the document image GK. .

  The image calculation unit 25 calculates the difference between the first image G1 and the second image G2 captured by the camera 13, and emphasizes the density of the additional image included in the second image G2 or the projection auxiliary image GH. For example, a calculation process necessary for creating the projection data DT is performed. Further, various image processing is performed on the image read by the document reading unit 34 and the input image.

  A user IF (user interface) 26 is a part that performs an interface with a user. The user IF 26 is, for example, a touch panel on which various operation buttons are displayed. The user inputs various commands and data to the image forming apparatus 11 via the user IF 26 and receives messages from the image forming apparatus 11.

  For example, the user sets the projector mode via the user IF 26, and instructs the camera 13 to shoot the first image G1 and the second image G2 in the projector mode. Further, in the projector mode, the projection of the projection image GT by the projector 12 is prohibited in order to prevent the projection image GT from appearing when the camera 13 is instructed to shoot. In addition, for illumination at the time of shooting by the camera 13, light projection with adjusted light amount is performed from the projector 12 or flash is emitted as necessary.

  Note that the user can remotely operate the image forming apparatus 11 using a browser of a personal computer via the LAN 17 or another network instead of using the user IF 26. For example, using the personal computers 15 and 16 or another external personal computer, a command is given to the image forming apparatus 11 using a browser, and the camera 13 is controlled to capture the first image G1, the second image G2, and the like. Is also possible.

  The projection data output unit 27 performs control for outputting the projection data DT input from the personal computer 15 and the projection data DT created in the image forming apparatus 11 to the projector 12.

  The camera control unit 28 controls the camera 13 in the projector mode.

  The projection data input unit 29 performs control for inputting projection data DT from the personal computer 15.

  The network IF 30 receives and transmits various data via the LAN 17 or the like. When the personal computer 15 is connected to the LAN 17, the projection data DT can be input via the network IF 30 without using the projection data input unit 29.

  The document reading unit 34 scans and reads images of documents of various sizes. The printing unit 35 prints an image read by the document reading unit 34 or an image sent from the personal computer 15 or 16 on a sheet by an electrophotographic process.

  Next, processing in the projector mode, particularly processing for generating the projection auxiliary image GH will be described.

  As described above, the image forming apparatus 11 can perform processing for creating the projection auxiliary image GH when the projector mode is set. The projection auxiliary image GH is used auxiliary to the projection image GT projected from the projector 12 onto the screen 14.

  That is, when the user is performing a presentation by projecting the projection image GT onto the screen 14 using the projector 12, the user writes a line or character on the screen on which the projection image GT is projected, or photographs or materials. A supplementary explanation may be given by pasting. As described above, the addition FB added to the projection image GT is digitized (image data) with the screen 14 as a background, and the projection auxiliary image GH is generated based on the digitization.

  The projection auxiliary image GH can be saved as an independent image, or can be synthesized so as to be added to the original projection image GT (original image GK) and saved as a synthesized image. Is possible. The projection auxiliary image GH can be used as an auxiliary to the original document image GK by projecting onto the screen 14 by the projector 12. Therefore, once the projection auxiliary image GH is created, the projection auxiliary image GH can be projected by the projector 12 in the next presentation, so that the projection auxiliary image can be projected without preparing the actual addition FB. A presentation can be performed using the additional image GF by GH.

  FIG. 3 shows a functional configuration of the projection auxiliary image generation apparatus HS realized in the image forming apparatus 11.

  In FIG. 3, the generation device HS includes a first image acquisition unit 41, a second image acquisition unit 42, a difference acquisition unit 43, a density enhancement processing unit 44, a synthesis unit 45, a projection data storage unit 46, and the like.

  The first image acquisition unit 41 acquires the first image G1 captured by the camera 13. The first image G1 is an image of the surface of the screen 14 taken by the camera 13 in a state where the projection image GT is not projected. That is, it is an image obtained by photographing only the screen 14, and the surface state of the screen 14 is an image as it is. If there is a stain on the surface of the screen 14, the stain appears as an image.

  The first image G1 is taken by the user giving a command to the image forming apparatus 11. The user gives an instruction to shoot the first image G1 after setting the operation mode of the image forming apparatus 11 to the projector mode. At the time of shooting, projection of the projection image GT of the projector 12 is prohibited. In order to illuminate the screen 14, light projection is performed by the projector 12 or the flash of the camera 13 is emitted as necessary.

The second image acquisition unit 42 acquires the second image G2 captured by the camera 13. The second image G2 is an image obtained by photographing the addition FB supplementarily added to the projection image GT for supplementary explanation or the like with the screen 14 as a background and without projecting the projection image GT. is there. That is, the second image G2 is an image obtained by photographing only the addition FB with the screen 14 as a background. The portion of the surface of the screen 14 that is not hidden by the additive FB appears as an image as it is. Therefore, when there is a stain on the surface of the screen 14, the stain appears as an image.

  The “addition” is information or information added to the projection image GT for supplementary explanation, such as lines and characters written on the screen 14, photographs or materials pasted on the screen 14. is there. An image obtained by photographing the “addition” with the camera 13 with the screen 14 as a background is the second image G2. Therefore, the second image G2 includes an “addition image” that is an image of the “addition”.

  The shooting of the second image G2 is performed by the user giving a command to the image forming apparatus 11. In the middle of the presentation, the user gives an instruction to shoot the second image G2 after adding the appendage FB to the projection image GT to the screen 14. Then, the image forming apparatus 11 prohibits the projection of the projection image GT by the projector 12, performs the same control for illumination as that of the first image G <b> 1, and performs imaging by the camera 13.

  The difference acquisition unit 43 acquires a difference image (difference image) GS between the first image G1 and the second image G2. The difference image GS is obtained, for example, by subtracting the first image G1 from the second image G2. Therefore, when the surface of the screen 14 is dirty, an image due to the stain appears in both the first image G1 and the second image G2, and therefore the image due to the stain is canceled (offset) in the difference image GS. Lost or greatly reduced.

  Further, in the first image G1 and the second image G2, the density of the image on the surface of the screen 14 usually has a significant value instead of 0. That is, a low-density gray image that is not pure white appears as indicating the density of the background image (background density). However, the background density is also canceled in the difference image GS. Accordingly, in the difference image GS, the density value of the portion other than the additional image GF is almost zero. On the other hand, in the difference image GS, the density value of the additional image GF decreases by a density corresponding to the background density.

  In the difference image GS, images other than the additional image GF are background images.

  The density enhancement processing unit 44 performs a process of enhancing the density of the additional image GF (density enhancement process). That is, in the present embodiment, density enhancement processing is performed on the additional image GF included in the difference image GS. The density enhancement process may be performed on the additional image GF included in the second image G2, or may be performed on the additional image GF included in the projection auxiliary image GH.

  When density enhancement processing is performed, first, an area (additional image area) where density enhancement is to be performed is detected. The additional image area is an area occupied by the additional image GF. In order to detect the additional image region, the region of the additional image GF and the region of the background image GB are separated and extracted. In order to demarcate the region of the background image GB, for example, the first image G1 and the second image G2 are compared, the region where the density does not change is set as the background region, and the region where the density changes is set as the additional image region. .

  As the density enhancement process, for example, the density of the additional image GF is increased by a predetermined ratio. Alternatively, a predetermined density is added to the density of the additional image GF. At that time, the multiplication rate (multiplication coefficient) or the addition value (addition coefficient) may be adjusted according to the density of the additional image GF before density enhancement processing. For example, when the density of the additional image GF is lower than a predetermined value, the multiplication rate or the addition value is increased, and when the density of the additional image GF is higher than the predetermined value, the multiplication rate or the addition value is decreased. Further, the density of the additional image GF may be set to a predetermined value. In this case, an average value, median value, maximum value, etc. of the density can be used as the density of the additional image GF. By performing the density enhancement process, the additional image GF becomes clearer.

  Therefore, for example, when the user writes light on the surface of the screen 14 or the line is thin, the additional image GF may become thin and unclear, but the density is increased by performing the density enhancement process. , The sharpness increases.

  The synthesizing unit 45 performs a process of synthesizing the projection auxiliary image GH and the original image GK1, and generates a synthesized image GP. The document image GK1 used here is the document image GK to be added with the appendage FB that is the source of the projection auxiliary image GH. The generated composite image GP is stored in the projection data storage unit 46, sent to the projector 12 in accordance with a user operation instruction, and projected onto the screen 14 as a projection image GT.

  The projection data storage unit 46 stores projection data DT that can be sent to the projector 12 for presentation or for generation of a projection auxiliary image GH. As the projection data DT, data such as a document image GK, a composite image GP, and a projection auxiliary image GH are stored. The document image GK includes an image obtained by reading the document with the document reading unit 34, an image sent from the personal computer 15, an image generated inside the image forming apparatus 11, and the like.

  Next, a process for generating the projection auxiliary image GH will be described based on a specific example.

  On the upper side of FIGS. 4A to 4D, there is an example in which the screen 14 in which the character “a” remains as the dirty image GY1 is used, and the material having the gray image is pasted on the screen region RS as the additional FB1. It is shown. In the screen region RS, the density value NT for the x coordinate along the x axis at the specific coordinate y1 on the y axis is shown below (A) to (D) in FIG.

  In FIG. 4A, the stain image GY1 considered to have been written in the previous presentation remains on the screen 14. In this state, the screen 14 is photographed to obtain the first image G1. Therefore, in the first image G1, the density value NT of one line in the x-axis direction at the coordinate y1 is the density value NT1 at the x coordinate position corresponding to the dirt image GY, and the density value is NT2 at the other background portions. It is.

  In FIG. 4B, the additional material FB1 is attached to the screen 14. In this state, the screen 14 is photographed to obtain a second image G2. Therefore, in the second image G2, the density value at the x-coordinate position corresponding to the additive FB1 is NT3.

  In FIG. 4C, a difference image GS1 obtained by subtracting the first image G1 from the second image G2 is shown. In the difference image GS, the density value NT for the background portion and the dirt image GY1 is canceled to 0 or almost 0, and the density value for the additive FB1 is NT4. The density value NT4 is a value obtained by subtracting the density value NT1 from the density value NT3.

  In FIG. 4D, density enhancement processing is performed on the additional image GF1 of the difference image GS, and a projection auxiliary image GH1 is obtained. In the projection auxiliary image GH1, the original density value is increased from NT4 to NT5. The density value NT5 is obtained, for example, by multiplying the original density value NT4 by a multiplication factor K1, or by adding a predetermined density value K2.

  Next, a process for generating the projection auxiliary image GH will be described with reference to a flowchart.

  In FIG. 5, when the projector mode is turned on (Yes in # 11), the generating device HS of the image forming apparatus 11 sends a shooting command to the camera 13 (# 12). The image G photographed by the camera 13 is captured as the first image G1 (# 13). Presentation is started, and projection of the projection image GT by the projector 12 is started (# 14).

  In the middle of the presentation, when the user gives an instruction for imaging (digitizing) the screen 14 (Yes in # 15), the transmission of the projection data DT to the projector 12 is stopped, and the projection of the projection image GT is interrupted ( # 16). Then, a shooting command is sent to the camera 13 (# 17). The image G photographed by the camera 13 is captured as the second image G2 (# 18). The transmission of the projection data DT to the projector 12 is resumed, and the presentation using the projection image GT is continued (# 19).

  While the presentation is continuing, the generation device HS obtains a difference image GS1 for the first image G1 and the second image G2 (# 20). In the difference image GS, an additional image region is extracted (# 21), and density enhancement processing is performed on the additional image GF (# 22). The image thus obtained is set as a projection auxiliary image GH (# 23). The projection auxiliary image GH and the original document image GK are synthesized to obtain a synthesized image GP (# 24). Note that the generation of the composite image GP in step # 24 may not be performed if it is not necessary.

  In FIG. 6, the camera 13 determines the presence or absence of a shooting command from the image forming apparatus 11 (# 31). When there is a shooting command (Yes in # 32), the shutter is controlled to perform shooting (# 33), and the image G obtained by shooting is transferred to the image forming apparatus 11 (# 34).

  As described above, according to the projector cooperation system 1 of the first embodiment, the first image G1 obtained by photographing the surface of the screen 14 in a state in which the projection image GT is not projected and the first image G1 obtained by photographing the additive FB with the same screen as the background. By taking the difference from the two images G2, the dirt image GY is cancelled.

  Therefore, even when the surface of the screen 14 has a stain such as an unwritten part, it is possible to obtain the projection auxiliary image GH that eliminates or reduces the influence of the stain.

  Further, by combining the projection auxiliary image GH with the original document image GK to generate a composite image GP, and using the composite image GP instead of the original document image GK, without using the actual addition FB, It is possible to make a presentation similar to that using the addition FB.

  Therefore, it becomes easy for the user to prepare for the presentation and to perform the presentation itself.

Next, the projector cooperation system 1B of 2nd Embodiment is demonstrated.
[Second Embodiment]
In the first embodiment, the difference between the first image G1 and the second image G2 is taken in order to cancel the dirt image GY. On the other hand, in the second embodiment, the first image G1 and the second image G2 are further compared, and a portion where the density is changed is determined to be the additional image area AF, and the density is not changed. It is determined that the portion is the background image area AB. Then, different processing is performed in the background image area AB and the additional image area AF.

  That is, in the background image area AB, the maximum density LH in the background image area AB is acquired, and a background image GB in which the entire background image area AB is uniform in density with the maximum density LH is generated. In the additional image area AF, density enhancement processing is performed on the additional image GF.

  Then, the background image GB and the additional image GF are combined to generate a fourth image G4. In a state where the fourth image G4 is projected from the projector 12 onto the screen 14, the fifth image G5 is obtained by photographing with the camera 13, and the projection auxiliary image GH is obtained based on the fifth image G5.

  By projecting the fourth image G4 including the background image GB onto the screen 14, the dirt image GY becomes invisible or difficult to see on the screen 14. This is photographed by the camera 13 to generate a projection auxiliary image GH.

  In the projector cooperation system 1B of the second embodiment, a part of the processing and control in the generation device HSB of the image forming apparatus 11B is different from the first embodiment. However, other parts are the same as those in the first embodiment. Therefore, in description of 2nd Embodiment, about the part similar to 1st Embodiment, suppose that drawing and description used in 1st Embodiment are used, and description is abbreviate | omitted or simplified. In addition, each element of the second embodiment may be indicated by adding “B” to the reference numeral used in the first embodiment. In the following, differences will be mainly described.

  FIG. 7 shows a functional configuration of the projection auxiliary image generation apparatus HSB of the second embodiment realized in the image forming apparatus 11B.

  In FIG. 7, the generation device HSB includes a first image acquisition unit 41, a second image acquisition unit 42, a density enhancement processing unit 44, a synthesis unit 45, a projection data storage unit 46B, a background image acquisition unit 47, and a fourth image generation unit. 48 and a fifth image acquisition unit 49.

  The first image acquisition unit 41 acquires the first image G1 as in the first embodiment. Similarly, the second image acquisition unit 42 acquires the second image G2. The density enhancement processing unit 44 performs density enhancement processing on the additional image GF included in the difference image GS.

  The synthesizer 45B synthesizes the document image GK1 and the auxiliary projection image GH to generate a synthesized image GP. The projection data storage unit 46B stores various projection data DT. However, in the second embodiment, a fourth image G4 is added as the projection data DT in addition to the document image GK, the composite image GP, and the projection auxiliary image GH.

  The background image acquisition unit 47 acquires a background image GB. The background image acquisition unit 47 performs the following processing to acquire the background image GB. That is, first, the first image G1 and the second image G2 are compared, and a portion having a change in density is determined to be the additional image region AF, and a portion having no change in density is determined to be the background image region AB. To do.

  In this case, if there is a portion where the density of the second image G2 is lower than that of the first image G1, it is determined that the portion is the background image area AB. That is, when the dirt image GY in the first image G1 is erased in the second image G2, the density of the portion of the second image G2 is lower than that of the first image G1. That portion is treated as the background image area AB.

  Then, the maximum density LH in the background image area AB is acquired, and a background image GB is generated in which the entire background image area AB has a uniform density with the maximum density LH.

  Since the background image GB is an image (for example, a gray image) whose entire density is uniformed by the maximum density LH, when the background image GB is projected onto the dirty image GY on the screen 14, the dirty image GY is It becomes invisible or difficult to see behind the background image GB.

  Note that the background image acquisition unit 47 refers to the result of the difference acquisition unit 43 in order to determine whether or not the density is unchanged.

  The fourth image generation unit 48 includes the background image GB acquired by the background image acquisition unit 47 and the additional image GF output from the density enhancement processing unit 44 (that is, the image obtained by performing density enhancement processing on the difference image GS). The fourth image G4 is synthesized. The fourth image G4 is temporarily stored in the projection data storage unit 46B.

  The fifth image acquisition unit 49 captures the fourth image G4 on the screen 14 from the projector 12 and acquires the fifth image G5 by photographing with the camera 13. In the fifth image G5, the background image GB is projected on the screen 14 so that the dirt image GY is not noticeable, and the additional image GF is sharpened by the density enhancement process. The fifth image G5 becomes the projection auxiliary image GH as it is or after appropriate processing is performed, and is stored in the projection data storage unit 46B.

  Note that, when the second image G2 is acquired, when the image is taken in a state where the stain on the surface of the screen 14 is erased, the density in the background image GB is not set to the maximum density LH, and the density is lower than that. It is good to do.

  Further, when the fourth image G4 is generated, the background image GB and the additional image GF are not simply combined, but the density of the portion corresponding to the smear image GY in the background image GB is subtracted to be low. When the image is projected on the screen 14, the background image GB is combined with the dirt image GY on the screen 14, so that the entire density of the background image area AB becomes uniform so that the dirt image GY does not stand out. May be.

  Next, the process for generating the projection auxiliary image GH in the second embodiment will be described based on the specific examples shown in FIGS.

  First, as described in FIG. 4 in the first embodiment, the first image G1 and the second image G2 are acquired, the difference image GS1 is generated, and the density enhancement process is performed.

  Then, the maximum density LH in the background image area AB is acquired. For example, the density value NT2 in FIG. 4A or 4B is set as the maximum density LH.

  As shown in FIG. 8A, the obtained maximum density LH is used to generate a background image GB1 in which the entire background image area AB has a uniform density by the maximum density LH.

  As illustrated in FIG. 8B, the background image GB1 and the additional image GF1 are combined to generate a fourth image G4.

  The fourth image G4 is projected on the screen 14 from the projector 12, and is photographed by the camera 13 to obtain the fifth image G5.

  FIG. 9A shows a screen 14 having a dirt image GY1. FIG. 9B shows an appendage FB2 handwritten on the screen 14 having the dirt image GY1. When the fourth image G4 is projected on the screen 14 shown in FIG. 9A, the dirt image GY becomes inconspicuous due to the background image GB as shown in FIG. 9C.

  Moreover, you may project the 4th image G4 on the screen 14 on which the addition FB was stuck as FIG.9 (B) shows. In that case, the position of the actual additional object FB2 and the additional image GF included in the fourth image G4 may be matched.

  In FIG. 9B, the dirty image GY1 remains on the screen 14 on which the additional object FB2 is handwritten, but if the dirty image GY1 has been erased by this time, the gray image There is no need to generate such a background image GB1. In that case, the fourth image G4 may be generated using the background image GB having a lower density or a zero density. Therefore, in this case, when the fourth image G4 is projected on the screen 14, the background image GB does not become a gray image as shown in FIG. 9C, and the density is lower or the density is 0. A bright background image GB is obtained.

  Next, a process for generating the auxiliary projection image GH according to the second embodiment will be described with reference to a flowchart.

  In FIG. 10, steps # 31 to 40 are the same as steps # 11 to 20 in FIG. In step # 41, the background image area AB and the additional image area AF are discriminated. For the background image area AB, a background image GB is generated using the maximum density LH (# 42).

  Density enhancement processing is performed on the additional image GF in the additional image area AF (# 43). The fourth image G4 is generated by synthesizing the background image GB and the additional image GF (# 44). The fourth image G4 is projected on the screen 14 by the projector 12 (# 45). A shooting command is sent to the camera 13 (# 46). The image G photographed by the camera 13 is captured as the fifth image G5 (# 47). The fifth image G5 is stored as the projection auxiliary image GH (# 48). The projection auxiliary image GH and the original document image GK are synthesized to obtain a synthesized image GP (# 49).

  Also in the second embodiment, when the surface of the screen 14 is contaminated such as an unwritten part, it is possible to obtain the projection auxiliary image GH in which the influence due to the dirt is eliminated or the influence is reduced.

  In the embodiment described above, the generation apparatuses HS and HSB can be realized by hardware, software, or a combination thereof in the image forming apparatuses 11 and 11B. Such software, that is, a program (computer program) is recorded on an appropriate recording medium, installed in the memory 22 or HDD 23 of the image forming apparatuses 11 and 11B, and provided in the operation control unit 21 and the image calculation unit 25. It can be executed by a CPU or the like.

  In addition, the configurations, processing contents, operations, and the like of the generation devices HS and HSB can be various configurations other than those described above. Further, the configuration, structure, circuit, shape, dimensions, number, material, processing content, processing order, etc. of the whole or each part of the image forming apparatus 11, 11B or the projector cooperation system 1 are appropriately changed in accordance with the spirit of the present invention. can do.

DESCRIPTION OF SYMBOLS 1 Projector cooperation system 11, 11B Image forming apparatus 12 Projector 13 Camera 14 Screen 15 Personal computer 21 Operation control part 25 Image operation part 27 Projection data output part 28 Camera control part 29 Projection data input part 41 First image acquisition part 42 2nd Image acquisition unit 43 Difference acquisition unit 44 Density enhancement processing unit 45 Composition unit 46 Projection data storage unit 47 Background image acquisition unit 48 Fourth image generation unit 49 Fifth image acquisition unit HS Generation device GT Projection image DT Projection data GH Projection auxiliary image G1 1st image G2 2nd image G4 4th image G5 5th image GS Difference image GP Composite image GK Original image

Claims (8)

A method for generating a projection auxiliary image used as an auxiliary to a projection image projected on a screen from a projection device,
Capturing an image of the surface of the screen in a state where the projection image is not projected with a photographing device to obtain a first image;
Photographing an additional material added for assistance to the projection image with the screen as a background and not projecting the projection image to obtain a second image;
Comparing the first image with the second image to obtain the difference, and obtaining a difference image including an additional image that is an image of the appendage;
The first image and the second image are compared, and a portion having a change in density is determined to be an additional image region that is an area occupied by the additional image, and a portion having no change in density is determined other than the additional image. Determining a background image area that is an area occupied by a background image of the image;
Obtaining a maximum density in the background image region;
Combining a background image in which the entire background image region has a uniform density with the maximum density and the additional image included in the difference image to generate a fourth image;
Photographing the fourth image from the projection device onto the screen to obtain a fifth image by photographing with the photographing device;
Obtaining the projection auxiliary image based on the fifth image ;
A method for generating a projection auxiliary image, comprising: A step of enhancing the density of the additional image;
The method of generating a projection auxiliary image according to claim 1. In the step of determining, if there is a portion where the second image is lower in concentration than the first image, determines the portion to be the background image area,
The method for generating a projection auxiliary image according to claim 1 . Each of the steps is executed by an image forming apparatus that controls the projection apparatus and the imaging apparatus.
The method of generating the projected auxiliary image according to any one of claims 1 to 3. An apparatus for generating a projection auxiliary image used as an auxiliary to a projection image projected on a screen from the projection apparatus by controlling a projection apparatus and an imaging apparatus,
Means for obtaining a first image by causing the photographing device to photograph an image of the surface of the screen in a state in which the projection image is not projected;
Means for obtaining a second image by causing the photographing apparatus to photograph an additional object added to the projected image for assistance with the screen as a background and without projecting the projected image;
Means for comparing the first image with the second image to obtain the difference, and obtaining a difference image including an additional image that is an image of the appendage;
The first image and the second image are compared, and a portion having a change in density is determined to be an additional image region that is an area occupied by the additional image, and a portion having no change in density is determined other than the additional image. Means for determining that the background image area is an area occupied by the background image of the image;
Means for obtaining a maximum density in the background image region;
Means for synthesizing a background image in which the entire background image region has a uniform density by the maximum density and the additional image included in the difference image to generate a fourth image;
Means for obtaining a fifth image by causing the photographing device to shoot the fourth image projected on the screen from the projection device;
Means for acquiring the projection auxiliary image based on the fifth image ;
A projection auxiliary image generating apparatus comprising: Means for enhancing the density of the additional image;
The projection auxiliary image generating apparatus according to claim 5 . The means for generating the fourth image is combined with the background image after density enhancement processing is performed on the additional image.
The projection auxiliary image generating apparatus according to claim 5 . It claims 5 to with a generating device of the projected auxiliary image according to any one 7 of the image forming apparatus.

Images