JP6540732B2 - INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, PROGRAM, AND INFORMATION PROCESSING SYSTEM - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, a program, and an information processing system.

  2. Description of the Related Art In recent years, softphones implemented by application software have become widespread in place of telephones implemented by hardware as in the past. Since the softphone is implemented by software, the function addition to the softphone and the cooperation between the softphone and other application software can be realized relatively easily. Therefore, various application technologies related to soft phones have been proposed.

  For example, according to Patent Document 1, when a person appearing in the captured image is touched on the display screen of the captured image obtained from the camera, the person is identified by face recognition, and the telephone number of the person is acquired. A technique for making a call to a person is disclosed.

Unexamined-Japanese-Patent No. 2007-208863

  However, Patent Document 1 does not disclose switching between and displaying a captured image in which a wider area is captured and a captured image in which a narrower area is captured.

  Therefore, the present invention is a novel and improved information processing apparatus, information processing method, and program capable of switching and displaying a captured image in which a wider area is captured and a captured image in which a narrower area is captured. , And provide an information processing system.

According to the present invention, the real space displaying the display screen of the first display mode for displaying the first captured image relating to the first region of the real space captured by one of the plurality of imaging devices is displayed on the display unit An information providing unit, wherein the real space information providing unit is configured to select one of the plurality of imaging devices in accordance with the user's designation of a position in the first captured image by the user on the display screen of the first display mode. A display screen of a second display mode for displaying a second captured image related to a second area of the real space which is narrower than the first area and obtained by the one imaging apparatus along with a change in the orientation of the two imaging apparatuses An information processing apparatus is provided, which switches a display screen of the first display mode and causes the display unit to display the display screen.

  The second area may be an area including a position in the real space corresponding to a position in the first captured image specified by the user.

  The second area may be an area centered on a position in the real space corresponding to a position in the first captured image designated by the user.

  At least the first captured image may be a captured image of a part or the whole of the first area.

  At least the second captured image may be a captured moving image.

  The display screen of the second display mode may include a button display for switching the display mode from the second display mode to the first display mode.

  In the display screen of the second display mode, the button display may be displayed in the vicinity of the second captured image without being superimposed on the second captured image.

Wherein each of the first image and the second captured image may be generated captured image based on the sensing by the one imaging device among the plurality of imaging devices arranged in the real space.

The information processing apparatus may further include a communication unit connected to a network, and the communication unit may obtain each of the first captured image and the second captured image via the network.

The communication unit may obtain each of the first captured image and the second captured image from the server connected to the network via the network.

The real space information provision unit changes the orientation of one of the plurality of imaging devices so that the second region can be imaged according to the user's designation of a position in the first captured image by the user. The communication unit may transmit the request to the server to the server.

  The network may be a LAN and / or the Internet.

  The information processing apparatus may further include the display unit, and a position in the first captured image may be designated by the user's touch operation on the display unit.

The information processing apparatus includes an operating system (OS), the real space is a space in a building, and the real space information providing unit is software operating on the operating system (OS), and after startup, The real space information provision unit may display the display screen of the first display mode on the display unit, the first display mode may be a bird's-eye view mode, and the second display mode may be a proximity mode.
The information processing apparatus may be a tablet, a smartphone, a PC (Personal Computer), or a telephone.

The real space information provision unit sets the zoom ratio of the optical zoom or digital zoom of one imaging device among the plurality of imaging devices to the designated zoom ratio according to the user's designating operation of the zoom ratio on the display unit. A request to be changed is transmitted to the server to the communication unit, and the communication unit is generated based on imaging by one imaging device of the plurality of imaging devices after changing to the designated zoom ratio via the network When the second captured image is acquired, the real space information providing unit displays the display screen of the second display mode for displaying the second captured image after being changed to the designated zoom ratio on the display unit. You may

Further, according to the present invention, the display unit is caused to display the display screen in the first display mode for displaying the first captured image related to the first region of the real space captured by one of the plurality of imaging devices. According to the step and the designation of the position in the first captured image by the user on the display screen of the first display mode, the one of the plurality of imaging devices is changed along with the change of the direction of the one imaging device. Switching the display screen of the first display mode to a display screen of a second display mode for displaying a second pickup image relating to a second area of the real space which is obtained by one imaging device and is narrower than the first area And B. displaying on the display unit.

Further, according to the present invention, a display screen of a first display mode for displaying a first captured image relating to a first region of a real space captured by one imaging device among a plurality of imaging devices according to the present invention is displayed on a display unit The program for functioning as a real space information providing unit to be displayed on the screen, wherein the real space information providing unit designates a position in the first captured image by the user on the display screen of the first display mode. In accordance with the change in the direction of the one imaging device among the plurality of imaging devices, the second imaging device being associated with the second region of the real space, which is narrower than the first region , obtained by the one imaging device (2) A program is provided which switches the display screen of the first display mode to the display screen of the second display mode for displaying a captured image and causes the display unit to display the display screen.

Further, according to the present invention, the display unit is caused to display the display screen in the first display mode for displaying the first captured image related to the first region of the real space captured by one of the plurality of imaging devices. A real space information provision unit, the real space information provision unit being configured to select one of the plurality of imaging devices according to the user's designation of the position in the first captured image by the user on the display screen of the first display mode; In a second display mode for displaying a second captured image relating to a second area of the real space which is narrower than the first area and obtained by the one imaging apparatus along with the change of the direction of the one imaging apparatus An information processing system is provided in which a display screen of the first display mode is switched to a display screen and displayed on the display unit.
Further, according to the present invention, the first image generated based on the first captured image relating to the first region of the real space captured by one of the plurality of imaging devices is provided to the information processing device Control unit, the control unit being configured to obtain the actual area narrower than the first region obtained by the one imaging device along with the change in the orientation of the one imaging device among the plurality of imaging devices. The second image generated based on the second captured image related to the second region of the space is displayed according to the user's designation of the position in the first captured image by the user on the display screen displayed on the information processing device. A server is provided which is provided to the information processing apparatus.
Further, according to the present invention, there is provided an information processing apparatus and a server, wherein the server is a first captured image relating to a first region of real space captured by one of a plurality of imaging apparatuses. A controller configured to provide a first image generated based on the information processing apparatus to the information processing apparatus, wherein the control section is configured to change the direction of the one imaging apparatus among the plurality of imaging apparatuses . In a display screen displayed on the information processing device, a second image generated based on a second captured image related to a second region of the real space which is obtained by the imaging device and is narrower than the first region. An information processing system is provided, which is provided to the information processing apparatus in accordance with designation of a position in the first captured image by a user.
Further, according to the present invention, there is provided a program and a server, wherein the server is based on a first captured image relating to a first region of real space captured by one of a plurality of imaging devices. A control unit that provides the generated first image to an information processing apparatus that executes the program, and the control unit is configured to change the direction of the one imaging device among the plurality of imaging devices. An information processing apparatus that executes the program, on a second image generated based on a second captured image related to a second area of the real space, which is obtained by one imaging apparatus and is narrower than the first area. An information processing system is provided, which is provided to an information processing apparatus that executes the program in accordance with designation of a position in the first captured image by a user on a displayed display screen.

  As described above, according to the present invention, it is possible to switch and display a captured image in which a wider area is captured and a captured image in which a narrower area is captured.

It is an explanatory view showing an example of rough composition of an information processing system concerning one embodiment. It is a block diagram showing an example of the hardware constitutions of the terminal unit concerning one embodiment. It is a block diagram showing an example of functional composition of a terminal unit concerning one embodiment. It is an explanatory view for explaining an example of a display screen displayed in an eyebrow mode. It is an explanatory view for explaining an example of a display screen displayed in proximity mode. It is an explanatory view for explaining the 1st example of a position in a proximity image pick-up picture specified by a user. It is explanatory drawing for demonstrating the 2nd example of the position in the proximity | contact captured image designated by the user. It is explanatory drawing for demonstrating the 1st example of the three-dimensional virtual space corresponding to a center office. FIG. 9 is an explanatory diagram for describing an example of selection of an object arranged in the three-dimensional virtual space shown in FIG. 8; It is explanatory drawing for demonstrating the 2nd example of the three-dimensional virtual space corresponding to a center office. FIG. 11 is an explanatory diagram for describing an example of selection of an object arranged in the three-dimensional virtual space shown in FIG. 10; It is an explanatory view for explaining an example of a display screen displayed in conversation mode. It is a transition diagram for demonstrating an example of transition of a display mode. It is a block diagram which shows an example of the software configuration of the terminal device 100 which concerns on one Embodiment. It is a block diagram showing an example of the hardware constitutions of the information management server concerning one embodiment. It is a block diagram showing an example of functional composition of an information management server concerning one embodiment. It is a flow chart which shows an example of a rough flow of information processing concerning one embodiment. It is a flow chart which shows an example of a rough flow of starting processing concerning one embodiment.

  The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. In the present specification and the drawings, components having substantially the same functional configuration will be assigned the same reference numerals and redundant description will be omitted.

  From then on, <1. Schematic Configuration of Information Processing System>, <2. Configuration of Terminal Device>, <3. Configuration of Information Management Server>, <4. Processing Flow> The embodiment of the present invention will be described in the order of

<< 1. Schematic configuration of information processing system >>
First, a schematic configuration of an information processing system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is an explanatory view showing an example of a schematic configuration of an information processing system according to the present embodiment. Referring to FIG. 1, the information processing system exists, for example, over a plurality of bases. In this example, the information processing system exists across the center office 10 and the satellite office 20 (or the home office 20). The center office 10 is a relatively large office, and the satellite office 20 (or home office and 20) is a relatively small office.

  The information processing system includes a camera 11, a microphone 13, a sensor 15, a media distribution server 17, an information management server 200, and a LAN (Local Area Network) 19 in the center office 10. The information processing system also includes a terminal device 100, a display 21, and a LAN 23 in the satellite office 20 (or home office and 20). The information processing system further includes a PBX (Private Branch eXchange) 40.

(Camera 11)
The camera 11 captures an area in the direction in which the camera 11 is directed (that is, the imaging direction). In the center office 10, a plurality of cameras 11 are installed. And each installed camera 11 images a part or whole of a center office from each installation position. Thus, in the information processing system, the center office is imaged from various positions. The captured image generated through the camera 11 may be a still image or a moving image (i.e., an image).

  Also, the camera 11 can automatically change its direction, for example. Furthermore, the camera 11 has, for example, a zoom function. The zoom function may be an optical zoom function or a digital zoom function.

  Also, the camera 11 may be capable of changing its position. As an example, the camera 11 may be movable by dolly. That is, the camera 11 may be movable along the rails. In this case, the camera 11 may move back and forth by controlling the motor to move along the rails. Thus, even with one camera 11, it is possible to generate a captured image in which a subject is captured from different positions.

  In addition, when the camera 11 can change the position, the zoom function may be a zoom function by changing the position of the camera 11. As one example, the zoom function may be a dolly zoom function. For example, zooming in may be performed by moving the camera 11 toward the subject, and zooming out may be performed by moving the camera 11 in a direction away from the subject. Note that the dolly zoom does not have to be a fine zoom like optical zoom or digital zoom. For example, in zoom-in, a captured image in which a subject is captured in a larger size may be generated, and in zoom-out, a captured image in which a subject is captured in a smaller size may be generated.

(Microphone 13)
The microphone 13 collects sound around the microphone 13. In the center office 10, a plurality of microphones 13 are installed. And each installed microphone 13 collects the sound around each installation position in a center office. As described above, in the information processing system, sounds at various positions in the center office 10 are collected.

(Sensor 15)
Sensor 15 may include various types of sensors. For example, the sensor 15 includes a seat sensor that determines whether a person is in a seat. The seat sensor is installed in each seat and determines whether a person is sitting in each seat. The seat sensor is, for example, any sensor that can detect a press.

(Media distribution server 17)
The media distribution server 17 distributes media (for example, audio, video, etc.) to the terminal device in response to the request.

(Information management server 200)
The information management server 200 manages various information used in the information processing system. That is, the information management server 200 stores the various information and updates the various information in a timely manner.

  For example, the information management server 200 manages parameters related to the camera 11, the microphone 13, and the sensor 15. Specifically, for example, the information management server 200 stores and updates information such as the installation position of the camera 11, the imaging direction (for example, the direction perpendicular to the lens of the camera), and the zoom ratio as parameters of the camera 11. .

  Also, for example, the information management server 200 manages data of a three-dimensional virtual space corresponding to a real space. The three-dimensional virtual space is, for example, a three-dimensional virtual space imitating the center office 10. In addition, an object is arranged in the three-dimensional virtual space. For example, the object corresponds to a person. Then, the object is arranged at a three-dimensional virtual position in the three-dimensional virtual space corresponding to the position of each seat of the center office 10. That is, when the person is sitting in the seat, the object is placed at a three-dimensional virtual position corresponding to the position where the person would be. As an example, the object is a cylindrical object. The three-dimensional virtual space will be described later.

(LAN 19)
The LAN 19 is a network that connects the devices in the center office 10. The LAN 19 also connects devices in the center office 10 to devices outside the center office 10 via the external network 30. The external network 30 includes, for example, the Internet.

(Terminal device 100)
The terminal device 100 is used by a user. For example, the terminal device 100 provides the user with a function for performing communication such as telephone calls and mails. The terminal device 100 is a tablet terminal as an example. The terminal device 100 may be another device having a display function and a communication function, such as a smartphone, a PC (Personal Computer), and a telephone with a display, instead of the tablet terminal.

(Display 21)
The display 21 displays one of the screens. For example, the display 21 displays a screen including a captured image generated through the camera 11. Thereby, many persons including the user of the terminal device 100 can view the state of the center office 10 through the display 21.

  The display 21 also outputs, for example, any sound. For example, the display 21 outputs the sound collected by the microphone 13. Thereby, many persons including the user of the terminal device 100 can hear the sound of the center office 10 through the display 21.

(LAN23)
The LAN 23 is a network connecting devices in the satellite office 20 (or home office 20). Further, the LAN 23 connects each device in the satellite office 20 to a device outside the satellite office 20 via the external network 30.

(PBX 40)
The PBX 40 enables communication between devices via the external network 30. For example, in the H. It operates according to H.323 or SIP (Session Initiation Protocol).

  Specifically, for example, the PBX 40 stores communication identification information (for example, a telephone number) and an IP (Internet Protocol) address in association with each other. Then, in response to the request, the PBX 40 converts communication identification information into an IP address, and provides the IP address to the request source.

  The PBX 40 may be connected to the LAN 19 or the LAN 23.

<< 2. Terminal Configuration >>
Then, with reference to FIGS. 2-14, an example of a structure of the terminal device 100 which concerns on this embodiment is demonstrated.

<2-1. Hardware configuration >>
First, an example of the hardware configuration of the terminal device 100 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the hardware configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 2, the terminal device 100 includes a central processing unit (CPU) 801, a read only memory (ROM) 803, a random access memory (RAM) 805, a bus 807, a storage device 809, a communication interface 811, a camera 813 and a microphone. 815, a speaker 817 and a touch panel 820 are provided.

  The CPU 801 executes various processes in the terminal device 100. The ROM 803 also stores programs and data for causing the CPU 801 to execute processing in the terminal device 100. The RAM 805 temporarily stores programs and data when the CPU 801 executes processing.

  The bus 807 connects the CPU 801, the ROM 803, and the RAM to one another. Further, a storage device 809, a communication interface 811, a camera 813, a microphone 815, a speaker 817, and a touch panel 820 are connected to the bus 807. The bus 807 includes, for example, a plurality of types of buses. As one example, the bus 807 includes a high speed bus connecting the CPU 801, the ROM 803 and the RAM 805, and one or more other buses slower than the high speed bus.

  The storage device 809 stores data to be temporarily or permanently stored in the terminal device 100. The storage device 809 may be, for example, a magnetic storage device such as a hard disk, or an EEPROM (Electrically Erasable and Programmable Read Only Memory), a flash memory, a MRAM (Magnetoresistive Random Access Memory). The memory may be a non-volatile memory such as a ferroelectric random access memory (FeRAM) and a phase change random access memory (PRAM).

  The communication interface 811 is communication means included in the terminal device 100, and communicates with an external device via a network (or directly). The communication interface 811 may be an interface for wireless communication, and in this case, may include, for example, a communication antenna, an RF circuit, and other circuits for communication processing. The communication interface 811 may be an interface for wired communication, and in this case, may include, for example, a LAN terminal, a transmission circuit, and other circuits for communication processing.

  The camera 813 captures an image of a subject. The camera 813 includes, for example, an optical system, an imaging device, and an image processing circuit.

  The microphone 815 collects surrounding sounds. The microphone 815 converts ambient sound into an electrical signal and converts the electrical signal into digital data.

  The speaker 817 outputs sound. The speaker 817 converts digital data into an electrical signal and converts the electrical signal into sound.

  The touch panel 820 includes a touch detection surface 821 and a display surface 823.

  The touch detection surface 821 detects a touch position on the touch panel 820. More specifically, for example, when the user touches the touch panel 820, the touch detection surface 821 senses the touch, generates an electric signal according to the position of the touch, and detects the electric signal at the touch position. Convert to information. The touch detection surface 821 may be formed, for example, according to an arbitrary touch detection method such as an electrostatic capacitance method, a resistive film method, or an optical method.

  The display surface 823 displays an output image (that is, a display screen) from the terminal device 100. The display surface 823 can be realized using, for example, a liquid crystal, an organic light-emitting diode (OLED), a cathode ray tube (CRT), or the like.

2-2. Functional configuration >>
Next, an example of a functional configuration of the terminal device 100 according to the present embodiment will be described. FIG. 3 is a block diagram showing an example of a functional configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 3, the terminal device 100 includes a communication unit 110, an input unit 120, an imaging unit 130, a sound collection unit 140, a display unit 150, an audio output unit 160, a storage unit 170, and a control unit 180.

(Communication unit 110)
The communication unit 110 communicates with other devices. For example, the communication unit 110 is directly connected to the LAN 23 and communicates with each device in the satellite office 20. The communication unit 110 also communicates with each device in the center office 10 via the external network 30 and the LAN 19. Specifically, for example, the communication unit 110 communicates with the camera 11, the microphone 13, the sensor 15, the media distribution server 17, and the information management server 200. The communication unit 110 may be implemented by, for example, the communication interface 811.

(Input unit 120)
The input unit 120 receives an input by the user of the terminal device 100. Then, the input unit 120 provides the input result to the control unit 180.

  For example, the input unit 120 detects the position designated by the user on the display screen. More specifically, for example, the input unit 120 is mounted by the touch detection surface 821 and detects a touch position on the touch panel 820. Then, the input unit 120 provides the detected touch position to the control unit 820.

(Imaging unit 130)
The imaging unit 130 captures an image of a subject. For example, the imaging unit 130 captures an area in the front direction of the terminal device 100. In this case, the imaging unit 130 captures an image of the user of the terminal device 100. The imaging unit 130 provides the control unit 180 with an imaging result (that is, a captured image). The imaging unit 130 may be mounted by, for example, the camera 813.

(Sound collecting unit 140)
The sound collecting unit 140 collects sounds around the terminal device 100. For example, the sound collection unit 140 collects the voice of the user of the terminal device 100. The sound collection unit 140 provides the control unit 180 with the sound collection result (i.e., audio data). The sound collection unit 140 may be implemented by the microphone 815, for example.

(Display unit 150)
The display unit 150 displays an output image (that is, a display screen) from the terminal device 100. The display unit 150 displays a display screen according to control by the control unit 180. Note that the display unit 150 can be mounted by, for example, the display surface 823.

(Voice output unit 160)
The voice output unit 160 outputs voice from the terminal device 100. The voice output unit 160 outputs voice in accordance with control by the control unit 180. The audio output unit 160 may be implemented by, for example, the speaker 817.

(Storage unit 170)
The storage unit 170 stores programs and data for the operation of the terminal device 100. The storage unit 170 may be implemented by, for example, the storage device 809.

  For example, the storage unit 170 stores data of a three-dimensional virtual space corresponding to a real space. Specifically, for example, since the information management server 200 stores data of the three-dimensional virtual space corresponding to the center office 10, the control unit 180 can communicate with the three-dimensional virtual space via the communication unit 110. Get data Then, the storage unit 170 stores the acquired data of the three-dimensional virtual space.

(Control unit 180)
The controller 180 provides various functions of the terminal device 100. The control unit 180 includes a real space information providing unit 181, a position acquisition unit 183, an object selection unit 185, an ID acquisition unit 187, and a telephone unit 189. The control unit 180 can be implemented by, for example, the CPU 801, the ROM 803, and the RAM 805.

(Real space information provider 181)
The real space information provision unit 181 provides the user of the terminal device 100 with information of the real space.

  For example, the real space information provision unit 181 causes the display unit 150 to display a display screen of a captured image of the real space. More specifically, for example, the captured image is a captured image generated through the camera 11 in the real space (center office 10). The captured image may be a captured image generated by the camera 11 or may be a captured image generated by processing the captured image generated by the camera 11. Further, the display screen is a screen including the captured image in part or in whole.

  The real space information provision unit 181 acquires a captured image generated by the camera 11 via the communication unit 110. Then, the real space information provision unit 181 generates a display screen including the captured image, and causes the display unit 150 to display the display screen.

  Also, for example, the captured image is a captured image generated through a selected one of the plurality of imaging devices in the real space. More specifically, for example, the captured image is a captured image generated through one camera 11 selected from the plurality of cameras 11 disposed in the center office 10. A specific method of how the user selects the camera 11 will be described later. Since the camera 11 can be selected, the user can view a captured image from a desired position. And when specifying the position of a captured image so that it may mention later, a user can specify a position with a more desirable captured image.

  Also, for example, the display screen includes a captured image according to the display mode. More specifically, for example, in the first display mode, the display screen includes a first captured image obtained by capturing a first area of the real space, and in the second display mode, the first area. A narrower second region includes the second captured image captured. That is, the real space information provision unit 181 causes the display unit 150 to display the first captured image in the first display mode, and displays the second captured image on the display unit 150 in the second display mode. Let

  More specifically, for example, the first captured image is a captured image corresponding to a first zoom ratio. The second captured image is a captured image corresponding to a second zoom ratio that is larger than the first zoom ratio. For example, the real space information providing unit 181 requests the camera 11 to perform zoom (optical zoom, digital zoom, or zoom by changing the position of the imaging device (for example, zoom by dolly)) via the communication unit 110. Thus, the captured image corresponding to the first zoom factor or the captured image corresponding to the second zoom factor is acquired. Alternatively, the real space information provision unit 181 may generate a captured image corresponding to the first zoom ratio or a captured image corresponding to the second zoom ratio by digital zoom on the captured image generated by the camera 11 . Note that the zoom ratio here does not have to be a precise value such as 1.5 times or 2 times, and may be a value that directly or indirectly indicates the extent to which the subject appears in the captured image. . For example, particularly when zoom by changing the position of the camera 11 (for example, zoom in and zoom out by dolly) is used, the zoom factor is not a minute value such as 1.5 times or 2 times, but the size of the object. Something that directly indicates the degree of size (eg, a parameter that indicates the degree of approximate size of the subject, etc.), or that indirectly indicates the degree of size of the subject (eg, the position of the camera 11 on the rail , Etc.). The captured image corresponding to the first zoom ratio is a captured image in which the subject is captured smaller, and the captured image corresponding to a second zoom ratio larger than the first zoom ratio is captured in a larger object. The captured image is sufficient.

  As an example, the display screen includes a bird's-eye view image generated by the camera 11 in imaging at a zoom ratio of X (for example, X = 1) in the bird's-eye mode, and in the proximity mode It includes a close-up captured image generated by the camera 11 in imaging at Y> X). That is, the overhead image is a captured image in which a wider area of the center office 10 is captured, and the close-up captured image is a captured image in which a narrower area of the center office 10 is captured. Hereinafter, this example will be described with reference to FIGS. 4 and 5.

-Display Screen Displayed in Eyebrow Mode FIG. 4 is an explanatory diagram for describing an example of a display screen displayed in the eyelid mode. Referring to FIG. 4, a display screen 60 displayed in the overhead mode is shown. The display screen 60 includes an eyelid captured image 61, a button image 63, a presence icon 65, a balloon image 67, and a map image 69.

  The bird's-eye view pickup image 61 is, for example, a pickup image generated by the camera 11 by imaging at a zoom ratio of X times. As an example, X = 1. That is, the overhead view captured image 61 is a captured image generated by the camera 11 by imaging without zooming.

  Also, for example, when the user specifies the position of the overhead view captured image 61, the real space information provision unit 181 switches the display mode from the overhead mode to the proximity mode. More specifically, for example, when the user touches the position of the eyelid captured image 61 and the touch position corresponding to the eyelid captured image 61 is detected, the real space information provision unit 181 brings the display mode closer to the eyelid mode. Switch to mode.

  The button image 63 is an image for selecting another camera 11. For example, when the user specifies the position of the button image 63, the real space information provision unit 181 acquires a bird's-eye view captured image generated by another camera 11, and causes the display unit 150 to display the bird's-eye view captured image. More specifically, for example, when the user touches the position of the button image 63 and the touch position corresponding to the button image 63 is detected, the real space information providing unit 181 generates an image generated by another camera 11. A captured image is acquired, and the overhead view captured image is displayed on the display unit 150. For example, when the position of the button image 63A is designated by the user, the camera 11 located on the left side of the current camera 11 is selected. In addition, when the position of the button image 63B is designated by the user, the camera 11 located on the right side of the current camera 11 is selected. Then, the real space information provision unit 181 acquires the overhead view captured image generated by the selected camera 11, and causes the display unit 150 to display the overhead view captured image.

  Further, the presence icon 65 is an icon indicating the degree of busyness of the person shown in the captured image 61. For example, the presence icon 65 changes color according to the busyness of the person. As an example, the presence icon indicates that the degree of busyness is high when it is red, it indicates that the degree of busyness is normal when it is yellow, and that the degree of busyness is low when it is blue. As described later, since it can be known where in the captured image 61 a person should be shown, it is possible to display such an icon.

  For example, as described above, the display screen 60 includes, for example, information related to a person shown in the overhead view captured image 61 (hereinafter, referred to as “person related information”). The person related information includes, for example, state information indicating the state of the person. As described above, the state information is the presence icon 65 as an example. The person related information may include a state history indicating the state of the person at two or more points in time. As an example, the state history information may include a history of busyness of a person shown in the captured image 61. That is, on the display screen 60, a history of busyness of a person may be displayed. The real space information provision unit 181 acquires, for example, the person-related information or information necessary for displaying the person-related information from the information management server 200 via the communication unit 110.

  Such person-related information allows the user to more accurately grasp the situation in which the person is placed. In addition, the state information allows the user to more accurately determine whether to contact a person. In addition, since the state history allows the user to grasp not only the instantaneous state of the user but also the state of the user in a certain period, the user can more accurately determine whether or not a person may be contacted. be able to.

  In addition, the balloon image 67 is an image including character information presented by a person shown in the captured image 61. The balloon image 67 is also an example of person-related information.

  Further, the map image 69 is an image showing a map of the center office 10. The map image 69 further indicates the camera 11 in use by an icon. Note that the map image 69 may be omitted, especially if only one or a few cameras 11 are installed in the center office 10.

Display Screen Displayed in Proximity Mode FIG. 5 is an explanatory diagram for describing an example of a display screen displayed in the proximity mode. Referring to FIG. 5, a display screen 70 displayed in the proximity mode is shown. The display screen 70 includes a close-up captured image 71, a button image 73 and a map image 75.

  The close-up captured image 71 is, for example, a captured image generated by the camera 11 by imaging at a zoom ratio of Y (Y> X). As an example, Y = 1.5. That is, the overhead view captured image 61 is a captured image generated by the camera 11 by capturing at a 1.5 × zoom.

  The button image 73 is an image for switching the display mode from the proximity mode to the overhead mode. For example, when the user specifies the position of the button image 73, the real space information provision unit 181 switches the display mode from the proximity mode to the overhead mode. More specifically, for example, when the user touches the position of button image 73 and the touch position corresponding to button image 73 is detected, real space information provision unit 181 changes the display mode from the proximity mode to the overhead mode. Switch.

  Moreover, the map image 75 is an image which shows the map of the center office 10 similarly to the map image 69 in the eyebrow mode. The map image 75 further shows the camera 11 in use. For example, in the proximity mode, the icon of the camera in use in the map image 75 is displayed at a more advanced position to symbolically indicate that the zoom is performed. It should be noted that the map image 75 may be omitted as well as the map image 69 in the overhead mode, especially if only one or a few cameras 11 are installed in the center office 10.

  The display screen 70 displayed in the proximity mode may also include person-related information such as a presence icon and a balloon image.

  As described above, according to the display mode, the user can view the general situation of the real space by displaying a captured image in which a wider area is captured, and it is easy to find a specific person. Can. Then, by displaying a captured image in which a narrower area is captured, the user can more easily specify the position of a specific person. Also, since only the display mode is switched, only easy operation is required for the user. In addition, since the captured image in which the wider area is captured and the captured image in which the narrower area is captured have different zoom ratios, the user intuitively and easily grasps the relationship between these captured images. be able to. Therefore, even if the display mode is switched, the user can easily find a specific person and designate the position of the specific person.

-Other real space information As mentioned above, the real space information provision part 181 provides the visual information of real space by making the display part 150 display the display screen of the captured image of real space. Furthermore, the real space information provision unit 181 may also provide auditory information of real space. That is, the real space information provision unit 181 may cause the sound output unit 160 to output the sound of the sound data obtained by sound collection in the real space.

  For example, the real space information provision unit 181 selects the microphone 13 close to the selected camera 11. Then, the real space information provision unit 181 acquires voice data obtained by sound collection at the center office 10 from the microphone 13 via the communication unit 110. Then, the real space information provision unit 181 may cause the sound output unit 160 to output the sound of the acquired sound data.

(Position acquisition unit 183)
The position acquisition unit 183 acquires the position in the captured image specified by the user on the display screen of the captured image in the real space. More specifically, for example, when the input unit 120 detects a position in the captured image specified by the user on the display screen of the captured image in real space, the position acquisition unit 183 acquires the position. Then, the position acquisition unit 183 provides the object selection unit 185 with the position in the captured image.

  As an example, when acquiring any position in the close-up captured image 71, the position acquisition unit 183 provides the position to the object selection unit 185. Hereinafter, this example will be described with reference to FIGS. 6 and 7.

  FIG. 6 is an explanatory diagram for describing a first example of the position in the close-up captured image specified by the user. Referring to FIG. 6, the proximity captured image 71 included in the display screen displayed in the proximity mode and the hand 3 of the user are shown. Further, a person 77 appearing in the close-up captured image 71 is shown. Then, the user touches the position of the person 77 with the hand 3 to specify the position of the person 77 in the close-up captured image 71. In this case, the input unit 120 detects the above-mentioned position of the person 77 in the close-up captured image 71, and the position acquisition unit 183 acquires the position. Then, the input unit 120 provides the acquired position to the object selection unit 185.

  FIG. 7 is an explanatory diagram for describing a second example of the position in the close-up captured image specified by the user. Referring to FIG. 7, persons 77A and 77B appearing in the close-up captured image 71 are shown. Then, the user touches the position of the person 77A with the hand 3 to specify the position of the person 77A in the close-up captured image 71. In this case, the input unit 120 detects the position of the person 77A in the close-up captured image 71, and the position acquisition unit 183 acquires the position. Then, the input unit 120 provides the acquired position to the object selection unit 185.

(Object selection unit 185)
The object selection unit 185 selects an object arranged in the three-dimensional virtual space corresponding to the real space based on the position in the acquired captured image. For example, when the position acquisition unit 183 acquires a position in the captured image specified by the user on the display screen of the captured image in the real space, the object selection unit 185 corresponds to the real space based on the position. Select an object placed in the 3D virtual space.

  The object is an object disposed at a three-dimensional virtual position in the three-dimensional virtual space corresponding to the position in the captured image. Also, for example, the captured image is a captured image included in the display screen in the second mode (for example, the proximity mode).

  As an example, when the position acquisition unit 183 acquires the position in the close-up captured image generated by the camera 11, the object selection unit 185 selects one of the objects arranged in the three-dimensional virtual space corresponding to the center office 10. Select an object corresponding to the above position. The object selection unit 185 acquires, for example, data of a three-dimensional virtual space corresponding to the center office 10 from the storage unit 170.

-One Example when One Object is Arranged Hereinafter, with reference to FIG. 8 and FIG. 9, a concrete example when one object is arranged in the three-dimensional virtual space will be described.

  FIG. 8 is an explanatory diagram for explaining a first example of a three-dimensional virtual space corresponding to the center office 10. As shown in FIG. Referring to FIG. 8, a three-dimensional virtual space 90 corresponding to the center office 10 is shown. Further, in the three-dimensional virtual space 90, an object 91 is disposed. The object 91 corresponds to a person (for example, Mr. A). Then, the object is arranged at a three-dimensional virtual position corresponding to the position of the seat of the person (for example, Mr. A) of the center office 10. That is, the object 91 is placed at a three-dimensional virtual position corresponding to the position where the person would be present when the person is sitting in the seat. In this example, the object 91 is a cylindrical object. The cylindrical object is a cylindrical object with radius R and height H. The radius R and the height H are, for example, predetermined. The data of the three-dimensional virtual space 90 also includes information related to each camera 11. For example, the information related to each camera 11 includes a three-dimensional virtual position corresponding to the installation position of each camera 11, an imaging direction (for example, a direction perpendicular to the lens of the camera), an angle of view, and the like.

  FIG. 9 is an explanatory diagram for explaining an example of selection of an object arranged in the three-dimensional virtual space shown in FIG. Referring to FIG. 9, the positional relationship in the horizontal plane in the three-dimensional virtual space 90 is shown for easy understanding. Specifically, an object 91 disposed in the three-dimensional virtual space 90, a three-dimensional virtual position (hereinafter referred to as a "virtual camera position") O corresponding to the installation position of the camera 11 used for imaging, and imaging of the camera 11 An axis y corresponding to a direction (e.g., a direction perpendicular to the lens of the camera) and an axis x orthogonal to the axis y are shown. In this example, for easy understanding, it is assumed that the camera 11 is installed such that the imaging direction of the camera 11 is parallel to the horizontal plane.

  Furthermore, the angle of view θ of the camera 11 is also shown. Further, a virtual surface 93 is shown which is perpendicular to the axis y corresponding to the imaging direction of the camera 11 and has a width corresponding to the angle of view θ. Also, the virtual plane 93 is separated from the virtual camera position O by the distance I. The virtual surface 93 is a square surface and has the same aspect ratio as the captured image. That is, the virtual surface 93 is a surface corresponding to a captured image.

  The object selection unit 185 converts, for example, the position in the captured image specified by the user as shown in FIG. 6 into a three-dimensional virtual position A. Then, the object selection unit 185 specifies an object that intersects a straight line passing the virtual camera position O and the three-dimensional virtual position A. In this example, the object selection unit 185 specifies the object 91. Then, the object selection unit 185 selects the object 91.

  In the example of FIG. 9, for example, the position in the captured image converted into the three-dimensional virtual position between the three-dimensional virtual position B and the three-dimensional virtual position D in the virtual surface 93 is the user in the captured image. When designated by the, the object 91 is selected. Note that such a position is generally a position where a person corresponding to the object 91 is shown in the captured image.

  The distance I is determined so that the virtual plane 93 is located between the virtual camera position O and the object 91. As an example, the distance I is the focal length of the camera 11, but of course is not limited thereto.

  In addition, although the method of selecting an object by focusing on the horizontal surface of the three-dimensional virtual space 90 has been described for easy explanation, naturally, even if a vertical direction (for example, z axis) is added, Can be converted into a three-dimensional virtual position to specify the object 91 from the three-dimensional virtual position.

  For example, as described above, an object corresponding to a position in a captured image is selected. In the example described above, the object corresponding to the position in the captured image is specified by converting the position in the captured image into a three-dimensional virtual position, but a method for specifying the object corresponding to the position in the captured image Not limited to this.

  As an example, the object selecting unit 185 may project the object 91 on the virtual plane 93 with the virtual camera position O as an origin, and convert the projection range of the object 91 into a range in the captured image. Then, the object selection unit 185 may select the object 91 when the position in the captured image specified by the user is included in the range.

  As another example, a rendering image is generated by rendering the three-dimensional virtual space 90 using the virtual camera position O, the axis y, and the angle of view θ, and the object 91 in the rendering image is captured. The range in the captured image corresponding to the object 91 may be obtained from the range. Then, the object selection unit 185 may select the object 91 when the position in the captured image specified by the user is included in the range.

-Selection of Object Considering the State Further, for example, the object 91 arranged in the three-dimensional virtual space 90 corresponds to state information indicating any one of a plurality of states. For example, the object 91 corresponds to a person. Then, the object 91 corresponds to state information (hereinafter referred to as “presence / absence information”) indicating whether or not the person is sitting in the seat. The presence / absence information indicates that a person is sitting in a seat or a person is not sitting in a seat. The object selection unit 185 acquires, for example, the determination result as to whether or not the person corresponding to the object 91 is sitting in the seat from the sensor 15 via the communication unit 110. And the said determination result becomes presence or absence information.

  Then, for example, the object selecting unit 185 selects the object corresponding to the state information indicating the predetermined state among the plurality of states, which is the object arranged in the three-dimensional virtual space. For example, the object selecting unit 185 selects an object arranged in the three-dimensional space, which corresponds to state information indicating a state in which a person is sitting in a seat. That is, when it is determined by the sensor 15 that a person is sitting in the seat, the object selection unit 185 can select an object corresponding to the person, but the sensor 15 determines that the person is not sitting in the seat When it is determined, the object corresponding to the person is not selected.

  In this way, since the object is selected according to the state of the person, it is possible to avoid selecting an object that should not be selected. For example, it is possible to prevent the object corresponding to the person from being selected when there is no person.

-Example in which two objects are arranged In addition, two or more objects may be arranged in the three-dimensional virtual space. Hereinafter, a specific example in which two objects are arranged will be described with reference to FIG.

  FIG. 10 is an explanatory diagram for explaining a second example of the three-dimensional virtual space corresponding to the center office 10. As shown in FIG. Referring to FIG. 10, a three-dimensional virtual space 90 corresponding to the center office 10 is shown. Further, in the three-dimensional virtual space 90, an object 91A and an object 91B are arranged. The object 91A corresponds to a certain person (for example, Mr. A), and is arranged at a three-dimensional virtual position corresponding to the position of the seat of the certain person in the center office 10. Further, the object 91 B is arranged at a three-dimensional virtual position corresponding to a certain person (for example, Mr. B) and corresponding to the position of the seat of the certain person in the center office 10. Similar to the example of FIG. 8, the object 91 is a cylindrical object with a radius R and a height H.

  When two or more objects are arranged in the three-dimensional virtual space in this manner, the plurality of objects may correspond to positions in the captured image specified by the user. For example, in the example of FIG. 10, both the object 91A and the object 91B may correspond to the position in the captured image specified by the user. As an example, as shown in FIG. 7, when two persons can appear in a captured image, when the user designates a position where the two persons overlap in the captured image, the two persons are designated. Corresponding two objects may correspond to the above position.

  Therefore, in such a case (that is, when the plurality of objects correspond to the position of the captured image specified by the user), the object selection unit 185 selects one of the plurality of objects. select. That is, when there are a plurality of objects respectively arranged in the three-dimensional virtual position in the three-dimensional virtual space corresponding to the position in the captured image specified by the user, the object selecting unit 185 Select one of the objects.

  For example, the captured image is generated through an imaging device in real space. Then, the object selection unit 185 selects an object closer to the three-dimensional virtual position in the three-dimensional virtual space corresponding to the imaging device among the plurality of objects. More specifically, for example, the captured image is generated through the camera 11 in the center office 10. Then, when there are a plurality of objects corresponding to the position in the captured image specified by the user, the object selecting unit 185 selects the three-dimensional virtual position (that is, the virtual camera position O) corresponding to the installation position of the camera 11 Select a close object. Hereinafter, a specific example will be described with reference to FIGS. 10 and 11 regarding this point.

  FIG. 11 is an explanatory diagram for explaining an example of selection of an object arranged in the three-dimensional virtual space shown in FIG. Referring to FIG. 11, the positional relationship in the horizontal plane in the three-dimensional virtual space 90 is shown for easy understanding. Specifically, an object 91A and an object 91B arranged in the three-dimensional virtual space 90 are shown. Also, as in FIG. 9, a virtual camera position O, an axis y, an axis x, an angle of view θ, and a virtual surface 93 are shown. Also in this example, as in FIG. 9, for easy understanding, it is assumed that the camera 11 is installed such that the imaging direction of the camera 11 is parallel to the horizontal plane.

  For example, as shown in FIG. 7, the position in the captured image is designated by the user. In this case, according to the method described with reference to FIG. 9, the position in the captured image is converted into a three-dimensional virtual position between the three-dimensional virtual position B ′ and the three-dimensional virtual position D. In the above, both the object 91A and the object 91B are specified as objects corresponding to the positions in the captured image. Then, the object selection unit 185 selects an object 91A closer to the virtual camera position O among the object 91A and the object 91B.

  When the position in the captured image specified by the user is converted into a three-dimensional virtual position between the three-dimensional virtual position B and the three-dimensional virtual position B ′, the object 91A is the above-described captured image It is identified and selected as the object corresponding to the position in the When the position in the captured image specified by the user is converted to a three-dimensional virtual position between the three-dimensional virtual position D and the three-dimensional virtual position D ′, the object 91 B is the above-described captured image It is identified and selected as the object corresponding to the position in the

  By selecting one object from a plurality of objects as described above, it is possible to prevent a plurality of objects from being selected and an error (an error due to acquiring a plurality of communication IDs) occurring in the subsequent processing. be able to. Further, by selecting an object close to the virtual camera position corresponding to the camera 11 among a plurality of objects, for example, even when people overlap in the captured image, it corresponds to the person appearing in the foreground Objects to be selected. Thus, an object corresponding to the person intended by the user is selected.

  As described above, the object selection unit 185 selects an object. Then, the object selection unit 185 provides the ID acquisition unit 187 with identification information of the selected object (hereinafter referred to as “object ID”). The object ID may be identification information of a person corresponding to the selected object, or may be a mere number assigned to the selected object.

(ID acquisition unit 187)
The ID acquisition unit 187 acquires identification information corresponding to the selected object. For example, the identification information is identification information for communication (hereinafter referred to as “communication ID”) corresponding to the selected object. The communication ID is, for example, a telephone number.

  Specifically, for example, when the object selection unit 185 selects an object, the ID acquisition unit 187 acquires the object ID of the selected object. Then, the object selection unit 185 transmits the object ID to the information management server 200 via the communication unit 110, and acquires a communication ID corresponding to the object ID. The communication ID is a communication ID of a person corresponding to the selected object.

  For example, as described above, the ID acquisition unit 187 acquires a communication ID corresponding to the selected object. The communication ID is a communication ID of the communication device of the person corresponding to the object (that is, the person appearing at the position of the captured image specified by the user). Then, the ID acquisition unit 187 provides the acquired communication ID to the telephone unit 189.

  As described above, when the position in the captured image is designated by the user, the object corresponding to the position is selected, and the communication ID corresponding to the object is acquired. This enables the user to contact the target person with an intuitive operation. Further, regardless of how the person is shown in the captured image, the object corresponding to the person is selected and the communication ID is acquired, so that the person can be more reliably contacted.

(Telephone part 189)
The telephone unit 189 provides a function for making a call. For example, the telephone unit 189 provides a softphone function.

  For example, when the telephone unit 189 acquires the communication ID provided by the ID acquisition unit 187, the telephone unit 189 uses the communication ID to make a call. More specifically, for example, when the telephone unit 189 acquires a communication ID, the telephone unit 189 provides the communication ID to the PBX 40 via the communication unit 110, and acquires an IP address from the PBX 40. Then, the telephone unit 189 executes a series of sequences for establishing a session with the communication apparatus having the IP address (that is, the communication apparatus at the destination).

  In this manner, the telephone unit 189 makes a call for a call to a person appearing at a position in the captured image specified by the user on the display screen. That is, the telephone unit 189 makes a call to the communication device of the person.

  When voice data from the communication apparatus of the other party of the telephone call is received by the communication unit 110, the telephone unit 189 causes the voice output unit 160 to output the voice of the voice data. In addition, the telephone unit 189 causes the communication unit 110 to transmit the audio data provided by the sound collection unit 140 to the communication device at the other end of the telephone. Also, the telephone unit 189 causes, for example, the communication unit 110 to also transmit a captured image provided by the imaging unit 130 (for example, a captured image in which a user of the terminal device 100 is shown) to the communication device of the other party of the telephone.

  Further, the telephone unit 189 causes the display unit 150 to display a display screen at the time of a telephone call. For example, a display mode for displaying a display screen at the time of telephone call is called a conversation mode. In this case, when the telephone unit 189 acquires the communication ID provided by the ID acquisition unit 187, the telephone unit 189 switches the display mode from the proximity mode to the conversation mode. Hereinafter, a concrete example of the display screen in the conversation mode will be described with reference to FIG.

  FIG. 12 is an explanatory diagram for describing an example of a display screen displayed in the conversation mode. Referring to FIG. 12, a display screen 80 displayed in the conversation mode is shown. The display screen 60 includes the other-side captured image 81, the button image 83, and the own-side captured image 85.

  The other party side captured image 81 is, for example, a captured image acquired from the communication device of the other party of the telephone. For example, when the captured image from the communication device of the other party of the telephone call is received by the communication unit 110, the telephone unit 189 uses the captured image as the other-side captured image 81.

  Also, the button image 63 is an image for ending the telephone call. For example, when the user specifies the position of the button image 83, the telephone unit 189 terminates the call. More specifically, for example, when the user touches the position of the button image 83 and the touch position corresponding to the button image 83 is detected, the telephone unit 189 executes a sequence for call termination including disconnection of the session. Do. Also, for example, the telephone unit 189 switches the display mode from the conversation mode to the eyebrow mode.

  The self-side captured image 85 is a captured image provided by the imaging unit 130.

(Other: Transition of display mode)
Here, a specific example of the transition of the habit mode, the proximity mode, and the conversation mode will be described with reference to FIG.

  FIG. 13 is a transition diagram for explaining an example of transition of the display mode. Referring to FIG. 13, for example, when connection processing with the camera 11, the microphone 13, the sensor 15, the information management server 200, and the like is performed at the time of activation of the software, the display mode is the bird's eye mode 301 (ENTRY).

  In the overhead mode 301, overhead mode processing is performed (DO). Then, when the user specifies the position of the overhead view captured image 61, mode change processing is performed (EXIT), and the display mode is switched from the overhead mode 301 to the proximity mode 303. The mode change process includes the zoom process of the camera 11 (ENTRY).

  In the proximity mode 303, proximity mode processing is performed (DO). Then, when the user designates the position where the person in the close-up shot image 71 is taken, mode change processing is performed (EXIT), and the display mode is switched from the close mode 303 to the conversation mode 305. The mode change process in this case includes a process for a call (ENTRY). Further, when the user designates the position of the button image 73, mode change processing is performed (EXIT), and the display mode is switched from the proximity mode 303 to the overhead mode 301. The mode change process in this case includes the connection process described above (ENTRY).

  In the conversation mode 305, conversation mode processing is performed (DO). Further, when the user designates the position of the button image 83, mode change processing is performed (EXIT), and the display mode is switched from the conversation mode 305 to the overhead mode 301. The mode change process in this case includes the connection process described above (ENTRY).

(Others: Another example of the display screen)
-A captured image according to the display mode As described above, for example, in the first display mode (for example, the eyelid mode), the real space information provision unit 181 is the first image in which the first region of the real space is imaged. A captured image (for example, a bird's-eye view captured image) is displayed on the display unit 150. Further, in the second display mode (for example, the proximity mode), the real space information provision unit 181 is a second captured image (for example, the proximity captured image) in which the second area narrower than the first area is captured. ) Is displayed on the display unit 150. Then, in the above-described example, the first captured image in which the first region of the real space is captured is a captured image corresponding to the first zoom ratio, and the second image is narrower than the first region. The second captured image obtained by capturing the area of is a captured image corresponding to a second zoom ratio that is larger than the first zoom ratio. However, the first captured image and the second captured image are not limited to this.

  For example, the first captured image is a captured image generated through the first imaging device of the plurality of imaging devices in real space, and the second captured image is the image of the plurality of imaging devices. It may be a captured image generated through the second imaging device. In this case, for example, the real space information provision unit 181 selects the camera 11 that captures a wide area of the center office 10 in the overhead mode, and uses the captured image generated by the camera 11 as the overhead captured image get. Then, in the proximity mode, the real space information provision unit 181 selects a camera 11 (for example, a camera further ahead) that images a narrower area of the center office 10, and the image generated by the camera 11 An image is acquired as a close-up image.

  Thereby, depending on the arrangement of the camera 11, it becomes easier to specify the position of the person. In addition, the camera 11 may not request the zoom. Therefore, as in the case of requesting an optical zoom or a zoom by dolly, for example, requests from individual terminal devices do not compete, and no waiting state occurs in any of the terminal devices. In addition, as in the case of using digital zoom, for example, the amount of processing does not increase.

-A captured image generated by imaging under more free conditions In the above-described example, although an example in which the display mode is switched has been described, the display screen is not limited to this. For example, instead of switching the display mode, a captured image may be generated by imaging under more free conditions, and a display screen including the captured image may be displayed.

  For example, the captured image of the real space may be a captured image corresponding to the selected zoom ratio of the plurality of zoom ratios. In this case, for example, the real space information provision unit 181 requests the camera 11 via the communication unit 120 for the zoom factor specified by the user via the input unit 120. As a result, the camera 11 changes the zoom factor according to the request, and provides the terminal device 100 with a captured image generated by imaging at the changed zoom factor. Then, the real space information providing unit 181 causes the display unit 150 to display a display screen including the provided captured image. Then, when the user designates a position in the captured image, the position acquisition unit 183 acquires the position. Then, the position acquisition unit 183 provides the object selection unit 185 with the position in the captured image.

  Thus, the user can specify a fine zoom factor to display a desired captured image. Thus, the user can more easily specify the position of a specific person.

  Note that, as described above, the zoom ratio here does not have to be a precise value such as 1.5 times or 2 times, and directly or indirectly indicates the degree of the size of the subject in the captured image. It is a thing. For example, particularly when zoom by changing the position of the camera 11 (for example, zoom in and zoom out by dolly) is used, the zoom factor is not a minute value such as 1.5 times or 2 times, but the size of the object. Something that directly indicates the degree of size (eg, a parameter that indicates the degree of approximate size of the subject, etc.), or that indirectly indicates the degree of size of the subject (eg, the position of the camera 11 on the rail , Etc.).

<2-3. Software configuration>
Next, an example of the software configuration of the terminal device 100 according to the present embodiment will be described. FIG. 14 is a block diagram showing an example of the software configuration of the terminal device 100 according to the present embodiment. Referring to FIG. 14, the terminal device 100 includes an operating system (OS) 840 and application software. Then, the terminal device 100 includes, as application software, a soft phone 851, an ultra-realism client 853, and a call origination control function 855.

  The OS 840 is software that provides basic functions for operating the terminal device 100. The OS 840 executes each application software.

  The softphone 851 is application software for making a call using the terminal device 100. The telephone unit 189 may be implemented by, for example, the softphone 851.

  The ultra-realism client 853 is application software for providing the terminal device 100 with information of real space. The real space information provision unit 181 may be implemented by, for example, the ultra-realism client 853.

  Note that the ultra-realism client 853 may acquire state information indicating the state of a person who is in the real space (for example, the center office 10), and may provide the softphone 851 via the OS. Then, the softphone 851 may control call origination based on the state information.

  In addition, the telephone transmission control function 855 is application software for acquiring a communication ID of a communication device of a person shown in a captured image in a display screen. The position acquisition unit 183, the object selection unit 185, and the ID acquisition unit 187 may be implemented by the telephone transmission control function 855.

  When the call origination control function 855 acquires the communication ID, it provides the softphone 851 via the OS. Then, the softphone 851 makes a call by using the communication ID.

<< 3. Information Management Server Configuration >>
Subsequently, an example of the configuration of the information management server 200 according to the present embodiment will be described with reference to FIGS. 15 and 16.

<3-1. Hardware configuration >>
First, an example of a hardware configuration of the information management server 200 according to the present embodiment will be described with reference to FIG. FIG. 15 is a block diagram showing an example of the hardware configuration of the information management server 200 according to the present embodiment. Referring to FIG. 15, the information management server 200 includes a CPU 901, a ROM 903, a RAM 905, a bus 907, a storage device 909, and a communication interface 911.

  The CPU 901 executes various processes in the information management server 200. The ROM 903 also stores programs and data for causing the CPU 901 to execute processing in the information management server 200. The RAM 905 temporarily stores programs and data when the CPU 901 executes processing.

  The bus 907 interconnects the CPU 901, the ROM 903 and the RAM. Further, a storage device 909 and a communication interface 911 are connected to the bus 907. The bus 907 includes, for example, a plurality of types of buses. As one example, the bus 907 includes a high-speed bus connecting the CPU 901, the ROM 903 and the RAM 905, and one or more other buses slower than the high-speed bus.

  The storage device 909 stores data to be temporarily or permanently stored in the information management server 200. The storage device 909 may be, for example, a magnetic storage device such as a hard disk, or a non-volatile memory such as an EEPROM, a flash memory, an MRAM, an FeRAM, and a PRAM.

  The communication interface 911 is a communication unit included in the information management server 200, and communicates with an external device via a network (or directly). The communication interface 911 may be an interface for wireless communication, and in this case, may include, for example, a communication antenna, an RF circuit, and other circuits for communication processing. The communication interface 911 may be an interface for wired communication, and in this case, may include, for example, a LAN terminal, a transmission circuit, and other circuits for communication processing.

<3-2. Functional configuration >>
Next, an example of a functional configuration of the information management server 200 according to the present embodiment will be described. FIG. 16 is a block diagram showing an example of a functional configuration of the information management server 200 according to the present embodiment. Referring to FIG. 16, the information management server 200 includes a communication unit 210, a storage unit 220, and a control unit 230.

(Communication unit 210)
The communication unit 210 communicates with other devices. For example, the communication unit 210 is directly connected to the LAN 19 and communicates with each device in the center office 10. Specifically, for example, the communication unit 210 communicates with the camera 11, the microphone 13, the sensor 15, and the media distribution server 17. The communication unit 210 also communicates with each device in the satellite office 20 via the external network 30 and the LAN 23. Specifically, for example, the communication unit 210 communicates with the terminal device 100 and the display 21. The communication unit 210 may be implemented by, for example, the communication interface 911.

(Storage unit 220)
The storage unit 220 stores a program and data for the operation of the information management server 200.

  In particular, in the present embodiment, the storage unit 220 stores various information used in the information processing system.

  As a first example, the storage unit 220 stores parameters related to the camera 11, the microphone 13, and the sensor 15. The specific content of the parameter is as described above.

  As a second example, the storage unit 220 stores data of a three-dimensional virtual space corresponding to a real space. The three-dimensional virtual space is, for example, a three-dimensional virtual space imitating the center office 10. The specific content of the three-dimensional virtual space is as described above.

  As a third example, the storage unit 220 stores person related information. The person-related information is, for example, person-related information of a person at the center office 10. The specific content of the person-related information is as described above.

  As a fourth example, the storage unit 220 stores the object ID of the object arranged in the three-dimensional virtual space and the communication ID in association with each other. The specific contents of the object ID and the communication ID are as described above.

(Control unit 230)
The control unit 230 provides various functions of the information management server 200.

  For example, the control unit 230 provides various information used in the information processing system in response to a request. For example, in response to a request from the terminal device 100, the control unit 230 includes data of a three-dimensional virtual space, person-related information, a communication ID corresponding to an object ID, and parameters related to the camera 11, the provide.

  Also, for example, the control unit 230 appropriately updates various information used in the information processing system. The control unit 230 updates the information automatically or in response to a manual instruction.

<< 4. Flow of processing >>
Subsequently, an example of the information processing according to the present embodiment will be described with reference to FIG. FIG. 17 is a flowchart showing an example of a schematic flow of information processing according to the present embodiment.

  In step S401, the position acquisition unit 183 determines whether the position in the captured image on the display screen of the captured image in the real space is designated by the user. More specifically, for example, the position acquisition unit 183 determines whether the position in the close-up captured image is designated by the user. If the above position is designated, the process proceeds to step S403. Otherwise, the process repeats step S401.

  In step S403, the position acquisition unit 183 acquires the position in the captured image specified by the user.

  In step S405, the object selection unit 185 acquires data of the three-dimensional virtual space corresponding to the real space from the storage unit 170.

  In step S407, an object arranged in the three-dimensional virtual space is selected based on the acquired position in the captured image.

  In step S409, the ID acquisition unit 187 acquires a communication ID corresponding to the selected object as a communication ID on the called side.

  In step S411, the ID acquisition unit 187 acquires a communication ID for the transmission side (that is, the ID for communication of the terminal device 100).

  In step S413, the telephone unit 189 makes a call by using the communication ID on the called side. Thereafter, the telephone unit 189 performs various processes for the telephone, and the process ends.

  As described above, the information processing according to the present embodiment is performed. Furthermore, an example of the activation process performed before the start of the information processing will be described with reference to FIG.

  FIG. 18 is a flowchart showing an example of a schematic flow of activation processing according to the present embodiment.

  In step S501, a process of activating the softphone 851 is performed. Thus, the softphone 851 is activated.

  In step S503, registration processing for the softphone 851 is performed. For example, as one of the registration processing of the softphone 851, registration (for example, SIP REGISTRATION) in the PBX 40 is performed.

  In step S505, the process of activating the ultra-realistic client is performed. For example, the camera 11, the microphone 13, the sensor 15, the media distribution server 17, the information management server 200, and the like used in the ultra-realism client are specified.

  In step S507, the crawling mode processing of the ultra-realism client is executed. Then, the series of activation processing ends.

  Although the preferred embodiments of the present invention have been described above with reference to the accompanying drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the appended claims, and of course these also fall within the technical scope of the present invention. It is understood.

  For example, although the example in which the three-dimensional virtual space of the center office is prepared as the three-dimensional virtual space corresponding to the real space has been described, the present invention is not limited thereto. For example, a plurality of three-dimensional spaces may be prepared. As an example, a three-dimensional virtual space corresponding to real space may be prepared for each of a plurality of offices. For example, a three-dimensional virtual space may also be provided for satellite offices, home offices, other center offices, etc. In this case, the three-dimensional virtual space of each office may be a three-dimensional virtual space having a size corresponding to the size of each office. In addition, the object selection unit of the terminal device may acquire data of a desired three-dimensional virtual space among a plurality of three-dimensional virtual spaces. Also, a three-dimensional virtual space other than the office may be prepared.

  Further, although the example has been described in which the object corresponding to the person is selected only when the person is sitting in the seat, the present invention is not limited thereto. For example, the object may be selected even when the person is not seated in the seat. As an example, when a person is sitting in a seat, the communication ID of the communication device installed in the seat is acquired, and when the person is not sitting in the seat, the ID for communication of the portable terminal of the person May be acquired.

  Although the example in which the communication ID is a telephone number has been described, the present invention is not limited to this. The communication ID may be an ID other than a telephone number. As one example, the communication ID may be a softphone ID other than the telephone number. As another example, the communication ID may be an ID for communication other than telephone. For example, the communication ID may be a mail address or an ID for a short message. In this case, a mail may be sent or a short message may be sent using the communication ID.

  Further, although an example has been described in which the communication ID corresponding to the object is acquired when the object is selected, the present invention is not limited to this. For example, when an object is selected, any identification information corresponding to the object may be acquired. As one example, when an object is selected, any identification information of a person corresponding to the object may be obtained. Then, for example, the communication ID may be acquired from the identification information.

  Further, although the example in which the object (object selected by the object selection unit) disposed in the three-dimensional virtual space corresponding to the real space corresponds to a person and is a cylindrical object has been described, the present invention is not limited thereto . For example, the object may not be a cylindrical object but an object of another shape. Also, for example, an object may correspond to something other than a person. As an example, an object may correspond to a region of real space. Specifically, for example, the object may be arranged at a three-dimensional virtual position corresponding to the seat and corresponding to the position of the seat. Then, the communication ID of the communication device installed in the seat corresponds to the object, and when the object is selected, the communication ID may be acquired. Also, the object may correspond to an area wider than the seat, and may exist over a three-dimensional virtual area corresponding to the area of the area. Then, the communication ID of the communication device installed in the area corresponds to the object, and when the object is selected, the communication ID may be acquired.

  In addition, although an example in which the position of the captured image on the display screen is specified by a touch by the user has been described, the present invention is not limited to this. For example, the position of the captured image may be designated by the user using an input unit other than the touch panel. For example, the position of the captured image may be specified by clicking with a mouse, or may be specified using another input means such as a button or a keyboard.

  Also, an example has been described in which the captured image generated by the camera, the audio data generated by the microphone, and the determination result by the sensor are directly provided to the terminal device by the camera, the microphone and the sensor, respectively. Is not limited to this. For example, these data may be provided by another device. As an example, any server (for example, media distribution server) may obtain these data and provide these data to the terminal device.

  Moreover, although the example in which the functions such as the position acquisition unit, the object selection unit, and the ID acquisition unit are provided by the terminal device has been described, the present invention is not limited thereto. For example, these functions may be provided by devices other than the terminal device. As an example, these functions may be provided by any server.

  Moreover, although the example in which the display screen of the captured image of real space was displayed by the terminal device was demonstrated, this invention is not limited to this. For example, the display screen may be displayed by another device. As one example, the display screen may be displayed by a display installed at a satellite office. Then, the user may designate the position of the captured image included in the display image on the display.

  In addition, the processing steps in the information processing of the present specification may not necessarily be performed in chronological order according to the order described in the flowchart. For example, the processing steps in the information processing may be performed in an order different from the order described as the flowchart, or may be performed in parallel.

  In addition, it is possible to create a computer program for causing hardware such as a CPU, a ROM, and a RAM built in an information processing apparatus (for example, a terminal apparatus) to exhibit the same function as each configuration of the information processing apparatus. A storage medium storing the computer program is also provided.

11 Camera 13 Microphone 15 Sensor 17 Media Distribution Server 19, 23 LAN (Local Area Network)
21 Display 30 External Network 40 PBX (Private Branch eXchange)
60 Display screen (俯瞰 mode)
61 俯瞰 captured image 63, 73, 83 button image 70 display screen (proximity mode)
71 Close-up image 80 Display screen (conversation mode)
81 Remotely-captured image 81
85 Own side captured image 85
90 three-dimensional virtual space 91 object 93 virtual plane 100 terminal device 180 control unit 181 real space information provision unit 183 position acquisition unit 185 object selection unit 187 ID acquisition unit 189 telephone unit

Claims (22)

A real space information providing unit that causes a display unit to display a display screen in a first display mode for displaying a first captured image relating to a first region of the real space captured by one of the plurality of imaging devices;
Equipped with
The real space information provision unit changes the direction of the one imaging device among the plurality of imaging devices in accordance with the designation of the position in the first captured image by the user on the display screen of the first display mode. To a display screen of a second display mode for displaying a second captured image relating to a second region of the real space which is narrower than the first region captured by the one imaging device with the first display mode An information processing apparatus, switching a display screen and displaying the same on the display unit.   The information processing apparatus according to claim 1, wherein the second area is an area including a position in the real space corresponding to a position in the first captured image specified by the user.   The information processing apparatus according to claim 2, wherein the second area is an area having a position in the real space corresponding to a position in the first captured image specified by the user.   The information processing apparatus according to claim 3, wherein at least the first captured image is a captured image of a part or the whole of the first area.   The information processing apparatus according to claim 4, wherein at least the second captured image is a captured moving image.   The information processing apparatus according to claim 5, wherein the display screen of the second display mode includes a button display for switching a display mode from the second display mode to the first display mode.   The information processing apparatus according to claim 6, wherein the button display is displayed on the display screen of the second display mode without being superimposed on the second captured image and in the vicinity of the second captured image.   Each of the first captured image and the second captured image is a captured image generated based on imaging by one of the plurality of imaging devices disposed in the real space. The information processing apparatus according to claim 1. The information processing apparatus is
And a communication unit connected to the network,
The information processing apparatus according to claim 8, wherein the communication unit acquires each of the first captured image and the second captured image via the network.   The information processing apparatus according to claim 9, wherein the communication unit acquires each of the first captured image and the second captured image from the server connected to the network via the network.   The real space information provision unit changes the orientation of one of the plurality of imaging devices so that the second region can be imaged according to the user's designation of a position in the first captured image by the user. The information processing apparatus according to claim 10, wherein the communication unit causes the communication unit to transmit a request to cause the server to make a request.   The information processing apparatus according to claim 11, wherein the network is a LAN and / or the Internet. The information processing apparatus further includes the display unit.
The information processing apparatus according to claim 12, wherein the position in the first captured image is designated by a touch operation of the user on the display unit. The real space information provision unit sets the zoom ratio of the optical zoom or digital zoom of one imaging device among the plurality of imaging devices to the designated zoom ratio according to the user's designating operation of the zoom ratio on the display unit. Have the communication unit send the request to be changed to the server;
When the communication unit acquires the second captured image generated based on imaging by one imaging device of the plurality of imaging devices after being changed to the designated zoom ratio through the network, the real space information The information processing apparatus according to claim 13, wherein the providing unit causes the display unit to display a display screen of the second display mode for displaying the second captured image after being changed to the designated zoom rate. The information processing apparatus includes an operating system (OS),
The real space is a space in a building,
The real space information provision unit is software operating on the OS (Operating System),
After activation, the real space information provision unit causes the display unit to display a display screen of the first display mode,
The first display mode is an eyebrow mode,
The information processing apparatus according to any one of claims 1 to 14, wherein the second display mode is a proximity mode.   The information processing apparatus according to any one of claims 1 to 15, wherein the information processing apparatus is a tablet, a smartphone, a PC (Personal Computer), or a telephone. Displaying on the display unit a display screen in a first display mode for displaying a first captured image relating to a first region of the real space captured by one of the plurality of imaging devices;
The one imaging device accompanied by a change in the direction of the one imaging device among the plurality of imaging devices in accordance with the designation of the position in the first captured image by the user on the display screen of the first display mode the display unit switches the display screen of the second display mode the first display mode to the display screen for displaying the second image, according to a second region of a narrow the real space than the first region imaged by Step to display
Information processing methods, including: Computer,
A real space information providing unit that causes a display unit to display a display screen in a first display mode for displaying a first captured image relating to a first region of the real space captured by one of the plurality of imaging devices;
A program to function as
The real space information provision unit changes the direction of the one imaging device among the plurality of imaging devices in accordance with the designation of the position in the first captured image by the user on the display screen of the first display mode. To a display screen of a second display mode for displaying a second captured image relating to a second region of the real space which is narrower than the first region captured by the one imaging device with the first display mode A program for switching a display screen to display on the display unit. A real space information providing unit that causes a display unit to display a display screen in a first display mode for displaying a first captured image relating to a first region of the real space captured by one of the plurality of imaging devices;
Equipped with
The real space information provision unit changes the direction of the one imaging device among the plurality of imaging devices in accordance with the designation of the position in the first captured image by the user on the display screen of the first display mode. To a display screen of a second display mode for displaying a second captured image relating to a second region of the real space which is narrower than the first region captured by the one imaging device with the first display mode The information processing system which switches a display screen and displays it on the said display part. A control unit that provides a first image generated based on a first captured image related to a first region of the real space captured by one of the plurality of imaging devices, to the information processing device;
Equipped with
The control unit relates to a second area of the real space, which is narrower than the first area , obtained by the one imaging device along with a change in the direction of the one imaging device among the plurality of imaging devices. The second image generated based on the second captured image is provided to the information processing device in accordance with the designation of the position in the first captured image by the user on the display screen displayed on the information processing device. Do the server. An information processing apparatus and a server;
The server is
A control unit that provides a first image generated based on a first captured image related to a first region of a real space captured by one imaging device among a plurality of imaging devices, to the information processing device;
Equipped with
The control unit relates to a second area of the real space, which is narrower than the first area , obtained by the one imaging device along with a change in the direction of the one imaging device among the plurality of imaging devices. The second image generated based on the second captured image is provided to the information processing device in accordance with the designation of the position in the first captured image by the user on the display screen displayed on the information processing device. Information processing system. Have a program and a server,
The server is
A control unit that provides a first image generated based on a first captured image related to a first region of a real space captured by one imaging device among a plurality of imaging devices to an information processing device that executes the program ,
Equipped with
The control unit relates to a second area of the real space, which is narrower than the first area , obtained by the one imaging device along with a change in the direction of the one imaging device among the plurality of imaging devices. The program according to a user's designation of a position in the first captured image by a user on a display screen displayed on the information processing apparatus that executes the program, with the second image generated based on the second captured image. An information processing system provided to an information processing apparatus that executes

Images