WO2020248682A1 - Display device and virtual scene generation method - Google Patents

Abstract

Provided is a display device, characterized by comprising: a display for displaying a game application-related interface; a camera configured to acquire a photographed image, including a portrait, of a user; and a controller for acquiring, according to the position of a virtual character corresponding to the user in a game interface, a portrait-related part in the photographed image, and controlling the display to display the portrait-related part in a manner where same floats at the position of the virtual character in the game interface.

Description

Display device and virtual scene generation method

This disclosure claims the priority of a Chinese patent application filed with the Chinese Patent Office on June 10, 2019, the filing date is 201910497519.8, and the application title is "a method and device for generating virtual scenes", the entire content of which is incorporated into this disclosure by reference in.

Technical field

This application relates to the technical field of smart TVs, and in particular to a display device and a virtual scene generation method.

Background technique

With the development of science and technology, the functions of various types of terminal equipment are becoming more abundant, so as to meet the diverse needs of users. For example, current terminal devices such as smart phones and TVs can provide game functions to meet the entertainment needs of users.

When a user plays a game through a terminal device, the terminal device often displays the game scene corresponding to the game for the user, so that the user can watch the game screen and understand the game process. Generally, in a game scene, the actual user portrait cannot be applied to the actual scene, but only in the game scene to show the preset animation effect. This is not conducive to the user experience.

Summary of the invention

An exemplary embodiment of the present application provides a display device for dynamically displaying a game interface according to a user's portrait on the TV side.

According to the first aspect of the exemplary embodiment, the present application provides a display device, including:

The display is configured to present a game application related interface;

A camera configured to obtain a photographed image of a user, the photographed image including a portrait of the user;

The controller is configured as:

According to the position of the virtual character corresponding to the user in the game interface, perform sideways processing on the portrait in the captured image to obtain a sideways image including the sideways processed portrait, wherein the virtual character is in the game interface Is the target position;

Acquire a virtual scene interface superimposed with the side image and the game interface, and control the display to present the virtual scene interface, wherein the side image is located on the upper layer of the virtual character in the game interface.

In the second aspect, an exemplary embodiment of the present application provides a virtual scene generation method, including:

Acquiring a photographed image of the user in real time, the photographed image including a portrait of the user;

According to the position of the virtual character corresponding to the user in the game interface, perform sideways processing on the portrait in the captured image to obtain a sideways image including the sideways processed portrait, wherein the virtual character is in the game interface Is the target position;

Obtain a virtual scene interface superimposed with the sideways image and the game interface, and display the virtual scene interface, wherein the sideways image is located on the upper layer of the game interface, and the sideways processing included in the sideways image The rear portrait is located directly above the target location.

In a third aspect, an exemplary embodiment of the present application provides a virtual scene generation device, including:

An image acquisition module, configured to acquire a photographed image of a user in real time, and the photographed image includes a portrait of the user;

The sideways processing module is configured to perform sideways processing on the portrait in the captured image according to the position of the virtual character corresponding to the user in the game interface, and obtain a sideways image including the sideways processed portrait, wherein the virtual character The position in the game interface is the target position;

The interface overlay module is used to obtain a virtual scene interface superimposed with the sideways image and the game interface, and display the virtual scene interface, wherein the sideways image is located on the upper layer of the game interface, and the sideways The sideways processed portrait included in the image is located directly above the target position.

In a fourth aspect, an exemplary embodiment of the present application provides a display device, including:

The display is configured to present a game application related interface;

A camera configured to obtain a photographed image of a user, the photographed image including a portrait of the user;

The controller is configured as:

Acquiring the portrait-related part in the captured image according to the position of the virtual character corresponding to the user in the game interface, wherein the position of the virtual character in the game interface is the target position;

Controlling the display to display the portrait-related part floating on the target position of the game interface.

The above is a brief summary of the present disclosure to explain certain aspects of the application. This summary is not a comprehensive or exhaustive overview of the application and its various aspects, examples and/or configurations. Its purpose is neither to determine the main or key elements of the application, It does not describe the scope of the application, but briefly introduces certain concepts of the application. As an introduction to the detailed description below, it should be understood that other aspects, examples and/or configurations of this disclosure can be used alone or in combination with the above statements or A feature or pairs of fine lines detailed.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

FIG. 1 exemplarily shows a schematic diagram of an operation scene between the display device 200 and the control device according to the embodiment;

FIG. 2 exemplarily shows a configuration block diagram of the software system in the display device 200 according to the embodiment;

FIG. 3 exemplarily shows a schematic diagram of application programs included in the application layer of the display device according to the embodiment;

FIG. 4 exemplarily shows a schematic diagram of a user interface in the display device 200 according to the embodiment;

5 is a schematic diagram of a workflow of a method for generating a virtual scene disclosed in an embodiment of the application;

FIG. 6 is a schematic diagram of an application scenario of a virtual scene generation method disclosed in an embodiment of the application;

FIG. 7 is a portrait after sideways processing in a method for generating a virtual scene disclosed in an embodiment of the application;

Fig. 8 is a schematic structural diagram of a virtual scene generating apparatus disclosed in an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Hereinafter, each embodiment of the present application will be described in detail with reference to the drawings of the specification. It should be noted that the display order of the embodiments of the present application only represents the sequence of the embodiments, and does not represent the pros and cons of the technical solutions provided by the embodiments.

The term "module" used in the various embodiments of this application can refer to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or/and software code that can execute related to the component Function.

The term "remote control" used in the various embodiments of this application refers to a component of an electronic device (such as the display device disclosed in this application), which can generally control the electronic device wirelessly within a short distance. This component can generally use infrared and/or radio frequency (RF) signals and/or Bluetooth to connect with electronic devices, and can also include functional modules such as wireless network WiFi, wireless USB, Bluetooth, and motion sensors. For example, a handheld touch remote control uses a user interface in a touch screen to replace most of the physical built-in hard keys in general remote control devices.

The term "gesture" used in the embodiments of the present application refers to a user's behavior through a change of hand shape or hand movement to express expected ideas, actions, goals, and/or results.

FIG. 1 exemplarily shows a schematic diagram of an operation scene between the display device 200 and the control device according to the embodiment. As shown in FIG. 1, the user can operate the display device 200 by controlling the device 100.

Wherein, the control device 100 may be a remote controller 100A, which can communicate with the display device 200 through infrared protocol communication, Bluetooth protocol communication, ZigBee protocol communication or other short-distance communication methods for wireless or other communication The display device 200 is controlled in a wired manner. The user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, etc. For example, the user can control the display device 200 by inputting corresponding control commands through the volume plus and minus keys, channel control keys, up/down/left/right movement keys, voice input keys, menu keys, and switch buttons on the remote control. Function.

The control device 100 can also be a smart device, such as a mobile terminal 100B, a tablet computer, a computer, a notebook computer, etc., which can be connected through a local area network (LAN, Wide Area Network), a wide area network (WAN, Wide Area Network), and a wireless local area network ((WLAN) , Wireless Local Area Network) or other networks communicate with the display device 200, and realize the control of the display device 200 through an application program corresponding to the display device 200. For example, use an application program running on a smart device to control the display device 200 The application can provide users with various controls through an intuitive user interface (UI, User Interface) on the screen associated with the smart device.

For example, both the mobile terminal 100B and the display device 200 can be installed with software applications, so that the connection and communication between the two can be realized through a network communication protocol, thereby realizing one-to-one control operation and data communication. For example, the mobile terminal 100B can establish a control command protocol with the display device 200, synchronize the remote control keyboard to the mobile terminal 100B, and control the display device 200 by controlling the user interface of the mobile terminal 100B; or the mobile terminal 100B The audio and video content displayed on the screen is transmitted to the display device 200 to realize the synchronous display function.

As shown in FIG. 1, the display device 200 can also communicate with the server 300 through multiple communication methods. In various embodiments of the present application, the display device 200 may be allowed to communicate with the server 300 via a local area network, a wireless local area network, or other networks. The server 300 may provide various contents and interactions to the display device 200.

Illustratively, the display device 200 transmits and receives information, interacts with an Electronic Program Guide (EPG, Electronic Program Guide), receives software program updates, or accesses a remotely stored digital media library. The server 300 may be a group or multiple groups, and may be one or more types of servers. The server 300 provides other network service content such as video on demand and advertising services.

The display device 200 may be a liquid crystal display, an organic light emitting diode (Organic Light Emitting Diode, OLED) display, a projection display device, or a smart TV. The specific display device type, size, aspect ratio, etc. are not limited, and those skilled in the art can understand that the display device 200 can make some changes in performance and configuration as required.

In addition to providing the broadcast receiving TV function, the display device 200 may additionally provide a smart network TV function that provides a computer support function. Examples include Internet TV, Smart TV, Internet Protocol TV (IPTV, Interactive Personality TV), etc.

As shown in Figure 1, the display device may be connected or provided with a camera, which is used to present the picture captured by the camera on the display interface of the display device or other display devices to realize interactive chats between users. Specifically, the picture captured by the camera can be displayed on the display device in full screen, half screen, or in any optional area.

As an optional connection method, the camera is connected to the monitor rear shell through a connecting plate, and is fixedly installed on the upper middle of the monitor rear shell. As an installable method, it can be fixedly installed at any position of the monitor rear shell to ensure its It is sufficient that the image capture area is not blocked by the rear shell, for example, the image capture area and the display device have the same orientation.

As another optional connection method, the camera can be connected to the display rear shell through a connecting plate or other conceivable connectors. The connector is equipped with a lifting motor. When the user wants to use the camera or has an application to use the camera When the camera is not needed, it can be raised above the display. When the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.

As an embodiment, the camera used in this application may have 16 million pixels to achieve the purpose of ultra-high-definition display. In actual use, a camera with higher or lower than 16 million pixels can also be used.

When a camera is installed on the display device, the content displayed in different application scenarios of the display device can be merged in many different ways, so as to achieve functions that cannot be achieved by traditional display devices.

Exemplarily, the user can video chat with at least one other user while watching a video program. The presentation of the video program can be used as the background picture, and the video chat window is displayed on the background picture. Visually, you can call this function "watch and chat".

Optionally, in the "watching and chatting" scenario, while watching live video or online video, at least one video chat is performed across terminals.

In another example, the user can video chat with at least one other user while entering the education application for learning. For example, students can realize remote interaction with teachers while learning content in educational applications. Visually, you can call this function "learning and chatting".

In another example, when a user is playing a card game, a video chat is conducted with players entering the game. For example, when a player enters a game application to participate in a game, it can realize remote interaction with other players. Visually, you can call this function "watch and play".

Optionally, the game scene is integrated with the video picture, and the portrait in the video picture is cut out and displayed on the game picture to improve user experience.

Optionally, in somatosensory games (such as ball games, boxing games, running games, dancing games, etc.), human body postures and movements are acquired through the camera, body detection and tracking, and the detection of human bone key points data, and then the game Animations are integrated to realize games such as sports and dance scenes.

In another example, the user can interact with at least one other user in video and voice in the K song application. Visually, you can call this function "watch and sing". Preferably, when at least one user enters the application in the chat scene, multiple users can jointly complete the recording of a song.

In another example, the user can turn on the camera locally to obtain pictures and videos, which is vivid, and this function can be called "look in the mirror". The mirror application is a newly added application in the social TV project. The application supports camera image preview, face recognition, adding beauty filters, taking pictures, photo sharing and other functions; the method of taking pictures mentioned in this embodiment is mirroring Realized in application.

In other examples, more functions can be added or the aforementioned functions can be reduced. This application does not specifically limit the function of the display device.

FIG. 2 exemplarily shows a configuration block diagram of a software system in the display device 200 according to an exemplary embodiment.

As shown in FIG. 2, the operating system 2911 includes operating software for processing various basic system services and performing hardware-related tasks, and serves as a medium for completing data processing between application programs and hardware components.

In some embodiments, part of the operating system kernel may include a series of software to manage the hardware resources of the display device and provide services for other programs or software codes.

In some other embodiments, part of the operating system kernel may include one or more device drivers, and the device drivers may be a set of software codes in the operating system to help operate or control devices or hardware associated with the display device. The drive may contain code to manipulate video, audio, and/or other multimedia components. Examples include displays, cameras, Flash, WiFi, and audio drivers.

Among them, the accessibility module 2911-1 is used to modify or access the application program, so as to realize the accessibility of the application program and the operability of its display content.

The communication module 2911-2 is used to connect to other peripherals via related communication interfaces and communication networks.

The user interface module 2911-3 is used to provide objects that display the user interface for access by various applications, and can realize user operability.

The control application 2911-4 is used to control process management, including runtime applications.

The event transmission system 2914 can be implemented in the operating system 2911 or in the application 2912. In some embodiments, it is implemented in the operating system 2911 on the one hand, and implemented in the application program 2912 at the same time, for monitoring various user input events, and responding to the recognition results of various events or sub-events according to various events. And implement one or more sets of pre-defined operation procedures.

Among them, the event monitoring module 2914-1 is used to monitor input events or sub-events of the user input interface.

The event recognition module 2914-1 is used to input the definitions of various events to various user input interfaces, recognize various events or sub-events, and transmit them to the processing to execute the corresponding one or more groups of processing programs .

Among them, the event or sub-event refers to the input detected by one or more sensors in the display device 200 and the input of an external control device (such as the control device 100). Such as: various sub-events of voice input, gesture input sub-events of gesture recognition, and sub-events of remote control button command input of control devices. For example, one or more sub-events in the remote control include multiple forms, including but not limited to one or a combination of pressing up/down/left/right/, confirming keys, and pressing keys. And the operations of non-physical buttons, such as moving, pressing, and releasing.

The interface layout management module 2913, which directly or indirectly receives various user input events or sub-events monitored by the event transmission system 2914, is used to update the layout of the user interface, including but not limited to the position of each control or sub-control in the interface, and the container The size, position, level, etc. of the interface are related to various execution operations.

FIG. 3 exemplarily shows a schematic diagram of the application programs included in the application program layer of the display device according to the exemplary embodiment, and the application program layer of the display device includes various application programs that can be executed on the display device 200.

The application layer 2912 may include, but is not limited to, one or more applications, such as: video-on-demand application, application center, game application, etc., live TV application, media center application, etc.

Live TV applications can provide live TV through different sources. For example, a live TV application can provide a TV signal using input from cable TV, wireless broadcasting, satellite services, or other types of live TV services. And, the live TV application can display the video of the live TV signal on the display device 200.

Video-on-demand applications can provide videos from different storage sources. Unlike live TV applications, VOD provides video display from certain storage sources. For example, the video on demand can come from the server side of cloud storage, and from the local hard disk storage that contains the stored video programs.

Media center applications can provide various multimedia content playback applications. For example, the media center can provide services that are different from live TV or video on demand, and users can access various images or audio through the media center application.

Application center, can provide storage of various applications. The application program may be a game, an application program, or some other application program that is related to a computer system or other device but can be run on a display device. The application center can obtain these applications from different sources, store them in the local storage, and then run on the display device 200.

FIG. 4 exemplarily shows a schematic diagram of a user interface in the display device 200 according to an exemplary embodiment. As shown in FIG. 4, the user interface includes a view display area 201. For example, the view display area 201 includes layout of one or more different items. And the user interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.

In some embodiments, the view display area 201 is a zoomable view display. "Scalable" can mean that the size or proportion of the view display area 201 on the screen is scalable, or that the size or proportion of the items in the view display 201 on the screen is scalable.

"Items" refer to visual objects displayed in the view display area of the user interface of the display device 200 to represent corresponding content such as icons, thumbnails, and video clips. For example, an item may represent image content or video clips of movies, TV shows, audio content of music, applications, or other user access content history information.

In addition, the item may indicate an interface or a set display of interfaces through which the display device 200 is connected to an external device, or may indicate the name of an external device connected to the display device, or the like. Such as: signal source input interface collection, or high-definition multimedia interface (High Definition Multimedia Interface, HDMI), USB interface, PC terminal interface, etc.

It should be noted that: the view display area can present the content of the video chat project or the content of the application layer project (for example, webpage video, video on demand (VOD) display, application screen, etc.).

"Selector" is used to indicate that any item has been selected, such as cursor or focus object. Positioning the selection information input according to the icon or menu position touched by the user in the display 200 can enable the display device 200 to display the movement of the focus object to select the control item, and select or control one or more items.

The focus object refers to the object that moves between items based on user input. For example, draw a thick line on the edge of the item to realize or identify the position of the focus object. In other embodiments, the focus form is not limited to examples. It can be a tangible or intangible form that can be recognized by the user, such as a cursor, such as a 3D deformation of the item, and the border line and size of the text or image of the focused item can also be changed. , Color, transparency, outline, and/or font.

The event transmission system 2914 can monitor the user input of each predefined event or sub-event, and directly or indirectly provide the interface layout management module 2913 with the control of the recognized event or sub-event.

The interface layout management module 2913 is used to monitor the state of the user interface (including the position and/or size, change process, etc. of the view partition, item, focus or cursor object, etc.), and according to the event or sub-event, the view display area can be modified The size, location, level and other layout of the display area, and/or, adjust or modify the size or/and location, quantity, type, content and other layouts of various items in the view display area. In some embodiments, modifying and adjusting the layout includes displaying or not displaying each view partition or the content of items in the view partition on the screen.

The user input interface is used to send the user's input signal to the controller, or to transmit the signal output from the controller to the user. Exemplarily, the control device (such as a mobile terminal or a remote control) may send input signals input by the user, such as a power switch signal, a channel selection signal, and a volume adjustment signal, to the user input interface, and then the user input interface forwards the input signal to the controller; Alternatively, the control device may receive output signals such as audio, video, or data output from the user input interface processed by the controller, and display the received output signal or output the received output signal as audio or vibration.

In some embodiments, the user may input a user command on a graphical user interface (GUI) displayed on the display 200, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through the sensor to receive the user input command.

In order to meet the entertainment needs of users, various types of terminal devices are usually able to provide users with game functions. Among them, when the user is playing certain games through the terminal device, the virtual character corresponding to the user may be displayed on the game interface. In this case, the user can select the image that he or she is satisfied with from the various images provided by the game, or select the image that he or she is satisfied with from the album stored in the terminal device, and then the game interface of the terminal device displays The image selected by the user is used as the virtual character corresponding to the user.

However, in this case, during the course of the game, the virtual character in the game interface will remain fixed and remain the image selected by the user, resulting in a poor user experience during the game.

In order to overcome the above-mentioned shortcomings, this application discloses a virtual scene generation method and device through the following embodiments. The virtual scene generation method and device are applied to a terminal device. The terminal device can be a smart phone, a TV, a tablet computer, etc. The embodiment does not limit this.

In addition, the terminal device can communicate with other terminal devices through a local area network (LAN, Local Area Network), a wide area network (WAN, Wide Area Network), a wireless local area network ((WLAN, Wireless Local Area Network) or other networks.

In addition, the terminal device may be connected or provided with a camera for presenting the screen shot by the camera on the display interface of the terminal device or the display interface of other terminal devices to realize interactive chat between users. Specifically, the picture captured by the camera can be displayed on the terminal device in full screen, half screen, or in any optional area.

As an exemplary connection method, the camera is connected to the rear shell of the terminal device through a connecting plate, and is fixedly installed on the upper middle of the display rear shell of the terminal device. As an installable method, it can be fixedly installed on the display of the terminal device. Any position of the rear case can ensure that the image acquisition area is not blocked by the rear case. For example, the image acquisition area and the display orientation of the terminal device are the same.

As another exemplary connection method, the camera can be connected to the rear shell of the terminal device through a connecting plate or other conceivable connectors. A lifting motor is installed on the connector. When the user wants to use the camera or has an application When the camera is used, it is raised above the terminal device. When the camera is not needed, it can be embedded behind the back shell to protect the camera from damage.

In addition, FIG. 4 exemplarily shows a schematic diagram of a display interface in an embodiment of the present application. As shown in FIG. 4, the display interface includes multiple view display areas, for example, a first view display area 201 and a play screen 202, where the play screen includes layout of one or more different items. In addition, the display interface also includes a selector indicating that the item is selected, and the position of the selector can be moved through user input to change the selection of different items.

It should be noted that multiple view display areas can present display screens of different levels. For example, the first view display area can present video chat item content, and the second view display area can present application layer item content (eg, webpage video, VOD display, application program screen, etc.).

As an exemplary embodiment, the presentation of different view display areas has different priorities, and the display priorities of the view display areas are different between view display areas with different priorities. For example, the priority of the system layer is higher than that of the application layer. When the user uses the acquisition selector and screen switching in the application layer, the screen display in the view display area of the system layer is not blocked; and the application layer is enabled according to the user's selection When the size and position of the view display area change, the size and position of the view display area of the system layer will not be affected.

In addition, the same level of display screens can also be presented. At this time, the selector can switch between the first view display area and the second view display area, and when the size and position of the first view display area change, the first view display area The size and position of the second view display area can be changed at any time.

In order to clarify the solutions disclosed in the present application, the solutions of the present application are introduced through different embodiments respectively below.

The first embodiment of the present application discloses a method for generating a virtual scene. Referring to the schematic diagram of the workflow shown in FIG. 5, the method includes the following steps:

Step S11: Acquire a photographed image of a user in real time, and the photographed image includes a portrait of the user.

Wherein, the user is each participant of the game.

Step S12: According to the position of the virtual character corresponding to the user in the game interface, perform sideways processing on the portrait in the photographed image to obtain a sideways image including the sideways processed portrait, wherein the virtual character is in the The position in the game interface is the target position.

Among them, during the sideways processing, according to the position of the virtual character in the game interface, it is determined in the actual game scene whether the virtual character corresponding to each user is displayed on the front or side in the game interface, and then the captured image Perform corresponding sideways processing on the portrait of, so that the sideways processed portrait conforms to the actual game scene.

In the game interface, in order to increase the user’s sense of experience, there are often virtual characters corresponding to the user. For example, in a chess and card game, if there are three people A, B and C participating, the game interface often displays the virtual characters of these three people. character. Wherein, when the virtual character is displayed, usually only the head portion of the virtual character is displayed. Of course, the entire virtual character can also be displayed, which is not limited in the embodiment of the present application.

In addition, the positions of different virtual characters in the game interface are different. Accordingly, after the sideways processing, the angles of the sideways processed portraits are different, so that the sideways processed portraits can match the actual game scene. For example, in an example, user A is on the front of the chess table, user B is on the left side of the chess table, and user C is on the right side of the chess table. In this case, in order to conform to the actual game scene, the user passes sideways After processing, make the processed portrait of user A sideways face the game interface, make the processed portrait of user B sideways display the right body, and make the processed portrait of user C sideways display the left body.

In step S12, according to the position of the avatar corresponding to the user in the game interface, corresponding sideways processing is performed on the portrait in the captured image, so that the sideways processed portrait can conform to the game scene.

Step S13: Obtain a virtual scene interface superimposed with the side image and the game interface, and display the virtual scene interface, wherein the side image is located on the upper layer of the game interface, and the side image includes The sideways processed portrait is located directly above the target position.

The terminal device of the embodiment of the present application can display multiple levels of display screens. In step S13, the side image and the game interface are display screens of different levels. In addition, the sideways image is located on the upper layer of the game interface, and the sideways processed portrait included in the sideways image is located directly above the target position. In this case, the sideways image is superimposed on the game interface. In the virtual scene interface, the sideways processed portraits cover the virtual characters in the game interface, so that when watching the screen displayed on the terminal device, the user no longer sees the image of the virtual character provided by the game interface, but handles it sideways Portrait after.

In the virtual scene generation method provided by the embodiments of the present application, the user’s photographed image is first obtained in real time; then, the portrait in the photographed image is processed sideways according to the position of the virtual character corresponding to the user in the game interface. Acquiring a sideways image including the sideways processed portrait; acquiring a virtual scene interface superimposed with the sideways image and the game interface, and displaying the virtual scene interface, wherein the sideways image is located on the upper layer of the game interface, In addition, the sideways processed portrait included in the sideways image is located directly above the target position.

Wherein, the portrait included in the side image is a portrait obtained after processing the portrait in the captured image according to the position of the virtual character corresponding to the user in the game interface, and the captured image is an image obtained in real time. In this case, the sideways processed portrait included in the sideways image can be changed according to the real-time captured image and the position of the virtual character in the game interface.

In addition, when the user views the virtual scene interface through the solution of the embodiment of the present application, what he sees is the portrait processed sideways, instead of viewing the image of the virtual character provided by the game interface. In this case, even if the user image in the game interface remains fixed, the user can view the sideways image after processing the user’s portrait sideways through the virtual scene interface, so that the user can watch the changed virtual scene. The character solves the problem in the prior art that the virtual character in the game interface remains fixed during the game, thereby improving the user's game experience.

Further, in the embodiment of the present application, the user's photographed image can be obtained in various ways.

In one of the ways, the real-time acquisition of the user's captured image includes:

The captured image transmitted by the first terminal device on the user side is acquired in real time.

In the application scenarios of the embodiments of the present application, multiple users are often playing games. In this case, the first terminal device on each user side can shoot the user through its own camera, obtain the captured image, and then shoot The image is transmitted to the terminal device that executes the method of the embodiment of the present application.

For example, when three users of A, B, and C are playing a game, and the terminal devices of these three users execute the method disclosed in the embodiment of this application, the first terminal devices on the side of the three users (ie, A and B respectively) And the terminal equipment of the three users of user C) to shoot, obtain the photographed image, and then transmit the photographed image to other terminal equipment, that is, the terminal equipment of user A will transmit the photographed image including user A to the terminals of users B and C Device, the terminal device of user B will transmit the captured images of user A to the terminal devices of users A and C, and the terminal device of user C will transmit the captured images of user A to the terminals of users B and A Device, so that the terminal device that executes the method disclosed in the embodiment of the present application can obtain the photographed image of each user.

In another manner, the real-time acquisition of a photographed image of the user includes:

Acquiring the photographed video transmitted by the first terminal device on the user side in real time, and each frame of images constituting the photographed video includes the portrait of the user;

According to a portrait segmentation algorithm, each frame of images constituting the captured video is segmented into a portrait and a background, and a captured image including the portrait is obtained.

In this manner, the first terminal device is used to transmit the captured video in real time to the terminal device that executes the method disclosed in the embodiment of the present application. Then, the terminal device can perform portrait and background image on each frame of the captured video. Divide to obtain captured images. Wherein, the portrait segmentation algorithm is a traditional portrait segmentation algorithm, for example, it may be a portrait segmentation algorithm based on a convolutional neural network, etc., which is not limited in this application embodiment.

In addition, in a feasible example, referring to the schematic diagram of the application scenario shown in FIG. 6, in FIG. 6, the terminal device 100 and the first terminal device 200 that execute the method disclosed in the embodiment of the present application are included, and further include the respective The cloud server 300 connected to the terminal device 100 and the first terminal device 200. When the terminal device 100 receives the captured image or the captured video transmitted by the first terminal device 200, it can be forwarded through the cloud server 300. In this case, the first terminal device 200 transmits the captured image or captured video to the cloud server 300, and the cloud server 300 transmits the captured image or captured video to the terminal device 100 that executes the method disclosed in the embodiment of the present application.

Among them, the first terminal device 200 can transmit the captured image or the captured video to the cloud server 300 through a standard multi-channel video call transmission method. Of course, the first terminal device 200 may also use other methods to transmit to the cloud server 300, which is not limited in the embodiment of the present application. Then, the cloud server 300 transmits the captured image or the captured video to the terminal device 100 that executes the method disclosed in the embodiment of the present application.

The cloud server 300 may be a server of the game that the user is playing, or may also be a server of call software (such as WeChat and QQ instant messaging software). The cloud server 300 can be connected to the first server through a network connection. The terminal device 200 and the terminal device 100 that executes the method disclosed in the embodiments of the present application function as a relay during the transmission of the captured image or the captured video, and realize the transmission of the captured image or the captured video.

Through the above two methods, a photographed image can be acquired, wherein the photographed image includes a portrait of the user. In this case, before the sideways processing is performed on the portrait in the captured image, the following steps are further included:

Perform beautification processing on the captured image according to the received beautification setting parameters, so as to perform sideways processing on the captured image after the beautification processing.

Before performing sideways processing on the captured image, you can also perform beauty processing on the captured image. During the beautification processing, the beautification processing can be performed according to the received beautification setting parameters. The beauty setting parameters may include the degree of whitening, the degree of eye enlargement, and the degree of face-lifting. Among them, the user can control the terminal device to make the terminal device receive the type and specific value of the beauty setting parameter.

In this case, the terminal device can perform sideways processing on the captured images after the beautification processing, so that the portrait after the sideways processing is more beautiful.

In addition, in step S12 of the embodiment of the present application, it is disclosed that according to the position of the virtual character corresponding to the user in the game interface, sideways processing is performed on the portrait in the captured image, and the sideways image including the sideways processed portrait is obtained. Technical characteristics. The sideways processing can be implemented by a traditional 2.5D algorithm, of course, it can also be implemented by other methods, which is not limited in the embodiment of the present application.

Sideways processing refers to determining the angle of the virtual character corresponding to the user during the game according to the position of the virtual character corresponding to the user in the game interface, and processing the portrait in the captured image according to this angle. In this case, the angle presented by the sideways processed portrait is the angle that the virtual character corresponding to the user needs to present.

For example, during the game, if the angle of the virtual character corresponding to a certain user is frontal and the portrait in the captured image is frontal, the portrait in the captured image can be treated as a sideways portrait; in addition, if a certain The angle that the virtual character corresponding to a user needs to present is the left side, and the portrait in the captured image is the front, then after the sideways processing, the left side portrait is presented as the sideways processed portrait. In this case, the portrait after the sideways treatment can be as shown in Figure 7.

In one of the methods, when performing sideways processing, the captured image can be copied in a quadrilateral grid, and then the quadrilateral grid can be processed to deform the quadrilateral grid to a certain degree. The quadrilateral grid appears slightly sideways.

Specifically, when the effect of the left body needs to be reflected, the left side of the quadrilateral grid can be compressed to make the left side shorter, and the right side of the quadrilateral grid can be stretched to make the right side change Long, which reflects the effect of slightly to the left side. When the effect of the right side body needs to be reflected, the right side of the quadrilateral mesh can be compressed to make the right side shorter, and the left side of the quadrilateral mesh can be stretched to make the left side longer. It shows the effect of turning sideways slightly.

In another way, the skeletal model corresponding to different sideways angles can be set in the terminal device in advance, and the user’s makeup model can be created through the obtained multiple frames of captured images. The makeup model is used to reflect the user’s current Makeup and hair during the game. In this case, in the process of sideways processing, the corresponding bone model is determined by the angle of the sideways, and then, according to the makeup model, the corresponding makeup content is filled on the bone model to obtain the sideways processed Portrait.

Specifically, when the effect of the left body needs to be reflected, the skeleton model corresponding to the left body can be obtained, and then the corresponding makeup and hair are filled on the skeleton model corresponding to the left body, so as to obtain the portrait of the left body.

In addition, if the above-mentioned method is used to obtain the sideways image, when the shooting angle of the multi-frame captured image is relatively single, it may cause the corresponding bone model to be filled with makeup, and there is no proper makeup on some areas of the bone model. The content is filled.

In this case, in the embodiment of the present application, after the user's makeup model is created based on the multi-frame captured images, the user can also access the server to compare various makeup types provided by the server with the makeup model. By comparison, the makeup and hair type most suitable for the makeup and hair model is determined, and the makeup and hair model is further improved accordingly.

For example, when you need to reflect the effect of the left body, you can obtain the bone model corresponding to the left body. In this case, if you fill in makeup on the bone model corresponding to the left body, it is determined that the area of the partial bone model has no corresponding The content of the corresponding makeup model. In this case, by accessing the server, the makeup model and the various types of makeup provided by the server are compared with the makeup model. If the ponytail hairstyle provided by the server is compared with the makeup model If the hair model has the highest similarity, it is determined that the hair makeup model is a ponytail. In this case, ponytail hairstyles can be filled when filling makeup on the bone model corresponding to the left body.

Further, in step S13, the operation of obtaining a virtual scene interface superimposed with the side image and the game interface is disclosed, and this operation can be implemented in various ways.

In one of the ways, the acquiring a virtual scene interface superimposed with the side image and the game interface includes the following steps:

In the first step, the number of pixels occupied by the side image is adjusted according to the number of pixels occupied by the virtual character included in the game interface, and the number of pixels occupied by the virtual character included in the game interface is adjusted. The coordinates in the game interface are adjusted to the position of the side processed portrait included in the side profile image, so that the portrait included in the adjusted side profile image can cover the virtual character included in the game interface .

If the sideways processed portrait included in the sideways image occupies a small number of pixels, the sideways processed portrait will be small and cannot cover the virtual character displayed in the game interface. In this step, the number of pixels occupied by the sideways image is adjusted to enlarge the sideways processed portrait, so that the sideways processed portrait included in the sideways image can cover the virtual character displayed in the game interface.

If the position occupied by the sideways processed portrait included in the sideways image is different from the position occupied by the virtual character included in the game interface, after direct superimposition, the sideways processed portrait and the virtual character are often not in the same display area. Cover the virtual characters displayed in the game interface. Therefore, in this step, according to the coordinates of the virtual character included in the game interface in the game interface, the position of the sideways processed portrait included in the sideways image can be adjusted to adjust The sideways processed portrait included in the sideways image can cover the virtual character displayed in the game interface.

The sideways processed portrait included in the sideways image occupies a large number of pixels and often occupies a larger position in the game interface, which is not conducive to the user's viewing. Therefore, in the process of adjusting the number of pixels occupied by the sideways image, the magnification threshold can be set in advance so that the sideways image with the adjusted number of pixels does not occupy too many pixels.

For example, the magnification threshold can be set to 3 pixels. In this case, after adjusting the number of pixels occupied by the sideways image, the number of pixels per row of the sideways processed portrait included in the sideways image is greater than that of the game. The virtual character in the interface occupies 3 pixels more pixels, so that the sideways processed portrait can cover the virtual character, and the sideways processed portrait can avoid occupying more pixels and affect the user to view other screens.

The second step is to adjust pixels in other regions of the sideways image to a transparent state, where the other regions are the portrait regions of the sideways image that do not include the user.

In order to prevent other areas of the sideways image from interfering with the user's line of sight, in this embodiment of the present application, the pixels in other areas of the sideways image are adjusted to a transparent state.

The third step is to set the adjusted side image on the upper layer of the game interface.

The terminal device that executes the method disclosed in the embodiment of the present application can present display screens of different levels in the same view display area. In this case, the adjusted side image is a one-level display screen, and the game interface is a display screen located below the adjusted side image. In this case, the user sees these two images. Display the picture that appears after the picture is superimposed.

In another manner, the acquiring a virtual scene interface superimposed with the sideways image and the game interface includes:

The first step is to perform matting processing on the sideways image to obtain the sideways processed portrait included in the sideways image, where the sideways processed portrait is a sideways portrait;

The second step is to adjust the number of pixels occupied by the side profile portrait according to the number of pixels occupied by the virtual character included in the game interface, and adjust the number of pixels occupied by the virtual character included in the game interface. For the coordinates in the game interface, the adjusted side profile portrait is set on the upper layer of the game interface, so that the adjusted side profile portrait can cover the virtual characters included in the game interface.

In this method, cutout processing is performed on the sideways image, and the sideways portrait after the cutout processing is superimposed on the position of the virtual character on the game interface.

In addition, if the number of pixels occupied by the sideways portrait is small, the sideways portrait often cannot cover the game interface

Virtual characters displayed. In this step, the number of pixels occupied by the sideways portrait is adjusted to enlarge the sideways portrait, so that the sideways portrait can cover the virtual characters displayed in the game interface.

If the sideways portrait occupies a large number of pixels, it will often occupy a larger position in the game interface, which is not conducive to users' viewing. Therefore, in the process of adjusting the number of pixels occupied by the sideways portrait, the magnification threshold can be preset so that the sideways portrait after adjusting the number of pixels does not occupy too many pixels.

For example, the magnification threshold can be set to 3 pixels. In this case, after adjusting the number of pixels occupied by the side profile portrait, the number of pixels in each row of the side profile portrait is greater than that occupied by the virtual character in the game interface. There are 3 more pixels, so that the side profile portrait can cover the virtual character, and avoid the side profile portrait occupying more pixels, which will affect the user to view other pictures.

Correspondingly, in another embodiment of the present application, a virtual scene generation device is disclosed. Referring to the schematic structural diagram shown in FIG. 8, the device includes: an image acquisition module 110, a sideways processing module 120, and an interface overlay module 130.

Wherein, the image acquisition module 110 is configured to acquire a captured image of a user in real time, and the captured image includes a portrait of the user;

The sideways processing module 120 is configured to perform sideways processing on the portrait in the captured image according to the position of the virtual character corresponding to the user in the game interface, and obtain the sideways image including the sideways processed portrait, where all The position of the virtual character in the game interface is the target position;

The interface overlay module 130 is configured to obtain a virtual scene interface superimposed with the side image and the game interface, and display the virtual scene interface, wherein the side image is located on the upper layer of the game interface, and, The sideways processed portrait included in the sideways image is located directly above the target position.

When the user views the virtual scene interface through the solution of the embodiment of the present application, what he sees is the portrait processed sideways instead of the image of the virtual character provided by the game interface. In this case, even if the user image in the game interface remains fixed, the user can view the sideways image after processing the user’s portrait sideways through the virtual scene interface, so that the user can watch the changed virtual scene. The character solves the problem in the prior art that the virtual character in the game interface remains fixed during the game, thereby improving the user's game experience.

In specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, and the program may include some or all of the steps in each embodiment of the calling method provided by the present invention when the program is executed. The storage medium may be a magnetic disk, an optical disc, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM), etc.

Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention can be embodied in the form of software products, which can be stored in a storage medium, such as ROM/RAM. , Magnetic disks, optical disks, etc., including a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of the present invention.

The same or similar parts in the various embodiments in this specification can be referred to each other. In particular, as for the embodiment of..., since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the description in the method embodiment.

The embodiments of the present invention described above do not constitute a limitation on the protection scope of the present invention.

Claims (9)

A display device, characterized by comprising: The display is configured to present a game application related interface; A camera configured to obtain a photographed image of a user, the photographed image including a portrait of the user; The controller is configured as: According to the position of the virtual character corresponding to the user in the game interface, perform sideways processing on the portrait in the captured image to obtain a sideways image including the sideways processed portrait, wherein the virtual character is in the game interface Is the target position; Acquire a virtual scene interface superimposed with the side image and the game interface, and control the display to present the virtual scene interface, wherein the side image is located on the upper layer of the virtual character in the game interface. The display device according to claim 1, wherein the camera acquiring a photographed image of the user comprises: Acquiring a photographed image transmitted by the first terminal device on the user side; or, Acquiring the photographed video transmitted by the first terminal device on the user side in real time, and each frame of images constituting the photographed video includes the portrait of the user; The display device according to claim 2, wherein the controller is further configured to: According to a portrait segmentation algorithm, each frame of images constituting the captured video is segmented into a portrait and a background, and a captured image including the portrait is obtained. The display device according to claim 1, wherein before the controller performs sideways processing on the portrait in the captured image, the controller further comprises: Performing beautification processing on the captured image according to the received beautification setting parameters, so as to perform sideways processing on the captured image after the beautification processing. The display device according to claim 1, wherein the controller executes acquiring a virtual scene interface superimposed with the side image and the game interface, which specifically includes: According to the number of pixels occupied by the virtual character included in the game interface, the number of pixels occupied by the side image is adjusted, and the number of pixels occupied by the virtual character included in the game interface is adjusted in the game interface. Adjust the position of the sideways processed portrait included in the sideways image, so that the portrait included in the adjusted sideways image can cover the virtual characters included in the game interface; Adjusting pixels in other regions of the sideways image to a transparent state, where the other regions are areas in the sideways image that do not include a portrait of the user; The adjusted side image is set on the upper layer of the game interface. The method according to claim 1, wherein the controller executes acquiring a virtual scene interface superimposed with the side image and the game interface, which specifically comprises: Performing matting processing on the sideways image to obtain the sideways processed portrait included in the sideways image, wherein the sideways processed portrait is a sideways portrait; According to the number of pixels occupied by the virtual character included in the game interface, the number of pixels occupied by the side profile portrait is adjusted, and according to the virtual character included in the game interface in the game interface The adjusted side profile portrait is set on the upper layer of the game interface, so that the adjusted side profile portrait can cover the virtual characters included in the game interface. A method for generating a virtual scene, characterized in that it comprises: Acquiring a photographed image of the user in real time, the photographed image including a portrait of the user; According to the position of the virtual character corresponding to the user in the game interface, perform sideways processing on the portrait in the captured image to obtain a sideways image including the sideways processed portrait, wherein the virtual character is in the game interface Is the target position; Obtain a virtual scene interface superimposed with the sideways image and the game interface, and display the virtual scene interface, wherein the sideways image is located on the upper layer of the game interface, and the sideways processing included in the sideways image The rear portrait is located directly above the target location. A virtual scene generating device is characterized in that it comprises: An image acquisition module, configured to acquire a photographed image of a user in real time, and the photographed image includes a portrait of the user; The sideways processing module is configured to perform sideways processing on the portrait in the captured image according to the position of the virtual character corresponding to the user in the game interface, and obtain a sideways image including the sideways processed portrait, wherein the virtual character The position in the game interface is the target position; The interface overlay module is used to obtain a virtual scene interface superimposed with the sideways image and the game interface, and display the virtual scene interface, wherein the sideways image is located on the upper layer of the game interface, and the sideways The sideways processed portrait included in the image is located directly above the target position. A display device, characterized by comprising: The display is configured to present a game application related interface; A camera configured to obtain a photographed image of a user, the photographed image including a portrait of the user; The controller is configured as: Acquiring the portrait-related part in the captured image according to the position of the virtual character corresponding to the user in the game interface, wherein the position of the virtual character in the game interface is the target position; Controlling the display to display the portrait-related part floating on the target position of the game interface.

Images