JP7696941B2 - Image processing method and device, electronic device and readable storage medium - Google Patents

Description

The present invention relates to the technical field of image processing, and in particular to an image processing method and device, an electronic device, and a readable storage medium.

Cameras on electronic devices such as smartphones include a rear camera that is installed behind the display for taking pictures, and a front camera that is installed on the same side as the display for taking selfies. Since the front camera is installed on the same side as the display, it is necessary to avoid the display, which makes it difficult to realize a full-screen design in which the entire surface of the electronic device is a display. In order to realize a full-screen design for electronic devices, a bezel-type design in which the front camera is installed on the outer bezel of the display can be adopted, but the position of the camera makes it difficult to operate. There are also notch-type, dot-drop-type, and punch-hole-type designs in which a hole is made in part of the display to install the front camera, but a good full-screen design cannot be guaranteed. In addition, a pop-up-type design in which the front camera is stored in the side wall of the electronic device and pops out when taking a picture also has the problem of poor reliability.

To further realize the full-screen design of electronic devices, the design of placing the front camera under the display is increasingly being applied. The transmittance of the display directly above the front camera is increased to ensure the amount of light required for shooting, but increasing the transmittance of the display reduces the display contrast, leading to a decrease in display quality. In addition, there are problems with the images captured by the under-screen camera, such as low image quality due to the diffraction effect.

At least one embodiment of the present invention provides an image processing method and device, an electronic device, and a readable storage medium that can improve the quality of images captured by an underscreen camera.

In order to solve the above-mentioned technical problems, the present invention is realized as follows.

In a first aspect, an embodiment of the present invention provides an image processing method applied to an electronic device having an underscreen camera installed under a display. The method includes acquiring initial images captured by the underscreen camera at multiple different angles of view, determining first image information in which the multiple initial images are affected by the diffraction phenomenon based on difference information between the multiple initial images, and performing complementation processing on the first image information based on second image information in which the multiple initial images are not affected by the diffraction phenomenon to obtain a target image.

In a second aspect, an embodiment of the present invention provides an image processing device applied to an electronic device having an underscreen camera installed under a display. The device includes an acquisition module for acquiring initial images captured by the underscreen camera at multiple different angles of view, a first processing module for determining first image information in which the multiple initial images are affected by the diffraction phenomenon based on difference information between the multiple initial images, and a second processing module for performing complementation processing on the first image information based on second image information in which the multiple initial images are not affected by the diffraction phenomenon to obtain a target image.

In a third aspect, an embodiment of the present invention provides an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable by the processor, the program or instructions being executed by the processor to realize the steps of the image processing method described in the first aspect above.

In a fourth aspect, an embodiment of the present invention provides a readable storage medium having stored thereon a program or instructions which, when executed by a processor, cause the steps of the image processing method described in the first aspect above to be realized.

Compared to the conventional technology, the image processing method and device, electronic device, and readable storage medium according to the embodiments of the present invention, in the case of an electronic device having an underscreen camera installed under a display, acquire initial images captured by the underscreen camera at multiple different angles of view, determine first image information in which the multiple initial images are affected by the diffraction phenomenon based on difference information between the multiple initial images, and perform complementation processing on the first image information based on second image information in which the multiple initial images are not affected by the diffraction phenomenon to obtain a target image, thereby ensuring the quality of the image captured by the underscreen camera and reducing the impact of the diffraction phenomenon on the image quality, and avoiding the problem of a reduced display effect due to increasing the transmittance of the display and improving the quality of the image captured by the underscreen camera.

1 is a first flowchart of an image processing method according to an embodiment of the present invention. 1 is a schematic diagram showing an under-screen camera installed under a display according to an embodiment of the present invention; FIG. 1 is a schematic diagram of a device for changing the angle of view of an underscreen camera according to an embodiment of the present invention. 1 is a schematic diagram of a driving device for an ultrasonic motor according to an embodiment of the present invention. 4 is a flowchart of the movement driving of the under-screen camera by the ultrasonic motor according to the embodiment of the present invention. 1 is a schematic diagram of a stepping motor driving device according to an embodiment of the present invention. 4 is a flowchart of the movement driving of the under-screen camera by the stepping motor according to the embodiment of the present invention. 2 is a second flowchart of the image processing method according to the embodiment of the present invention. 3 is a third flowchart of the image processing method according to the embodiment of the present invention. 4A and 4B are schematic diagrams of different measurement frames within an imaging region according to an embodiment of the present invention; FIG. 2 is a schematic diagram of a contrast measurement according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a device for changing the angle of view of an underscreen camera according to an embodiment of the present invention. 4 is a fourth flowchart of the image processing method according to the embodiment of the present invention. 5 is a fifth flowchart of the image processing method according to the embodiment of the present invention. 1 is a block diagram of an image processing apparatus according to an embodiment of the present invention; FIG. 2 is a hardware configuration diagram of an electronic device according to an embodiment of the present invention.

The technical means in the embodiments of the present application are described below clearly and completely with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, and are not all of them. All other embodiments that can be made by a person skilled in the art based on the embodiments of the present application without any creative work are all within the scope of protection of the present application.

The terms "first", "second", etc. in the specification and claims of this application are used to distinguish between similar goals, and not necessarily to describe a particular order or priority. It should be understood that the data used in this manner may be interchanged so that the embodiments of this application can be practiced in orders other than those shown or described herein, and that the goals distinguished by the terms "first", "second", etc. are generally of the same type and do not limit the number of goals. For example, the first goal may be one or more. Note that "and/or" in the specification and claims represents at least one of the connected goals. The character "/" generally indicates that the related goals before and after are in an "or" relationship.

To further realize a full-screen design for electronic devices, an under-screen camera has been proposed, in which a front camera is placed under the display and selfies are taken through the display. Research into under-screen cameras has shown that if it is possible to increase the transmittance of the display directly above the front camera to ensure the amount of light required for shooting while also ensuring sufficient display contrast when the display is in use, it will be possible to realize electronic devices with a full-screen design without issues such as cost, thickness, and reliability.

Since the camera takes pictures through a display, it is necessary to increase the transmittance of the display where the camera is installed. However, increasing the transmittance of the display reduces the contrast of the display and the display quality. There is a limit to how much the transmittance of the display can be increased, as this can cause the display to lose uniformity with other parts. Therefore, the disadvantages of under-screen cameras include the following issues of image quality degradation: (1) Decreased sensitivity due to a decrease in the amount of light transmitted through the display. (2) Degraded resolution due to the diffraction effect of the display pixels. (3) Occurrence of flare and ghosting due to light scattering in the display component layers. (4) Occurrence of color shifts, artifacts, false colors, etc. due to the wavelength dependency of the display transmittance and phase changes in the component layers. (5) Image distortion due to non-uniformity of the display component layers.

To address these issues, the embodiments of the present application provide an image processing method and device, an electronic device, and a readable storage medium to avoid transmittance loss in the display and improve the quality of images captured by an under-screen camera.

As shown in FIG. 1, an embodiment of the present invention provides an image processing method applied to an electronic device having an under-screen camera installed under a display. The method specifically includes the following steps:

In step 11, the underscreen camera acquires initial images taken at multiple different angles of view.

Optionally, the underscreen camera can move under the display of the electronic device. For example, the direction of movement of the underscreen camera under the display includes one dimension (X or Y direction) or two dimensions (X and Y directions). Here, when the underscreen camera moves to a different position under the display, the angle of view of the captured image is different. Therefore, the underscreen camera can obtain initial images captured at multiple different angles of view.

In step 12, first image information in which the influence of the diffraction phenomenon occurs in the multiple initial images is determined based on difference information between the multiple initial images.

Optionally, the difference information indicates a difference between a plurality of initial images, such as difference information between a plurality of the initial images obtained by a registration process. Specifically, before step 12, the method includes performing registration processing on the plurality of the initial images based on feature point information for each of the initial images and movement amount information between different initial images, and determining difference information between the plurality of the initial images based on the plurality of the initial images after the registration processing. Here, the movement amount information is movement amount information of the underscreen camera when capturing different initial images.

Optionally, after obtaining difference information between the multiple initial images, the difference in the occurrence state of the diffraction phenomenon can be detected by the extracted filter coefficient, and only image information affected by the diffraction phenomenon can be extracted. Specifically, step 12 includes determining first image information in which the multiple initial images are affected by the diffraction phenomenon based on the difference information between the multiple initial images and the filter coefficient. Here, the filter coefficient is determined based on a difference level associated with the exposure information of the underscreen camera. For example, a corresponding filter coefficient is extracted based on a difference level linked to the exposure information of the underscreen camera.

In step 13, a target image is obtained by performing an interpolation process on the first image information based on second image information in which the multiple initial images are not affected by the diffraction phenomenon.

Optionally, step 13 includes determining, based on a target image area corresponding to the first image information, second image information in which the influence of the diffraction phenomenon does not occur in the multiple initial images corresponding to the target image area, and performing an interpolation process on the first image information based on the second image information to obtain the target image.

In the above aspect, in the case of an electronic device having an underscreen camera installed under a display, the underscreen camera acquires initial images captured at multiple different angles of view, and based on difference information between the multiple initial images, first image information in which the multiple initial images are affected by the diffraction phenomenon is determined, and based on second image information in which the multiple initial images are not affected by the diffraction phenomenon, a complement process is performed on the first image information to obtain a target image, thereby ensuring the quality of the image captured by the underscreen camera and reducing the effect of the diffraction phenomenon on the image quality, and avoiding the problem of a reduced display effect due to increasing the transmittance of the display and improving the quality of the image captured by the underscreen camera.

Selectably, acquiring initial images captured by the underscreen camera at multiple different angles of view includes acquiring initial images captured when the underscreen camera moves to multiple different positions along a first direction, acquiring initial images captured at maximum contrast when the underscreen camera moves to multiple different positions along the first direction, acquiring initial images captured when the underscreen camera moves to multiple different positions along the first direction and the second direction, acquiring initial images captured at maximum contrast when the underscreen camera moves to multiple different positions along the first direction and the second direction.

Here, the first direction and the second direction are different directions parallel to the display. For example, the first direction and the second direction are directions perpendicular to each other, such that the first direction is one of the X and Y directions and the second direction is the other of the X and Y directions.

Optionally, the underscreen camera moving to a plurality of different positions includes the underscreen camera moving to a plurality of different positions by a fixed amount of movement, or the underscreen camera moving to a plurality of different positions by an amount of movement determined by contrast. For example, the underscreen camera moves by a fixed amount of movement in a direction parallel to the display to obtain images with different angles of view. Here, the fixed amount of movement is set by the system or is defined by the user, and is not limited thereto in the embodiment of the present application. Also, for example, the contrast component is measured to obtain the difference in the diffraction effect depending on the position of the underscreen camera, and the amount of movement of the underscreen camera can be set based on the contrast component to obtain images with different angles of view.
Example 1

In the embodiment of the present application, the electronic device is a smartphone, and a front under-screen camera is installed under the display, and multiple cameras with different light receiving wavelengths are used. As shown in FIG. 2, a high-transmittance area 210 with increased light transmittance is provided above the under-screen camera 22 on the display 21. In this way, the camera is placed under the high-transmittance area of the display, and captures an image by receiving light that has passed through the high-transmittance area of the display.

The underscreen camera can be selectively moved within the highly transparent area below the display. FIG. 3 shows a device for changing the angle of view of the underscreen camera. The underscreen camera 22 can move (e.g., in the horizontal direction X) relative to the planar space in which the display exists, and can capture images with different camera angles of view. For example, the arrow X0 indicates the horizontal change direction of the angle of view.

Specifically, one implementation method is as follows. FIG. 4 shows a drive device for an ultrasonic motor. An ultrasonic motor 41 for driving the movement of the underscreen camera 22 is arranged on one side horizontal to the movement direction of the underscreen camera 22, and a guide 42 supporting the underscreen camera 22 and a position detection magnet 43 for detecting the position (amount of movement) are arranged on the opposite side. A magnetic sensor 44 is arranged on the same horizontal plane as the position detection magnet 43 of the underscreen camera 22. FIG. 5 shows a flowchart of the movement driving of the underscreen camera by an ultrasonic motor. Specifically, the magnetic field of the position detection magnet 43 is detected by the magnetic sensor 44 to calculate the position (amount of movement) by feedback control, and the underscreen camera 22 is driven to move by the ultrasonic motor 41.

Another implementation method is as follows. Figure 6 shows a stepping motor drive device. A stepping motor 61 for driving is arranged on one side horizontal to the movement direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 is arranged on the opposite side. Figure 7 shows a flowchart of the movement drive of the under-screen camera by the stepping motor. Specifically, the position (movement amount) is calculated by feedforward control by monitoring the pulses generated by the stepping motor 61 for driving, and the under-screen camera 22 is driven to move by the stepping motor 61.

Figure 8 shows the flow of image processing to remove the effects of diffraction and perform image fusion using output images with different angles of view to solve the problem of resolution degradation caused by diffraction.

Specifically, the method includes obtaining initial images captured when the underscreen camera moves to a plurality of different positions along a first direction, determining first image information in which the initial images are affected by the diffraction phenomenon based on difference information between the initial images, and performing complementation processing on the first image information based on second image information in which the initial images are not affected by the diffraction phenomenon to obtain a target image.

For example, the underscreen camera is moved a certain amount to capture multiple image data with different angles of view, and alignment processing is performed based on the movement amount information for capturing the different image data and the feature point information for each image data. The aligned image data are compared to generate difference information. Next, by setting a difference level extraction filter coefficient linked to the exposure information of the underscreen camera, it is possible to accurately detect differences in the occurrence state of the diffraction phenomenon and extract only the image information affected by diffraction. Next, the parts of the image information affected by diffraction obtained here are complemented and output from image data that is not affected by diffraction, thereby obtaining an image from which the effects of diffraction have been removed, thereby improving the quality of the images output by the underscreen camera.

In this embodiment, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted by taking into account the exposure information of the underscreen camera. An image in which the influence of diffraction is removed can be obtained by complementing the part where the diffraction effect occurs from image data where the diffraction effect does not occur. The sensitivity can be improved by superimposing multiple images.
Example 2

In the embodiment of the present application, the electronic device is a smartphone, and a front under-screen camera is installed under the display, and multiple cameras with different light receiving wavelengths are used. Continuing to refer to FIG. 2, a high-transmittance area 210 with increased light transmittance is provided above the under-screen camera 22 on the display 21. In this way, the camera is placed under the high-transmittance area of the display, and captures an image by receiving light that has passed through the high-transmittance area of the display.

The underscreen camera can be selectively moved within the highly transparent area below the display. FIG. 3 shows a device for changing the angle of view of the underscreen camera. The underscreen camera 22 can move (e.g., in the horizontal direction X) relative to the planar space in which the display exists, and can capture images with different camera angles of view. For example, the arrow X0 indicates the direction in which the angle of view changes.

Specifically, one implementation method is as follows. FIG. 4 shows a drive device for an ultrasonic motor. An ultrasonic motor 41 for driving the movement of the underscreen camera 22 is arranged on one side horizontal to the movement direction of the underscreen camera 22, and a guide 42 supporting the underscreen camera 22 and a position detection magnet 43 for detecting the position (amount of movement) are arranged on the opposite side. A magnetic sensor 44 is arranged on the same horizontal plane as the position detection magnet 43 of the underscreen camera 22. FIG. 5 shows a flowchart of the movement driving of the underscreen camera by an ultrasonic motor. Specifically, the magnetic field of the position detection magnet 43 is detected by the magnetic sensor 44 to calculate the position (amount of movement) by feedback control, and the underscreen camera 22 is driven to move by the ultrasonic motor 41.

Another implementation method is as follows. Figure 6 shows a stepping motor drive device. A stepping motor 61 for driving is arranged on one side horizontal to the movement direction of the under-screen camera 22, and a guide 42 for supporting the under-screen camera 22 is arranged on the opposite side. Figure 7 shows a flowchart of the movement drive of the under-screen camera by the stepping motor. Specifically, the position (movement amount) is calculated by feedforward control by monitoring the pulses generated by the stepping motor 61 for driving, and the under-screen camera 22 is driven to move by the stepping motor 61.

Figure 9 shows the flow of image fusion image processing and the removal of diffraction effects using output images with different angles of view to solve the problem of resolution degradation caused by diffraction.

Specifically, the method includes obtaining initial images captured at the highest contrast when the underscreen camera moves to multiple different positions along the first direction, determining first image information in which the initial images are affected by the diffraction phenomenon based on difference information between the initial images, and performing complementation processing on the first image information based on second image information in which the initial images are not affected by the diffraction phenomenon to obtain a target image.

For example, when the underscreen camera moves to a plurality of different positions along the first direction, obtaining initial images each captured at the highest contrast specifically includes obtaining an optimal image for synthesis processing based on measuring the difference between the camera position and the diffraction effect by different arbitrary measurement frames in the imaging area. Referring to FIG. 9, the difference in the diffraction effect due to the underscreen camera position can be obtained by measuring the contrast component. Referring to FIG. 10, the underscreen camera is moved to the position with the highest contrast in each of different arbitrary measurement frames A, B, and C in the imaging area 101, and an image is obtained by capturing the image. The positions and number of measurement frames for measuring the contrast component at this time can be set arbitrarily. For example, as shown in FIG. 11, the underscreen camera is moved to the corresponding positions according to the positions of the set plurality of measurement frames, and then the peak value of the contrast is measured according to the contrast of the corresponding measurement frame and captured.

After capturing an optimal image for image processing from the contrast components, multiple image data with different angles of view are aligned based on the movement amount information and feature point information of each image data. The aligned image data are compared to generate difference information. Next, by setting a difference level extraction filter coefficient linked to the exposure information of the underscreen camera, it is possible to accurately detect differences in the occurrence state of the diffraction phenomenon and extract only the information affected by diffraction. Furthermore, by complementing and outputting the parts of the image information affected by diffraction obtained here from image data that is not affected by diffraction, an image with the effects of diffraction removed is obtained, improving the quality of the images output by the underscreen camera.

In this embodiment, by taking into account the exposure information of the underscreen camera, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the part where the diffraction effect occurs with image data where it does not occur, it is possible to obtain an image where the influence of diffraction is removed. By superimposing multiple images, it is possible to improve sensitivity. By controlling the contrast, it is possible to obtain an image with the diffraction effect that is optimal for image processing, and high-performance correction of resolution degradation and removal of flare and ghosting can be realized.
Example 3

In the embodiment of the present application, the electronic device is a smartphone, and a front under-screen camera is installed under the display, and multiple cameras with different light receiving wavelengths are used. Continuing to refer to FIG. 2, a high-transmittance area 210 with increased light transmittance is provided above the under-screen camera 22 on the display 21. In this way, the camera is placed under the high-transmittance area of the display, and captures an image by receiving light that has passed through the high-transmittance area of the display.

The underscreen camera can be selectively moved within a highly transparent area below the display. FIG. 12 shows a device for changing the angle of view of the underscreen camera. The underscreen camera moves (e.g., in the horizontal direction X and the vertical direction Y) relative to the planar space in which the display exists, making it possible to capture images with different angles of view of the underscreen camera. For example, arrow X0 indicates the horizontal change direction of the angle of view, and arrow Y0 indicates the vertical change direction of the angle of view.

Figure 13 shows the flow of image fusion image processing and the removal of diffraction effects using output images with different angles of view to solve the problem of resolution degradation caused by diffraction.

Specifically, the method includes obtaining initial images captured when the underscreen camera moves to a plurality of different positions along a first direction and a second direction, determining first image information in which the initial images are affected by the diffraction phenomenon based on difference information between the initial images, and performing complementation processing on the first image information based on second image information in which the initial images are not affected by the diffraction phenomenon to obtain a target image.

For example, the underscreen camera is moved a certain amount on each of the X and Y axes to capture multiple image data with different angles of view, and the multiple image data with different angles of view are aligned based on the movement amount information and feature point information of each image. The aligned image data are compared to generate difference information. Next, by setting a difference level extraction filter coefficient linked to the exposure information of the underscreen camera, it is possible to accurately detect differences in the occurrence state of the diffraction phenomenon and extract only the image information affected by diffraction. Next, the parts of the image information affected by diffraction obtained here are complemented and output from image data that is not affected by diffraction, thereby obtaining an image from which the effects of diffraction have been removed, thereby improving the quality of the images output by the underscreen camera.

In this embodiment, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted by taking into account the exposure information of the underscreen camera. An image in which the influence of diffraction is removed can be obtained by complementing the part where the diffraction effect occurs from image data where the diffraction effect does not occur. Sensitivity can be improved by superimposing multiple images. By acquiring an image with changes in the angle of view in two directions of the X axis and the Y axis, it is possible to extract and synthesize the analysis phenomenon with high accuracy.
Example 4

In the embodiment of the present application, the electronic device is a smartphone, and a front under-screen camera is installed under the display, and multiple cameras with different light receiving wavelengths are used. Continuing to refer to FIG. 2, a high-transmittance area 210 with increased light transmittance is provided above the under-screen camera 22 on the display 21. In this way, the camera is placed under the high-transmittance area of the display, and captures an image by receiving light that has passed through the high-transmittance area of the display.

Optionally, the underscreen camera can be moved within a highly transparent area below the display. FIG. 12 shows an apparatus for changing the angle of view of the underscreen camera. The camera can be moved (e.g., horizontally and vertically) relative to the planar space in which the display exists, allowing images to be captured with different camera angles of view.

Figure 14 shows the flow of image processing to remove the effects of diffraction and perform image fusion using output images with different angles of view to solve the problem of resolution degradation caused by diffraction.

Specifically, the method includes obtaining initial images captured at the highest contrast when the underscreen camera moves to multiple different positions along a first direction and a second direction, determining first image information in which the initial images are affected by the diffraction phenomenon based on difference information between the initial images, and performing complementation processing on the first image information based on second image information in which the initial images are not affected by the diffraction phenomenon to obtain a target image.

For example, when the underscreen camera moves to a plurality of different positions along the first direction and the second direction, obtaining initial images each captured at the highest contrast specifically includes obtaining an optimal image for synthesis processing based on measuring the difference between the camera position and the diffraction effect by different arbitrary measurement frames in the imaging area. Continuing to refer to FIG. 9, the difference in the diffraction effect due to the underscreen camera position can be obtained by measuring the contrast component. Continuing to refer to FIG. 10, the underscreen camera is moved to the position with the highest contrast in each of different arbitrary measurement frames A, B, and C in the imaging area 101, and an image is obtained by photographing. The positions (e.g., different positions moved along the first direction and the second direction) and the number of measurement frames for measuring the contrast component at this time can be set arbitrarily. For example, as shown in FIG. 11, the underscreen camera is moved to the corresponding positions according to the positions of the set plurality of measurement frames, and then the peak value of the contrast is measured according to the contrast of the corresponding measurement frame and photographed.

Then, the underscreen camera is moved along the X-axis and Y-axis, respectively, to capture an image for optimal image processing based on the contrast component, and the captured image data with multiple different angles of view are aligned based on the movement amount information and feature point information of each image data. The aligned image data are compared to generate difference information. Next, by setting a difference level extraction filter coefficient linked to the exposure information of the underscreen camera, it is possible to accurately detect differences in the occurrence state of the diffraction phenomenon and extract only the information affected by diffraction. Furthermore, by complementing and outputting the parts of the image information affected by diffraction obtained here from image data that is not affected by diffraction, an image with the effects of diffraction removed is obtained, improving the quality of the images output by the underscreen camera.

In this embodiment, by taking into account the exposure information of the camera, the influence of the diffraction effect according to the magnitude of the diffraction effect due to the exposure amount is appropriately extracted. By complementing the part where the diffraction effect occurs with image data where it does not occur, it is possible to obtain an image where the influence of diffraction has been removed. By superimposing multiple images, it is possible to improve sensitivity. By acquiring an image with changes in the angle of view in two directions, the X axis and the Y axis, it is possible to extract and synthesize the analysis phenomenon with high accuracy. By controlling the contrast, it is possible to acquire an image with the diffraction effect that is optimal for image processing, and high-performance correction of resolution degradation and removal of flare and ghosting can be realized.

As shown in FIG. 15, an embodiment of the present invention provides an image processing device 1500. The electronic device has an underscreen camera installed under a display. The device 1500 includes an acquisition module 1510 for acquiring initial images captured by the underscreen camera at multiple different angles of view, a first processing module 1520 for determining first image information in which the multiple initial images are affected by the diffraction phenomenon based on difference information between the multiple initial images, and a second processing module 1530 for performing a complementation process on the first image information based on second image information in which the multiple initial images are not affected by the diffraction phenomenon to obtain a target image.

Optionally, the acquisition module 1510 includes a first acquisition unit for acquiring initial images taken when the underscreen camera moves to a plurality of different positions along the first direction, or includes a second acquisition unit for acquiring initial images taken at the highest contrast when the underscreen camera moves to a plurality of different positions along the first direction, or includes a third acquisition unit for acquiring initial images taken when the underscreen camera moves to a plurality of different positions along the first direction and the second direction, or includes a fourth acquisition unit for acquiring initial images taken at the highest contrast when the underscreen camera moves to a plurality of different positions along the first direction and the second direction.

Optionally, the first direction and the second direction are different directions parallel to the display.

Selectably, the underscreen camera moving to a plurality of different positions includes the underscreen camera moving to a plurality of different positions each by a fixed amount of movement, or the underscreen camera moving to a plurality of different positions each by an amount of movement determined by contrast.

Optionally, the image processing device 1500 further includes a third processing module for performing alignment processing on the multiple initial images based on feature point information for each initial image and movement amount information between different initial images, and a fourth processing module for determining difference information between the multiple initial images based on the multiple initial images after the alignment processing, where the movement amount information is movement amount information of the underscreen camera when capturing the different initial images.

Optionally, the first processing module 1520 includes a first processing unit for determining first image information in which the plurality of initial images are affected by a diffraction phenomenon based on difference information between the plurality of initial images and a filter coefficient, where the filter coefficient is determined based on a difference level associated with exposure information of the underscreen camera.

Optionally, the second processing module 1530 includes a second processing unit for determining, based on a target image area corresponding to the first image information, second image information in which the influence of the diffraction phenomenon does not occur in the multiple initial images corresponding to the target image area, and a third processing unit for performing a complementation process on the first image information based on the second image information to obtain the target image.

The above-mentioned device according to an embodiment of the present invention, in the case of an electronic device having an underscreen camera installed under a display, acquires initial images captured by the underscreen camera at multiple different angles of view, determines first image information in which the initial images are affected by the diffraction phenomenon based on difference information between the multiple initial images, and performs complementation processing on the first image information based on second image information in which the initial images are not affected by the diffraction phenomenon to obtain a target image, thereby ensuring the quality of the image captured by the underscreen camera and reducing the effect of the diffraction phenomenon on the image quality, and avoiding the problem of a decrease in the display effect of the display by increasing the transmittance of the display and improving the quality of the image captured by the underscreen camera.

As shown in FIG. 16, an embodiment of the present invention further provides an electronic device 1600 including a processor 1601, a memory 1602, and a program or instruction stored in the memory 1602 and executable by the processor 1601, which, when executed by the processor 1601, realizes each process of the embodiment of the image processing method described above and can achieve the same technical effect, and therefore will not be described again here to avoid redundancy.

The electronic devices in the embodiments of this application include mobile electronic devices and non-mobile electronic devices.

Furthermore, an embodiment of the present invention provides a readable storage medium having stored thereon a program or instructions. When executed by a processor, the program or instructions can realize each process of the embodiment of the image processing method described above and achieve the same technical effect, so that they will not be described repeatedly here to avoid duplication.

Although the present invention has been described above with reference to the drawings, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are illustrative and not limiting. Many forms that a person skilled in the art can implement without departing from the spirit of the present invention and the scope of the claims are all included in the scope of protection of the present invention.

Claims (10)

An image processing method applied to an electronic device having an under-screen camera installed under a display, comprising:
Obtaining initial images taken by an underscreen camera at a plurality of different angles of view;
determining first image information in which the influence of a diffraction phenomenon occurs in the initial images based on difference information between the initial images;
and performing an interpolation process on the first image information based on second image information in which the plurality of initial images are not affected by the diffraction phenomenon to obtain a target image .
The method is characterized in that the underscreen camera is installed movably, and the initial images captured at the multiple different angles of view are acquired by moving the underscreen camera . 2. The image processing method according to claim 1,
The underscreen camera acquires initial images captured at a plurality of different angles of view,
obtaining initial images taken when the underscreen camera moves to a plurality of different positions along a first direction;
Or,
acquiring initial images at a maximum contrast as the underscreen camera moves to a plurality of different positions along the first direction;
Or,
obtaining initial images taken when the underscreen camera moves to a plurality of different positions along a first direction and a second direction;
Or,
The method includes obtaining initial images captured at maximum contrast as the underscreen camera moves to a plurality of different positions along a first direction and a second direction. 3. The image processing method according to claim 2,
The first direction and the second direction are different directions parallel to the display. 3. The image processing method according to claim 2,
The underscreen camera is moved to a plurality of different positions.
The under-screen camera is moved to a plurality of different positions by a fixed amount of movement,
Or,
The underscreen camera may be moved to a plurality of different positions with a movement amount determined by contrast. 2. The image processing method according to claim 1,
before determining first image information in which the influence of a diffraction phenomenon occurs in the plurality of initial images based on difference information between the plurality of initial images,
performing a registration process on the initial images based on feature point information for each initial image and movement amount information between different initial images;
determining difference information between the plurality of initial images based on the plurality of initial images after the registration process;
Here, the movement amount information is information about the movement amount of the underscreen camera when capturing different initial images. 2. The image processing method according to claim 1,
Determining first image information in which the influence of a diffraction phenomenon occurs in the plurality of initial images based on difference information between the plurality of initial images includes:
determining first image information in which an effect of a diffraction phenomenon occurs in the initial images based on difference information between the initial images and a filter coefficient;
Here, the filter coefficients are determined based on a difference level associated with exposure information of the underscreen camera. 2. The image processing method according to claim 1,
performing an interpolation process on the first image information based on second image information in which the influence of the diffraction phenomenon does not occur in the plurality of initial images, to obtain a target image;
determining second image information, which is not affected by the diffraction phenomenon in the plurality of initial images, based on a target image area corresponding to the first image information;
The method further comprises performing a complementation process on the first image information based on the second image information to obtain the target image. An image processing device applied to an electronic device having an under-screen camera installed under a display,
an acquisition module for acquiring initial images taken by the underscreen camera at a plurality of different angles of view;
a first processing module for determining first image information in which the influence of a diffraction phenomenon occurs in the plurality of initial images based on difference information between the plurality of initial images;
a second processing module for performing an interpolation process on the first image information based on second image information in which the plurality of initial images are not affected by the diffraction phenomenon, to obtain a target image ;
The method is characterized in that the underscreen camera is installed movably, and the initial images captured at the multiple different angles of view are acquired by moving the underscreen camera . An electronic device,
A processor, a memory, and a program or instructions stored in the memory and executable by the processor;
The program or instructions, when executed by the processor, cause the steps of the image processing method according to any one of claims 1 to 7 to be realized. 1. A readable storage medium, comprising:
The readable storage medium stores a program or instruction,
The program or instructions, when executed by a processor, cause the program or instructions to implement the steps of the image processing method according to any one of claims 1 to 7.