JP7625439B2 - Image processing device and image processing method - Google Patents

Description

The disclosed technology relates to an image processing device and an image processing method.

Patent Literature 1 discloses an image processing device that displays image data of a subject's eye acquired by an ophthalmic examination device on a report and enlarges the image data by touch panel operations (e.g., pinch-out operations, tap operations) on the image data.

JP 2018-246 A

Here, depending on the type of image data of the subject's eye and the diagnostic purpose, simply enlarging or reducing the image data may cause problems for diagnosis.

One of the objectives of the disclosed technology is to perform enlargement processing of image data suitable for the type of image data of the subject's eye, diagnostic purpose, etc.

In addition to the above objective, the present invention also has another objective to achieve effects that cannot be obtained by conventional techniques, which are derived from the configurations shown in the detailed description of the invention described below.

One of the disclosed image processing devices is
An image processing device including a display control means for displaying a tomographic image of a subject's eye obtained by using an OCT optical system on a touch panel including a detection surface capable of detecting a touch panel operation,
When two fingers are placed side by side in contact with the short side of the tomographic image displayed on the touch panel before the touch panel operation, the display control means is capable of enlarging and displaying the tomographic image in the short side direction without enlarging it in the long side direction.

One of the disclosed technologies allows for enlargement processing of image data suitable for the type of image data of the subject's eye, diagnostic purpose, etc.

1 is a diagram illustrating a schematic functional configuration of an image processing apparatus according to a first embodiment. FIG. 11 is a flowchart showing a report display according to the first embodiment. FIG. 11 is a diagram illustrating an example of a report according to the first embodiment. FIG. 11 is a diagram showing a display magnification setting of an SLO image according to the first embodiment. FIG. 11 is a diagram showing a display magnification setting of an SLO image according to the first embodiment. FIG. 11 is a diagram illustrating an example of a report according to the second embodiment. FIG. 4 is a diagram showing the setting of the display magnification of a tomographic image according to the first embodiment. 11A and 11B are diagrams illustrating rotation of report display elements according to the first embodiment. 11A and 11B are diagrams illustrating translation of a report display element according to the first embodiment. 11A and 11B are diagrams illustrating brightness and contrast adjustment of report display elements according to the first embodiment. 13 is a diagram illustrating synchronization of the enlargement and reduction of report display elements according to the second embodiment. 13 is a diagram illustrating synchronization of the enlargement and reduction of report display elements according to the second embodiment. FIG. 11 is a diagram showing settings of enlargement/reduction positions of report display elements according to the second embodiment. FIG. 13 is a diagram showing a full-screen tomographic image according to the third embodiment.

Below, the embodiments will be explained with reference to the drawings. The same or equivalent components, parts, and processes shown in each drawing will be given the same reference numerals, and duplicate explanations will be omitted as appropriate. In addition, some components, parts, and processes that are not important for the explanation may be omitted in each drawing.

Example 1
An example of the image processing device of this embodiment includes a display control means for displaying a tomographic image of the subject's eye obtained using the OCT optical system or a report related to the tomographic image on a touch panel including a detection surface capable of detecting touch panel operation. The example of the image processing device of this embodiment can enlarge and display the tomographic image in the short direction without enlarging it in the longitudinal direction in response to a touch panel operation (e.g., a pinch-out operation in the short direction) on the tomographic image displayed on the touch panel. Note that, when the touch panel operation is a pinch-out operation in the diagonal direction of the tomographic image, the example of the image processing device of this embodiment may be configured to enlarge and display the tomographic image in both the longitudinal and short directions. The example of the image processing device of this embodiment may also be configured to enlarge a part of the tomographic image and display it in the report in response to a touch panel operation on the tomographic image in the report displayed on the touch panel. In addition, the image processing device of this embodiment may be configured to display a report generated from one test result on a touch panel, and control changes to the display of the test results, such as various images (e.g., different enlargement processes are performed) depending on the type of input (e.g., differences in the designated positions) made by an operator using two fingers on the touch panel to a display element on the report.

In this embodiment, the examination results of an OCT (Optical Coherence Tomography) imaging device are used as an example of an ophthalmic imaging device. The OCT imaging device of this embodiment is capable of acquiring one or more SLO (Scanning Laser Ophthalmoscopy) images during an examination. The OCT imaging device of this embodiment includes an OCT optical system. The image processing device of this embodiment may be configured to be able to communicate with the ophthalmic imaging device via a wired or wireless connection, or may be built into the ophthalmic imaging device.

First, referring to FIG. 1, a schematic functional configuration according to this embodiment will be described. As shown in FIG. 1, the acquisition unit 100 acquires the examination results from the ophthalmic imaging device. The examination results include information such as the subject's information such as the date of birth and gender, as well as the captured image, analysis results, and diagnosis results. In this embodiment, the images captured by the ophthalmic imaging device and the images reconstructed from OCT data are referred to as examination images. The examination results may be acquired from a storage device (not shown). The generation unit 101 generates a report based on the examination images and analysis results from the examination results acquired by the acquisition unit 100. The report includes display elements such as the examination image and analysis results. The display unit 102 is a touch panel capable of simultaneous multi-point input, and the report created by the creation unit 101 is displayed by the display control means. The touch panel includes, for example, a detection surface capable of detecting touch panel operations. At this time, the detection surface is capable of detecting at least one of a change in relative distance when at least two designated positions designated on the touch panel are moved relatively to each other, and a rotation amount. The image processing device of this embodiment may be configured to include an acquisition unit 100 and a generation unit 101. The display unit 102 may be provided in the image processing device or ophthalmic imaging device of this embodiment, or may be separate from these and configured to be able to communicate with them via wired or wireless communication, etc.

Also, referring to FIG. 2, a flow chart of report display according to this embodiment will be described. First, in step S200, a patient is selected from a patient screen (not shown) in response to an instruction from the operator. In step S201, one examination result is selected in response to an instruction from the operator. Here, the operator selects an examination result relating to the macular region photographed by scanning the fundus of the subject's eye using an OCT imaging device. In step S202, a report is generated using the examination result selected in step S201. Then, in step S203, the report generated in step S202 is displayed on the display unit 102.

An example of the report generated in step S202 is shown in FIG. 3. The front image display area 300 displays an SLO image 304. The image displayed in the front image display area 300 may be a front two-dimensional image other than the SLO image. For example, it may be a fundus image or a fluorescent fundus image captured by a separate fundus camera. The tomographic image display area 301 displays a tomographic image 306 of the examination result. The tomographic image displayed in the tomographic image display area 301 may be a pseudo tomographic image generated from a three-dimensional image along an arbitrary line on the XY surface. The pseudo front image display area 302 displays a pseudo front image 305 reconstructed based on the tomographic image of the examination. The three-dimensional image display area 303 displays a three-dimensional image based on the tomographic image of the examination. The analysis results may be superimposed on each image. For example, a layer boundary may be detected from a tomographic image, a layer thickness between desired layers may be calculated, and the layer thickness may be superimposed on the SLO image 304 as a thickness map. This makes it easier for the operator to visually check the distribution of layer thicknesses. Known methods can be used to detect layer boundaries.

Also, referring to FIG. 4, the display method change of the report display element according to this embodiment will be described by taking the enlargement/reduction (hereinafter, enlargement/reduction) setting of the display of the tomographic image as an example. FIG. 4(a) shows an example in which the operator performs a pinch-out operation in the vertical direction (the short side direction of the tomographic image) on the tomographic image display area 301 of the tomographic image 306 displayed on the touch panel. At this time, the image processing device of this embodiment can enlarge the tomographic image 306 in the vertical direction (the depth direction in the tomographic image) and display it on the touch panel, as shown in FIG. 4(b). Note that the tomographic image 306 may be configured not to be enlarged in the horizontal direction (the long direction of the tomographic image). Also, the pinch-out operation can be performed, for example, by moving two fingers in contact with the touch panel so that the distance between them becomes longer. Also, the enlargement magnification changes, for example, according to the amount of movement of the fingers in the pinch-out operation. As a result, a report in which the tomographic image 306 is enlarged in the tomographic image display area 301 can be displayed, as shown in FIG. 4(c). The image to be enlarged may be displayed at a higher resolution. The operator can confirm the image displayed in a reduced size as shown in FIG. 4(a) by performing a pinch-in operation on the display image in the state shown in FIG. 4(b). The pinch-in operation can be performed, for example, by moving two fingers in contact with the touch panel so that the distance between them becomes shorter. The reduction ratio changes according to the amount of movement of the fingers in the pinch-in operation.

Here, when a tomographic image is displayed, the length per pixel in the horizontal direction (photographing range) and the vertical direction (depth direction) may not be in a 1:1 ratio. This is because the horizontal shooting range is several mm to several tens of mm, while the thickness of the retina is several hundred μm, which is a different range. To observe the state of the retina in detail in the tomographic image, it is important to enlarge the depth direction. During a pinch-in/pinch-out operation (hereinafter, pinch operation) on the tomographic image 306, both or either one of a guide indicating the midpoint of the two fingers of the pinch operation (e.g., a rectangular frame indicated by a dotted line in the figure) and a guide indicating the magnification rate according to the movement amount of the two fingers (e.g., a rectangular frame indicated by a solid line in the figure) may be displayed. That is, the display control means of this embodiment may display a guide regarding the magnification process of the tomographic image on the tomographic image when it is detected on the detection surface that the operator's fingers are in contact with the tomographic image displayed on the touch panel. In addition, the display indicating the magnification rate may be displayed not only during the pinch operation but also during the zooming. For example, the shape of the guide may change depending on the position of the two fingers, and when the two fingers are placed vertically or horizontally, the shape is a rectangle, and when they are placed diagonally, the shape is a square. That is, the display control means of this embodiment may display a guide on the tomographic image so that it is possible to distinguish that different enlargement processes are being executed depending on the relative positions of the two fingers touching the tomographic image displayed on the touch panel. This allows the operator to know whether the currently displayed image is being enlarged or reduced at the same vertical and horizontal ratios, or whether the vertical and horizontal ratios are different.

Note that the zooming in and out by pinching is not limited to the vertical direction. A pinch in the horizontal direction may zoom in and out horizontally. The direction of zooming in and out may be determined by the direction of movement of the two fingers. Specifically, for example, when the operator pinches in a direction 60 degrees from the horizontal direction, the image is zoomed in and out horizontally and vertically at a ratio of 1:√3. The direction of zooming in and out may be approximately the same as the direction of movement of the finger, but does not have to be exactly the same. For example, the detection of the direction of movement of the finger may be set in 15 degree increments. This allows the operator to zoom in and out only vertically or horizontally when the operator pinches in a generally vertical or horizontal direction, and to zoom in and out while maintaining the aspect ratio when the operator pinches in a generally diagonal direction. The operator can perform the desired zooming in and out without performing strict operations. Alternatively, a unique zooming process may be performed regardless of the direction of movement of the operator's fingers in the pinch operation. For example, the zooming process may be performed in all directions even when the operator pinches in the vertical direction, or may be performed only vertically when the operator pinches diagonally.

Note that the reduction operation is performed so that the vertical or horizontal length of the tomographic image 306 does not become shorter than the tomographic image display area 301. Specifically, if the length of any side of the reduced tomographic image 306 is longer in both the vertical and horizontal directions than the length of the corresponding side of the tomographic image display area 301, the reduction process is performed according to the operation of the operator. On the other hand, if the length of any side of the reduced tomographic image 306 is shorter than the corresponding length of the tomographic image display area 301, the reduction process is performed so that the length of the relevant side is magnified to match the length of the corresponding side of the tomographic image display area 301. This makes it possible to prevent the tomographic image 306 from being displayed smaller than the operator intends.

When the analysis results are superimposed on the tomographic image 306, the superimposed analysis results are enlarged or reduced in synchronization with the enlargement or reduction process of the tomographic image 306. For example, when the boundary lines of layers are superimposed on the tomographic image, the image processing device of this embodiment can display the tomographic image 306 while maintaining the correspondence between the boundary lines of the superimposed layers. This allows the operator to observe each layer of the tomographic image in more detail, for example, thereby improving the efficiency of diagnosis.

For example, when a wide range of the retina is photographed, the retina is curved, so unless the tomographic image is photographed widely not only in the horizontal direction but also in the vertical direction (depth direction of the retina), the entire retina may not be photographed. Here, consider a case where a tomographic image obtained by photographing with a wide angle and a large depth range is displayed in the same display area as a tomographic image obtained by photographing with a normal photographing range (a tomographic image obtained by photographing with a narrow angle and a small depth range). In this case, the tomographic image obtained by photographing with a wide angle and a large depth range is displayed in a reduced size overall compared to the tomographic image obtained by photographing with a normal photographing range. At this time, by making it possible to enlarge and display the tomographic image vertically while keeping a wide range displayed horizontally, it is possible to observe each layer in detail while observing the entire retina over a wide range. This allows the operator to easily realize the enlarged and reduced display.

In this embodiment, the tomographic image is expanded or contracted, but the present invention is not limited to this. For example, the SLO image may be expanded or contracted by a similar operation. As shown in FIG. 7(a), the SLO image may be expanded as shown in FIG. 7(b) by performing a pinch operation on the SLO image. This operation displays a report in which the SLO image 304 is expanded in the front image display area 300, as shown in FIG. 7(c). When the SLO image 304 is expanded or contracted, each display element displaying the same area as the SLO image 304 may be expanded or contracted in a synchronized manner. For example, when the periphery of the macular area of the SLO image 304 is expanded, the periphery of the macular area of the pseudo front image 305 may be expanded to the same display magnification as the SLO image 304. When a thickness map or the like is superimposed on the SLO image 304, the thickness map expanded or contracted in a synchronized manner with the SLO image 304 and each display area displaying the same area may be expanded or contracted. In addition, when the display magnifications of the SLO image 304 and the pseudo front image 305 are different, the display magnifications may be enlarged or reduced to the same magnification. For example, when the display magnification of the SLO image 304 is 100% and the display magnification of the pseudo front image 305 is 150%, if the enlargement process enlarges the image by 1.5 times, the display magnification of the SLO image 304 may be set to 150% and the display magnification of the pseudo front image 305 may be set to 225%. This makes it easier for the operator to observe a desired portion of a plurality of display elements included in the report. Note that the enlargement or reduction process by the pinch operation shown in this embodiment can be performed on display elements such as analysis results and subject information displayed in the report. For example, the display elements may be motion contrast images and OCTA images generated from tomographic images, En-Face images projected onto a plane based on any two reference planes in the depth direction from a three-dimensional image, and fundus images captured by a fundus camera. In addition, the display elements may be graphs or tables displaying comparison maps with a normal database, thickness maps, thickness profiles, and analysis results in addition to the test results obtained from a measuring instrument such as a perimeter. It may also be examination information such as the reception number and examination date, or subject information such as the subject's name, date of birth, and gender. This allows the operator to zoom in and out on any display element, improving the efficiency of diagnosis.

In addition, in this embodiment, the configuration in which the display element is enlarged or reduced by a pinch operation with two fingers is shown, but this is not limited to this. For example, a configuration in which three or more fingers are used may be used. As shown in FIG. 5(a), on the front image display area 300 displaying the SLO image 304, the operator specifies a position 500 with one finger on the touch panel and performs a pinch operation with two fingers. At this time, the image processing device of this embodiment executes a process of enlarging or reducing the SLO image 304 with the position 500 as the center. A configuration in which the display element is enlarged or reduced by a double tap with two fingers may also be used. At this time, the operator executes the enlargement or reduction by double tapping the SLO image 304 on the front image display area 300. At this time, the enlargement or reduction magnification may be constant, or may be preset in the image processing device of this embodiment. Alternatively, a configuration in which the display element is enlarged or reduced by a double tap with multiple fingers may also be used. At this time, the SLO image 304 can be enlarged or reduced to the maximum or minimum magnification by double tapping with multiple fingers. With this configuration, the operator can easily realize an operation of switching the display to the maximum enlargement magnification or the minimum reduction magnification. In this way, various operations using a finger on a touch panel are examples of touch panel operations.

In this embodiment, the display method change of the report display element is described as an example of enlargement and reduction, but it is not limited to enlargement and reduction. For example, the display element may be rotated, moved in parallel, or adjusted in brightness and contrast. First, the rotation process of the SLO image will be described as an example of the rotation process of the report display element. For example, as shown in FIG. 8(a), the SLO image 304 on the front image display area 300 may be rotated clockwise as shown in FIG. 8(c) by performing a sliding operation to draw a circle clockwise while touching the SLO image 304 on the touch panel with a finger. Note that the sliding operation and the rotation rate of the SLO image 304 do not necessarily need to be synchronized. For example, by configuring the SLO image 304 to rotate by 15 degrees each time the finger makes one circle on the SLO image 304, the operator can rotate the SLO image 304 while making fine adjustments.

Also, as shown in FIG. 8B, a configuration using two fingers, the thumb and the index finger, may be used. In this case, the thumb is set as the center of rotation of the SLO image 304, and the SLO image 304 is rotated according to the amount of movement of the index finger, so that the operator can rotate the SLO image 304 around any position on the SLO image 304. This allows the operator to adjust the display angle so that the optic nerve and the macula are horizontal by rotating an image in which the optic nerve and the macula are simultaneously displayed, for example, around the optic nerve. In this case, when a grid having multiple analysis regions and an analysis result 800 in the analysis region are superimposed on the SLO image 304, the analysis result 800 superimposed and displayed may be rotated in synchronization with the rotation of the SLO image 304. Note that, if the analysis result superimposed includes characters, the characters displayed in the analysis result 800 may be displayed without being rotated, as shown in FIG. 8C. Note that the rotation process may be applied to any of the report display elements, as in the case of the enlargement/reduction process. Alternatively, a configuration in which only the analysis region is rotated may be used. For example, to observe the progression of glaucoma, the symmetry of layer structure and layer thickness, such as the nerve fiber layer, with respect to the line connecting the macula and the optic disc may be evaluated. In this case, it is possible to rotate only the grid on the SLO image 304 in order to place the grid parallel to the line connecting the macula and the optic disc. This makes it possible to rotate and set the analysis region in the desired direction.

Next, the configuration of the parallel movement processing of report display elements is shown. The parallel movement processing of an SLO image is described as an example. For example, as shown in FIG. 9(a), the SLO image 304 on the front image display area 300 may be pressed and held while being slid (hereinafter, referred to as a dragging operation) to translate the SLO image and display it as shown in FIG. 9(b). If analysis results are superimposed on this SLO image 304, the superimposed analysis results may be translated in sync with the translation of the SLO image 304. Note that the parallel movement processing may be applied to any report display element, similar to the zoom processing.

Furthermore, the configuration of brightness/contrast adjustment (hereinafter, B/C adjustment) processing is shown. B/C adjustment of a tomographic image is described as an example. For example, as shown in FIG. 10(a), the brightness or contrast can be adjusted by dragging the tomographic image 306 on the tomographic image display area 301 up and down or left and right. Also, as shown in FIG. 10(b), a configuration may be adopted in which the brightness and contrast can be adjusted simultaneously by dragging diagonally. Note that the B/C adjustment processing may be applied to any report display element, just like the zoom processing.

The above configuration allows the operator to zoom in and out on an SLO image or tomographic image to observe an area of interest, such as a diseased area, while still displaying all display elements necessary for diagnosis on the report, improving diagnostic efficiency.

Example 2
In the first embodiment, the display magnification setting of display elements in a report generated from one inspection result is described. In the second embodiment, the display magnification setting of display elements in a report generated from two inspection results is described. The image processing device of this embodiment can perform enlargement/reduction operations on images of multiple inspections simultaneously. Note that the general functional configuration of this embodiment is similar to that of the first embodiment, and therefore description thereof will be omitted.

First, referring to FIG. 2, a flow chart of report display according to this embodiment will be described. First, in step S200, a patient is selected from a patient screen (not shown) in response to an instruction from the operator. In step S201, one each of the left eye examination result and the right eye examination result for the same patient is selected in response to an instruction from the operator. Here, the examination result for the macula photographed by scanning the fundus of the subject eye with the OCT device is selected. The selection of the examination result may be performed automatically. For example, when the operator selects the examination result for the left eye or the right eye, the examination result for the opposite eye of the same patient on the same date and the same photographed site may be automatically selected. This allows the operator to view reports for both eyes without selecting multiple examination results. In step S202, a report is generated using the examination result selected in step S201. Then, in step S203, the report generated in step S202 is displayed on the display unit 102.

Here, FIG. 6 shows an example of a report generated in step S202 for simultaneously displaying multiple test results. In the right eye image display area 600, an SLO image 601 of a right eye test is displayed. In the left eye image display area 602, an SLO image 603 of a left eye test is displayed. Note that the images displayed in the right eye image display area 600 and the left eye image display area 602 may be frontal two-dimensional images other than SLO images. Note that the tomographic images, projected images, and three-dimensional images described in Example 1 are also displayed for the right eye test and the left eye test, respectively, in the same way as the SLO images. Note that the analysis results may be superimposed on each image. Note that the change in the display method of the report display elements in this embodiment is the same as in Example 1, so a description thereof will be omitted.

In addition, the synchronization of the display method change of the report display element according to this embodiment will be described. When the operator performs a zoom process on the SLO image 601 by pinching, a process of zooming in and out of the same area on the SLO image 603 is performed. Specifically, in the image processing device according to this embodiment, for example, when the operator zooms in on the periphery of the macular area of the SLO image 601, the image processing device automatically zooms in and displays the periphery of the macular area of the SLO image 603 in synchronization. For example, as an example of synchronization of the zooming of the front two-dimensional image, the synchronization of the zooming of the SLO image will be described. FIG. 11(a) is an SLO image displayed in the right eye image display area 600. Also, FIG. 11(b) is an SLO image displayed in the left eye image display area 602. At this time, when, for example, the first rectangular area 1100 is zoomed in response to the touch panel operation on the SLO image, the second rectangular area 1101 is synchronously zoomed in. Here, the position of the first rectangular area 1100 in the SLO image for the right eye and the position of the second rectangular area 1101 in the SLO image for the left eye are in a positional relationship that is symmetrical (with respect to the center of the face).

An example of a tomographic image obtained by scanning the fundus in the horizontal direction (left-right direction) will be described. FIG. 11(c) is a horizontal tomographic image of the right eye, with the left side of the image being the ear side of the fundus and the right side being the nasal side of the fundus. FIG. 11(d) is a horizontal tomographic image of the left eye, with the positional relationship reversed from FIG. 11(c). At this time, when the third rectangular area 1102 in FIG. 11(c) is enlarged in response to a touch panel operation on the horizontal tomographic image, the fourth rectangular area 1103 in FIG. 11(d) is enlarged in synchronization. Here, the position of the third rectangular area 1102 in the horizontal tomographic image of the right eye and the position of the fourth rectangular area 1103 in the horizontal tomographic image of the left eye are in a positional relationship that is symmetrical (based on the center of the face).

An example of a tomographic image scanned in a vertical direction (vertical direction) relative to the fundus is described below. FIG. 11(e) is a vertical tomographic image of the right eye, with the upper part of the fundus on the left side of the image and the lower part of the fundus on the right side of the image. FIG. 11(f) is a vertical tomographic image of the left eye, with the same positional relationship as FIG. 11(e). At this time, when the fifth rectangular area 1104 in FIG. 11(e) is enlarged in response to a touch panel operation on the vertical tomographic image, for example, the sixth rectangular area 1105 in FIG. 11(f) is enlarged in synchronization. Here, the sixth rectangular area 1105 is at a position corresponding to the position of the fifth rectangular area 1104 in the vertical tomographic image (for example, the same position relative to the common origin of both images).

In addition, when the angle of view of the image enlarged by the operator's operation and the image enlarged synchronously are different, the position may be enlarged by correcting the difference due to the angle of view. This makes it easier for the operator to observe the desired part in multiple examinations. In addition, this synchronization process is also performed on the display areas that are paired with the left eye examination and the right eye examination displayed in the report. In addition, although the same part in multiple display elements is configured to be enlarged and reduced synchronously, this is not limited. It may be configured to be enlarged and reduced synchronously such that the relative position in the area of the display element of the operator's operation position is equal. For example, as an example of synchronization of enlargement of a front two-dimensional image, synchronization of enlargement of an SLO image will be described. FIG. 12(a) is an SLO image of the left eye. Also, FIG. 12(b) is an SLO image obtained by photographing the left eye of the same patient at different times. At this time, when the first rectangular area 1200 in FIG. 12(a) is enlarged in response to the touch panel operation on the SLO image, for example, the second rectangular area 1201 in FIG. 12(b) is enlarged synchronously. Here, the second rectangular region 1201 is at a position that corresponds to the position of the first rectangular region 1200 in the SLO image (e.g., the same position relative to the common origin of both images).

An example of a tomographic image obtained by scanning the fundus in the horizontal direction (left-right direction) will be described. FIG. 12(c) is a horizontal tomographic image of the left eye, with the left side of the image being the nasal side of the fundus and the right side being the temporal side of the fundus. FIG. 12(d) is a horizontal tomographic image obtained by photographing the left eye of the same patient at a different time. At this time, when the third rectangular area 1202 in FIG. 12(c) is enlarged in response to a touch panel operation on the horizontal tomographic image, for example, the fourth rectangular area 1203 in FIG. 12(d) is enlarged in synchronization. Here, the fourth rectangular area 1203 is at a position corresponding to the position of the third rectangular area 1202 in the horizontal tomographic image (for example, the same position with respect to the common origin of both images).

An example of a tomographic image scanned in a vertical direction (vertical direction) relative to the fundus will be described. FIG. 12(e) is a vertical tomographic image of the right eye, with the left side of the image being the upper part of the fundus and the right side being the lower part of the fundus. FIG. 12(f) is a vertical tomographic image obtained by photographing the right eye of the same patient at different times. At this time, for example, when the fifth rectangular area 1204 in FIG. 12(e) is enlarged in response to a touch panel operation on the vertical tomographic image, the sixth rectangular area 1205 in FIG. 12(f) is enlarged in synchronization. Here, the sixth rectangular area 1205 is a position corresponding to the position of the fifth rectangular area 1204 in the vertical tomographic image (for example, the same position relative to the common origin of both images). Note that, when the angle of view of the image enlarged by the operator's operation and the image enlarged in synchronization are different, the position corrected for the difference due to the angle of view may be enlarged. This makes it possible to easily scale display elements, for example, in a report that displays multiple test results for the same side of the same patient taken on multiple dates and times (hereinafter referred to as a follow-up report).

In this embodiment, when multiple examinations are displayed and an image of one of the examinations is enlarged or reduced, the corresponding images in the other examinations are enlarged or reduced in synchronization, but synchronous enlargement and reduction is not essential. For example, the operator may select whether or not to synchronize. As in the first embodiment, the enlargement and reduction of each display element in the same examination may be synchronized. In this embodiment, the report display element is enlarged or reduced according to the operation position of the operator, and the same position relative to the report display element is automatically enlarged or reduced in synchronization, but this is not limited to this. The enlargement and reduction center of the operation image and the enlargement and reduction center of the synchronized image may be arbitrarily set. For example, an example of an operation in which the operator specifies the enlargement and reduction center by tapping is shown. FIG. 13(a) shows a tomographic image operated by the operator and a tomographic image enlarged and reduced in synchronization. FIG. 13(b) shows a tomographic image enlarged from FIG. 13(a) and a tomographic image enlarged and reduced in synchronization. The operator specifies the zoom center 1300 and zoom center 1301 in FIG. 13(a) and performs a zoom operation, thereby zooming in around the zoom center as shown in FIG. 13(b). Note that the operator does not have to specify the zoom center. The macular region or optic disc region identified by analysis of the test image may be used as the zoom center. This allows the operator to easily compare areas displayed in different relative positions between images.

In this embodiment, a report configuration based on two examinations is shown, but the present invention is not limited to this. For example, a follow-up observation report displaying three or more examinations may be used. In this case, the operator may perform a pinch operation on a display element of one of the examinations, which causes the display elements of the other examinations to be enlarged or reduced in synchronization. Note that in this embodiment, a configuration in which the enlargement or reduction of display elements is synchronized is described, but the present invention is not limited to this. For example, a configuration in which the rotation, translation, brightness/contrast adjustment, etc. of display elements described in Example 1 are synchronized may be used.

The above configuration allows the operator to view all display elements necessary for diagnosis on a report showing multiple examinations, and to zoom in and out to observe areas of interest, such as diseased areas, on SLO images or tomographic images, improving diagnostic efficiency.

Example 3
In the third embodiment, the display can be switched between a report displaying one test result and a screen displaying an enlarged version of the report display elements (hereinafter, referred to as a full-screen screen).

The general functional configuration of this embodiment, the flowchart for displaying reports, and changes to the display method of the generated reports and report display elements are the same as those in the first embodiment, so a description thereof will be omitted.

Regarding the switching between displaying the report and the full-screen screen (hereinafter, screen transition) according to this embodiment, the screen transition of the full-screen screen of the tomographic image will be described as an example. By double-tapping the tomographic image 306 on the tomographic image display area 301 in FIG. 3, the tomographic image full-screen screen shown in FIG. 13 is displayed. At this time, the tomographic image displayed on the full-screen screen is displayed by inheriting the display method of the tomographic image 306 before transitioning to the full-screen screen. For example, when the tomographic image 306 is in a state in which the macular portion is enlarged, the macular portion is also displayed in a state in which it is enlarged on the full-screen screen. In addition, when the full-screen screen is closed and the report in FIG. 3 is displayed again, the tomographic image 306 is displayed by inheriting the display method of the tomographic image before closing the full-screen screen. The display method includes the enlargement, rotation, translation, and brightness/contrast adjustment of the display elements. Note that the method of transitioning to the full-screen screen is not limited to the double-tap operation by the operator. It may be a long tap operation. In this embodiment, a configuration in which the tomographic image is displayed on the full-screen screen is shown, but this is not limiting. For example, a full-screen SLO image may be displayed by double-tapping the SLO image 304 on the front image display area 300 in FIG. 3. Alternatively, the display element may be a comparison map with a normal database, a thickness map, a thickness profile, or the like. This allows the operator to view a full-screen image of any display element without having to reset the display method from scratch in order to observe the display element in more detail, improving diagnostic efficiency.

(Variation 1)
In this embodiment, the touch panel operation using fingers has been described, but the input method is not limited to this. Operation may be performed using multiple stylus pens instead of fingers. Furthermore, the input device does not need to be pen-shaped, and may be any input device that the touch panel responds to. For example, in the case of a capacitive touch panel, the input device is not limited to any particular shape as long as the contact surface with the touch panel is made of a conductive material.

(Variation 2)
In this embodiment, the zooming of an image is described by pinching in and pinching out, but this is not limited to this. For example, it is possible to assign any operation by switching the operation mode. For example, in the first embodiment, an example of adjusting the brightness or contrast of a tomographic image by dragging is described, but by switching between the brightness contrast mode and the zooming mode in the tomographic image, it is possible to perform different operations with the same movement. That is, when the zooming mode is selected in the tomographic image, the zooming may be performed by dragging up and down, and the image may be moved by dragging left and right. This allows operation with one finger or one pen.

Other Examples
The present invention can also be realized by executing the following process. That is, the present invention can also be realized by supplying software (programs) that realize one or more functions of the various embodiments and modifications described above to a system or device via a network or a storage medium, and having a computer (or a CPU, MPU, etc.) of the system or device read and execute the programs. The computer has one or more processors or circuits, and may include multiple separate computers or a network of multiple separate processors or circuits to read and execute computer-executable instructions.

In this case, the processor or circuitry may include a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), or a field programmable gateway (FPGA). The processor or circuitry may also include a digital signal processor (DSP), a data flow processor (DFP), or a neural processing unit (NPU).

Claims (12)

An image processing device including a display control means for displaying a tomographic image of a subject's eye obtained by using an OCT optical system on a touch panel including a detection surface capable of detecting a touch panel operation,
The display control means is an image processing device that is capable of enlarging and displaying the tomographic image in the short direction without enlarging it in the long direction when two fingers are placed side by side in contact with the short direction of the tomographic image displayed on the touch panel before the touch panel operation. The image processing device according to claim 1, wherein the display control means, when the touch panel operation is a pinch-out operation in the short-side direction, enlarges and displays the tomographic image in the short-side direction without enlarging it in the long-side direction, and when the touch panel operation is a pinch-out operation in the diagonal direction of the tomographic image, enlarges and displays the tomographic image in both the long-side direction and the short-side direction. The image processing device according to claim 1 or 2, wherein the display control means, when it is detected on the detection surface that the operator's finger is in contact with the tomographic image displayed on the touch panel, displays a guide regarding the enlargement process of the tomographic image on the tomographic image. The image processing device according to claim 3, wherein the display control means displays the guide on the tomographic image in such a way that it is identifiable that different enlargement processes are performed depending on the positional relationship of two fingers touching the tomographic image displayed on the touch panel. An image processing device including a display control means for displaying a medical image of a subject's eye obtained by using an OCT optical system on a touch panel including a detection surface capable of detecting a touch panel operation,
The display control means is an image processing device that, when the touch panel operation is a rotation operation and an image in which the analysis results of the medical image are superimposed on the medical image is displayed on the touch panel, is capable of rotating and displaying the analysis area of the analysis result without rotating and displaying the characters contained in the analysis result. The image processing device according to any one of claims 1 to 5, wherein the display control means changes the display of one of a plurality of images obtained by photographing the test eye at different times by operating a touch panel, and synchronously changes the display of the corresponding images photographed at the different times. The image processing device according to claim 6, wherein the display control means synchronously changes the display of a plurality of images obtained by photographing the test eye at different times, or a plurality of images including an image of the left eye and an image of the right eye, at symmetrical positions corresponding to the type of image operated on the touch panel. The image processing device according to claim 1 , wherein the display control means performs different enlargement processes depending on the type of image in response to a touch panel operation on the image displayed on the touch panel. The image processing device according to claim 1 , wherein the detection surface is capable of detecting at least one of a change in a relative distance and an amount of rotation when at least two designated positions designated on the touch panel are moved relatively to each other. A step of displaying a tomographic image of the subject's eye obtained by using the OCT optical system on a touch panel including a detection surface capable of detecting a touch panel operation;
a step of enlarging and displaying the tomographic image in the short-side direction without enlarging the tomographic image in the long-side direction when two fingers are in contact with the tomographic image displayed on the touch panel side by side before the touch panel operation ;
An image processing method comprising: A step of displaying a medical image of the subject's eye obtained using the OCT optical system on a touch panel including a detection surface capable of detecting a touch panel operation;
When the touch panel operation is a rotation operation and an image in which an analysis result of the medical image is superimposed on the medical image is displayed on the touch panel, a step of rotating an analysis area of the analysis result without rotating characters included in the analysis result for display;
An image processing method comprising: A program for causing a computer to execute the image processing method according to claim 10 or 11 .

Images