JP6994587B2 - System performance monitor with graphical user interface - Google Patents

Description

The inventions described herein belong to the field of system performance monitors with a graphical user interface.

There has long been a need for a system performance monitor that can graphically show how fast a particular system or subsystem within the particular system can perform a given task. This system can be a fully automated system, such as a computer system for delivering messages over the Internet. The system may also include people performing tasks such as employees of a particular company. A subsystem may include a group of automated appliances such as servers in a server farm. Also, a subsystem can include a group of people, such as a department within an organization. A subsystem may include both an automated device and a person, such as a group of people each working on a workstation connected to a network.

The disclosure of the present invention is provided as a guide for understanding the present invention. The disclosure of the invention does not necessarily describe the most comprehensive embodiment of the invention or another embodiment of the present invention.

FIG. 1 is a conceptual diagram of a system performance monitor 100 adapted to display the performance of a system 150 performing one or more tasks (eg R ₁ , R ₂ , R ₃ ). This system performance monitor
a) A frontend 110 realized by a computer,
i) Microprocessor and
ii) Fixed storage and
iii) Input device 111 and
iv) Frontend 110, including output device 160, and
b) The backend 120 realized by a computer,
c) One or more subsystem schema databases 140 implemented by a computer, each including one or more subsystem schema databases 140 associated with a subsystem schema.

A system 150 that performs one or more tasks is systematized in one or more of the above subsystem schemas 151 (eg, A, B, and C). Each of the subsystem schemas contains one or more members with member IDs (eg, A ₁ , A ₂ , A ₃ , B ₁ , B ₂ , B ₃ , C ₁ , C ₂ , C ₃ ). Each member performs at least a portion of one or more of the above tasks (eg, R ₁ , R ₂ , R ₃ ).

Each task has a related task record 131 stored in the task database 130. Each task record is
i) Task index 132 and
ii) Metadata 133 containing one or more member IDs (eg, A ₁ , B ₃ , C ₂ ) of one or more members performing at least part of the task associated with the task record.
iii) Each event record contains one or more event records (eg 134) associated with each event.
1) The event type (for example, 135) of the above related event and
2) Includes a time stamp (eg 136) at the time the related event occurred.

Each of the subsystem schema databases (eg 141) associated with the subsystem schema
i) A member record (eg, 147) associated with each member of the relevant subsystem schema above.
Includes.

Each of the member records
i) A member ID index (eg 142) associated with a member of the relevant subsystem schema above, and
ii) Contains at least a portion (eg, 143) of each task record associated with a task that is at least partially performed by the member associated with the member index.

The front end includes computer-readable instructions stored on the fixed storage device of the front end, and the computer-readable instructions of the front end are sent to the microprocessor of the front end i) the above. A selection of one of the subsystem schemes (eg 115) and a selection of one of the members of the selected subsystem scheme (eg A3) ₍ 112) is received from the user via the input device. Steps and
ii) Invoking the backend with the selected subsystem schema and selected members (114) and
iii) A formatted member record 123 containing zero or more selected task records 144 associated with the selected member and zero or more non-selected task records 146 associated with the selected member. With the steps received through the backend above from the subsystem schema database associated with the selected subsystem schema above.
iv) It is possible to physically execute the (161) step of formatting the mark (for example, 164) of the selected task record and displaying it on the output device.

FIG. 1 is a conceptual diagram of a system performance monitor. FIG. 2 is a time cloud scatter plot displayed on the output device of the system performance monitor.

This detailed description describes a non-limiting exemplary embodiment. Each individual feature can be combined with other features, at least if required by different uses to obtain the benefits described herein. As used herein, the term "about" means plus or minus 10% of a particular value, unless otherwise indicated.

Some of the disclosures in this patent document include matters for which copyright claims have been made. The copyright holder does not object to the facsimile reproduction of the patent file or document of the Patent and the disclosure of this patent on record by the Patent and Trademark Office, but all other copyrights. We reserve this for anything.

As used herein, a "computer system" is an input device for receiving data (eg, an electronic digital input from a keyboard, touch screen, or other device) and for outputting data. An output device (eg, a digital connection to a printer, computer screen or other device), a fixed digital memory for storing data, computer code or other digital instructions, and a digital processor for executing digital instructions. Includes. The digital instruction stored in the fixed storage device reads data via the input device, processes the data in the microprocessor, and outputs the processed data via the output device. Is physically made to be the above-mentioned digital processor.

As used herein, the term "shape" has that particular shape as an overall appearance, even if there are slight variations to the pure form of the particular shape. Means that.

As used herein, the term "generally", when referring to a shape, means that an object is in its shape, even if it is slightly variable from that shape by a typical observer. It means to understand that you have.

As used herein, "top," "bottom," "top," "bottom," "left," "right," "vertical," "horizontal," "distal," and "distal." Relative directional terms such as "proximal" are defined with respect to the initial description of the object, even if the object is subsequently described in another direction, unless otherwise noted. It means the same part of the object.

As used herein, unless otherwise indicated, singular disclosures suggest plural disclosures and vice versa.

Seeing FIG. 1 again, the front-end computer-readable instructions are:
a) It may further include displaying a time cloud scatter plot (eg, 161) on the output device 160 in response to receiving the formatted member record. This time cloud scatter plot is
i) X-axis 162 and
ii) Y-axis 163 and
iii) It may contain a data point mark (eg 164) for each of the selected task records in the formatted member record.

The selected task record in the formatted member record may contain a time stamped start event and a time stamped end event, respectively. Each of the displayed data point marks has an X value based on the time stamp of the end event. Each of the displayed data point marks has a Y value based on the time stamp of the start event.

The X-axis may extend horizontally. The Y-axis may extend vertically. The amazing advantage of having this X-axis is that the user can read the time cloud diagram from left to right to understand how the performance of the selected members is changing over time. Alternatively, the Y-axis may extend horizontally and the X-axis may extend vertically. The advantage of this direction is that the horizontal axis corresponds to the point in time when the event started.

The displayed data point mark may be translucent. As a result, when two data point marks overlap (for example, 166), the overlapping range becomes visible.

The time cloud scatter plot may have a first mark line 165 at a position where Y is equal to X. This line represents the current time. The data point mark on the above line corresponds to the task completed immediately after starting.

The time cloud scatter plot has a second mark line 171 at the position where Y is equal to the value obtained by adding the expected time amount 173 from the start event to the end event for a specific task to X. May be good. This allows the user to immediately see if the task has been completed within the expected amount of time by looking at the time cloud diagram. If the data point mark is located below the second mark line (eg 172), the user may want to investigate further to determine what was the cause of this delay. ..

The data point mark may be interactive to the user. For example, when the user activates the data point mark, each of the data point marks may display the metadata 167 associated with the related task of the related data point mark.

The time cloud scatter plot may further include a statistic item 169 based on the selected task record of the selected member and the non-selected task record of the selected member. This statistic item helps to provide the status of the displayed data point marks. For example, the displayed statistic may be the ratio of selected task records to all task records (ie, selected task records plus non-selected task records). This allows the user to know whether the number of displayed data point marks is large or small.

Presents a display of selected members 168 on the time cloud scatter plot so that the user can show the time cloud scatter plot to others and the other person can see which member is selected. You may.

In addition to or instead of this, the member display and statistical items may be displayed outside the time cloud scatter plot, eg, in the margins.

The system performance monitor is a computer feasible data manger 124,
a) Microprocessor and
b) A fixed storage device to the microprocessor of the data manger.
i) A step of reading one of the above task records (for example, 131) stored in the above task database, and
ii) A step of generating a member record for each of the above members (for example, A ₁ , B ₃ , C ₂ ) in the above metadata (for example, 133) of the read task record, and
iii) Computer readable to physically execute the step of storing the generated member records (eg A ₁ : r ₁ , B ₃ : r ₁ , C ₂ : r ₁ ) in the system schema database. A data manger 124 including a fixed storage device including instructions may be further provided.

The backend may also be configured to authenticate and route the call 114 from the frontend (121). The backend may also be configured to call the appropriate subsystem schema database (122) based on the routing request by the selected subsystem schema.

The frontend may also be configured to determine which subsystem schema (eg 115, 116, 117).

Example 1
The United States Patent and Trademark Office (USPTO) is a system that performs one or more tasks. One of the above tasks is the examination of patent applications. This examination is at least partially performed by the patent examiner. One of the patent office's subsystem schemas is the technical classification assigned to a patent application. Members of the patent classification schema contain different classes (eg class 706-artificial intelligence). Another subsystem schema is the art unit to which patent applications are assigned. Members of the technical review department schema include individual technical review departments (eg, AU 2121). Another subsystem schema is the patent examiner. Members of the Patent Examiner Schema are individual examiners. Other schemas such as law firm schema, applicant schema, or inventor schema can be considered.

Each patent application is assigned an application number (eg SN 12 / 345,678). The USPTO is aware of all events that occur during the examination of a patent application (eg, processing history or application bag). These events have an event type (eg, "not the last notice of reasons for refusal") and a time stamp (eg 05-05-2011 or month-day-year).

The USPTO stores metadata and records for events associated with patent application numbers in a publicly available database called the "Patent Examination Data System" (PEDS). Records in PEDS are indexed by application number. The metadata for each record contains a subsystem label such as a 3-digit technical class number for the patent classification schema, a 4-digit technical examination department number for the technical examination department schema, and an alphabetic name for the examiner schema. The USPTO provides a search engine for selecting applications assigned to a particular patent classification class, technical examination department, or examiner, but it takes to get a set of records for a particular search. The time can be very long (eg 10 minutes or more for large patent classification classes). Moreover, after the record has been returned by the search engine, the time required to process the record and generate a time cloud diagram can be very long. The data file for one patent classification class can be 1GB. The time required to process data for one patent classification class can be several hours on one workstation. The patent classification schema has about 700 members. It takes several days on a huge cloud-based data processing server to process data for all 700 members.

To solve this problem, the system performance monitor described above was developed. This system performance monitor is generally cloud-based away from the user's client terminal. The frontend is located on a web server (Netlify.com). The user's client terminal connected to Netlify is considered to be part of the front end. The backend is located on the application server, Amazon Web Server (aws.amazon.com). The subsystem schema database is located on the cloud-based service MongoDB Atlas (www.mongodb.com/cloud/atlas). Those of skill in the art will appreciate that other cloud-based services and dedicated resources such as mainframes and server farms can be used to host front-end, back-end, and subsystem schema databases. As such, the inventions described herein are not limited to a particular computer platform.

The frontend is provided with one input field for receiving the selection of members of the subsystem schema from the user. The drop-down menu displays the members available from all three schemas, depending on what the user has entered. The user can then select the desired member from the drop-down menu. Member names are specific to each subsystem schema, so the backend can determine which subsystem schema is selected and therefore which subsystem schema database to query. The choice will be sufficient. In this example, if the selected member ID has alphabetic characters, the selected subsystem schema will be the examiner. If the selected member ID is a 3-digit number, the selected subsystem schema is a patent classification class. If the selected member ID is a 4-digit number, the selected subsystem schema will be the technical review department. In this example, the frontend simply forwards the selected member ID to the backend, which determines the selected subsystem schema.

At the back end, the data manger is written using a programming language. The data manger reads the entire PEDS database (more than 1TB). The data manger then selects the metadata and events of interest for each application record, reformatts the data, and stores the data in the subsystem schema database. Three subsystem schema databases are set up. One subsystem schema database is set up for the patent classification class, one subsystem schema database is set up for the technical review department, and one subsystem schema database is set up for the examiner. Users such as registered US patent attorneys or lawyers will be interested in the performance of any member of these schemas. The user can, for example, determine whether the performance of a particular examiner matches the performance of all examiners in a particular technical review department. Similarly, it is possible to compare the performance of different technical examination departments examining patent applications of the same patent classification class. Similarly, managers within the USPTO may be particularly interested in the existence of significant discrepancies between examiners, technical review departments, or classifications that should perform similarly.

Filter PEDS data to include only the metadata of interest (eg, application number, invention name, classification, technical review department, examiner) and event data (eg, non-final rejection, final rejection, permission notice). Then, the amount of data is reduced by 35 times. However, since there are three subsystem schema databases, and each database has duplicate data indexed by the members of each subsystem schema, the overall data reduction is about 12 times. Still, the delay before selecting the subsystem schema, selecting the members of the above subsystem schema, and showing the time cloud diagram on the output device is even for a time cloud diagram showing 100,000 data point marks. , Only about 10 seconds or less.

FIG. 2 is an exemplary time cloud scatter plot 202 displayed on the output device 200 of the system performance monitor. This subsystem schema was a patent classification class. The selected member of the Patent Classification class was Class 706-Artificial Intelligence. The selection record was the record of all applications that received the permission notice for this patent classification class. Non-selected records were records of all applications that did not receive permission notice. The non-selected record may be for an abandoned patent application or for a pending patent application without notice of permission. If there were multiple permission notices in a record, all permission notices were selected and each was shown as a data point mark.

Y-axis 231 is the filing date of the patent application. X-axis 232 is the permission notification date. Both axes start on January 1, 2000 and end on the day the PEDS data was downloaded (ie March 15, 2019). Here, after the first download of the entire PEDS database, subsequent downloads may contain only records for applications with new events since the previous download. The data from subsequent downloads can be used to update the subsystem schema database.

The first mark line 223 is at a position where Y is equal to X. The second mark line 224 is at a position where Y is equal to the value obtained by adding 3 years to X. Three years is the maximum amount of time an applicant will need to receive a permit notice under US Patent Law for not delaying the examination of a patent application.

A data point mark (eg 211) is displayed on the time cloud scatter plot for each of the selected records (ie, all applications for which a permit notice has been received). A total of about 11,000 data point marks are shown. The data point mark is solid black, has no border, and has a size of 3 points. The transparency of the data points is 80%. As a result, one data point mark (for example, 212) can be clearly seen, and at the same time, a dense region (for example, 213) can be recognized.

When any data point mark (eg 212) is selected by the user, a pop-up 214 showing the relevant metadata (eg, application number and invention name) of the application associated with that data point mark is displayed.

The selected patent classification class (item 204) is displayed on the time cloud scatter plot. In another embodiment, instead, the patent classification class is displayed in the margin of the time cloud scatter plot.

In addition, the calculated statistical item APOA ₁₂ value (item 206) is shown on the time cloud scatter plot. APOA means "allowances per office action" for one notice of reasons for refusal. The subscript "12" indicates that this APOA is for a notice of reasons for refusal during the 12 months prior to the date the data was downloaded or last updated. The notice of reasons for refusal includes a notice of reasons for refusal that is not the last of the non-selected records, such as an application that did not receive the notice of permission, and a notice of the last reason for refusal. For this reason, APOA ₁₂ is intended to indicate to the user how many times the user is likely to need to respond to the notice of reasons for refusal for all applications in the portfolio in order to obtain one notice of permission. Become. APOA ₁₂ indicates that the applicant had to respond to about four notices of reasons for refusal in order to obtain one permit notice in the last 12 months.

Vertical grid lines 221 and horizontal grid lines 222 are shown on a time cloud scatter plot so that the user can easily see when a particular application is granted and when it is filed, respectively.

Looking at this time cloud, users noticed that the data point marks have faded since the beginning of 2017 (242). The user then generated another time cloud diagram based on the case where the application containing the not last notice of reasons for refusal or the last notice of reasons for refusal is represented by a data point mark. Users use pop-ups for some applications that have been denied grounds after 2017, and many applications will be in the United States if the application has not previously received a reason for refusal under Article 101 of the U.S. Patent Act. It was determined that the refusal was received under Article 101 of the Patent Act. Therefore, users have become pessimistic about filing applications that would be assigned to class 706. However, upon closer inspection, users found that in the first few months of 2019, the data point marks on permit notifications suddenly darkened (243). The USPTO issued new guidelines in January 2019 regarding the proper examination of patent applications under Section 101 of the US Patent Act. When the user confirms the application bag of the application approved in early 2019, the examiner of this classification has confirmed that the application that was rejected before January 2019 is permitted based on the new guidelines. I got you. For this reason, it encourages users to continue filing applications that belong to this category. Users can see both small and rapid changes in the performance of different members of different subsystem schemas to more effectively prepare manuscripts for patent applications and proceed with their examination more effectively. ..

CONCLUSIONS Although the present disclosure has been described with reference to one or more different exemplary embodiments, various modifications can be made without departing from the scope of the present disclosure, substituting its elements for equivalents. It will be understood by those skilled in the art that it can be done. In addition, numerous modifications can be made to adapt to a particular condition without departing from its essential scope or teachings. Accordingly, the present disclosure is not limited to the particular embodiments disclosed as being considered best for carrying out the invention.

Claims (13)

A system performance monitor adapted to display the performance of a system that performs one or more tasks.
a) A front end realized by a computer
i) Microprocessor and
ii) Fixed storage and
iii) Input device and
iv) With the front end, including the output device,
b) The back end realized by the computer and
c) One or more subsystem schema databases realized by a computer, each of which includes one or more subsystem schema databases associated with a subsystem schema.
d) The system that performs one or more tasks is organized into one or more of the subsystem schemas.
e) Each of the above subsystem schemas contains one or more members with a member ID.
f) Each of the above members performs at least a portion of one or more of the above tasks.
g) Each of the above tasks has a related task record stored in the task database.
h) Each task record is
i) Task index and
ii) Metadata that contains one or more member IDs of one or more members that perform at least part of the task associated with the task record.
iii) Each event record contains one or more event records associated with each event.
1) Event types of the above related events and
2) Including the time stamp when the above related event occurred
i) Each of the above subsystem schema databases associated with the subsystem schema is
i) Contains the member records associated with each member of the relevant subsystem schema above.
j) Each of the above member records
i) The member ID index associated with the member of the relevant subsystem schema above, and
ii) Containing at least a portion of each task record associated with a task that is at least partially performed by the member associated with the member ID index.
k) The frontend includes computer-readable instructions stored on the fixed storage device of the frontend, and the computer-readable instructions of the frontend are sent to the microprocessor of the frontend i). A step of receiving a selection of one of the subsystem schemes and a selection of one of the members of the selected subsystem scheme from the user via the input device.
ii) Invoking the backend with the selected subsystem schema and selected members,
iii) A formatted member record containing zero or more selected task records associated with the selected member and zero or more non-selected task records associated with the selected member is selected above. The steps received from the subsystem schema database associated with the subsystem schema via the above backend, and
iv) It is possible to physically execute the steps of formatting the mark of the selected task record and displaying it on the output device.
System performance monitor. The computer-readable instructions on the frontend are sent to the microprocessor on the frontend.
a) In response to the receipt of the formatted member record, it is possible to further physically perform the step of displaying the time cloud scatter plot on the output device, which is a time cloud scatter plot.
i) X-axis and
ii) Y-axis and
iii) Includes a data point mark for each of the selected task records in the formatted member record
b) The selected task record in the formatted member record contains a time stamped start event and a time stamped end event, respectively.
c) Each of the data point marks has an X value based on the time stamp of the end event.
d) Each of the data point marks has a Y value based on the time stamp of the start event.
The system performance monitor according to claim 1. The X-axis extends horizontally and the Y-axis extends vertically.
The system performance monitor according to claim 2. The system performance monitor according to claim 2 or 3 , wherein the displayed data point mark is translucent. The above time cloud scatter plot is
a) The first mark line where Y is equal to X, and
b) Further includes a second mark line at a position where Y is equal to the value obtained by adding the amount of time expected from the start event to the end event for a specific task to X.
The system performance monitor according to any one of claims 2 to 4 . The system performance monitor according to any one of claims 2 to 5, wherein the data point mark is interactive to the user. The system performance monitor of claim 6, wherein when the user activates the data point mark, each of the data point marks displays the metadata associated with the related task of the related data point mark. The time cloud scatter plot according to any one of claims 2 to 7 , further comprising statistical items based on the selected task record of the selected member and the non-selected task record of the selected member. System performance monitor. The system performance monitor according to any one of claims 1 to 8, wherein the system for executing the task includes a person who performs at least a part of the task. The system performance monitor of claim 9 , wherein at least one of the above subsystem schemas comprises the group of said persons. A data manger that can be realized by a computer
a) Microprocessor and
b) A fixed storage device to the microprocessor of the data manger.
i) A step to read one of the above task records stored in the above task database, and
ii) A step to generate a member record for each of the above members in the above metadata of the above read task record, and
iii) Claims 1-10 further comprising a data manger including a fixed storage device including computer-readable instructions for physically performing the steps of storing the generated member records in the system schema database. The system performance monitor described in any one of the above. a) The above member IDs are unique to each subsystem schema so that the selection of the member ID is also the selection of the subsystem schema.
b) The computer-readable instructions on the front end are sent to the microprocessor on the front end.
i) With the step of presenting one input field on the input device of the frontend to receive the selected member ID.
ii) It is possible to further physically execute the step of transferring the member ID to the backend.
c) The backend is configured to determine the selected subsystem schema from the selected member IDs.
The system performance monitor according to any one of claims 1 to 11 . a) The first subsystem schema is a classification of applications for intellectual property rights .
b) The second subsystem schema is the examination department for applications for intellectual property rights .
c) The third subsystem schema is the examiner of the application for intellectual property rights ,
The system performance monitor according to claim 12.

Images