T 0755/18 (Semi-automatic answering/3M INNOVATIVE PROPERTIES) 11-12-2020
Téléchargement et informations complémentaires:
User feedback in semi-automatic question answering systems
Summary of Facts and Submissions
I. The appeal lies from the decision of the examining division to refuse European patent application No. 11827611.2, which was filed as international application PCT/US2011/052983 published as WO 2012/040578.
II. With effect from 27 November 2020, the application was transferred to 3M Innovative Properties Company, which thereby obtained the status of appellant.
III. The application was refused for lack of inventive step in the subject-matter of claim 1 of the main request and of the auxiliary request over a standard general purpose computer. In the decision under appeal, the examining division explained how it had interpreted the phrases "a set of forward logic" and "applied to first and second concept extraction components", which it considered unclear.
IV. In the statement of grounds of appeal, the appellant requested that the decision be set aside and that a patent be granted on the basis of the main request or the auxiliary request, both requests considered in the appealed decision.
V. In a communication accompanying a summons to oral proceedings, the board explained that even though it had some doubts as to whether the reasoning of the decision under appeal was sufficient, especially for the auxiliary request, it intended to deal with the substantive merits of the case. The board expressed its preliminary opinion that the subject-matter of claim 1 of both requests was not clearly defined and did not involve an inventive step over a general purpose computer.
VI. With a letter of reply, the appellant filed a new auxiliary request 1 and requested that the previous auxiliary request be dealt with as a second auxiliary request.
VII. Oral proceedings were held as scheduled, at the end of which the Chair announced the board's decision.
VIII. The appellant's final request was that the contested decision be set aside and that a patent be granted on the basis of the main request filed with the grounds of appeal or, in the alternative, the first auxiliary request filed by letter of 11 November 2020, or the second auxiliary request filed as auxiliary request with the grounds of appeal. The main request and second auxiliary request correspond to the requests considered in the decision under appeal.
IX. Claim 1 of the main request reads as follows:
"A method performed by at least one computer processor executing computer program instructions tangibly stored on at least one non-transitory computer-readable medium, the method for use with a system including a data source and a first billing code, the first billing code being derived from a set of forward logic applied to first and second concept extraction components, the concept extraction components able [sic] to extract concepts from the data source, the method comprising:
(A) receiving input from a user, wherein the input represents a verification status of the first billing code;
(B) applying first inverse logic to the input, the billing code, and the set of forward logic, to identify the first and second concept extraction components; and
(C) applying reinforcement to the first and second concept extraction components, comprising: (C)(1) determining whether the verification status indicates that the first billing code is accurate; (C)(2) if the verification status indicates that the first billing code is inaccurate, then applying negative reinforcement to the first and second concept extraction components, comprising apportioning the negative reinforcement between the first and second concept extraction components."X. Claim 1 of the first auxiliary request reads as follows:"A method performed by at least one computer processor executing computer program instructions tangibly stored on at least one non-transitory computer-readable medium, the method for improving the accuracy of billing codes to be generated by a system, the system including a data source and having generated a first billing code, the first billing code being derived from a set of forward logic applied to the output of first and second concept extraction components, the concept extraction components being able to extract concepts from the data source, the method comprising: (A) receiving input from a user, wherein the input represents a verification status of the first billing code; (B) applying first inverse logic to the input, the billing code, and the set of forward logic, to identify the first and second concept extraction components, and (C) applying a reinforcement output to the first and second concept extraction components to thereby improve the accuracy of billing codes to be generated by the system, the applying comprising: (C)(1) determining whether the verification status indicates that the first billing code is accurate; (C)(2) if the verification status indicates that the first billing code is inaccurate, then applying a negative reinforcement output to the first and second concept extraction components, comprising apportioning the negative reinforcement output between the first and second concept extraction components."XI. Claim 1 of the second auxiliary request differs from that of the main request in that the text "and to generate concept codes" has been inserted after "the concept extraction components able to extract concepts from the data source", the word "and" at the end of step (B) has been removed, and the following text has been inserted at the end of the claim:", wherein a first reliability score is associated with the first concept extraction component, wherein the first reliability score represents an estimate of a first degree to which the first concept extraction component generates concept codes accurately, and wherein applying the negative reinforcement comprises associating a second reliability score with the first concept extraction component, wherein the second reliability score represents an estimate of a second degree to which the first concept extraction component generates concept codes accurately, wherein the second degree is lower than the first degree;
(D) determining whether the first concept extraction component is unreliable at generating concept codes; and
(E) if the first concept extraction component is determined to be unreliable at generating concept codes, then:
(E)(1) at the first concept extraction component, when generating a further concept code requiring human review of the further concept code before adding the further concept code to the data source."
XII. The appellant's arguments, where relevant to this decision, are addressed in detail below.
Reasons for the Decision
Application
1. The present application is concerned with the generation of billing codes to be used in medical billing. Billing codes may, for instance, be associated with a hospital stay of a patient based on a collection of the documents containing information about the medical procedures that were performed on the patient during the stay and other billable activities performed by hospital staff in connection with the patient during the stay. This set of documents may be viewed as a corpus of evidence for the billing codes that need to be generated and provided to an insurer for reimbursement (see international publication, page 1, paragraphs [0001] and [0002]).
1.1 As explained on pages 1 and 2, paragraphs [0003] and [0004], computer-based support systems have been developed to guide human coders through the process of generating billing codes.
The application describes such a system which includes "concept extraction components" to extract concepts from data sources, for example extracting "allergy" or "prescription" from a medical report, to generate concept codes for these concepts and then generate a "draft transcript" (page 4, paragraph [0016], to page 7, paragraph [0010], Figure 1A). A reasoning module (also referred to as "inference engine") generates or selects appropriate billing codes on the basis of the content of the draft transcript and/or other data sources (page 9, paragraph [0018], Figure 1A). The reasoning module includes "forward logic components", each of which implements a distinct set of logic for mapping document content to billing codes (page 9, paragraph [0020]).
1.2 The invention relates in particular to improving the accuracy of billing codes generated by such a system (page 2, paragraph [0005]).
A human operator provides input specifying whether the codes generated by the system are accurate. The user input may be a "verification status" value selected from a set of permissible values, such as "accurate" and "inaccurate" or "true" and "false". The feedback provided by the user may be captured and interpreted automatically to assess the performance of the automatic billing coding system. In particular, the reasoning process may be inverted in a probabilistic way to assign blame and/or praise for an incorrectly/correctly generated billing code to the constituent logic clauses which led to the generation of the billing code (page 20, paragraphs [0053] and [0054], Figure 4). In order to achieve that, "inverse logic components" may be used to implement logic for reasoning backwards over the rule set of the reasoning module (pages 27 and 28, paragraphs [0079] to [0081]).
The system identifies its components, for instance concept extraction components, responsible for generating the billing codes corresponding to the feedback and assigns positive or negative reinforcement to each of those components (page 20, paragraph [0055]).
Associating praise (positive reinforcement) with a particular component may include increasing a reliability score counter associated with the component or assigning a particular reliability score to the component. Similarly, associating blame (negative reinforcement) may include decreasing a reliability score counter associated with the component or assigning a particular reliability score to the component (pages 23 and 24, paragraph [0067]).
The system may take any of a variety of actions in response to concluding that a component is unreliable, for example subsequently and automatically requiring the human operator to review and approve any concept codes (subsequently and/or previously) generated by the unreliable concept extraction component (page 36, paragraph [0108]).
Main request
2. Claim 1 - interpretation
2.1 Claim 1 defines a method for use in a software system for generating billing codes, the system including two concept extraction components and forward logic. The system receives input from a user to determine whether the generated billing code is inaccurate and identifies two concept extraction components which were involved in the generation of the billing code. If the billing code is considered inaccurate, it applies a negative reinforcement to the identified concept extraction components.
2.2 The appellant argued that the claimed method affected future iterations of the otherwise unclaimed forward process and therefore improved the system so that it would generate more accurate billing codes in the future. Subsequent execution of the forward process was improved because the concept extraction components which provided inaccurate billing codes were negatively reinforced based on the previous iteration.
2.3 Claim 1 does not define what happens as a consequence of or after applying the negative reinforcement. However, for the sake of argument, in the following the board interprets claim 1 of the main request, in accordance with the appellant's arguments, as specifying a method in which negative reinforcement output is applied to the first and second extraction components to improve the accuracy of billing codes to be generated by the system in the future.
3. Inventive step - claim 1
3.1 Claim 1 specifies a computer-implemented method for improving the accuracy of automatically generated billing codes. The method is specified in terms of computer program features of the implementation of the task of modifying the computer program which generates billing codes in order to improve the accuracy of the generated billing codes.
Since a computer program as such is not patentable under Article 52(2) and (3) EPC, computer program features which do not make a technical contribution are not to be taken into account for inventive step.
A billing code is non-technical administrative data which may take the form of a textual representation, for instance "Unspecified diabetes" (see paragraph [0050] of the international publication). Generating a billing code (see also point 1. above) is a cognitive task (paragraphs [0002] and [0015]). The process of generating a billing code on the basis of documents is thus a non-technical administrative task, which, as such, is not patentable pursuant to Article 52(2) and (3) EPC.
In the grounds of appeal, the appellant argued that simply because a claimed set of features offered a solution to an administrative, economic or business problem, it did not in and of itself prohibit those same features from simultaneously solving a technical problem for which an applicant was entitled to seek protection. The examining division had applied the COMVIK approach incorrectly, which required, as a first step, an assessment of which features conferred technical character to the claimed subject-matter. Only then could features not contributing to the solution of a technical problem be ignored.
The board agrees that the presence of non-technical features in the claim does not mean that the claimed subject-matter is not patentable and that features which are non-technical when taken in isolation but which interact with technical features of the invention to solve a technical problem should be taken into account in assessing inventive step. It is therefore important to take into account all the claim features to identify those making a technical contribution. In order to assess inventive step, either a "conventional approach", starting with a selection of the prior art, or an approach relying on an initial analysis of the technical character of the claim features may be adopted, depending on the circumstances (see T 697/17 of 17 October 2019, Reasons 4.2 and 4.3; T 258/03, OJ EPO 2004, 575, Reasons 3.5 and 3.6; T 756/06 of 18 April 2008, Reasons 5; G 3/08, OJ EPO 2011, 10, Reasons 10.13.2). Therefore, the board does not agree with the appellant that the first step has to be that of identifying the features making a technical contribution.
3.2 The appellant argued that the invention used machine-learning techniques to improve the accuracy of the machine output. According to the appellant, the invention was technical because it improved the system so that it would generate more accurate billing codes in the future.
In the board's opinion, if neither the output of a learning-machine computer program nor the machine output's accuracy contributes to a technical effect, an improvement of the machine achieved automatically through supervised learning for producing a more accurate output is not in itself a technical effect. In this case, the learning machine's output is a billing code, which is non-technical administrative data. The accuracy of the billing code refers to "administrative accuracy" regarding, for example, whether the billing code is consistent with information represented by a spoken audio stream or a draft transcript (paragraph [0051]) or is "justified by the given corpus of documents, considering applicable rules and regulations" (paragraph [0002]). Therefore, improving the learning machine to generate more accurate billing codes or, equivalently, improving the accuracy of the billing codes generated by the system, is as such not a technical effect.
3.3 A technical contribution could reside in a technical solution to the problem of achieving the effect of improving the accuracy of the billing codes generated by the system.
The appellant argued that the invention had the advantage of avoiding wasting system resources. If an inaccurate billing code was produced which could not be used, a negative reinforcement was applied, which caused the system to improve the future generation of billing codes. In this way, inaccurate billing codes were used to their maximum utility. Moreover, by generating more accurate billing codes, resources were saved because less iterations were necessary to obtain the desired result.
The board does not find these arguments convincing. Even though the case law generally recognises a technical contribution by non-technical features if they are causally linked to a technical effect, such as reducing resource usage, not every such physical change qualifies as a technical effect. A physical change caused by non-technical features is to be regarded as a technical effect for the purpose of assessing inventive step if the non-technical features are based on technical considerations aimed at controlling that physical change (T 697/17, Reasons 5.2.2; T 2230/10 of 3 July 2015, Reasons 3.7). The physical change has to be purposively used in the solution of a technical problem (T 258/97 of 8 February 2002, Reasons 6; T 886/17 of 6 March 2020, Reasons 5.4.1). The computer program features of claim 1 are not purposively directed to reducing the system resources. Instead, they are the result of non-technical administrative and programming considerations. The set of forward logic and the concept extraction components are computer program components which mimic the administrative procedure of generating a billing code from input data. They are the result of non-technical administrative considerations by the administrative expert about how to generate a billing code and non-technical programming considerations about how to program a computer to generate a billing code according to the chosen administrative procedure.
Similarly, the steps of receiving verification and applying reinforcement of features (A) to (C)(3) correspond to administrative steps of improving the process of generating billing codes by identifying the sources of inaccuracy and negatively reinforcing them. The decisions to let a human expert evaluate the accuracy of the billing code generated by the system and apply a negative reinforcement on the persons or entities responsible for the inaccuracy are taken by the non-technical administrative expert. Implementing this solution in the system by using inverse logic to identify the responsible components, receiving input and applying negative reinforcement to the identified components is the result of a non-technical programming task. Features (A) to (C)(3) are thus the result of non-technical administrative considerations about how to improve the generation of a billing code and programming considerations about how to program a computer to improve the billing code generation according to the chosen administrative procedure.
3.4 The appellant further argued that providing billing codes was not the main aspect of the invention, which relied on the system architecture rather than on the type of data or the cognitive content of the data. The technical contribution of the invention was independent of the type of data dealt with.
The board does not find this argument relevant, because the claim is limited to the generation of billing codes.
3.5 The appellant also argued that the choices to provide the claimed system were not administrative. Instead, they were technical choices made by a computer engineer in the realm of machine learning who, when seeking to solve the technical problem of improving the accuracy of billing codes generated by the system, recognised the need to minimise the use of limited computer resources.
Since computer programming involves technical and non-technical aspects (G 3/08, Reasons 13.5.1; T 1463/11 of 29 November 2016, point 21), the test of whether program features would have been formulated by a software expert is not sufficient to conclude whether those features are technical (T 697/17, Reasons 5.2.4; T 817/16, Reasons 3.12). In the present case, the question of whether the features of the claimed method were made by a "computer engineer in the realm of machine learning" is not decisive for the question of whether the features make a technical contribution. Since the work of an expert in machine learning includes non-technical computer-programming tasks, which are not patentable under Article 52(2) and (3) EPC, what is decisive is whether those features were the result of "technical considerations beyond 'merely' finding a computer algorithm to carry out some procedure" (see T 697/17, Reasons 5.2.4; G 3/08, Reasons 13.5). As the board explained above, in the present case, the features of the invention are based only on non-technical administrative and programming considerations.
According to the appellant, the considerations underlying the invention were not about which lines of code to use in a particular programming language, such as Java or HTML, but technical considerations regarding which components to use. The board is however of the opinion that the claim does not describe any non-trivial technical characteristics of the "components", e.g. the concept extraction components, and that the choice of the components is not driven by technical constraints.
3.6 Claim 1 specifies an automated method of performing the administrative task of improving the accuracy of the generated billing codes. Using a computer to automate an administrative task is well known. From the above it follows that, with the exception of its implementation using a general purpose computer, the method is specified in claim 1 in terms of non-technical features which are not to be taken into account for inventive step.
3.7 Therefore, the subject-matter of claim 1 is not inventive over a general purpose computer (Article 56 EPC).
First auxiliary request
4. Claim 1 of the first auxiliary request differs from that of the main request in that it clarifies that the claimed method and step (C) improve the accuracy of billing codes to be generated by the system, the system having generated a first billing code, that the set of forward logic is applied to the output of the extraction components, and that a reinforcement output is applied to the concept extraction components.
5. Admission into the proceedings
The first auxiliary request was filed in response to the board's communication under Article 15(1) RPBA 2020, which raised new objections under Article 84 EPC. The request reflects a legitimate attempt by the appellant to overcome those objections, was filed at the earliest opportunity, and was accompanied by arguments explaining why the appellant considered it to overcome the new objections. In exercising its discretion under Article 13(1) and (2) RPBA 2020, the board therefore acknowledges the presence of exceptional circumstances which justify admitting this request. Consequently, the board admits the first auxiliary request into the appeal proceedings.
6. Inventive step
6.1 The amendments made to claim 1 of the first auxiliary request address the clarity objections raised by the board. In the inventive-step assessment of claim 1 of the main request given above, the board interpreted claim 1 of the main request as clarified by claim 1 of the first auxiliary request (see point 2. above).
6.2 Therefore, for the same reasons as given for the main request, the subject-matter of claim 1 is not inventive and the first auxiliary request does not fulfil the requirements of Article 56 EPC.
Second auxiliary request
7. Claim interpretation - claim 1
7.1 The additional features of claim 1 of the second auxiliary request essentially specify that the concept extraction components generate concept codes and that
- applying negative reinforcement comprises decreasing a reliability score associated with the first concept extraction component to reflect an estimated lower degree of accuracy of the concept codes generated by that component (features added to (C)(2)),
- if the first concept extraction component is determined to be unreliable at generating concept codes, then it requires human review before adding further generated concept codes to the data source (features (D) to (E)(1)).
8. Inventive step - claim 1
8.1 With respect to the second auxiliary request, at the oral proceedings the appellant referred to the arguments given for the higher ranking requests. In its grounds of appeal, the appellant argued that the additional features of claim 1 of the second auxiliary request enabled the identification of further concept codes generated by a concept extraction component that had previously been found to be unreliable and had to be moderated by a human reviewer. Subsequent to the execution of steps (D) and (E)(1), the concept extraction components were subdivided into two categories, only one of which required attention by the human reviewer. Therefore, features (C)(2) to (E)(1) solved the technical problem of streamlining the review process.
The board, however, is of the opinion that avoiding user review for further automating a task is trivial. The additional features of (C)(2) to (E)(1) are also related to the problem of improving the accuracy of the billing codes generated by the system. The concept codes now mentioned in features (C)(2) to (E)(1) are administrative codes, and a reliability score reflecting an estimated degree of accuracy is a non-technical algorithmic choice. The board is thus of the opinion that features (C)(2) to (E)(1) of claim 1 of the second auxiliary request are based on non-technical administrative and algorithmic considerations regarding the process of generating the billing code from the source documents, what affects the accuracy of the billing code, and how to avoid inaccurate concept codes being used further in the process of determining the billing code.
8.2 Therefore, the subject-matter of claim 1 of the second auxiliary request is not inventive (Article 56 EPC).
Concluding remark
9. Since none of the requests is allowable, the appeal is to be dismissed.
Order
For these reasons it is decided that:
The appeal is dismissed.