T 0975/20 (Visual localization method/NAVVIS) 08-02-2023

1. The application relates to a visual localisation method for a mobile device (see abstract and claim 1 of the main request).

Main request

2. Added subject-matter

2.1 In its communication under Article 15(1) RPBA 2020 the board questioned whether the specific combination of features of claim 1 was directly and unambiguously derivable from the passages cited in the first-instance proceedings as a basis for claim 1, namely originally filed claims 1 and 4 and the description, page 4, lines 25 and 28, and pages 6 and 7. The cited passages of the description appeared to refer to a specific embodiment based on the "Bag-of-Features" (BoF) approach, which reformulated an image retrieval problem into a text retrieval problem by quantizing the high-dimensional feature descriptors into "visual words" (see the description, page 5, lines 1 to 20). However, claim 1 did not appear to be limited to this embodiment. Consequently, claim 1 of the main request appeared to infringe Article 123(2) EPC.

2.2 In its written response to the board's communication, the appellant did not argue against the board's objection under Article 123(2) EPC for the main request, but filed an amended auxiliary request IV as a direct response to the board's fresh objections under Articles 123(2) and 84 EPC.

2.3 At the oral proceedings, the appellant argued that claim 1 of the main request had a basis in the application as filed. According to the appellant, the board's preliminary opinion did not list all the passages cited as the basis in the first-instance proceedings.

The full basis starting from claim 1 as originally filed was as follows. Feature (a) of claim 1 under discussion was based on claim 1 as originally filed. Step (b) was based on claim 2 as originally filed, which also provided a basis for the amendment made in step (c) of claim 1 under discussion ("on the basis of the estimated location of the mobile device"). Step (d) was based on claim 1 as originally filed, but the word "transmitting" had been amended to "pushing" on the basis of the disclosure on page 4, line 25 of the description as originally filed. Feature (e) was based on page 4, line 28 ("preloaded adaptive vocabularies") of the description as originally filed. Moreover, the expressions "reference data" and "vocabulary" were used as synonyms in the application as filed (see page 7, lines 1 to 5 of the description as originally filed). Step (f) of claim 1 was also based on the passages cited for feature (e). The last feature of claim 1 ("wherein [...] and (d).") was based on claim 4 as originally filed.

2.4 At the oral proceedings, the board informed the appellant that it had doubts that originally filed claim 2 could be a basis for steps (b) and (c) of claim 1 under discussion. Moreover, the board considered that the description disclosed a single embodiment relating to the BoF approach, starting on page 5 and continuing on pages 6 and 7. This was evident, for example, from the first sentence on page 6 ("It has been found that the performance of a BoF-based mobile location recognition system can be improved based on the above mentioned feature selection and by making use of prior knowledge of the user's location").

2.5 At the oral proceedings, the appellant contested the board's reading of the application as filed, insisting that the application as filed disclosed the BoF approach and the use of an approximate position as two different, independent concepts to determine a location. The BoF approach served to process the reference images into visual features. The appellant conceded that originally filed claim 2 disclosed that the locations of the mobile device were estimated on the server, which was not stated in claim 1. The appellant requested an opportunity to file a further request if that particular aspect were to be decisive

for Article 123(2) EPC since this issue had been raised for the first time at the oral proceedings.

2.6 Even though the appellant's arguments were submitted at a late stage in the oral proceedings, the board admitted them into the appeal proceedings since the board had raised a fresh objection under Article 123(2) EPC in its communication and since, as a basis for claim 1 under discussion, the appellant had referred essentially to passages of the application as filed that had already been cited in the first-instance proceedings.

2.7 However, the board is not convinced that the skilled person would directly and unambiguously derive steps (b) and (c) of claim 1 of the main request from originally filed claims 1 and 2 or any other passage of the application as filed cited by the appellant.

2.7.1 Claim 2 as originally filed specifies in more detail how a subset of reference data (see step (b) of claim 1 as originally filed) is determined. According to originally filed claim 2, the subset is determined on the basis of (1) data received from the mobile device representative of one or more visual features or (2) information indicative of the approximate position of the mobile device, for example a cell ID, or on the basis of the combination of options (1) and (2). Claim 1 under discussion does not refer to data received from the mobile device representative of one or more visual features, or to "information indicative of the approximate position of the mobile device", which is not the same as using an estimated location. Consequently, neither of options (1) and (2) is present in claim 1.

2.7.2 At the oral proceedings, the appellant argued that step (b) of claim 1 under discussion corresponded to the estimation of the location of the mobile device by the server as disclosed in originally filed claim 2, and that it could be claimed as a separate step (b) since this step was logically carried out before the subsets were determined. However, this argument does not explain why it was directly and unambiguously derivable to perform the estimation without using any of options (1) and (2) disclosed in originally filed claim 2. For this reason alone, the board is not convinced that claim 1 of the main request meets the requirements of Article 123(2) EPC.

2.8 Nor is the board convinced that the skilled person would directly and unambiguously derive feature (e) in the overall context of claim 1 from the application as filed (Article 123(2) EPC). Feature (e) specifies that the mobile device is preloaded with "adaptive partial reference data". The cited passage on page 4, line 28, refers to "preloaded adaptive partial vocabularies", i.e. it refers to the BoF approach, which, according to page 5, lines 11 to 14, "reformulates the image retrieval problem into a text retrieval problem by quantizing the high-dimensional feature descriptors into so called visual words with the aid of the k-means algorithm". Since the "vocabulary" corresponds to a set of "visual words", it is part of the BoF approach, but this is not specified in claim 1.

2.8.1 The appellant's argument that the expressions "reference data" and "vocabulary" were used as synonyms in the application as filed is not convincing. For example, page 2, line 10 refers to "reference data" in the context of the prior art, which is not linked to "visual words" or a "vocabulary". By contrast, the expression "visual words" is used in the context of the BoF approach in the description (see e.g. page 3, lines 8 and 9; page 4, lines 12 to 16; page 5, lines 11 to 17). The description introduces the concept of "adaptive partial vocabularies" starting from page 6, line 1, where it is explicitly presented as a way of improving the performance of the BoF approach. The terminology of "visual words" and "vocabulary" in the context of image retrieval stems from the use of retrieval techniques developed originally for text retrieval, i.e. for documents consisting of words and document collections defining a vocabulary. As the application itself explains on page 5, lines 11 to 12, the "BoF approach reformulates the image retrieval problem into a text retrieval problem". Consequently, the skilled person reading the application as filed would understand that using visual words and a vocabulary is directly linked to the BoF approach and would not directly and unambiguously infer that visual words and a vocabulary are concepts separate from the BoF approach that could be used in isolation or even generalised to other content-based image retrieval approaches using "adaptive partial reference data" in general.

The fact that the application as filed also uses the expression "reference data" on page 7, lines 1 to 5 means merely that it uses this more general expression in the context of the BoF approach. However, the skilled person would not directly and unambiguously derive from this passage that "reference data" and "vocabulary" are used as synonyms; at best they would infer that a "vocabulary" is a specific example of "reference data".

2.9 In view of the above, the board concludes that the subject-matter of claim 1 of the main request extends beyond the content of the application as filed (Article 123(2) EPC).

Auxiliary request I

3. Added subject-matter

3.1 The appellant argued that the additional feature "a pose estimation" (see point X. above) was disclosed on page 4, lines 27 to 28 of the application as filed. The cited sentence reads: "The pose estimation is performed on the mobile device utilizing these preloaded adaptive partial vocabularies." In the board's view, this sentence appears to be insufficient as a basis since it refers to adaptive partial vocabularies whereas step (e) of claim 1 refers to adaptive partial reference data (see also points 2.8 and 2.8.1 above).

3.2 Since the appellant did not present any written arguments against this objection and relied only on its written submissions at the oral proceedings, the board concludes that claim 1 of auxiliary request I infringes Article 123(2) EPC.

Auxiliary request II

4. Claim 1 of auxiliary request II differs from claim 1 of the main request in that it adds the text "to allow for a local pose estimation on the mobile device within a limited area" at the end of step (d) and amends step (f) as in auxiliary request I (see point XI. above).

5. Article 123(2) EPC

5.1 The objection under Article 123(2) EPC against auxiliary request I also applies to auxiliary request II as the contested part of the wording of claim 1 is identical in both requests. Since the appellant did not present any written arguments against this objection and relied on its written submissions for auxiliary request II at the oral proceedings, the board concludes that claim 1 of auxiliary request II infringes Article 123(2) EPC.

Auxiliary request III

6. Admissibility under Article 12(4) RPBA 2020

6.1 Current auxiliary request III differs from auxiliary request III considered in the decision under appeal in that it amends step (f) as detailed above (see point XII.).

6.2 The board has discretion under Article 12(4) RPBA 2020 to admit auxiliary request III. The amendment corresponds to an attempt to address a ground for refusal of the decision under appeal, and the board could easily deal with this request in a procedurally efficient manner in the circumstances of this case. In view of this, the board admitted auxiliary request III into the appeal proceedings.

7. Added subject-matter

7.1 The appellant cited page 8, lines 13 to 16 as a basis for the amendments made in auxiliary request III.

7.2 In its preliminary opinion, the board was not convinced that the amendments made in claim 1 were directly and unambiguously derivable from the cited passage (Article 123(2) EPC). Page 8, lines 8 to 16 of the description discloses sending relevant visual words together with their inverted files at about 10 frames per second (fps) to the client to allow for local pose estimation. Moreover, the cited passage refers to neighbouring panoramas with a distance ranging between 12 and 17 metres. The wording of claim 1 deviates substantially from this disclosure. For example, it does not mention a pose estimation, neighbouring panoramas or visual words and their inverted files.

7.3 The appellant did not submit any arguments in reply to the board's objection against auxiliary request III. Consequently, claim 1 of auxiliary request III infringes Article 123(2) EPC.

Auxiliary request IV

8. Admissibility under Article 13(2) RPBA 2020

Auxiliary request IV was filed after oral proceedings had been arranged, so its admittance is subject to the board's discretion on account of the stringent criteria set out in Article 13(2) RPBA 2020. Auxiliary request IV adds a number of features to independent claim 1 which had not been present in the independent claims of the claim requests filed with the statement of grounds of appeal. Nevertheless, as auxiliary request IV constitutes an attempt to overcome objections under Articles 84 and 123(2) EPC raised for the first time in the board's communication under Article 15(1) RPBA 2020, the board admits auxiliary request IV into the appeal proceedings.

9. Clarity - Article 84 EPC

9.1 At the oral proceedings, the board raised, for the first time, several clarity objections against claim 1 of auxiliary request IV. Since it is only necessary to decide on one of the relevant objections in order to arrive at a conclusion on auxiliary request IV, the board will in the following only consider the objection that the wording "a sufficiently high probability" in step (e) of claim 1 of auxiliary request IV is unclear.

9.2 At the oral proceedings, the appellant replied to this objection, stating that the contested wording had been taken from page 9, lines 4 to 6 of the description. The appellant confirmed that this wording could indeed be unclear and did not present any arguments against that objection. Rather, it requested time to draft and file a further auxiliary request, i.e. auxiliary request V, in response to the clarity objections raised by the board.

9.3 After deliberation, the board confirmed its preliminary opinion that a "sufficiently high probability" was not clear in the context of claim 1. Consequently, claim 1 of auxiliary request IV does not meet the requirements of Article 84 EPC.

Auxiliary request V

10. Admissibility under Article 13(2) RPBA 2020

10.1 Under Article 13(2) RPBA 2020 any amendment to a party's appeal case made after notification of a summons to oral proceedings is, in principle, not to be taken into account unless there are exceptional circumstances, which have been justified with cogent reasons by the party concerned.

10.2 Auxiliary request V was filed at the oral proceedings after the appellant had been heard on several clarity objections raised by the board against claim 1 of auxiliary request IV. Consequently, auxiliary request V was filed after notification of the summons to oral proceedings and Article 13(2) RPBA 2020 applies.

10.3 According to the appellant, auxiliary request V was filed in direct response to clarity objections raised by the board for the first time at the oral proceedings, with a view to overcoming those objections. This course of action in the appeal proceedings constituted exceptional circumstances. Consequently, the appellant's request should be admitted into the appeal proceedings under Article 13(2) RPBA 2020 in view of the appellant's right to be heard.

10.4 With regard to the admissibility of auxiliary request V, the board considers that the fact that it raised fresh objections under Article 84 EPC against claim 1 of auxiliary request IV at the oral proceedings does not constitute exceptional circumstances under Article 13(2) RPBA 2020, the reasons being as follows.

Auxiliary request IV was filed for the first time in response to the board's communication under Article 15(1) RPBA 2020, which had raised fresh issues under Article 84 EPC and Article 123(2) EPC. Consequently, the oral proceedings were the first opportunity for the board to raise objections against auxiliary request IV, and the board could be expected to raise fresh objections (in particular under Articles 84 and 123(2) EPC) against this request at the oral proceedings, in the event that said request gave rise to such objections.

Moreover, the board's clarity objection against auxiliary request IV (see point 9.1 above) falls under the board's clarity objection raised in point 9.3 of its communication under Article 15(1) RPBA 2020 against the word "adaptive" in feature (e) of claim 1, which the appellant tried to overcome with an amendment containing the contested wording "sufficiently high probability" (see the appellant's letter of 9 January 2023, page 5, third paragraph). When the appellant argued in favour of the admissibility of auxiliary request V, it did not explain why the board's clarity objection did not fall under the objections raised in the board's summons.

In the circumstances of the case in hand, the fact that the board raised clarity objections against claim 1 of auxiliary request IV at the oral proceedings merely represents the normal course of action in appeal proceedings that the appellant had to have expected (see also decision T 2214/15, Reasons 5.3 and 5.4). Consequently, no exceptional circumstances can be seen in the course of action taken at the oral proceedings before the board that could justify the admission of auxiliary request V under Article 13(2) RPBA 2020.

10.5 The appellant did not invoke any other exceptional circumstances, nor can the board identify any. For these reasons, auxiliary request V was not admitted into the proceedings under Article 13(2) RPBA 2020.

10.6 The board remarks that for the above consideration of admissibility it is irrelevant whether claim 1 of auxiliary request V at least prima facie overcomes any or all of the clarity objections raised by the board for auxiliary request IV.

The objection raised under Rule 106 EPC

11. At the oral proceedings, the appellant raised an objection under Rule 106 EPC (see above, point VII.) since, in its opinion, the board had violated its right to be heard under Article 113 EPC during the oral proceedings before the board.

11.1 The appellant submitted that the amended features were in part based on original claims 2 and 3 and argued that the board had raised fresh objections under Article 84 EPC regarding features of claim 1 of auxiliary request IV that had been part of the claims as originally filed and had never been objected to before.

In the board's view, this argument is irrelevant for the clarity objection considered above under point 9.1, which concerned the expression "a sufficiently high probability", i.e. an amended part of step (e) of claim 1 of auxiliary request IV that was based on the description, page 9, lines 4 to 6. The board remarks that this argument does not apply to further clarity objections raised by the board at the oral proceedings against claim 1 of auxiliary request IV either, in particular the wording "which is computed using inverted files" and the wording of step (b) and step (c), first three lines, in combination, since the contested wording of claim 1 of auxiliary request IV was not present in the claims as originally filed.

11.2 The appellant also argued that the clarity objections raised "were not identified in a specific and conclusive manner, but by way of example only".

However, the board is under no obligation to raise all possible clarity (or other) objections against a request newly filed during the appeal proceedings, the admissibility of which is to be assessed under Article 13(2) RPBA 2020 and which has been drafted in a way that gives rise to numerous new clarity objections. Hence, it is irrelevant that the board informed the appellant at the oral proceedings that it had identified additional clarity objections as it was sufficient and appropriate to raise only those which the board considered to be particularly relevant for the decision. Moreover, it was apparent to the board that the appellant had understood which features of claim 1 had been objected to and that it had been able to file a further request - auxiliary request V - aimed at overcoming all of the objections raised. Consequently, contrary to the appellant's assertions, it had evidently had enough time to consider the board's objections. Indeed, the appellant requested and obtained a break in the oral proceedings to prepare its auxiliary request V and was able to file this request.

11.3 The board disagrees with the appellant's argument that not admitting auxiliary request V violated the appellant's right to be heard under Article 111 EPC. Article 13(2) RPBA 2020 explicitly specifies that late-filed requests are, in principle, not to be taken into account unless there are exceptional circumstances; there were no exceptional circumstances that needed to be considered for the admissibility of auxiliary request V.

11.4 It is also evident from the reasons given in points 12. and 13. below that, at the oral proceedings, the appellant was heard on its two procedural requests, i.e. the request for the application to be remitted for further prosecution before the department of first instance and the request for the appeal proceedings to be continued in writing. As there is no absolute right for either a remittal or for continuation in writing, the board's decision to reject the appellant's procedural requests does not constitute a violation of its right to be heard under Article 113(1) EPC.

11.5 In view of the above, the board concludes that there has been no violation of the appellant's right to be heard in the appeal proceedings. Consequently, the appellant's objection under Rule 106 EPC is dismissed.

The appellant's request for the case to be remitted for further prosecution to the department of first instance

12. Under Article 111(1) EPC, the board has discretion as to whether it decides a case itself or remits it to the department of first instance. There is no absolute right to have an issue decided by two instances (see, for example, decisions T 133/87, Reasons 2, and R 9/10, Reasons 8 to 10). In the interests of procedural economy, the board may decide the case even if a decisive issue has not been dealt with by the department of first instance (see, for example, decision T 1376/07, Reasons 3.2).

12.1 Under Article 11 RPBA 2020, a case is not remitted to the department whose decision was appealed for further prosecution, unless special reasons present themselves for doing so. As a rule, fundamental deficiencies apparent in the proceedings before that department constitute such special reasons.

12.2 The appellant argued that the decision under appeal was based only on inventive step. Consequently, the board only had to review inventive step in the appeal proceedings. However, the board had raised fresh objections under Articles 84 and 123(2) EPC and admitted auxiliary request IV filed in response to those fresh objections. Since the first-instance proceedings dealt with completely different objections from those raised by the board, it was justified to remit the case to the department of first instance.

12.3 The board does not consider that the fresh clarity objections raised by the board against auxiliary request IV could be regarded as a special reason as per Article 11 RPBA 2020 for remitting the case. In this context, the board notes that auxiliary request IV was admitted by the board but gave rise to fresh clarity objections. The board consequently raised them as it considered that doing so was appropriate and justified in order to deal with the appellant's request and bring the proceedings to an end.

Furthermore, the appellant could have submitted its request to remit the case for further prosecution on the basis of auxiliary request IV as early as with its response to the board's preliminary opinion. Since the appellant filed a new request, it had to expect that this request could give rise to fresh objections. Yet it did not submit its request for remittal until a very late stage at the oral proceedings, after it had been heard on both the clarity issues raised against auxiliary request IV and the admissibility of auxiliary request V. Consequently, the appellant's request was also filed late, which is another reason for rejecting it.

12.4 Under these circumstances, the board considers it justified to reach a decision on the appeal case without remittal and rejects the appellant's request for the appeal case to be remitted to the department of first instance for further prosecution.

The appellant's request for the appeal proceedings to be continued in writing

13. The appellant requested that the appeal proceedings be continued in writing, thus giving the appellant sufficient time to deal with the board's fresh clarity objections.

13.1 However, at the end of the oral proceedings, the board was in a position to decide on all of the appellant's substantive requests, including auxiliary requests IV and V, so there was no reason to continue the appeal proceedings in writing and delay a decision on the appellant's case. Allowing the appellant's request would have clearly run counter to the need for procedural economy and the public interest that the appeal case be settled in due time. Moreover, it appears that the appellant had had enough time at the oral proceedings to file a further request, namely auxiliary request V, in order to deal with all of the board's objections. The fact that the board considered auxiliary request V to be inadmissible for procedural reasons cannot be a reason to continue the appeal proceedings in writing in order to give the appellant more time to formulate and submit further requests.

13.2 In view of the above, the appellant's request for the appeal proceedings to be continued in writing is rejected (see also Article 15(6) RPBA 2020).

Conclusion

14. Since none of the appellant's substantive requests can form the basis for the grant of a patent and the appellant's procedural requests are rejected, the appeal is to be dismissed.