T 0634/01 (Method for extracting images/FUJI) 18-01-2005
Download and more information:
Method for extracting object images and method for detecting movements thereof
Inventive step (main and first and second auxiliary requests: no)
Remittal to first instance for further prosecution on the basis of third auxiliay request (yes)
I. This is an appeal against the decision of the examining division to refuse European patent application No. 92115724.4.
II. According to the decision appealed the method of extracting an object image according to claim 1 (in the version as originally filed) was obvious in view of documents D2 (J. S. J. Lee et al., "An Intelligent Real-Time Multiple Moving Object Tracker", SPIE vol. 937, 1988, pp. 328-335) or D3 (G. A. Weller et al., "A gradient comparison method for tracking and focussing purposes", Time-Varying Image Processing And Moving Object Recognition, ed. V. Cappellini, Elsevier 1990, pp. 316-322). Furthermore, since claim 1, which involved causing a view window to travel to a candidate object, was silent on how this was done, it even applied to piecing together a jig-saw puzzle.
III. On appeal, the applicant requested that the decision under appeal be set aside and a patent be granted on the basis of claim 1 as originally filed (main request and first auxiliary request) or on the basis of a newly filed second auxiliary request.
IV. In a communication the Board remarked on what appeared to be a fundamental problem with claim 1, viz. that it did not state how object images were found and recognised ("extracted") but mainly that they were found. The claim seemed to be little more than a reformulation of the problem of "extracting an object image, wherein a predetermined object image is extracted accurately and appropriately from a given image" (as stated in the description, p. 6, l.20, 21). Moreover, the main features of the claim (centre point, view window, extraction area) were not necessarily technical, as demonstrated by the examining division's argumentation.
The Board went on to note that claim 1 of auxiliary request 2 differed from claim 1 of the main request in the feature "wherein said center point is decided based on the information of the shape of the whole object or of a part of the object which falls in a region of the view window". This feature did not seem to change much since, if it referred to the final position of the view window, it was obvious that this position was determined by information about the object to which the window should travel.
V. The appellant replied to the communication of the Board and filed amended claims. Oral proceedings were appointed for 18 January 2005.
VI. With letter dated 20 December 2004 the appellant filed new claims according to a main request and three auxiliary requests.
VII. Claim 1 of the main request read:
"A computer-implemented method for extracting an object image, in which an extraction area for extraction of a candidate for a predetermined object image from an image is determined, the method for extracting an object image comprising the steps of:
i) defining a view window having a center point and a predetermined size on the image,
ii) obtaining information from an area within the view window, and calculating vectors based on the obtained information, wherein each vector indicates a shift amount and a moving direction of the view window;
iii) determining a next position of the center point of the view window based on the obtained information and the calculated vectors so that the center point approaches to the candidate and re-defining the view window centered on said next position of the center point,
iv) repeating the above steps ii) and iii) until the whole candidate is contained in the area within the view window, and
v) determining said extraction area within the view window in accordance with the size and/or the shape of said candidate for the predetermined object image, the center point of said view window being taken as a reference during the determination of said extraction area."
Claim 1 of the first auxiliary request was identical.
Claim 1 of the second auxiliary request contained an additional limitation (below in italics) in feature iii):
"iii) determining a next position of the center point of the view window based on the obtained information of the shape of the whole object or of a part of the object which falls in a region of the view window and the calculated vectors, so that the center point approaches to the candidate, and re-defining the view window centered on said next position of the center point".
VIII. Oral proceedings were held on 18 January 2005. The appellant filed a new version of the set of claims 1 to 63 according to the third auxiliary request.
Claim 1, the only independent claim, read:
"A computer-implemented method for extracting an object image, in which an extraction area for extraction of a candidate for a predetermined object image from an image is determined,
the method for extracting an object image comprising the steps of:
i) cutting out an image, which falls in a region inside of a view window having a predetermined size, from said image,
ii) detecting a contour line of said candidate for the predetermined object image, which line extends in a predetermined direction, from said cut-out image;
iii) extracting all of components of said detected contour line, which are tilted at a predetermined angle with respect to circumferential directions of concentric circles surrounding the center point of said view window, from said detected contour line of said candidate for the predetermined object image,
iv) detecting azimuths and intensities of said extracted components with respect to the center point of said view window, the azimuths and the intensities being detected as azimuth vectors;
v) composing a vector from said azimuth vectors, a vector for a travel of said view window being thereby determined;
vi) causing the center point of said view window to travel in accordance with said vector for the travel of said view window;
vii) repeating the above steps ii) to vi), until the whole candidate is contained in the area within the view window, and
viii) determining an extraction area in accordance with the size and/or the shape of said candidate for the predetermined object image, the center point of said view window, which has thus been caused to travel, being taken as a reference during the determination of said extraction area".
IX. The appellant requested that the decision under appeal be set aside and that a patent be granted on the basis of claim 1 of the main request and first auxiliary request or, alternatively, on the basis of claim 1 of the second auxiliary request, all claim versions as filed with the letter dated 20 December 2004 or to remit the case to the first instance for further prosecution on the basis of the claims 1 to 63 of the third auxiliary request submitted at the oral proceedings.
X. At the end of the oral proceedings the Board announced its decision.
The main and first auxiliary requests
1. The invention relates to a computer-implemented method for locating predetermined objects in a picture, such as human faces. A view window is placed on the picture and the information within it is analyzed. On the basis of that information a vector is calculated and the window is moved accordingly. If something which might be a face or part of a face is identified, the window will eventually be centred on it. The window can adapt to the size or shape of the object found such that the object fills the window to a desired degree (see also fig. 1 and associated text).
2. Claim 1 according to the main and first auxiliary requests being identical, these requests will be treated together.
3. The examining division compared the claimed method with the operation of human vision and this parallel will also be adopted by the Board. The closest prior art is thus taken to be the way a human being searches for an object within his field of view. Assume, for example, that a person standing in a street is looking for a certain building of which he only knows that it is tall and white. He may now be conscious of a whitish surface at the periphery of his field of vision. Turning around in order to examine the object, he fixes his gaze on it. The house being tall he might choose to watch it from a larger distance to get a better view.
4. Considering that previous research in the area of pattern recognition has been strongly inspired by human vision (see the section "Description of the Prior Art" in the present application), it was an obvious aim to design a computer-implemented method for searching for an object which is part of an image by automating steps which a human being would perform in a similar situation. Clad in the language of claim 1 and with the Board's comments in italics, a method inspired by the example outlined above would comprise the steps of:
i) defining a view window (corresponding to the field of vision of the human eye) having a center point and a predetermined size on the image,
ii) obtaining information from an area within the view window (unavoidable since only this information is available) and calculating vectors based on the obtained information, wherein each vector indicates a shift amount and a moving direction of the view window (unavoidable in order to move the view window at all),
iii) determining a next position of the center point of the view window based on the obtained information and the calculated vectors so that the center point approaches the candidate and re-defining the view window centered on said next position of the center point (discrete machine steps corresponding to the movement of the human eye),
iv) repeating the above steps ii) and iii) until the whole candidate is contained in the area within the view window (clearly necessary), and
v) determining said extraction area within the view window in accordance with the size and/or the shape of said candidate for the predetermined object image, the center point of said view window being taken as a reference during the determination of said extraction area (corresponding to the human being looking straight at the object and varying the distance to it).
5. The appellant has argued that the invention is different from human vision in that the eye motion "include saccadic movements", as pointed out in the description, p. 4, l. 2,3. Nevertheless, it appears to the Board that claim 1 corresponds to any eye movement which has the effect of bringing an object from the periphery of the field of view to its centre. How a machine can achieve such centring is perhaps a complex issue, but the solution according to claim 1 is merely to move the view window in accordance with calculated "vectors based on the obtained information". In the Board's view, such a general suggestion is clearly trivial.
6. Thus, the Board finds that the features of claim 1 are merely those which a skilled person would make a machine perform in order to correspond to the way a human being would accomplish the same task of "extracting" objects within his field of vision. Thus, the claimed method amounts to nothing more than the mere idea of automating human behaviour and hence does not involve an inventive step (Article 56 EPC).
The second auxiliary request
7. Compared with the foregoing requests, claim 1 of the second auxiliary request additionally specifies that a next position is determined from "information of the shape of the whole object or of a part of the object which falls in a region of the view window". It might appear doubtful whether this formulation is fully supported by the description since, as shown in connection with figures 39 and 40, information about the position of the candidate mainly determines where to move the view window (this must be so since the view window should approach the candidate), whereas the shape mainly determines what object image is searched for. However, even if this possible objection is neglected and the feature is understood as indicating that the view window will (only) approach objects of a predetermined shape, a human being would in a similar situation do nothing else. For example, as the examining division has pointed out, "a person piecing together a jig-saw puzzle is sequentially looking for candidate puzzle pieces that have particular form (shape, size)..." (decision, p. 3). In any case, the idea of moving the window towards objects having the required shape seems merely to correspond to the desired result to be achieved. Thus, also the invention as defined by claim 1 of the second auxiliary request is obvious (Article 56 EPC).
The third auxiliary request
8. Claim 1 of the third auxiliary request, based on independent claim 3 as initially filed, contains a number of additional features which set out in more detail how the travel vector is computed. The examining division has not yet had an opportunity to examine this considerably limited subject-matter. It is therefore appropriate to remit the case to the first instance, as requested by the appellant, for examination on the basis of the claims of this request (Article 111(1) EPC).
ORDER
For these reasons it is decided that:
1. The decision under appeal is set aside.
2. The case is remitted to the first instance for further prosecution.