T 0823/13 (Reproducing device/SONY) 04-07-2018

1. The appeal complies with the provisions referred to in Rule 101 EPC and is therefore admissible.

The invention

2. The application relates to the playback of multiplexed audiovisual (AV) streams. The technical background of the invention is that AV content based on the DVD (Digital Versatile Disc) video standard is recorded in the form of an MPEG (Moving Picture Experts Group) program stream. In the MPEG2 program stream, as shown in Figure 3 in the application, a video stream, a plurality of audio streams and a plurality of sub-picture streams are multiplexed such that the audio streams and the sub-picture streams are AV-synchronised with the video stream (description, paragraph [0003] of the English publication). Generally, a plurality of audio streams are used for recording sound in different languages, and a plurality of sub-picture streams are used for recording subtitles in different languages. The user can interactively select sound or subtitles in a desired language while video is being played back (description, paragraph [0004]).

The invention addresses the problem that according to the known DVD video systems, when switching sound or subtitles while playing back a video program stream, the user can select only from audio streams or sub-picture streams multiplexed into the program stream. Even if another stream having audio streams and subtitles different from a program stream which is currently being played back is available, the user cannot switch sound or subtitles to the audio streams or subtitles in the different stream. Accordingly, the "extensibility in selecting streams" is low (description, paragraphs [0011] and [0012]).

In a practical application scenario, the first AV stream is a movie for which a second AV stream contains the director's commentary. In prior-art player devices, the user cannot switch the sound from the movie sound to the director's commentary (description, paragraph [0067]).

3. The invention proposes solving this problem by allowing the user to select a combination of a first stream of a first multiplexed AV stream (see description, paragraphs [0043] to [0051], Figure 5) and a second stream of a second multiplexed AV stream on the basis of selection information included in the playback management information (description, paragraph [0016]). The first and the second stream are of the same stream type (e.g. audio or video; see description, paragraph [0018]). The playback management information comprises playlists. A playlist specifies the playback order and is a set of playback zones of an AV stream. A playback zone, which is also known as a PlayItem, is indicated by a playback start point and a playback end point on a time axis (description, paragraphs [0047], [0049] and [0050]). A playback path including at least one PlayItem in a playlist is referred to as a "main playback path", and a playback path including at least one SubPlayItem in a playlist is referred to as a "sub playback path". A sub playback path is to be played back in association with the main playback path (description, paragraph [0051], Figures 6 and 11).

The selection information (stored in a so-called stream number table (STN_table)) indicates which combinations of streams consisting of a first selectable stream of the first multiplexed AV stream and a second selectable stream of the second multiplexed AV stream, the first and second selectable streams being of the same stream type, are permissible combinations, i.e. can be combined by the user for playback at the same time (description, for example: paragraphs [0201] to [0211], [0287] to [0293], [0400] to [0418] and [0453] to [0485], Figures 37 to 39 and 44 to 48). One way of representing the permissibility of the combinations of streams is by using a bit (flag) for each combination, which indicates whether or not the combination is permissible. Combining two audio streams means mixing (superimposing) these two audio streams (description, paragraph [0210]). Two video streams can be combined and played back in a picture-in-picture display (description, paragraphs [0381], [0382] and [0442] to [0448]).

Main request - inventive step

4. The Examining Division refused claim 1 of the main request for lack of inventive step over document D4.

This document discloses a playback apparatus and a corresponding method for playing back a multiplexed AV stream read from a Blu-ray Disc (BD) drive. The playback apparatus (D4, paragraphs [0018], [0072]-[0074] and [0089]; Figure 12: BD drive 1; Figure 2: BD-ROM = Read Only Memory) reads playlist information from the BD-ROM. Hence, D4 discloses an apparatus comprising obtaining means for obtaining playback management information.

Document D4 discloses that the playback management information comprises a main playback path defining PlayItems/playback zones of a first multiplexed AV stream, being the main AV clip, comprising a multiplex of video, audio, presentation graphic and interactive graphic streams (D4, paragraphs [0021] to [0023], [0032] to [0038]; Figure 11).

D4 discloses that the playback management information comprises a sub playback path defined on a subclip (a second multiplexed AV stream) with SubPlayItems (D4, paragraphs [0022], [0032], [0033] and [0040] to [0048]; Figure 11).

The PlayItem information includes an IN_time field and an OUT_time field indicating a start and an end of the playback section on a presentation time axis. It also includes a stream number table field (D4, paragraph [0034]; Figure 5).

The stream number table (STN_table; paragraphs [0049] to [0068]) shows, among the elementary streams in the AV clip and its related subclip, elementary streams from which the playback apparatus can select for presentation of the PlayItem. An elementary stream available for presentation is mainly an elementary stream multiplexed in the AV clip specified by the PlayItem, but may also include an elementary stream (such as a textST stream for subtitles) recorded separately from but played together with the AV clip. For each PlayItem, the STN_table associates an entry of each elementary stream available for presentation with an attribute storing data related to the stream, such as a stream coding type (D4, Figures 8 and 9A to 9E; paragraph [0130]). The streams available for presentation are the streams selectable by users to be played back, the selection being enabled by receiving means for receiving a selection (D4, paragraphs [0096] and [0199] and Figure 12: user operation reception unit 29). A SubPlayItem of the sub playback path can be played back in synchronisation with a PlayItem of the main playback path (D4, paragraphs [0047] and [0091]).

4.1 The claimed invention therefore differs from the method of document D4 in that it includes the following features:

(a) the selection information includes information indicating which of at least one stream of a first AV stream can be combined with which of at least one stream of a second AV stream of the same stream type;

(b) receiving means for receiving a selection of a combination of a first stream of the at least one stream of the first AV stream and at least one second stream of the at least one stream of the second AV stream;

(c) combining means for combining said first stream and said at least one second stream so that the streams are played back at the same time.

4.2 In its statement of grounds of appeal, point 2.1, the appellant did not contest the Examining Division's finding that features (a) to (c) specified the differences over D4.

However, the appellant did disagree with the Examining Division's conclusion that there was no technical motivation, when starting from document D4, to combine streams of the same stream type of different AV streams. It argued that the application disclosed that prior-art players had the disadvantage of limited extensibility for selecting streams and that there was a need to increase this extensibility (description, paragraph [0012] and [0013]). Therefore the objective technical problem was how to provide a specific technical implementation for combining streams enabling higher extensibility.

The solution enabled this high extensibility through specifically designed selection information, which also allowed inconsistent combinations of streams to be avoided. There was no pointer to this solution in document D4, which was not concerned at all with combinations of different AV streams.

4.3 The differences over D4 provide the effect that the user can select compatible combinations of streams of the same type of different multiplexed AV streams for playback at the same time.

As disclosed in the application (see the above presentation of the invention), one example of a desirable combination could be to allow a director's commentary to be played back with streams of the movie.

Consequently, at least in general, the allowed combinations of streams of the same stream type are motivated by the content of the streams and not by technical considerations. Hence, the Board judges that the compatibility of the selectable streams, i.e. the permissible selections of combinations, is determined by the content designer of the AV media, in accordance with non-technical considerations such as determining which audio streams are compatible with a movie director's commentary.

In view of the above, the Board considers that in the context of the present case the wish to enable the user to select a combination of specific streams of a set of predetermined combinations of streams of different multiplexed AV streams is a non-technical aim that may legitimately appear in the formulation of the problem as part of the framework of the technical problem that is to be solved, in particular as a constraint that has to be met (see decision T 641/00, OJ EPO 2003, 352).

4.4 It follows that the technical problem to be solved by the skilled person when starting from document D4 can be formulated as how to implement in the playback apparatus of D4 the selection of a combination of two streams of different multiplexed AV streams according to predefined permissible combinations for streams of the same stream type.

4.5 The Board considers that an inventive step could still lie in the specific implementation details of the claimed technical solution. However, the implementation aspects expressed in features (a) to (c) of claim 1 are obvious routine developments which do not require the exercise of inventive skill.

Indeed, feature (a) does not add any implementation details beyond the given constraint.

Feature (b) specifies receiving means to receive the selection of a particular combination of streams from the user. Receiving means for user selections were however already known from D4 (D4, paragraph [0096] and Figure 12: user operation reception unit 29), and the skilled person would adapt these means as a matter of routine to receive a selection of a combination of streams.

Feature (c) adds combining means for combining the selected combination of streams. Combining means such as a mixer for audio streams (description, paragraph [0210]) or a picture-in-picture display for video streams (description, paragraphs [0442] to [0444] and [0453]) were however well-known at the effective filing date of the application, and the skilled person trying to solve the technical problem would have been able to select suitable combining means for the streams without the exercise of inventive skill.

4.6 As the implementation of the claimed solution was obvious for the skilled person, claim 1 of the main request lacks inventive step (Article 56 EPC).

First auxiliary request

5. Claim 1 according to the first auxiliary request adds to claim 1 of the main request the following feature of the selection information:

(d) "wherein said information is provided for each stream of the second AV-stream in respect to each stream of the first AV-stream as a flag".

6. Feature (d) is supported by the STN_table shown in Figure 37 in the application. This table comprises, for each stream of the second AV stream, a bitmap in the "Combination_of_Primary_and_Secondary" field. This bitmap comprises, for each stream of the first AV stream, a bit (flag) defining whether this stream can be combined with the stream of the second AV stream (see Figure 38 in the application).

7. In its statement of grounds of appeal, the appellant argued that the use of flags had the effect that a minimum amount of information was required to avoid the selection of a combination of inconsistent streams. The skilled person would find no motivation for this solution in D4, which did not disclose that the STN_table could include information regarding combinable streams. The specific technical implementation was not suggested.

8. The Board agrees with the contested decision that feature (d) is an obvious implementation detail. The use of a bitmap or of flags for an implementation is obvious, as the skilled person understands that the compatibility of streams can be expressed as a Boolean value and thus be stored in a bit. At the effective filing date, this was a routine design, as the use of flags and bitmaps belonged to the mental furniture of a software developer.

9. It follows that claim 1 of the first auxiliary request lacks inventive step (Article 56 EPC).

Second auxiliary request

10. Claim 1 according to the second auxiliary request amends claim 1 of the first auxiliary request by labelling the selection information relating to the first stream, the second stream and the combinations of streams as first, second and third information and adds that the first information and the second information are separate.

11. The appellant argued that this amendment was based on Figure 37 and paragraph [0401] in the application and had the technical advantage, disclosed in the same paragraph, that the number of streams that could be registered was "effectively used".

The Board understands from paragraph [0401] in the application that the STN_table in Figure 37 contains a set of entries for primary streams and a separate set of entries for secondary streams (both defined in Figure 37 with a "FOR" loop). For each secondary stream there is a bitmap showing the allowed combinations. Hence, this STN_table has a format that supports a compact representation of the primary and secondary streams and their combinations.

12. The features relating to the second type of audio stream ("audio_stream2" in Figure 37) are first introduced in the context of Figure 35 (paragraphs [0301] to [0328]). The primary audio streams ("audio_stream" in Figure 37) are audio streams which are played back preferentially over the secondary audio streams. According to paragraph [0327], "by separately defining two types of audio streams [...] to be played back, the user can select two audio streams as desired from the defined streams". Hence, in the context of the present case, the separate definition of primary and secondary audio streams is motivated by the wish to be able to combine specific audio streams for playback.

However, as discussed above, this wish is, at least in general, not technically motivated. Hence, how many primary or secondary audio streams are defined and which combinations of primary and secondary streams should be available are not issues to be decided by the skilled person. As the claim does not specify any details of the implementation of the "separate" definition, there are no implementation details which could have required the exercise of inventive skill.

For the sake of completeness, the Board remarks that the particular format of the STN_table in Figure 37 may, at least when the number of primary streams to be registered is fixed as stated in the cited paragraph [401], allow a compact representation of the primary and secondary audio streams and their combinations, but the design of such a table format is in any case a matter of routine design for the skilled person, as the advantages of bitmaps versus lists are considered to be well-known in the field of programming.

13. It follows that claim 1 of the second auxiliary request lacks inventive step (Article 56 EPC).

Third auxiliary request

14. Claim 1 according to the third auxiliary request adds to claim 1 of the second auxiliary request the following feature of the selection information:

(e) "controlling means for controlling, based on the selection information, an external display device to display for selection the combinations of the at least one stream of the first AV-stream and the at least one stream of the second AV-stream".

15. With respect to inventive step, the appellant argued that paragraphs [0411] to [0422] described the advantages of the controlling means. The effect was that the user could select a combination of audio streams from a list of predetermined combinations without the need to individually select two audio streams. Feature (e) was not suggested by D4, which was not concerned with such "high extensibility".

16. The Board considers that feature (e) allows users to select only consistent stream combinations, i.e. combinations that can be played back (description, paragraph [0420]). As the consistent combinations are preconfigured, it is straightforward to display them and allow users to select one. As users cannot know which combinations are consistent and thus are allowed combinations, it would be frustrating for them to select many combinations which are not allowed in order to identify an allowed one. It was straightforward to solve this problem by displaying only the allowed (consistent) combinations. The Board considers that, at the effective filing date, it was normal practice in program development to display only allowed input values for selection. Hence, feature (e) is an obvious implementation option.

17. It follows that claim 1 of the third auxiliary request lacks inventive step (Article 56 EPC).

Fourth auxiliary request

18. Claim 1 according to the fourth auxiliary request adds to claim 1 of the second auxiliary request the following features:

(f) the plurality of types of streams are audio streams;

(g) the streams to be combined are of the same coding type;

(h) decoding means for decoding the first stream and the at least one second stream, respectively;

(i) wherein the combining means is configured to perform said combining while the first stream and the at least one second stream are in the process of being decoded by the decoding means;

(j) wherein the combining means is configured to mix audio streams into each other.

19. In the contested decision, the Examining Division argued that the limitation of the combined streams to the same audio coding type (features (f) and (g)) was a normal design option to simplify the decoding operations of playback apparatus and that features (h), (i) and (j) "read on ordinary pipeline processing of a stream, wherein one stream segment is decoded while at the same time an already decoded second stream segment is being combined". Moreover, in point 5.1.4 of the reasons of the contested decision the Examining Division remarked that compressed-domain audio mixing (as disclosed in paragraphs [0299] and [0396] of the application) was known in the field of audio processing at the priority date. It referred to document D8 as an example.

20. As to the inventive merit of the amended subject-matter, the appellant argued that the skilled person would not have obviously implemented the combining of streams of the same coding type only. Nor would he have considered the combining of streams during stream decoding. It argued that the skilled person would not have considered document D8, as he received no motivation in document D4 for combining streams of different AV clips. Moreover, he would not have performed the multiple steps and modifications necessary without an inventive step.

21. The Board agrees with the Examining Division that a limitation to the same audio coding type (features (f) and (g) of claim 1) is an obvious simplification of the device, as it is evident that the combining of streams is less complex if they are of the same coding type.

The Board also agrees with the contested decision that features (h) to (j) can be viewed as pipelined stream processing, which was well known at the effective filing date. For example, document D4 discloses that video, interactive graphics and presentation graphics streams are decoded and then combined into video output (D4, playback apparatus of Figure 12: decoders 4, 9 and 13; adders 11 and 16). It is evident that the decoding and the combining are done concurrently for different parts of the respective streams. For example, D4 discloses in paragraph [0075] that the video decoder decodes video stream packets output from the demultiplexer to obtain uncompressed pictures and writes the uncompressed pictures to the video plane. This processing of video stream packets to obtain uncompressed pictures means that the video stream is processed using pipeline processing. Likewise, the audio decoder of D4 (Figure 12, audio decoder 20, paragraph [0088]) processes the audio stream in a pipelined manner. Document D4 discloses the combining of streams while they are decoded only for generating video output. However, the Board judges that the skilled person, being tasked with generating audio output based on a combination of audio streams, could and would apply the same principle to audio streams without the exercise of inventive skill and thereby arrive at the claimed solution.

22. Consequently, claim 1 of the fourth auxiliary request lacks inventive step (Article 56 EPC).

Fifth auxiliary request

23. Claim 1 according to the fifth auxiliary request adds to claim 1 of the fourth auxiliary request the following features:

(k) the selection information including fourth information indicating attributes of the at least one stream of the first AV stream and the at least one stream of the second AV stream;

(l) means for determining, on the basis of the fourth information in the selection information, whether the playback apparatus has a function of playing back a stream, wherein the determining means are configured to omit said determination for the at least one second stream.

24. In the contested decision, the Examining Division discussed the inventiveness of features (k) and (l) in the reasons for refusing the then pending third auxiliary request. It argued that document D4 already disclosed a functionality check for the reproducibility of streams (D4, paragraphs [0131] to [0138] and [0163]). Moreover, the omission of a reproducibility check was an obvious trade-off between complexity and functionality.

25. The appellant argued that the skilled person would not have arrived at the subject-matter of claim 1 without the exercise of inventive skill, as the omission of the functionality check as disclosed in paragraph [0251] of the description could not have been suggested by D4. The skilled person could not have known that the extensibility of the streams of the second AV stream was low and that, consequently, the possibility that it would contain extended or multiple channel streams was also low.

26. The Board agrees with the Examining Division that the further difference from D4 lies in the omission of the reproducibility check. It agrees that this omission might lead to the disadvantage that the player could not play back the second stream correctly. The introduction of a foreseeable disadvantage cannot be the basis for acknowledging inventive step. Moreover, the appellant's argument that the skilled person could not know that the extensibility for the second audio stream was limited does not take into account that the underlying decision to introduce such a limitation is based on non-technical rather than technical considerations. The Board also agrees with the Examining Division that claim 1 does not specify any such limitation for the second audio stream. Hence, the appellant's arguments do not convince the Board of the inventive merit of claim 1.

27. Hence, claim 1 of the fifth auxiliary request lacks inventive step (Article 56 EPC).

Sixth auxiliary request

28. In its statement of grounds of appeal, the appellant submitted that the claims of the sixth auxiliary request were a combination of the claims of the main request and the first to fifth auxiliary requests and addressed the goal of providing high extensibility in selecting streams to be combined, wherein selecting and combining would be as simple and effective as possible.

29. The Board notes that claim 1 of the sixth auxiliary request differs from claim 1 of the fifth auxiliary request in particular in that it adds feature (e) of the third auxiliary request. The Board considers that this combination of features does not provide any synergistic effect. Hence, in view of its assessment of claim 1 of the higher-ranking requests, the Board judges that the subject-matter of claim 1 lacks inventive step over document D4 (Article 56 EPC).

Conclusion

30. As none of the appellant's requests can form the basis for the grant of a patent, the appeal is to be dismissed.