movenc: add fallback audio track tref support

Message ID 1446593392-28458-1-git-send-email-stebbins@jetheaddev.com
State New
Headers show

Commit Message

John Stebbins Nov. 3, 2015, 11:29 p.m.
This feature allows making associations between audio tracks
that apple players recognize.  E.g. when an ac3 track has a
tref that points to an aac track, devices that don't support
ac3 will automatically fall back to the aac track.

Apple used to *guess* these associations, but new products
(AppleTV 4) no longer guess and this association can only
be made explicitly now using the "fall" tref.
---
 libavformat/movenc.c | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

Uncrustify

Comments

Luca Barbato Nov. 4, 2015, 6:37 a.m. | #1
On 04/11/15 00:29, John Stebbins wrote:
> This feature allows making associations between audio tracks
> that apple players recognize.  E.g. when an ac3 track has a
> tref that points to an aac track, devices that don't support
> ac3 will automatically fall back to the aac track.
> 
> Apple used to *guess* these associations, but new products
> (AppleTV 4) no longer guess and this association can only
> be made explicitly now using the "fall" tref.
> ---
>  libavformat/movenc.c | 18 ++++++++++++++----
>  1 file changed, 14 insertions(+), 4 deletions(-)
> 
> Uncrustify
> 
> diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> index 572e781..a6a9d09 100644
> --- a/libavformat/movenc.c
> +++ b/libavformat/movenc.c
> @@ -2285,10 +2285,20 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
>              mov->tracks[i].tref_id  = mov->tracks[mov->chapter_track].track_id;
>          }
>      for (i = 0; i < mov->nb_streams; i++) {
> -        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
> -            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
> -            mov->tracks[i].tref_id =
> -                mov->tracks[mov->tracks[i].src_track].track_id;
> +        MOVTrack *track = &mov->tracks[i];
> +        if (track->tag == MKTAG('r','t','p',' ')) {
> +            track->tref_tag = MKTAG('h','i','n','t');
> +            track->tref_id = mov->tracks[track->src_track].track_id;
> +        } else if (track->enc->codec_type == AVMEDIA_TYPE_AUDIO) {
> +            AVDictionaryEntry *fallback;
> +            fallback = av_dict_get(track->st->metadata, "fallback", NULL, 0);
> +            if (fallback != NULL) {
> +                int index = strtol(fallback->value, NULL, 0);
> +                if (index >= 0 && index < mov->nb_streams) {
> +                    track->tref_tag = MKTAG('f','a','l','l');
> +                    track->tref_id = mov->tracks[index].track_id;
> +                }
> +            }
>          }
>      }
>  
> 

Martin, if it is ok for you I'd get this in tonight.
Martin Storsjö Nov. 4, 2015, 6:12 p.m. | #2
On Wed, 4 Nov 2015, Luca Barbato wrote:

> On 04/11/15 00:29, John Stebbins wrote:
>> This feature allows making associations between audio tracks
>> that apple players recognize.  E.g. when an ac3 track has a
>> tref that points to an aac track, devices that don't support
>> ac3 will automatically fall back to the aac track.
>>
>> Apple used to *guess* these associations, but new products
>> (AppleTV 4) no longer guess and this association can only
>> be made explicitly now using the "fall" tref.
>> ---
>>  libavformat/movenc.c | 18 ++++++++++++++----
>>  1 file changed, 14 insertions(+), 4 deletions(-)
>>
>> Uncrustify
>>
>> diff --git a/libavformat/movenc.c b/libavformat/movenc.c
>> index 572e781..a6a9d09 100644
>> --- a/libavformat/movenc.c
>> +++ b/libavformat/movenc.c
>> @@ -2285,10 +2285,20 @@ static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
>>              mov->tracks[i].tref_id  = mov->tracks[mov->chapter_track].track_id;
>>          }
>>      for (i = 0; i < mov->nb_streams; i++) {
>> -        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
>> -            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
>> -            mov->tracks[i].tref_id =
>> -                mov->tracks[mov->tracks[i].src_track].track_id;
>> +        MOVTrack *track = &mov->tracks[i];
>> +        if (track->tag == MKTAG('r','t','p',' ')) {
>> +            track->tref_tag = MKTAG('h','i','n','t');
>> +            track->tref_id = mov->tracks[track->src_track].track_id;
>> +        } else if (track->enc->codec_type == AVMEDIA_TYPE_AUDIO) {
>> +            AVDictionaryEntry *fallback;
>> +            fallback = av_dict_get(track->st->metadata, "fallback", NULL, 0);
>> +            if (fallback != NULL) {
>> +                int index = strtol(fallback->value, NULL, 0);
>> +                if (index >= 0 && index < mov->nb_streams) {
>> +                    track->tref_tag = MKTAG('f','a','l','l');
>> +                    track->tref_id = mov->tracks[index].track_id;
>> +                }
>> +            }
>>          }
>>      }
>>
>>
>
> Martin, if it is ok for you I'd get this in tonight.

Hmm, well, metadata is a bit uncomfortable for this, but I don't have a 
better suggestion either. Anton?

// Martin
John Stebbins Nov. 5, 2015, 12:34 a.m. | #3
On Wed, 2015-11-04 at 20:12 +0200, Martin Storsjö wrote:
> On Wed, 4 Nov 2015, Luca Barbato wrote:
> 
> > On 04/11/15 00:29, John Stebbins wrote:
> > > This feature allows making associations between audio tracks
> > > that apple players recognize.  E.g. when an ac3 track has a
> > > tref that points to an aac track, devices that don't support
> > > ac3 will automatically fall back to the aac track.
> > > 
> > > Apple used to *guess* these associations, but new products
> > > (AppleTV 4) no longer guess and this association can only
> > > be made explicitly now using the "fall" tref.
> > > ---
> > >  libavformat/movenc.c | 18 ++++++++++++++----
> > >  1 file changed, 14 insertions(+), 4 deletions(-)
> > > 
> > > Uncrustify
> > > 
> > > diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> > > index 572e781..a6a9d09 100644
> > > --- a/libavformat/movenc.c
> > > +++ b/libavformat/movenc.c
> > > @@ -2285,10 +2285,20 @@ static int mov_write_moov_tag(AVIOContext
> > > *pb, MOVMuxContext *mov,
> > >              mov->tracks[i].tref_id  = mov->tracks[mov
> > > ->chapter_track].track_id;
> > >          }
> > >      for (i = 0; i < mov->nb_streams; i++) {
> > > -        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
> > > -            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
> > > -            mov->tracks[i].tref_id =
> > > -                mov->tracks[mov->tracks[i].src_track].track_id;
> > > +        MOVTrack *track = &mov->tracks[i];
> > > +        if (track->tag == MKTAG('r','t','p',' ')) {
> > > +            track->tref_tag = MKTAG('h','i','n','t');
> > > +            track->tref_id = mov->tracks[track
> > > ->src_track].track_id;
> > > +        } else if (track->enc->codec_type == AVMEDIA_TYPE_AUDIO)
> > > {
> > > +            AVDictionaryEntry *fallback;
> > > +            fallback = av_dict_get(track->st->metadata,
> > > "fallback", NULL, 0);
> > > +            if (fallback != NULL) {
> > > +                int index = strtol(fallback->value, NULL, 0);
> > > +                if (index >= 0 && index < mov->nb_streams) {
> > > +                    track->tref_tag = MKTAG('f','a','l','l');
> > > +                    track->tref_id = mov->tracks[index].track_id;
> > > +                }
> > > +            }
> > >          }
> > >      }
> > > 
> > > 
> > 
> > Martin, if it is ok for you I'd get this in tonight.
> 
> Hmm, well, metadata is a bit uncomfortable for this, but I don't have
> a 
> better suggestion either. Anton?

The other option I considered was AVStream side_data.  It's a bit more
work to do it that way, but if you would like that better *shrug*.
Anton Khirnov Nov. 5, 2015, 12:05 p.m. | #4
Quoting John Stebbins (2015-11-05 01:34:41)
> On Wed, 2015-11-04 at 20:12 +0200, Martin Storsjö wrote:
> > On Wed, 4 Nov 2015, Luca Barbato wrote:
> > 
> > > On 04/11/15 00:29, John Stebbins wrote:
> > > > This feature allows making associations between audio tracks
> > > > that apple players recognize.  E.g. when an ac3 track has a
> > > > tref that points to an aac track, devices that don't support
> > > > ac3 will automatically fall back to the aac track.
> > > > 
> > > > Apple used to *guess* these associations, but new products
> > > > (AppleTV 4) no longer guess and this association can only
> > > > be made explicitly now using the "fall" tref.
> > > > ---
> > > >  libavformat/movenc.c | 18 ++++++++++++++----
> > > >  1 file changed, 14 insertions(+), 4 deletions(-)
> > > > 
> > > > Uncrustify
> > > > 
> > > > diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> > > > index 572e781..a6a9d09 100644
> > > > --- a/libavformat/movenc.c
> > > > +++ b/libavformat/movenc.c
> > > > @@ -2285,10 +2285,20 @@ static int mov_write_moov_tag(AVIOContext
> > > > *pb, MOVMuxContext *mov,
> > > >              mov->tracks[i].tref_id  = mov->tracks[mov
> > > > ->chapter_track].track_id;
> > > >          }
> > > >      for (i = 0; i < mov->nb_streams; i++) {
> > > > -        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
> > > > -            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
> > > > -            mov->tracks[i].tref_id =
> > > > -                mov->tracks[mov->tracks[i].src_track].track_id;
> > > > +        MOVTrack *track = &mov->tracks[i];
> > > > +        if (track->tag == MKTAG('r','t','p',' ')) {
> > > > +            track->tref_tag = MKTAG('h','i','n','t');
> > > > +            track->tref_id = mov->tracks[track
> > > > ->src_track].track_id;
> > > > +        } else if (track->enc->codec_type == AVMEDIA_TYPE_AUDIO)
> > > > {
> > > > +            AVDictionaryEntry *fallback;
> > > > +            fallback = av_dict_get(track->st->metadata,
> > > > "fallback", NULL, 0);
> > > > +            if (fallback != NULL) {
> > > > +                int index = strtol(fallback->value, NULL, 0);
> > > > +                if (index >= 0 && index < mov->nb_streams) {
> > > > +                    track->tref_tag = MKTAG('f','a','l','l');
> > > > +                    track->tref_id = mov->tracks[index].track_id;
> > > > +                }
> > > > +            }
> > > >          }
> > > >      }
> > > > 
> > > > 
> > > 
> > > Martin, if it is ok for you I'd get this in tonight.
> > 
> > Hmm, well, metadata is a bit uncomfortable for this, but I don't have
> > a 
> > better suggestion either. Anton?
> 
> The other option I considered was AVStream side_data.  It's a bit more
> work to do it that way, but if you would like that better *shrug*.

I think side data is the proper way to do this. Metadata is intended to
store only information about the content itself, not the technical
details about how it is encoded.
John Stebbins Nov. 5, 2015, 3:32 p.m. | #5
On Thu, 2015-11-05 at 13:05 +0100, Anton Khirnov wrote:
> Quoting John Stebbins (2015-11-05 01:34:41)
> > On Wed, 2015-11-04 at 20:12 +0200, Martin Storsjö wrote:
> > > On Wed, 4 Nov 2015, Luca Barbato wrote:
> > > 
> > > > On 04/11/15 00:29, John Stebbins wrote:
> > > > > This feature allows making associations between audio tracks
> > > > > that apple players recognize.  E.g. when an ac3 track has a
> > > > > tref that points to an aac track, devices that don't support
> > > > > ac3 will automatically fall back to the aac track.
> > > > > 
> > > > > Apple used to *guess* these associations, but new products
> > > > > (AppleTV 4) no longer guess and this association can only
> > > > > be made explicitly now using the "fall" tref.
> > > > > ---
> > > > >  libavformat/movenc.c | 18 ++++++++++++++----
> > > > >  1 file changed, 14 insertions(+), 4 deletions(-)
> > > > > 
> > > > > Uncrustify
> > > > > 
> > > > > diff --git a/libavformat/movenc.c b/libavformat/movenc.c
> > > > > index 572e781..a6a9d09 100644
> > > > > --- a/libavformat/movenc.c
> > > > > +++ b/libavformat/movenc.c
> > > > > @@ -2285,10 +2285,20 @@ static int
> > > > > mov_write_moov_tag(AVIOContext
> > > > > *pb, MOVMuxContext *mov,
> > > > >              mov->tracks[i].tref_id  = mov->tracks[mov
> > > > > ->chapter_track].track_id;
> > > > >          }
> > > > >      for (i = 0; i < mov->nb_streams; i++) {
> > > > > -        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
> > > > > -            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
> > > > > -            mov->tracks[i].tref_id =
> > > > > -                mov->tracks[mov
> > > > > ->tracks[i].src_track].track_id;
> > > > > +        MOVTrack *track = &mov->tracks[i];
> > > > > +        if (track->tag == MKTAG('r','t','p',' ')) {
> > > > > +            track->tref_tag = MKTAG('h','i','n','t');
> > > > > +            track->tref_id = mov->tracks[track
> > > > > ->src_track].track_id;
> > > > > +        } else if (track->enc->codec_type ==
> > > > > AVMEDIA_TYPE_AUDIO)
> > > > > {
> > > > > +            AVDictionaryEntry *fallback;
> > > > > +            fallback = av_dict_get(track->st->metadata,
> > > > > "fallback", NULL, 0);
> > > > > +            if (fallback != NULL) {
> > > > > +                int index = strtol(fallback->value, NULL, 0);
> > > > > +                if (index >= 0 && index < mov->nb_streams) {
> > > > > +                    track->tref_tag = MKTAG('f','a','l','l');
> > > > > +                    track->tref_id = mov
> > > > > ->tracks[index].track_id;
> > > > > +                }
> > > > > +            }
> > > > >          }
> > > > >      }
> > > > > 
> > > > > 
> > > > 
> > > > Martin, if it is ok for you I'd get this in tonight.
> > > 
> > > Hmm, well, metadata is a bit uncomfortable for this, but I don't
> > > have
> > > a 
> > > better suggestion either. Anton?
> > 
> > The other option I considered was AVStream side_data.  It's a bit
> > more
> > work to do it that way, but if you would like that better *shrug*.
> 
> I think side data is the proper way to do this. Metadata is intended
> to
> store only information about the content itself, not the technical
> details about how it is encoded.
> 

Ok.  I will modify the patch to use side data.

Patch

diff --git a/libavformat/movenc.c b/libavformat/movenc.c
index 572e781..a6a9d09 100644
--- a/libavformat/movenc.c
+++ b/libavformat/movenc.c
@@ -2285,10 +2285,20 @@  static int mov_write_moov_tag(AVIOContext *pb, MOVMuxContext *mov,
             mov->tracks[i].tref_id  = mov->tracks[mov->chapter_track].track_id;
         }
     for (i = 0; i < mov->nb_streams; i++) {
-        if (mov->tracks[i].tag == MKTAG('r','t','p',' ')) {
-            mov->tracks[i].tref_tag = MKTAG('h','i','n','t');
-            mov->tracks[i].tref_id =
-                mov->tracks[mov->tracks[i].src_track].track_id;
+        MOVTrack *track = &mov->tracks[i];
+        if (track->tag == MKTAG('r','t','p',' ')) {
+            track->tref_tag = MKTAG('h','i','n','t');
+            track->tref_id = mov->tracks[track->src_track].track_id;
+        } else if (track->enc->codec_type == AVMEDIA_TYPE_AUDIO) {
+            AVDictionaryEntry *fallback;
+            fallback = av_dict_get(track->st->metadata, "fallback", NULL, 0);
+            if (fallback != NULL) {
+                int index = strtol(fallback->value, NULL, 0);
+                if (index >= 0 && index < mov->nb_streams) {
+                    track->tref_tag = MKTAG('f','a','l','l');
+                    track->tref_id = mov->tracks[index].track_id;
+                }
+            }
         }
     }