lpcm: fix sample size calculation for 20bit LCPM.

Message ID 20120314210243.004C65E088@aruru.libav.org
State New
Headers show

Commit Message

Janne Grunau March 14, 2012, 9:02 p.m.
Module: libav
Branch: release/0.8
Commit: e1b4614ab463f8ef4e350e0750fdefddea392135

Author:    Ronald S. Bultje <rsbultje@gmail.com>
Committer: Reinhard Tartler <siretart@tauware.de>
Date:      Tue Mar  6 20:08:17 2012 -0800

lpcm: fix sample size calculation for 20bit LCPM.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f1320dc3bed281bb2f3c5531c52b6a6246e2394a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>

---

 libavcodec/pcm-mpeg.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

Patch

diff --git a/libavcodec/pcm-mpeg.c b/libavcodec/pcm-mpeg.c
index 9ab6fc3..f010b97 100644
--- a/libavcodec/pcm-mpeg.c
+++ b/libavcodec/pcm-mpeg.c
@@ -156,7 +156,7 @@  static int pcm_bluray_decode_frame(AVCodecContext *avctx, void *data,
 
     /* There's always an even number of channels in the source */
     num_source_channels = FFALIGN(avctx->channels, 2);
-    sample_size = (num_source_channels * avctx->bits_per_coded_sample) >> 3;
+    sample_size = (num_source_channels * (avctx->sample_fmt == AV_SAMPLE_FMT_S16 ? 16 : 24)) >> 3;
     samples = buf_size / sample_size;
 
     /* get output buffer */