vmnc: check that subrectangles fit into their containing rectangles

Message ID 20160818151755.6BE3B5DACB@aruru.libav.org
State New
Headers show

Commit Message

Janne Grunau Aug. 18, 2016, 3:17 p.m.
Module: libav
Branch: master
Commit: f5d46d332258dcd8ca623019ece1d5e5bb74142b

Author:    Anton Khirnov <anton@khirnov.net>
Committer: Anton Khirnov <anton@khirnov.net>
Date:      Sun Aug 14 10:18:39 2016 +0200

vmnc: check that subrectangles fit into their containing rectangles

Fixes possible invalid writes with corrupted files.

CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

---

 libavcodec/vmnc.c |   16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

Patch

diff --git a/libavcodec/vmnc.c b/libavcodec/vmnc.c
index 3ef2134..7a01f1e 100644
--- a/libavcodec/vmnc.c
+++ b/libavcodec/vmnc.c
@@ -287,12 +287,24 @@  static int decode_hextile(VmncContext *c, uint8_t* dst, GetByteContext *gb,
                     return AVERROR_INVALIDDATA;
                 }
                 for (k = 0; k < rects; k++) {
+                    int rect_x, rect_y, rect_w, rect_h;
                     if (color)
                         fg = vmnc_get_pixel(gb, bpp, c->bigendian);
                     xy = bytestream2_get_byte(gb);
                     wh = bytestream2_get_byte(gb);
-                    paint_rect(dst2, xy >> 4, xy & 0xF,
-                               (wh>>4)+1, (wh & 0xF)+1, fg, bpp, stride);
+
+                    rect_x = xy >> 4;
+                    rect_y = xy & 0xF;
+                    rect_w = (wh >> 4) + 1;
+                    rect_h = (wh & 0xF) + 1;
+
+                    if (rect_x + rect_w > bw || rect_y + rect_h > bh) {
+                        av_log(c->avctx, AV_LOG_ERROR, "Invalid subrect\n");
+                        return AVERROR_INVALIDDATA;
+                    }
+
+                    paint_rect(dst2, rect_x, rect_y,
+                               rect_w, rect_h, fg, bpp, stride);
                 }
             }
         }