rtmp: Don't try to do av_malloc(0)

Message ID 1306339818-16874-1-git-send-email-martin@martin.st
State Superseded
Headers show

Commit Message

Martin Storsjö May 25, 2011, 4:10 p.m.
Some received packets can have size 0. The return value from
av_malloc(0) may be NULL, which is ok if the size was 0. On
OS X, however, the returned pointer is non-null but leads to
crashes when trying to free it.
---
 libavformat/rtmppkt.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

Comments

Kostya Shishkov May 25, 2011, 4:30 p.m. | #1
On Wed, May 25, 2011 at 07:10:18PM +0300, Martin Storsjö wrote:
> Some received packets can have size 0. The return value from
> av_malloc(0) may be NULL, which is ok if the size was 0. On
> OS X, however, the returned pointer is non-null but leads to
> crashes when trying to free it.
> ---
>  libavformat/rtmppkt.c |    4 ++--
>  1 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavformat/rtmppkt.c b/libavformat/rtmppkt.c
> index 63b0628..fd8fdc8 100644
> --- a/libavformat/rtmppkt.c
> +++ b/libavformat/rtmppkt.c
> @@ -233,8 +233,8 @@ int ff_rtmp_packet_write(URLContext *h, RTMPPacket *pkt,
>  int ff_rtmp_packet_create(RTMPPacket *pkt, int channel_id, RTMPPacketType type,
>                            int timestamp, int size)
>  {
> -    pkt->data = av_malloc(size);
> -    if (!pkt->data)
> +    pkt->data = size ? av_malloc(size) : NULL;
> +    if (!pkt->data && size)
>          return AVERROR(ENOMEM);
>      pkt->data_size  = size;
>      pkt->channel_id = channel_id;
> -- 

ok
Ronald Bultje May 25, 2011, 5:21 p.m. | #2
Hi,

On Wed, May 25, 2011 at 12:10 PM, Martin Storsjö <martin@martin.st> wrote:
> -    pkt->data = av_malloc(size);
> -    if (!pkt->data)
> +    pkt->data = size ? av_malloc(size) : NULL;
> +    if (!pkt->data && size)
>         return AVERROR(ENOMEM);

+ if (size) {
  pkt->data = av_malloc(size);
  if (!pkt->data)
      return AVERROR(ENOMEM);
+ }

+ appropriate reindent. Otherwise we could just as well have hacked
av_malloc() to allocate FFMIN(1, size)...

Ronald

Patch

diff --git a/libavformat/rtmppkt.c b/libavformat/rtmppkt.c
index 63b0628..fd8fdc8 100644
--- a/libavformat/rtmppkt.c
+++ b/libavformat/rtmppkt.c
@@ -233,8 +233,8 @@  int ff_rtmp_packet_write(URLContext *h, RTMPPacket *pkt,
 int ff_rtmp_packet_create(RTMPPacket *pkt, int channel_id, RTMPPacketType type,
                           int timestamp, int size)
 {
-    pkt->data = av_malloc(size);
-    if (!pkt->data)
+    pkt->data = size ? av_malloc(size) : NULL;
+    if (!pkt->data && size)
         return AVERROR(ENOMEM);
     pkt->data_size  = size;
     pkt->channel_id = channel_id;