[01/02] x11grab: add followmouse option.

Message ID CAFPbVP87yT+sa8Kr0E0nFEH=roRvKM7xk4W5Q6CJ=P0=Lu9bnw@mail.gmail.com
State Superseded
Headers show

Commit Message

Yu-Jie Lin July 30, 2011, 12:26 p.m.
Hi,

Example commands:

    ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse' /tmp/out.mpg
    ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse=100' /tmp/out.mpg

In first command, the grabbing region moves as mouse moves, mouse
pointer will be kept at center unless close to screen edges. The
second command, the region only moves when mouse pointer reaches
within 100 pixels to the edges of current grabbing region.

There will be a second patch to be posted right after this, which
renders a rectangle box to indicate current grabbing region.

(Aside: I have tried to submit a similar patch to FFmpeg-devel, but
after a week with no response, so I decided to continue making the
indication box and forgot the patch submission. Yesterday, I saw the
forwarded notice to FFmpeg-devel about the upcoming (or ongoing
already) major changes on libav. That's my first time heard of libav,
then I checked the logs of x11grabc, it seems the recent changes were
made from libav. So, I think I should try here instead of bumping my
post there.)

Sincerely yours,
Yu-Jie Lin

Comments

Luca Barbato July 30, 2011, 1:13 p.m. | #1
On 7/30/11 2:26 PM, Yu-Jie Lin wrote:
> Hi,
>
> Example commands:
>
>      ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse' /tmp/out.mpg
>      ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse=100' /tmp/out.mpg
>
> In first command, the grabbing region moves as mouse moves, mouse
> pointer will be kept at center unless close to screen edges. The
> second command, the region only moves when mouse pointer reaches
> within 100 pixels to the edges of current grabbing region.
>
> There will be a second patch to be posted right after this, which
> renders a rectangle box to indicate current grabbing region.

Using only the avoption should be ok.

Please fix

+    int follow_mouse;        /**< Set by a private option. */

so the var is aligned with the other in the struct.

Beside that the patch is quite interesting, thanks to submitting it.

lu
Anton Khirnov July 30, 2011, 1:36 p.m. | #2
Hi,

On Sat, 30 Jul 2011 20:26:32 +0800, Yu-Jie Lin <livibetter@gmail.com> wrote:
> Hi,
> 
> Example commands:
> 
>     ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse' /tmp/out.mpg
>     ./ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse=100' /tmp/out.mpg
> 
> In first command, the grabbing region moves as mouse moves, mouse
> pointer will be kept at center unless close to screen edges. The
> second command, the region only moves when mouse pointer reaches
> within 100 pixels to the edges of current grabbing region.
> 
> There will be a second patch to be posted right after this, which
> renders a rectangle box to indicate current grabbing region.
> 
> (Aside: I have tried to submit a similar patch to FFmpeg-devel, but
> after a week with no response, so I decided to continue making the
> indication box and forgot the patch submission. Yesterday, I saw the
> forwarded notice to FFmpeg-devel about the upcoming (or ongoing
> already) major changes on libav. That's my first time heard of libav,
> then I checked the logs of x11grabc, it seems the recent changes were
> made from libav. So, I think I should try here instead of bumping my
> post there.)
> 
> Sincerely yours,
> Yu-Jie Lin
> From 9d3505ae4a00f6576048f4058aaeaff902ee0157 Mon Sep 17 00:00:00 2001
> From: Yu-Jie Lin <livibetter@gmail.com>
> Date: Sat, 30 Jul 2011 18:46:36 +0800
> Subject: [PATCH 1/2] x11grab: add followmouse option.
> 
> Two modes of followmouse are added: a) appending ";followmouse" to input
> file name will move grabbing region to where mouse pointer at center;
> b) appending ";followmouse=PIXELES" will only move when pointer reaches
> with in PIXELES to the edge of grabbing region.
> 
> Signed-off-by: Yu-Jie Lin <livibetter@gmail.com>
> ---
>  doc/ffmpeg.texi       |   16 +++++++++++
>  doc/indevs.texi       |   14 ++++++++++
>  libavdevice/x11grab.c |   69 ++++++++++++++++++++++++++++++++++++++++++++++---
>  3 files changed, 95 insertions(+), 4 deletions(-)
> 
> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi
> index 4dca5d8..9b3728b 100644
> --- a/doc/ffmpeg.texi
> +++ b/doc/ffmpeg.texi
> @@ -903,6 +903,22 @@ ffmpeg -f x11grab -s cif -r 25 -i :0.0+10,20 /tmp/out.mpg
>  0.0 is display.screen number of your X11 server, same as the DISPLAY environment
>  variable. 10 is the x-offset and 20 the y-offset for the grabbing.
>  
> +@example
> +ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse' /tmp/out.mpg
> +@end example
> +
> +0.0 is display.screen number of your X11 server, same as the DISPLAY environment
> +variable. Grabbing region follows the mouse pointer, which stays at center of
> +grabbing region.
> +
> +@example
> +ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse=100' /tmp/out.mpg
> +@end example
> +
> +0.0 is display.screen number of your X11 server, same as the DISPLAY environment
> +variable. Grabbing region follows the mouse pointer, but only when mouse
> +pointer reaches within 100 pixels to the edge of grabbing region.
> +
>  @section Video and Audio file format conversion
>  
>  Any supported file format and protocol can serve as input to ffmpeg:
> diff --git a/doc/indevs.texi b/doc/indevs.texi
> index c5e04b0..3876807 100644
> --- a/doc/indevs.texi
> +++ b/doc/indevs.texi
> @@ -227,6 +227,7 @@ This device allows to capture a region of an X11 display.
>  The filename passed as input has the syntax:
>  @example
>  [@var{hostname}]:@var{display_number}.@var{screen_number}[+@var{x_offset},@var{y_offset}]
> +[;followmouse[=@var{follow_mouse}]]
>  @end example
>  
>  @var{hostname}:@var{display_number}.@var{screen_number} specifies the
> @@ -251,4 +252,17 @@ ffmpeg -f x11grab -r 25 -s cif -i :0.0 out.mpg
>  ffmpeg -f x11grab -25 -s cif -i :0.0+10,20 out.mpg
>  @end example
>  
> +If "followmouse" is specified, then the grabbing region follows the mouse
> +pointer and keeps pointer at center of region; and if @var{follow_mouse} is also
> +specified, then grabbing region only moves when mouse pointer reaches within
> +@var{follow_mouse} pixels to the edge of region.
> +
> +For example:
> +@example
> +ffmpeg -f x11grab -r 25 -s cif -i ':0.0;followmouse' out.mpg
> +
> +# Moving grabbing region only when mouse reaches within 100 pixels to edge
> +ffmpeg -f x11grab -r 25 -s cif -i ':0.0;followmouse=100' out.mpg
> +@end example
> +
>  @c man end INPUT DEVICES
> diff --git a/libavdevice/x11grab.c b/libavdevice/x11grab.c
> index 80507ab..c04c198 100644
> --- a/libavdevice/x11grab.c
> +++ b/libavdevice/x11grab.c
> @@ -71,6 +71,7 @@ struct x11_grab
>      int use_shm;             /**< !0 when using XShm extension */
>      XShmSegmentInfo shminfo; /**< When using XShm, keeps track of XShm infos */
>      int  draw_mouse;         /**< Set by a private option. */
> +    int follow_mouse;        /**< Set by a private option. */
>      char *framerate;         /**< Set by a private option. */
>  };
>  
> @@ -95,6 +96,9 @@ x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
>      XImage *image;
>      int x_off = 0;
>      int y_off = 0;
> +    int screen;
> +    int screen_w, screen_h;
> +    Window w;
>      int use_shm;
>      char *param, *offset;
>      int ret = 0;
> @@ -105,8 +109,15 @@ x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
>      if (offset) {
>          sscanf(offset, "%d,%d", &x_off, &y_off);
>          x11grab->draw_mouse = !strstr(offset, "nomouse");
> -        *offset= 0;
>      }
> +    x11grab->follow_mouse = 0;
> +    if (offset = strstr(param, ";followmouse")) {
> +        x11grab->follow_mouse = -1;
> +        sscanf(offset, ";followmouse=%d", &x11grab->follow_mouse);
> +    }
> +    /* trim dpyname to only [hostname]:<display_number>.<screen_number> */
> +    if (offset = strpbrk(param, "+;"))
> +        *offset = 0;
>  

As Luca already said, this part is unnecessary. Stacking unrelated
things onto "filename" is a relic from the time when we didn't have
demuxer-specific option. Now we do, so there's no need for this anymore.
fftools will automatically generate a commandline option '-follow_mouse
<number>' from the AVOption definition below.

>      if ((ret = av_parse_video_size(&x11grab->width, &x11grab->height, x11grab->video_size)) < 0) {
>          av_log(s1, AV_LOG_ERROR, "Couldn't parse video size.\n");
> @@ -141,6 +152,19 @@ x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
>      }
>      av_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */
>  
> +    screen = DefaultScreen(dpy);
> +    screen_w = DisplayWidth(dpy, screen);
> +    screen_h = DisplayHeight(dpy, screen);
> +

You can move those two along with their definitions inside the if(),
they're not used outside.

> +    if (x11grab->follow_mouse) {
> +        XQueryPointer(dpy, RootWindow(dpy, screen), &w, &w, &x_off, &y_off, &ret, &ret, &ret);
> +        x_off -= x11grab->width >> 1;
> +        y_off -= x11grab->height >> 1;

There's no shame in using / 2 ;)

> +        x_off = FFMIN(FFMAX(x_off, 0), screen_w - x11grab->width);
> +        y_off = FFMIN(FFMAX(y_off, 0), screen_h - x11grab->height);
> +        av_log(s1, AV_LOG_INFO, "followmouse is enabled, resetting grabbing region to x: %d y: %d\n", x_off, y_off);
> +    }
> +
>      use_shm = XShmQueryExtension(dpy);
>      av_log(s1, AV_LOG_INFO, "shared memory extension %s found\n", use_shm ? "" : "not");
>  
> @@ -171,7 +195,7 @@ x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
>              goto out;
>          }
>      } else {
> -        image = XGetImage(dpy, RootWindow(dpy, DefaultScreen(dpy)),
> +        image = XGetImage(dpy, RootWindow(dpy, screen),
>                            x_off,y_off,
>                            x11grab->width, x11grab->height,
>                            AllPlanes, ZPixmap);
> @@ -374,6 +398,13 @@ x11grab_read_packet(AVFormatContext *s1, AVPacket *pkt)
>      int x_off = s->x_off;
>      int y_off = s->y_off;
>  
> +    int screen;
> +    Window root, w;
> +    int screen_w;
> +    int screen_h;
> +    int x, y, _;
> +    int follow_mouse = s->follow_mouse;
> +

Again, variables only used inside the if(follow_mouse) should be
declared there. And the names could use some improving, like 'x' ->
'pointer_x' or something like that is more readable IMO/.

>      int64_t curtime, delay;
>      struct timespec ts;
>  
> @@ -400,12 +431,41 @@ x11grab_read_packet(AVFormatContext *s1, AVPacket *pkt)
>      pkt->size = s->frame_size;
>      pkt->pts = curtime;
>  
> +    screen = DefaultScreen(dpy);
> +    root = RootWindow(dpy, screen);
> +    if (follow_mouse) {
> +        screen_w = DisplayWidth(dpy, screen);
> +        screen_h = DisplayHeight(dpy, screen);
> +        XQueryPointer(dpy, root, &w, &w, &x, &y, &_, &_, &_);
> +        if (follow_mouse == -1) {
> +            // follow the mouse, put it at center of grabbing region
> +            if ((x_off <= screen_w - s->width && x_off >= 0) && x - s->width >> 1 != s->x_off)

I don't understand the purpose of those two if()s, if I understand this
correctly, the first condition should always be true, the second would
evaluate to 0 in the line below if it were false, so it's redundant.

> +                x_off += copysign(abs(x - s->width >> 1 - s->x_off), x - s->width >> 1 - s->x_off);

Isn't copysign(abs(foo), foo) == foo?

Patch

From 9d3505ae4a00f6576048f4058aaeaff902ee0157 Mon Sep 17 00:00:00 2001
From: Yu-Jie Lin <livibetter@gmail.com>
Date: Sat, 30 Jul 2011 18:46:36 +0800
Subject: [PATCH 1/2] x11grab: add followmouse option.

Two modes of followmouse are added: a) appending ";followmouse" to input
file name will move grabbing region to where mouse pointer at center;
b) appending ";followmouse=PIXELES" will only move when pointer reaches
with in PIXELES to the edge of grabbing region.

Signed-off-by: Yu-Jie Lin <livibetter@gmail.com>
---
 doc/ffmpeg.texi       |   16 +++++++++++
 doc/indevs.texi       |   14 ++++++++++
 libavdevice/x11grab.c |   69 ++++++++++++++++++++++++++++++++++++++++++++++---
 3 files changed, 95 insertions(+), 4 deletions(-)

diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi
index 4dca5d8..9b3728b 100644
--- a/doc/ffmpeg.texi
+++ b/doc/ffmpeg.texi
@@ -903,6 +903,22 @@  ffmpeg -f x11grab -s cif -r 25 -i :0.0+10,20 /tmp/out.mpg
 0.0 is display.screen number of your X11 server, same as the DISPLAY environment
 variable. 10 is the x-offset and 20 the y-offset for the grabbing.
 
+@example
+ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse' /tmp/out.mpg
+@end example
+
+0.0 is display.screen number of your X11 server, same as the DISPLAY environment
+variable. Grabbing region follows the mouse pointer, which stays at center of
+grabbing region.
+
+@example
+ffmpeg -f x11grab -s cif -r 25 -i ':0.0;followmouse=100' /tmp/out.mpg
+@end example
+
+0.0 is display.screen number of your X11 server, same as the DISPLAY environment
+variable. Grabbing region follows the mouse pointer, but only when mouse
+pointer reaches within 100 pixels to the edge of grabbing region.
+
 @section Video and Audio file format conversion
 
 Any supported file format and protocol can serve as input to ffmpeg:
diff --git a/doc/indevs.texi b/doc/indevs.texi
index c5e04b0..3876807 100644
--- a/doc/indevs.texi
+++ b/doc/indevs.texi
@@ -227,6 +227,7 @@  This device allows to capture a region of an X11 display.
 The filename passed as input has the syntax:
 @example
 [@var{hostname}]:@var{display_number}.@var{screen_number}[+@var{x_offset},@var{y_offset}]
+[;followmouse[=@var{follow_mouse}]]
 @end example
 
 @var{hostname}:@var{display_number}.@var{screen_number} specifies the
@@ -251,4 +252,17 @@  ffmpeg -f x11grab -r 25 -s cif -i :0.0 out.mpg
 ffmpeg -f x11grab -25 -s cif -i :0.0+10,20 out.mpg
 @end example
 
+If "followmouse" is specified, then the grabbing region follows the mouse
+pointer and keeps pointer at center of region; and if @var{follow_mouse} is also
+specified, then grabbing region only moves when mouse pointer reaches within
+@var{follow_mouse} pixels to the edge of region.
+
+For example:
+@example
+ffmpeg -f x11grab -r 25 -s cif -i ':0.0;followmouse' out.mpg
+
+# Moving grabbing region only when mouse reaches within 100 pixels to edge
+ffmpeg -f x11grab -r 25 -s cif -i ':0.0;followmouse=100' out.mpg
+@end example
+
 @c man end INPUT DEVICES
diff --git a/libavdevice/x11grab.c b/libavdevice/x11grab.c
index 80507ab..c04c198 100644
--- a/libavdevice/x11grab.c
+++ b/libavdevice/x11grab.c
@@ -71,6 +71,7 @@  struct x11_grab
     int use_shm;             /**< !0 when using XShm extension */
     XShmSegmentInfo shminfo; /**< When using XShm, keeps track of XShm infos */
     int  draw_mouse;         /**< Set by a private option. */
+    int follow_mouse;        /**< Set by a private option. */
     char *framerate;         /**< Set by a private option. */
 };
 
@@ -95,6 +96,9 @@  x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
     XImage *image;
     int x_off = 0;
     int y_off = 0;
+    int screen;
+    int screen_w, screen_h;
+    Window w;
     int use_shm;
     char *param, *offset;
     int ret = 0;
@@ -105,8 +109,15 @@  x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
     if (offset) {
         sscanf(offset, "%d,%d", &x_off, &y_off);
         x11grab->draw_mouse = !strstr(offset, "nomouse");
-        *offset= 0;
     }
+    x11grab->follow_mouse = 0;
+    if (offset = strstr(param, ";followmouse")) {
+        x11grab->follow_mouse = -1;
+        sscanf(offset, ";followmouse=%d", &x11grab->follow_mouse);
+    }
+    /* trim dpyname to only [hostname]:<display_number>.<screen_number> */
+    if (offset = strpbrk(param, "+;"))
+        *offset = 0;
 
     if ((ret = av_parse_video_size(&x11grab->width, &x11grab->height, x11grab->video_size)) < 0) {
         av_log(s1, AV_LOG_ERROR, "Couldn't parse video size.\n");
@@ -141,6 +152,19 @@  x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
     }
     av_set_pts_info(st, 64, 1, 1000000); /* 64 bits pts in us */
 
+    screen = DefaultScreen(dpy);
+    screen_w = DisplayWidth(dpy, screen);
+    screen_h = DisplayHeight(dpy, screen);
+
+    if (x11grab->follow_mouse) {
+        XQueryPointer(dpy, RootWindow(dpy, screen), &w, &w, &x_off, &y_off, &ret, &ret, &ret);
+        x_off -= x11grab->width >> 1;
+        y_off -= x11grab->height >> 1;
+        x_off = FFMIN(FFMAX(x_off, 0), screen_w - x11grab->width);
+        y_off = FFMIN(FFMAX(y_off, 0), screen_h - x11grab->height);
+        av_log(s1, AV_LOG_INFO, "followmouse is enabled, resetting grabbing region to x: %d y: %d\n", x_off, y_off);
+    }
+
     use_shm = XShmQueryExtension(dpy);
     av_log(s1, AV_LOG_INFO, "shared memory extension %s found\n", use_shm ? "" : "not");
 
@@ -171,7 +195,7 @@  x11grab_read_header(AVFormatContext *s1, AVFormatParameters *ap)
             goto out;
         }
     } else {
-        image = XGetImage(dpy, RootWindow(dpy, DefaultScreen(dpy)),
+        image = XGetImage(dpy, RootWindow(dpy, screen),
                           x_off,y_off,
                           x11grab->width, x11grab->height,
                           AllPlanes, ZPixmap);
@@ -374,6 +398,13 @@  x11grab_read_packet(AVFormatContext *s1, AVPacket *pkt)
     int x_off = s->x_off;
     int y_off = s->y_off;
 
+    int screen;
+    Window root, w;
+    int screen_w;
+    int screen_h;
+    int x, y, _;
+    int follow_mouse = s->follow_mouse;
+
     int64_t curtime, delay;
     struct timespec ts;
 
@@ -400,12 +431,41 @@  x11grab_read_packet(AVFormatContext *s1, AVPacket *pkt)
     pkt->size = s->frame_size;
     pkt->pts = curtime;
 
+    screen = DefaultScreen(dpy);
+    root = RootWindow(dpy, screen);
+    if (follow_mouse) {
+        screen_w = DisplayWidth(dpy, screen);
+        screen_h = DisplayHeight(dpy, screen);
+        XQueryPointer(dpy, root, &w, &w, &x, &y, &_, &_, &_);
+        if (follow_mouse == -1) {
+            // follow the mouse, put it at center of grabbing region
+            if ((x_off <= screen_w - s->width && x_off >= 0) && x - s->width >> 1 != s->x_off)
+                x_off += copysign(abs(x - s->width >> 1 - s->x_off), x - s->width >> 1 - s->x_off);
+            if ((y_off <= screen_h - s->height && y_off >= 0) && y - s->height >> 1 != s->y_off)
+                y_off += copysign(abs(y - s->height >> 1 - s->y_off), y - s->height >> 1 - s->y_off);
+        } else {
+            // follow the mouse, but only move the grabbing region when mouse
+            // reaches within certain pixels to the edge.
+            if (x_off < screen_w - s->width && x > x_off + s->width - follow_mouse) {
+                x_off += x - x_off - s->width + follow_mouse;
+            } else if (x_off > 0 && x < x_off + follow_mouse)
+                x_off -= x_off + follow_mouse - x;
+            if (y_off < screen_h - s->height && y > y_off + s->height - follow_mouse) {
+                y_off += y - y_off - s->height + follow_mouse;
+            } else if (y_off > 0 && y < y_off + follow_mouse)
+                y_off -= y_off + follow_mouse - y;
+        }
+        // adjust grabbing region position if it goes out of screen.
+        s->x_off = x_off = FFMIN(FFMAX(x_off, 0), screen_w - s->width);
+        s->y_off = y_off = FFMIN(FFMAX(y_off, 0), screen_h - s->height);
+    }
+
     if(s->use_shm) {
-        if (!XShmGetImage(dpy, RootWindow(dpy, DefaultScreen(dpy)), image, x_off, y_off, AllPlanes)) {
+        if (!XShmGetImage(dpy, root, image, x_off, y_off, AllPlanes)) {
             av_log (s1, AV_LOG_INFO, "XShmGetImage() failed\n");
         }
     } else {
-        if (!xget_zpixmap(dpy, RootWindow(dpy, DefaultScreen(dpy)), image, x_off, y_off)) {
+        if (!xget_zpixmap(dpy, root, image, x_off, y_off)) {
             av_log (s1, AV_LOG_INFO, "XGetZPixmap() failed\n");
         }
     }
@@ -452,6 +512,7 @@  static const AVOption options[] = {
     { "video_size", "A string describing frame size, such as 640x480 or hd720.", OFFSET(video_size), FF_OPT_TYPE_STRING, {.str = "vga"}, 0, 0, DEC },
     { "framerate", "", OFFSET(framerate), FF_OPT_TYPE_STRING, {.str = "ntsc"}, 0, 0, DEC },
     { "draw_mouse", "Draw the mouse pointer.", OFFSET(draw_mouse), FF_OPT_TYPE_INT, { 1 }, 0, 1, DEC },
+    { "follow_mouse", "Follow the mouse pointer.", OFFSET(follow_mouse), FF_OPT_TYPE_INT, { 0 }, -1, INT_MAX, DEC },
     { NULL },
 };
 
-- 
1.7.3.4