Evanescent Thoughts

Evanescent Thoughts

Flash Player 10.2 Workaround to Copy Video Files

with 12 comments


You might wanna check  this to get a context of what this post is about. With the previous versions of flash, I could get a copy of the videos being played in my browser by taking it from the /tmp directory. But with the new flash player, it deletes the file from /tmp after creating it. Why ?? I seriously have no  <insert blasphemy > idea !!. Generally when your browser plays a video using flash player, a process would be fetching the video from the internet, storing it in /tmp directory with a hashed value and the browser tab would be playing it. But now, we need to get around a lot of shit, just to copy your favorite videos. You may ask me why am I so bothered to do this. Answer is that, I generally stream the videos when I have an ultra fast net connection(which doesn’t happen to be my home), store them in my comp and then later watch it on vlc. You might also use some youtube-dl or other addon…. But those generally work only with youtube. Say you had a video which was directly uploaded on facebook or other website, I am not sure if you can copy that. Enough of trying to convince  you, that this is actually something interesting !! Lets get to the problem.

First, we may assume that the video was never saved on the disk. But who would be insane enough to  put videos that size to MBs in your main memory ?? So it has to be on the disk somewhere. When the video is being deleted from the /tmp directory, it doesn’t mean that its actually deleted. Lets take a small de-tour to the OS  basics. When two or more processes access the same file, they all get a file descriptor  and when one process deletes the file, it doesn’t actually get deleted unless all the other process close their own access through their file-descriptor. And here, we have a process which is buffering the video into the disk and later deletes the file from /tmp and another process ( your browser tab) which is still  playing the file. And voila !! We can still recover the video. Now this is no black magic !! You can see countless articles online about how to recover deleted files which are still open by some other process ( I think I have an a post on  that somewhere in my blog too). But what complicates the things ??

  1. Browsers nowadays run each tab as a separate process. So identifying how to access the file descriptors of the particular tab playing the video is the first concern.
  2. I generally stream  multiple videos. So  I sort of need to write a script that finds all deleted video files from /tmp and copy them back.

How to do this ??

  1. Get the PID of all the tabs of your browser, in my case chromium (Google chrome’s momma).
  2. List the open files of all the PIDs we got from above  and see if any of them have a link to a deleted file, specifically in /tmp
  3. Process the output of lsof which also gives the file descriptor.

Here is a run through

sathya@Phoenix:~$ pgrep chromium
2366
2449
2451
2696
2700
2702
2710
2719
2744
2764
4506
4515
4993
5039
5125

Now list the open files of all these process by appending them into a comma separated and grep them for deleted files with /tmp/Fl*. Assume for now that $PIDS has all the PIDs of the tabs with video open.

sathya@Phoenix:~$ lsof -np $PIDS | grep deleted | grep /tmp/Fl*
chromium- 4506 sathya   25r      REG        8,5 33356235   998384 /tmp/FlashXXK9gZKa (deleted)
chromium- 4506 sathya   32r      REG        8,5 10544205   998386 /tmp/FlashXXO0GYZw (deleted)

Now the column 2 gives the PID of the tab and the column 4 gives the file descriptor. I have opened both the videos from the same tab and  hence the PIDs are same. Now , the 4th column gives the file descriptor followed by the permission (which in this case is r (read)).  Now how to get to this fd ?? The /proc exposes the RAM which has all info about the processes to the user and we can get to the file using the file descriptor from there.

sathya@Phoenix:~$ cp /proc/4506/fd/25 ~/Desktop/GotItMF.flv

There !! We have the video file !!  Now all we need to do is, put all this together in a script, which I did.

for i in `pgrep chromium`
do
     PIDS=${PIDS}$i","
done
PIDS=`echo ${PIDS:0:${#PIDS}-1}`
export IFS=""
for i in `lsof -np $PIDS | grep deleted | grep /tmp/Fl*`
do
     PID=`echo $i | cut -d " " -f 2`
     FD=`echo $i | cut -d " " -f 6`
     FD=`echo ${FD:0:${#FD}-1}`
     cp /proc/$PID/fd/$FD ~/Desktop/$PID_$FD.flvdone
export IFS=" "

If you are too scared about meddling with the IFS, use the following which does some clever awk scripting.

for i in `pgrep chromium`
do
     PID=${PID}$i","
done
PID=`echo ${PID:0:${#PID}-1}`
lsof -np $PID | grep deleted | grep /tmp/Fl* | awk '{gsub(/[a-z]*/, "" ,$4)} { system("cp /proc/" $2 "/fd/" $4 " ~/Desktop/" $2 "_" $4 ".flv") } {print "Copied "$2"_"$4".flv"}'

Thanks to Hari for making me revisit bash strings and Jai for the AWKgasm !!

UPDATE : As Rik pointed out in comments, lsof with -n is much faster since a lot of time is wasted in host name lookup (which we obviously dont care abt).. -n makes it not convert the addresses to hostname. Makes it 10X faster !!

UPDATE : I recently switched back to firefox and to make it work on firefox, change the “pgrep  chromium” to “pgrep -f libflashplayer.so”

Advertisements

Written by Sathya Narayanan

February 15, 2011 at 10:23 pm

12 Responses

Subscribe to comments with RSS.

  1. dei! romba scene!

    amm

    February 16, 2011 at 1:44 am

  2. Very nice hacking da! I did not know about this use of /proc before. Cool!

    vijay03

    February 16, 2011 at 5:27 pm

    • @vijay : 🙂

      Sathya Narayanan

      February 16, 2011 at 7:02 pm

  3. In the past I wrote a script that would just move the file out of /tmp. The progressive download will continue without interruption. However, now because it is a copy, how do I know if it is done downloading? I could potentially copy a partially downloaded file…

    mopers

    February 17, 2011 at 10:13 am

    • I generally wait for some time and then run the script .. but you could also use the timestamp value .. see when it was last modified and when wait over it in a loop and then copy. Not sure though. tell me if that works !!

      Sathya Narayanan

      February 17, 2011 at 10:19 am

      • so the file size, 7th value, in lsof line does change as it is being progressively downloaded. so a looping script would just check if the size has change since it last iteration. if it has not, it means the file is ready to be copied. the result is a fairly automated monitor that will copy any progressively downloaded flash file; you don’t have to check manually in your browser whether the progressive download is complete. thanks for the original post….

        mopers

        February 17, 2011 at 9:04 pm

  4. use option “-n” for lsof. The script will run faster! without “-n” 5 secounds, with 0.06 😉

    rik

    February 24, 2011 at 11:58 am

    • Woah !! that did improve the speed drastically !! thanks rik !!

      Sathya Narayanan

      February 24, 2011 at 12:05 pm

  5. Does this work for firefox ??? i have arnd 7 tabs open n still pgrep firefox shows only one process

    Iceman

    September 14, 2011 at 11:32 am

    • check out the last update .. change the pgrep chromium to pgrep -f libflashplayer.so 🙂

      Sathya Narayanan

      September 14, 2011 at 12:43 pm

  6. No this does not work.I am getting error while using script 1 on your blog.If I open more than one youtube videos.

    cp: cannot stat `/proc/2768\n2768\n2768\n2768\n2768/fd/11u\n12u\n13r\n17u\n18′: No such file or directory

    James

    October 4, 2011 at 3:33 pm

    • looking at the \n, i think the shell u are using doesnt let the pids to be broken down and processed. could u give some more detail

      Sathya Narayanan

      January 30, 2012 at 5:21 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: