Evanescent Thoughts

You might wanna check  this to get a context of what this post is about. With the previous versions of flash, I could get a copy of the videos being played in my browser by taking it from the /tmp directory. But with the new flash player, it deletes the file from /tmp after creating it. Why ?? I seriously have no  <insert blasphemy > idea !!. Generally when your browser plays a video using flash player, a process would be fetching the video from the internet, storing it in /tmp directory with a hashed value and the browser tab would be playing it. But now, we need to get around a lot of shit, just to copy your favorite videos. You may ask me why am I so bothered to do this. Answer is that, I generally stream the videos when I have an ultra fast net connection(which doesn’t happen to be my home), store them in my comp and then later watch it on vlc. You might also use some youtube-dl or other addon…. But those generally work only with youtube. Say you had a video which was directly uploaded on facebook or other website, I am not sure if you can copy that. Enough of trying to convince  you, that this is actually something interesting !! Lets get to the problem.

First, we may assume that the video was never saved on the disk. But who would be insane enough to  put videos that size to MBs in your main memory ?? So it has to be on the disk somewhere. When the video is being deleted from the /tmp directory, it doesn’t mean that its actually deleted. Lets take a small de-tour to the OS  basics. When two or more processes access the same file, they all get a file descriptor  and when one process deletes the file, it doesn’t actually get deleted unless all the other process close their own access through their file-descriptor. And here, we have a process which is buffering the video into the disk and later deletes the file from /tmp and another process ( your browser tab) which is still  playing the file. And voila !! We can still recover the video. Now this is no black magic !! You can see countless articles online about how to recover deleted files which are still open by some other process ( I think I have an a post on  that somewhere in my blog too). But what complicates the things ??

1. Browsers nowadays run each tab as a separate process. So identifying how to access the file descriptors of the particular tab playing the video is the first concern.
2. I generally stream  multiple videos. So  I sort of need to write a script that finds all deleted video files from /tmp and copy them back.

How to do this ??

1. Get the PID of all the tabs of your browser, in my case chromium (Google chrome’s momma).
2. List the open files of all the PIDs we got from above  and see if any of them have a link to a deleted file, specifically in /tmp
3. Process the output of lsof which also gives the file descriptor.

Here is a run through

sathya@Phoenix:~$ pgrep chromium
2366
2449
2451
2696
2700
2702
2710
2719
2744
2764
4506
4515
4993
5039
5125

Now list the open files of all these process by appending them into a comma separated and grep them for deleted files with /tmp/Fl*. Assume for now that $PIDS has all the PIDs of the tabs with video open.

sathya@Phoenix:~$ lsof -np $PIDS | grep deleted | grep /tmp/Fl*
chromium- 4506 sathya   25r      REG        8,5 33356235   998384 /tmp/FlashXXK9gZKa (deleted)
chromium- 4506 sathya   32r      REG        8,5 10544205   998386 /tmp/FlashXXO0GYZw (deleted)

Now the column 2 gives the PID of the tab and the column 4 gives the file descriptor. I have opened both the videos from the same tab and  hence the PIDs are same. Now , the 4th column gives the file descriptor followed by the permission (which in this case is r (read)).  Now how to get to this fd ?? The /proc exposes the RAM which has all info about the processes to the user and we can get to the file using the file descriptor from there.

sathya@Phoenix:~$ cp /proc/4506/fd/25 ~/Desktop/GotItMF.flv

There !! We have the video file !!  Now all we need to do is, put all this together in a script, which I did.

for i in `pgrep chromium`
do
     PIDS=${PIDS}$i","
done
PIDS=`echo ${PIDS:0:${#PIDS}-1}`
export IFS=""
for i in `lsof -np $PIDS | grep deleted | grep /tmp/Fl*`
do
     PID=`echo $i | cut -d " " -f 2`
     FD=`echo $i | cut -d " " -f 6`
     FD=`echo ${FD:0:${#FD}-1}`
     cp /proc/$PID/fd/$FD ~/Desktop/$PID_$FD.flvdone
export IFS=" "

If you are too scared about meddling with the IFS, use the following which does some clever awk scripting.

for i in `pgrep chromium`
do
     PID=${PID}$i","
done
PID=`echo ${PID:0:${#PID}-1}`
lsof -np $PID | grep deleted | grep /tmp/Fl* | awk '{gsub(/[a-z]*/, "" ,$4)} { system("cp /proc/" $2 "/fd/" $4 " ~/Desktop/" $2 "_" $4 ".flv") } {print "Copied "$2"_"$4".flv"}'

Thanks to Hari for making me revisit bash strings and Jai for the AWKgasm !!

UPDATE : As Rik pointed out in comments, lsof with -n is much faster since a lot of time is wasted in host name lookup (which we obviously dont care abt).. -n makes it not convert the addresses to hostname. Makes it 10X faster !!

UPDATE : I recently switched back to firefox and to make it work on firefox, change the “pgrep  chromium” to “pgrep -f libflashplayer.so”