Evanescent Thoughts

Evanescent Thoughts

Archive for the ‘general’ Category

Facebook Internship

leave a comment »

Statuatory warning : Bear with me if this post is long or boring. Its been a long time since I blogged.

This summer, I interned at Facebook, Palo Alto. It also happened to be my very first experience outside academia, writing code that actually goes to production. I got the offer about a month after I started my graduate studies at Univ of Wisconsin, Madison. I had just moved into the US (late august, 2010)and there was a career fair around that time which I missed attending. A couple of my friends got internships from Qualcomm, but I decided to take some time to settle and then start looking for internships (after all, I had just moved in). Around mid september, facebook had come to college for a giving a talk about memcache. It happened to be that the person who was giving the talk  was a alumni of NIT Trichy and UW Madison. They asked us to forward our resumes if interested and I did. Within few weeks, the recruiting team was on campus again and they had a couple of interviews before offering me an internship. I was so excited that my first interview experience landed me with an offer and to top it off, it was FACEBOOK.

The internship was for 12 weeks, a very competitive offer that I didn’t consider sitting for any more companies. Besides google started their process very late (not that I would have been interested in taking it if I had been offered).  Started my internship on May 23rd. I worked for the Application Operations team under Scott and Alex,  with another fellow intern, Dustin (Alex’s intern). We worked on building a service called Freight  and later another thin layer of abstraction on top of it called PubSub (a brief report about it here) . It was an amazing experience to build something from scratch, have daily scrums, work agile and learn a shit load of new things, and the most important of all, to live and work in the the reality. Here goes a brief description of what happened in the next 12 weeks.

Week 1: Orientation, get your mac books and iPhones ( yes .. interns get brand new iPhone 4), get your dev environment setup, meet with your manager, discuss about the internship. We had a full team meeting discussing about Freight and what problems it should address. I should say Week 1 had my head in a whirlwind. A lot of things were pretty new for me. Though I was a little bit scared, I was happy that I would in the worst case, learn a LOT for sure.

Week 2: Started playing around with rsync and it was wiki time. Did a  _very_ dirty hack to rsync by mid second week to make it work for our end goal. It was so dirty that I dare not speak about it. But it works brilliantly!  Spent most of  the time saying hello to git. I had never worked much with any version control systems and I should say I was confused for half the time. But in the end, everything made sense. Dustin could relate to git since he had experience with other version control systems. So he was my 911 for git trouble. Also some dude stole my bike parked in the San Jose caltrain station 😐

Week 3: Dustin was making fast progress with the first draft of Conductor ready and I was understanding the facebook infrastructure and network topology. I had the basic monitoring framework ready. Integrated with the conductor.

Week 4: More scrums, more lines of code. Trains, henchman, shunter etc .. Dustin had a major change in the architecture of conductor and the shunter was born. I was working on anti-spikes. Started working on torrents.

Week 5: More features added to freight and I had the basic throttling framework ready with anti-spikes. Started helping out Dustin with some conductor stuff. Voila! Merge conflicts and git said it was doing a three-way !

Week 6: We sort of had a basic freight working and running ( I remember we did a fist pump). A lot of work was done in process management and clean shutdown by Dustin. Blocking / Non-Blocking servers explored. Had fun. I was also working meanwhile on the first draft of PubSub. Couple of experiments, trying to figure out how to do a cheap check-sum of huge files. Took the weekend off to go to this amazing 127 hour like trek in grand canyon.  (pic below.. also I thought it would be cool to drop a beautiful pic to wake up readers who have reached so far)

Week 7: Dustin started work on PubSub, now that we thought Freight was stable and a we had this long debate on phabricator about one of Dustin’s diffs. It was so long that both our managers never bothered reading it and just asked for summaries in the next scrum 😛

Week 8: Back to Freight, we started putting it to test with some small jobs, figured out a lot of changes need to be done. I discovered that transmission ( our torrent s/w ) had issues running stuff at the scale of our bandwidth. Ran a couple of tests and figured out what the problem was, source dived and fixed it ! It felt great.. was my first significant patch to a open source software. Co-incidentally, happened to be the one I had used countless number of times back in my undegrad for downloading what nots 😛

Week 9: Ran first of Cory’s  jobs. Discovered that more hacks need to be done for transmission ( the ones that cant be pushed upstream). Dustin spent some time working on stuff like peer-caching that was screwed up with our setting in transmission.

Week 10: More jobs for freight. We got couple of dedicated servers to host the service..  started work on making a front end( not fun at all, Dustin pitched in). Meanwhile, the conductor started freezing occasionally. Culprit being thrift’s python server implementation and python’s GIL causing performance bottlenecks. Mild architecture epiphany.. re design couple of things.  Dustin was working on some bugs in the python thrift server implementation were causing us problems .

Week 11: At this point of time, we had a mashed up version of freight running on the servers. It was a mash-up of hot fixes and the master was out of date for a while. Techincally, this was the last week we were supposed to push code. Making new changes to codebase in the last week was not encouraged.  Persistence was also achieved. Thank God !

Week 12: Last week .. team dinner at SF, final fixes to freight, running big jobs,  still trying to fix a lot of performance issues.. The last day, I realized another major performance bottleneck caused by transmission.. but hardly had time to source dive and think about fixing it . Meanwhile, got offered a full time position..  party time !! Also had to  take a flight to India the next day, showing up without any notice and surprise parents.

In all, it was exactly how I wished my internship would be. Dustin was a great guy to work with. Learnt tons of stuff from him. Though he was an undergrad, he had lots of experience working on stuff. At the end, we hated that we wrote most of it in python. Some small parts of it was written in c++ by  me, so I felt slightly better. One valuable lesson learnt was that, never write thread critical stuff in python .. stick to C/C++.

We touched a great deal of stuff in the course of internship .. the whole experience was overwhelming and Alex and Scott helped us a great deal in figuring out stuff.  We used to have daily scrums to discuss about the current progress and what has to be done next.  I had absolutely no time for anything else.  Rather, should say I didnt find anything else interesting in the course of the internship.

To cheer us up, facebook had a lot of intern events surfing, kayaking, go-karting, scaenger hunts, hiking. We also had a bbq party at Zuck’s house.

Throughout my internship, I never once felt like I was going to work. Its tough to call facebook working environment to be company-like. People go around the office in rip-sticks, have hackathons, work at crazy times on crazy things.. you could see hacks lying around everywhere, both in the code-base and the walls of the so called company. Every one works there because they feel so passionate about what they are doing. In my first week, I felt like every one around me was working 10 times faster than me. Took me a while to get used to it. We had amazing talks given by employees on programming. There was a series of c++ talks that just blew my mind off. I never thought about a language from such a perspective and I ended up knowing that the guy who gave the talk, sits few desks away from me and has a wiki page on him ! It was a fun working environment.. interns had their own weekly hackathons on thursdays.. pizzas would arrive sometime past midnight and we would gate crash into zuck’s acquarium ( his room.. more of  a room with glass walls, that looks like an aquarium from the outside when its occupied).. no one had cubicles.. you could just walk to any guy and start a conversation with him.

Food .. need I say ?? Cafe X \m/ .. It was so hard on  me that so much was offered and I had to restrict  myself from grabbing everything and piling it on my plates. I used to bike to work everyday (16 miles up and down) and that seemed to take care of all the desserts consumed everyday.  We had a different cuisine for every meal and friday evenings we had happy hour in the bball court after the Q & A with Zuck in the cafeteria.

I am sure I missed mentioning a ton of other fun stuff that we did .. the timeline might be slightly skewed by +/- 1 week .. My patience to sit and write a blog post has taken a dip, I should say.

Building something from scratch, hacking well known opensource software, pushing code everyday, scrums, whiteboard discussions, learning tons of stuff .. I miss all that now, sitting at home. Cant wait for the next summer and I apologize for the sloppy blog post!


Written by Sathya Narayanan

September 2, 2011 at 2:06 am

Flash Player 10.2 Workaround to Copy Video Files

with 12 comments

You might wanna check  this to get a context of what this post is about. With the previous versions of flash, I could get a copy of the videos being played in my browser by taking it from the /tmp directory. But with the new flash player, it deletes the file from /tmp after creating it. Why ?? I seriously have no  <insert blasphemy > idea !!. Generally when your browser plays a video using flash player, a process would be fetching the video from the internet, storing it in /tmp directory with a hashed value and the browser tab would be playing it. But now, we need to get around a lot of shit, just to copy your favorite videos. You may ask me why am I so bothered to do this. Answer is that, I generally stream the videos when I have an ultra fast net connection(which doesn’t happen to be my home), store them in my comp and then later watch it on vlc. You might also use some youtube-dl or other addon…. But those generally work only with youtube. Say you had a video which was directly uploaded on facebook or other website, I am not sure if you can copy that. Enough of trying to convince  you, that this is actually something interesting !! Lets get to the problem.

First, we may assume that the video was never saved on the disk. But who would be insane enough to  put videos that size to MBs in your main memory ?? So it has to be on the disk somewhere. When the video is being deleted from the /tmp directory, it doesn’t mean that its actually deleted. Lets take a small de-tour to the OS  basics. When two or more processes access the same file, they all get a file descriptor  and when one process deletes the file, it doesn’t actually get deleted unless all the other process close their own access through their file-descriptor. And here, we have a process which is buffering the video into the disk and later deletes the file from /tmp and another process ( your browser tab) which is still  playing the file. And voila !! We can still recover the video. Now this is no black magic !! You can see countless articles online about how to recover deleted files which are still open by some other process ( I think I have an a post on  that somewhere in my blog too). But what complicates the things ??

  1. Browsers nowadays run each tab as a separate process. So identifying how to access the file descriptors of the particular tab playing the video is the first concern.
  2. I generally stream  multiple videos. So  I sort of need to write a script that finds all deleted video files from /tmp and copy them back.

How to do this ??

  1. Get the PID of all the tabs of your browser, in my case chromium (Google chrome’s momma).
  2. List the open files of all the PIDs we got from above  and see if any of them have a link to a deleted file, specifically in /tmp
  3. Process the output of lsof which also gives the file descriptor.

Here is a run through

sathya@Phoenix:~$ pgrep chromium

Now list the open files of all these process by appending them into a comma separated and grep them for deleted files with /tmp/Fl*. Assume for now that $PIDS has all the PIDs of the tabs with video open.

sathya@Phoenix:~$ lsof -np $PIDS | grep deleted | grep /tmp/Fl*
chromium- 4506 sathya   25r      REG        8,5 33356235   998384 /tmp/FlashXXK9gZKa (deleted)
chromium- 4506 sathya   32r      REG        8,5 10544205   998386 /tmp/FlashXXO0GYZw (deleted)

Now the column 2 gives the PID of the tab and the column 4 gives the file descriptor. I have opened both the videos from the same tab and  hence the PIDs are same. Now , the 4th column gives the file descriptor followed by the permission (which in this case is r (read)).  Now how to get to this fd ?? The /proc exposes the RAM which has all info about the processes to the user and we can get to the file using the file descriptor from there.

sathya@Phoenix:~$ cp /proc/4506/fd/25 ~/Desktop/GotItMF.flv

There !! We have the video file !!  Now all we need to do is, put all this together in a script, which I did.

for i in `pgrep chromium`
PIDS=`echo ${PIDS:0:${#PIDS}-1}`
export IFS=""
for i in `lsof -np $PIDS | grep deleted | grep /tmp/Fl*`
     PID=`echo $i | cut -d " " -f 2`
     FD=`echo $i | cut -d " " -f 6`
     FD=`echo ${FD:0:${#FD}-1}`
     cp /proc/$PID/fd/$FD ~/Desktop/$PID_$FD.flvdone
export IFS=" "

If you are too scared about meddling with the IFS, use the following which does some clever awk scripting.

for i in `pgrep chromium`
PID=`echo ${PID:0:${#PID}-1}`
lsof -np $PID | grep deleted | grep /tmp/Fl* | awk '{gsub(/[a-z]*/, "" ,$4)} { system("cp /proc/" $2 "/fd/" $4 " ~/Desktop/" $2 "_" $4 ".flv") } {print "Copied "$2"_"$4".flv"}'

Thanks to Hari for making me revisit bash strings and Jai for the AWKgasm !!

UPDATE : As Rik pointed out in comments, lsof with -n is much faster since a lot of time is wasted in host name lookup (which we obviously dont care abt).. -n makes it not convert the addresses to hostname. Makes it 10X faster !!

UPDATE : I recently switched back to firefox and to make it work on firefox, change the “pgrep  chromium” to “pgrep -f libflashplayer.so”

Written by Sathya Narayanan

February 15, 2011 at 10:23 pm

The Indian Scenario

with 11 comments

The following is a random premature thought of what in my opinion is going wrong in India, which I am sure a lot of people think about.  Well I am  not gonna put on the Raja Ram Mohan Roy costume and lecture  about changing the society and  stuff ,  but I do  feel  that filling up a few bytes on the internet about this isn’t  a very bad idea either.

So, Whats wrong with India ??? I mean we are a body of 1.2 billion people constituting 20% of the world population. Which means out of every 10 guys on this planet, 2 of them are from India and  2 are from China and the rest 193 nations of the world constitute the remaining 6. Yet we just have nine Noble Laureates, and three of them are of foreign Origin. When we are 1/5 th of world population,  no matter how screwed up you were with  your 10th grade probability calculations, its not a tough job to figure out that Indians must be in top of everything.. not just population. But why is it that, when u take any field, its tough to find an Indian doing better at it ?? I am not talking just about Academia or Research for which every Indian’s instant response is the two worded phenomenon – “Brain Drain”.  Lets talk about sports.. How tough is to find a runner who can at least fetch a bronze medal in Olympics. Why is it that when we have Hockey as our national game, a majority of Indian kids don’t even know the names of the players(not like I know any of them). But every one does know that Sachin Tendulkar did start playing cricket at the age of 16 and has blah blah runs in his pocket and runs a restaurant in Mumbai. If we are so obsessed with cricket, why don’t we bloody change our national sport to Cricket !!!. Whats stopping us from doing that. Oh wait !! we are a democratic nation run by a bunch of corrupt politicians who roam around like zombies in wheel chair.

Population :

Population IS THE biggest feature of India. As a strength, it gives us diversity. You can go all around the US and  you will pretty much get bored of the same burger and pizza and the food varieties and culture it throws at you. Now back in India, you walk a few kilometers out of your district, you might actually end up in a totally different culture.. A different sect of people who talk a different language,  who listen to a different music, eat different food, have a whole array of local cinemas and cine-stars.

I sometimes get curious as to how India ended up with such a big population. I mean how ?? May be the so called Gods had such a plan you know, to stuff maximum number of people into this country and  to top it , they had to make the whole country  HOT and TROPICAL that when Mr Software-Engineer returns from work in the evening, he has to deal with both the HUGE traffic and import supernatural powers to tackle the hot weather outside. At the end of the day, all that is on top of anyone’s mind is  how to survive in this place. At times we even manage to have entertainment amidst all this and that’s why people all over the world respect Indians. They can almost survive any bullshit thrown at them. They are cheap labor who don’t mind sweating it out because compared to what they were doing back at home, anything that you throw at them sounds like luxury.

If India has a 1 billion population, it also has  an equal  number of  divine supernatural entities aka God. At times, you even wonder if they count the  number of Gods into the population stats. And Sachin Tendulkar does contribute under both the human and  divine category to the population stats.  How are you even supposed to survive in  a country with so much population??.  On one side, having a diverse population is a positive thing. But what actually happens in India is a different story.  Lets do a simple math. Say X guys were supposed to live under a km-square of area and it takes Y units of money to provide all necessary amenities to that area. Doesn’t it make sense that with the X very high in India, it would actually be easier to develop infrastructure and facilities ?? No .. that doesn’t happen because, One – we don’t pay our taxes properly. We just love painting our money black. And two – we cause more damage to infrastructure than the amount we contribute to build that.  And the moment we step into a foreign nation, we don’t spit,  we don’t urinate in public. We suddenly get a public sense of responsibility. We enforce that because every one else around you does so. In India, if you don’t cause any public nuisance, you get a nice portrait of yourself   in the “Hall of Clowns who follow rules”.  And nobody wants to change. We just continue living our lives putting up with all of the crap happening around us and contribute to it.

The Indian Rat Race at Schools  :

Lets get back to where all the talented people in India go to ??. The first thing that you notice is, every body in India wants a shortcut to success. By success we mean making more black money. How many of us actually think about living the life on our own ??. Half the Indian kids grow up listening to their parents. The parents mold  their kids into who they wanted to be when they were once in their childhood. Its  a sort of a pipeline where your dad wants you  live his dream and you want your kids to live your dream. You just throw your dreams onto your kids.  “Here kiddo, take all this books and digest them and become a doctor., Because, I once wanted to be a doctor and I didn’t have  facilities back in my childhood to achieve that.  And this will also make you rich and powerful in the society”.  I agree on the fact that parents have the responsibility to teach their kids of how to  become successful. But going a step further and choosing how they do it and in what field they do it is an over kill. And hence every kid gets lectured to be an engineer or a doctor. No one advices their kids or even talks to them about other areas like sports, literature, science, history,  economics, politics, art etc. Those fields are for kids who missed writing their 12th std board exams or those who forgot to bring a pencil to exam hall and lost out on  marks because they didn’t manage to underline the important points in their answers.

I feel like kids of this age are stereotyped into doing things. Each mom compares her kid with the other kids in the neighborhood and lectures him/her about how they need to take them as role models and score better marks in the next exam. And thus the rat race of engineers  begins with a big fat salary cheque on the other end. So most  of  the kids never get to  explore the opportunities in the other fields. A lot of talent actually comes  out in this Rat Race. Thats why when you go abroad and see a graduate student struggling with calculus, you just realize how great the Indian Schooling was. But most of these kids who could do calculus by just hand waving, don’t actually end up as successful engineers compared to what  US produces as undergrads and graduates. Thats just because once the Rat Race is over, we all take a well deserved rest during our undergraduate period. In the contrary, students abroad have a steep learning curve during Undergrad and graduate courses. Which just doesnt happen in India. Thats why even though we have a very successful schooling system, we have a very bad undergraduate experience ( IITs in my opinion can be exempt). There are several perspectives to this but lets rest on this.

Why Brain Drain??

So we only manufacture engineers in our country and what happens next ??  All these bright engineers run away to better places in the world because, those  not-so-talented kids who chose  the other important fields that constitute a society, do a bad job in their field and make the place a hell to live. To become a developed nation, India doesn’t need top notch computer scientists, but it needs talented people in the other fields who make India a better place to live. You need a good politician, a good economist a good city planner,  so on. But no one wants to choose these because they are poorly paid or have no clear shot at success in Indian terms and scenarios. So why not give these guys who do the less interesting but important jobs a better future. Lets lure the talented people into these fields.  One of my  batch mates  who graduated with an Instrumentation  & Control Engg degree, decided that Politics was the field he wants to be after he joined a company for work.  He even went on to enroll for some correspondence courses in politics as he works. Now if some one had put a thought in him when he was a kid that a politician was some one whom the society respects and needs,  he wouldn’t have wasted 4 years in NIT Trichy attending crappy engineering classes.  We would have had a  good politician who could have run the CWG without any corruption or not cause $36 Billion  loss to Indian government over the spectrum allocation. What India needs is not eradicating brain drain and stopping the talent from going outside the country. But  what India  needs is to make sure that  a fair share of the young talent pool goes on to contribute and develop the  other  fields that contribute  to the society. And of course yeah better politicians.

PS : All of the above are  random thoughts feel free to smirk them off if you are not in the mood to listen.


Written by Sathya Narayanan

November 18, 2010 at 12:55 am

Finding Invisible Friends and Dynamically Updating Your Status in Google Chat using XMPP Scripts #linux #python #ubuntu

with 24 comments

Read on, only if you Wanna do any of the following :

  1. Update your  google Chat status with info dynamically or fetch score from a website and display it every minute in your status msg.
  2. Find invisible friends in your chat list.
  3. Wanna change your status message as soon as some one signs on ?? aka make your own Buddy Pounce.
  4. Wish all your contacts by running a simple prog on important days. Like say, I wrote a script that wished  my  friends “Happy Friendship Day” when they appeared online 🙂 .
  5. Get a step closer to writing  your own back end for an IM client.
  6. Wanna remind someone about something over chat at a particular time or period when you might not  be online.
  7. And lots more that your imagination can limit.

Warning : I dont promise that everything in this is right. There might be  hell lot of mistakes. Feel free to point them out in the comments 🙂

Being someone who does a lot of IM and chatting, I was more curious about how a web chat works. How is it that you are instantly notified of your friend’s online presence and know when he is about to type something on your chat window. A little more digging and reading a couple of sites on XMPP  helped me figure out what I have been wanting to do a long time back in College. What basically you need for having a chat with your friend miles away is an XMPP Server (eXtensible Messaging and Presence Protocol) and an authenticated connection established to it. And of course an IM client (like google talk or pidgin or empathy) unless you wanna write your own command line scripts using some Python API for XMPP.

Firstly, I m not a big fan of gtalk, just cos it doesnt exist for Linux platform and who the hell uses an IM client just for google. An IM client is something that maintains all your chat accounts across various domains that u have a authentication on. An XMPP server listens to your xmpp requests on port 5222 and 5223. To give an idea about how the whole thing happens, imagine an XMPP server with which Alice and Bob have established their authentication. Unlike VOIP, chatting is done  with a man in the middle which happens to be the XMPP server. When Alice wants to send a IM(Instant Message) to Bob, Alice sends it to the server and the server sends it to  Bob. This sending and receiving is done in the form of XML packets. An XML stream runs between the Server  and Alice. A similar stream runs with Server and Bob. Whatever Alice puts on the XML stream is received by the server and the server reads the XML packet and looks at to whom the message is intended for and the server puts an XML packet to Bob’s XML stream and hence Bob receives the message Alice sent :). All this happens each time  u type something on ur IM client or change your status. Now this is  all about messaging. What about the status notifications and stuff ?? Thats where the presence part of XMPP comes to play. Basically a stanza in the XMPP stream between the server and the client can be etiher a Roster/Message/IQ/Presence. A Roster is used to notify the client about who all are online and registered with the server. A Message as I said above is a stanza that is used to communicate from one client to another through the server. A IQ is Info/Query where in u get specific info about a user. And finally a Presence is used to notify your type of availability to the other users (like busy , dnd  , away and put a status). This is the view of the XMPP  like how say you might see earth flying from an airplane . Each of these have their own specifications and finer details. For more, if you are interested, google it up .

When google opened a subdomain aka talk.google.com, users (the geeky ones) were able to authenticate and start chatting even before google officially announced that it has actually started an IM service :).  This happened back in 2004 when google first started the talk.google.com subdomain. Since then it kept on adding features to it chat client and led to group chat and stuff. But as I see, gtalk is not even close to an IM client. Its specific to google and has very limited capabilities when compared to pidgin or empathy. I am not talking about just using multiple accounts but about stuff that you can do with just your google accout like say control the presence notifications, add a  buddy pounce n stuff :). Well I can throw in more info with all my current enthu  about xmpp but I ll stop here and break straight to the part of writing cool scripts to do stuff with your google Chat that you cant do with your gtalk 🙂

Stuff that you might need : Python interpretor, xmpppy library for python. If you are in Linux , install python-xmpp and python-dnspython packages and you should be clear to go. If you are on windows, Keep breaking your head 😀

The first script I am gonna show here is how to send a message to a friend. This is how I started it. You might wanna cross check with the documentation here.

import xmpp
import getpass

passwd = getpass.getpass("Enter Password: ")

client= xmpp.Client('gmail.com',debug=[])
message = xmpp.Message(to,msg)

A quick look should tell you that Lines  1 and 2 import the necessary modules. The getpass module helps you to enter the password like the way you would enter it on a computer. Otherwise, you would have to put it somewhere and expose it (even better ways to do this are welcome in comments). I am declaring the variables in Lines 4,5,6,7. Line 9 , I create a xmpp Client with the parameters. When u specify the debug=[] option, the debug messages dont get thrown at you in the terminal. You connect to the server on port 5223 in Line 10 and authenticate in  11. Line 12 sends a Roster request and Initial Presence notification. You construct a message packet in the Line 12 and set its attributes in Line 13 and send it to server in Line 15 🙂 .

The next one I worked on was a script to dynamically update my status message with something like a clock or a cricket score. I am gonna leave to US on Aug 19th and thought how cool would it be to put a countdown on my status that updates how much time is left every minute(First I set it up every second, but it gets irritating to chat on the other end unless you have a very patient friend 😛 ).

import xmpp
import time
import datetime
import getpass

passwd = getpass.getpass("Enter Password: ")

client= xmpp.Client('gmail.com')

	diff = datetime.datetime(2010, 8, 19,4,40,00) - datetime.datetime.today()
	minutes=str(diff.seconds/60 - (diff.seconds/60/60 * 60))
	seconds= str(-seconds1+(diff.seconds))
	timeleft=days+" Days, "+hrs+" Hours, "+minutes+" Minutes more !!"
	pres=xmpp.Presence(priority=5, show='chat',status= timeleft )

Lines  upto 12 are self explanatory . I then start a while loop and calculate the difference in time from the current time to the time when I ll be boarding my flight. The diff gives back a structured variable with diff in days and seconds. The seconds difference is done mod 86400 (ie no of seconds in a day). So make appropriate calculations to count the days, minutes and hours and seconds left in lines 15-20 and I construct a XML Presence stanza by calling the xmpp.Presence() and send it on to the XMPP server. After this, I pause the loop for 60 seconds in Line 24 to send the next update packet after a minute. You could do something like put up Live cricket scores  and stuff here to keep your friends informed or put a horizontally scrolling status msg or whatever that comes to your mind !! 🙂 .

The next script that I have is a script to find invisible friends in gtalk 🙂 .  This I find pretty much useful. I used to do the other way around (that is to make myself invisible from pidgin by sending an XMPP req)  cos Pidgin’s Inivisible status doesnt work well. I found the info here when I was googling it back in my coll. Basically I put the following XML snippet in my Pidgin’s XMPP console to make myself go invisible.

<presence type="unavailable">

So I try to do the reverse of this from the Roster I get from the server and that should pretty much give the list of people who are invisible on gTalk :). Which I do in the following python code.

import xmpp
import getpass

user = "sathya.phoenix"
passwd = getpass.getpass("Enter Password: ")
server = "gmail.com"


def check(connec, event):
   if event.getType() == 'unavailable':
     print event.getFrom()

client.RegisterHandler('presence', check)

while client.Process(1):

Lines 1-10 again do the same authenticating and stuff while in Line 11 , instead of just sending the initPresence , we request for a Roster which gives us the status info of all my friends who have registered with the XMPP server. Now here is the magic. Even though you are invisible , your client registers with the XMPP server and sets the status as unavailable which the other Clients detect that and put you under the offline category(those who havent registered).  So all we got to do is to check for which of the registered users in the Roster are having a status “unavailable”. Pretty simple thing to break someone’s privacy huh 😉 .  So we have a handler to handle the presence notifications in the Roster and display the gmail ids that have status as “unavailable” and have registered with the server.  Now only disadvantage is that, if your friend is invi , he sure ends up on this list, but not vice versa. There are a couple more to check for that as well :).

So you could write handlers that can do a lot of stuff for you. Like you can send wishes to all online contacts by calling the client.send() in Line 15 and remove the if statement on Line 14. Or when some buddy of yours comes online , you can change your status message to soemthing else . Guess you can figure all that out by looking at the scripts above and combining them and making suitable modifications. Its all exciting !

PS : Any more cool ideas to do, put them on comments. Will try out 🙂 Also check this out if you wanna insert code snippets in your blog the easy way 🙂

Written by Sathya Narayanan

August 3, 2010 at 11:25 am

The Hyderabad Weekend

with one comment

The last time when I visited my cousins at Hyderabad was sometime around my First year. So I planned a small trip with mom over the weekend and it was one of the most exciting weekends for me after coll ended :). Not to forget the adventurous trip on the way back :P. Things that I did :

  1. Did a lot of shopping with cousins. Thanks to Kalapana and Kavitha akka, now 80% of my shopping work is over. We were right on time for the Sales at Lifestyle at In Orbit. I felt like I really did shop something for the first time in my life  (I never did any shopping till I was out of College :P).
  2. Met Srinivas, Abhinav and Mrinal at CCD near Hitec City. Abhinav has become like a hindi film star, mrinal is on race with Ankit to gain weight and Srinivas is all the more fair and handsome 😛 . Had a nice time with them though had to run back home quickly as one of  my cousins had to catch a flight to Chennai that evening.
  3. The new airport Ring road is Awesome ! 8 lanes and u can easily go on 140 kmph 🙂
  4. Went for a Rock and Jazz concert at Mariut Convention Center. This was my first concert if u dont count those we had in Coll during Festember and all. Was pure fun and was a pleasure meeting some of my cousin’s close friends :).
  5. Karthik picked me up on the way back at 12 in the night and went to his home :). NR and co were already there and Naga sir arrived early in the morning 😛 .. Tirth too came all the way at 1 in the night  and we had a nice chit chat till around 3 in the morning :). Not to forget the Zaveri call and the Hyderabadi briyani at some 2 in the night 🙂
  6. Spent the whole of next day at cousins place. Beautiful house .. .given that my cousin is an Interior Designer herself :).  I was always scared of dogs as pets. But I found  Deva aka the Lazy Labrador very very friendly. He nibbled me a couple of times and that was it. Such a cute one :).
  7. Went for lunch with family at  Mainland China. Went to the same one at Banglore few weeks back with Suren and co and guess what I am back :P. This one in Hyd had a better Chineese ambiance and the food was better too 🙂
  8. Got a Leather jacket on the way back and visited a Harley Davidson Show room. Some day , I shall own one 🙂
  9. Went on a Ride on the New Royal Enfield Classic 500 that Ramesh uncle got . It was so stud !! I always felt that bullets were more appealing to me than bikes(even the racing ones.. earlier on the same day , balu let me drive his R15 ).
  10. Got back on a train and reached Chitoor only to find that I am supposed to cross Andhra border and reach my home  through some weird way cos it was a State wide Bandh for Chandrababu Naidu’s Arrest. And this area seemed to be a TDP stronghold and they blocked all roads that led to city outskirts and TN border. Not even Cars were let to passby. Only emergency vehicles and 2 wheelers were allowed to pass. Had to take 2 autos to cross 2 such barricades ( the auto guys were also afraid to go cos they were asked strictly not to take on public transport). Cost me some 350 bucks to cross some 12 kms. On the last barricade at the check post at TN border, nothing worked and the auto guy also gave up. We walked all the way across the border with those heavy shopping bags. Luckily some nice guy gave me and my mom lift on a Hero Honda ( yes !!  we had no other option left ). So Triples it was. Me and my mom clinging on to that metallic fixtures on the back so that we dont fall off :P. Some how reached home after some 5 tensed hours 🙂

A great weekend in all … A memorable one too . Thanks to my cousins 🙂

Written by Sathya Narayanan

July 20, 2010 at 9:50 am

Posted in general, Its my life, Trips

My days at Theoretical Comp Sci Lab @ IIT-M (TCS Lab)

with 10 comments

I have been wanting to write this post for a very long time. The reason being simple enough, It changed a  lot in my life. When I joined fresh at NIT Trichy and completed my first year, I was more into programming and algorithms. Used to sit in I-Lab and do some of those SPOJ problems( yeah!! when I was in First year, I-Lab was good and it wasn’t filled with gultis watching videos on you tube and orkuting ). At that point of time, I was more interested into getting a Stud job once I complete my college. But now after 4 years at NIT-Trichy, I am dropping  an offer to join Cisco and heading for Masters at Univ of Wisconsin Madison. I can attribute all of it to Prof Pandurangan and his TCS Lab  🙂 .

Akhil and myself were on the same boat looking for profs to do project work on back in 4th Sem. I was more inclined to do an internship sort of thing at some startup or some small company but was also looking out on the academic sides. One afternoon, when we were busy spamming the inboxes of many such IIT Profs, Akhil mentioned  about Prof Pandurangan and  told me how his papers regularly appear in ACM’s Conf. And thats when I started mailing him and was fortunate enough to get his reply. Later he came down to our college( he was in the Board that certifies the Departments for some approval) for an official visit and said he would talk with all those interested in taking up internships at TCS lab. We were greatly amused by the way his lecture just kept us hooked on to the topic. One year into the department, I never felt like I was actually learning something new and exciting until I was attending his lecture(yes!! Mohan was taking a course in both the sems that year :P).  And thats when I decided to try out the internship and was very happy when he accepted to host a couple of us at IITM :).

So began the wonderful days at IITM TCS lab. The first time , some 8 of us were working under him and unfortunately he could only let us into the lab and not into the IIT M hostels :). So myself and Akhil stayed in the classrooms of KV School. The School was closed for vacations and we had the whole big black board all for us to scribble. Though technically, I should say we stayed at the Lab for 75% of the time in a day :).  TCS Lab was more than what we expected, given the condition of Labs at NITT( the 3rd semester, we didnt even have the labs ready and the next sem, I remember the server crashing most of the time :P) .

This was also when we were being taught what research work is. Until then I had no clue what it was to get a publication, a peer review, do a survey, Proof techniques etc. And a whole new exciting world of things was thrown at me. The  first time, I spent my time mostly reading papers and getting to know my area well. Akhil, myself and Bharathwaj(from IITB, a engg physics Student) were assigned to work on Multi Party Computation and Information Theoretic Security. In Research, you get excited when you know that the stuff you are working on is barely a few years old and there is a lot of scope for Research work.  Also Prof Pandurangan was taking a summer course on Algorithms and it was a Bliss!!. Till date, I havent enjoyed any class as much as I have enjoyed sitting in his classes :). Unlike other guys from my coll who were working on  Provable Security and trying to write a Survey, Prof let us work on a Problem right away :). We had to solve couple of Round Complexities in VSS and most of the day , it would be sitting in the TCS Lab and discussing if this works or that works with Akhil and Bharath(also a bit of chatting online with college friends and visiting the canteen freq :P). Bharathwaj was a step ahead of me and Akhil and always had a clear idea of whats wrong and whats right :). He helped me a lot in figuring out few mistakes too.  Whenever, we had a small idea we thought would work, we would run to Arpita and Ashish , who were doing  PhD under Prof. Now these two deserve more than a special thanks from me cos, I haven’t seen some one so dedicated as them :). In fact, whenever I consider my self if I have to do a PhD in future, I ll be weighing myself against them to check if I am ready :).

The one thing about TCS Lab is that you can find a lot of interesting people to discuss and chit chat including Prof Pandurangan. He would come around the labs, chatting with each one about his work in progress and crack a joke that would have us all in splits :).  He would also tell us what it takes to get a good publication out of students.After a few interactions with him, I was more than convinced to not just stop with B-Tech and get a stud job, but to definitely do a Masters.  Back then, the Lab would be full with so many  students that people who walk past the corridor, would mistake the place where we keep our foot wear for a shop that sells footwear :P. And some or the other old Student of Prof would visit the lab, not to forget the box of sweets that come with them. We had a tradition in TCS lab that who ever gets sweets for Prof  [which according to records, he never touches 😉 ], also gets a box of them for the lab inmates. All this experience with the Lab was something I could never dream of getting at NITT :). I dont think any Prof would comeback at Ten in the night to the lab and have discussions till late midnight at NITT ( many of them dont even come to dept in the day time !).

After our first internship, we were extended the opportunity to continue  working in the winter also. Prof was always insistent on making us realize  how Research is not a one-summer-outing thing. I still tell my juniors who are onto doing research project the same thing. Don’t think of it as a Summer Project or something you get to do for getting a Certificate.  A certificate can get you nothing in my opinion. So I said to myself, that I am not gonna give up unless I get a good publication out of this. Also Aishwarya and Anugrah joined me during the winter and we were looking at a new set of problems, hoping to solve them. By the end of it, I almost got some results and it so happened to be the new year eve.  During my  6th sem, I was mostly spending my time writing the paper for submission when one fine day, Arpita and Ashish spotted  a flaw in it :). And it rest there until the next summer when I went back to lab.

This time, I was totally working on a diff prob, AMPC. And suddenly one evening, I got an idea for the proof to work around the flaw in my previous result. Finally I had a perfect working one(though it was a very simple solution). And Prof wanted to send it for CANS 09   which had the submission deadline in  a week or so. So once again we were off to writing the protocols, changing a lot of stuff from old paper, adding new.. And we had a 23 page paper for a conf that allowed only 18 pages with appendix :P. And this is when Arpita and Ashish clearly demonstrated to us how many papers they have successfully authored. The draft before Arpita and Ashish did the corrections looked so amateurish. We had no idea what point to highlight and what to leave it for the reader to understand himself.  Using notations, skipping unwanted statements and proofs… learnt lots from them:). My desktop would have draft_1 draft_2 … and so on :P. We worked till some 2 in the labs. Did a lot of night-outs in lab (never heard of students even using labs outside the lab hours back in NITT). It felt exciting and like a whole diff world to me :).  Later in August, we were delighted when the paper got accepted. Should thank Sharmi and Vivek too for all those night outs. We used to go to Basera at 2 in the night after doing all the corrections to paper. At times, Sharmi n Vivek’s 2 yr old son Shvithin would come to lab and play around. Such a delight to watch prof and him talk :).

All this made me transform from a guy who aspired to get into a Stud company back in 3rd Sem  to a guy who wanted to go for higher studies by the end of 6th Sem.  Ironically, Akhil on the other hand who  wanted to end up doing research at the end of coll back then  is now heading to join Facebook :).  I still have no idea what it would have been if I hadn’t gone down this path but I am sure it would have been less exciting for me :). The admits that I got would never have been possible if not for Prof’s motivation and support:).  I owe it all to TCS lab , Prof  Pandurangan , and my other friends over there( Its actually a big list 😛 ). Those late night chats, running from TCS Lab – I to TCS Lab – II whenever we have something to discuss with Arpita and Ashish, Prof always trying to kalachify Shriram and Kalai, running to the Staff canteen for tea/cofee/icecream/lunch, the stay at KV school, latex work, Seminars, Prof’s jokes, Treats.. a long list of things I ll be missing when I go to Wisc Mad 🙂

Written by Sathya Narayanan

July 16, 2010 at 10:24 am

Ubuntu Lucid Lynx Tweaks

with 17 comments

I bought a new Dell Studio 1588 few days ago. It came with an Intel i5 450M processor, 4GB DDR3 Ram, 500GB SATA @7200rpm, 1GB ATI Raedon Graphic Card, 15″ HighDefinition LCD Display(1080p) with 1920×1080  resolutions and a back lit keyboard and a slot load DVD 🙂 .. A perfect config except that, it shipped with Windows 7 pre-installed.  The sales rep refused to even give me the lap without installing Windows sighting that they dont send OS DVD nowadays and the only way to give me the Windows 7 I am entitled for is by installing it in my comp. Fair enough I say, and gladly receive the laptop and start working on Windows for a couple of hours( I couldnt install Linux cos, at that time my modem was in repair and I had to wait)  only to find that 15hrs later, Windows screwed up my MBR !! Now on any other day , I would have fixed the MBR with a live CD of ubuntu. But since its brand new and still under the windows cloud, I thought of giving the customer care at Dell a ring to fix it and for the rest of the day, I kept staring at a slow green Progress bar. In the mean time,  I finished downloading  a  whole 2GB HD movie on a 512kbps BSNL connection ( Just to give u an idea of how long the progress bar made me wait 😛 ). Finally I gave up on the windows way of fixing, put my Live CD  to use and got done with it in 30 mins. Yet another times when you feel a live CD is like your swiss army  knife !

So I finally had my laptop ready for Lucid Lynx and it was just splendid  to see Ubuntu in HD screen 😀 .. Loads of UI improvement .. guess they are putting better designers at  the helm of the task though a lot needs to be  done, especially with nautilus’s usage of space and the Desktop icons alignment. Finally Ubuntu has a  Copy To and Move To added to the nautilus 🙂 More on the review of Lucid Lynx later.  In a matter of time, I upgraded my kernel, installed the updates, got my graphics card driver installed and copied the backed up data from old laptop. And then the actual problem began 🙂

The one Internet connection and Many Laptop Problem :

I had a BSNL connection for internet access and was comfortably using it on my old vostro laptop. Now that I have 2 laptops and I wanted to use both at same time with internet, I was checking out my options to get it working.

  1. The classic option was to get a router , a wireless one … But I wasnt interested in wasting my cash over it .
  2. IP Masquerading .. Make one comp route the traffic from eth0 ( the ethernet card connected to BSNL modem) to the other comp via a wlan0( Wireless card which had a private network established with my other laptop) … couple of iptable modification and this shud be up and running.. google and you would get sufficient info 🙂
  3. Setting up  ssh server and doing the above process without any changes to iptables..  You could try port binding ( ssh -L ) or a SOCKS proxy (ssh -d) to make things work .. but SOCKS proxy, I am  not sure how to export the proxy for apt-get to work .. (ideas welcome in comments)
  4. The next to easiest way of setting up  a http proxy server on one laptop …  I a had used in college was squid .. So I set it up in one of the laptops, changed the necessary ACL settings(access control lists) and http_access and done in a jiffy 🙂
  5. This is the best and simplest way …yet the most _not_so_interesting way .. in short the one click windows way !! (yes even ubuntu  is becoming bad !!) While creating the wireless connection, Under the ipv4 settings, put the method as”Shared to Other Computers”. This has to be done on the host laptop connected to the internet modem .  A very simple one click mechanism .. Think its there in windows too .

Any other interesting ways to do the same, please mention in comments 🙂 … Also note that to do all of the above you need to have two NIC s in your laptop.. The modern ones always ship with an ethernet card and an wireless adapter.. So that shud suffice.

No Sound in Head Phone Jack for Dell 1588 on Lucid Lynx :

When you plug in the head phones, the sound from the speakers stop but you dont hear anything on your  head phone .. Prob cos of your audio modules not properly configured. Times like this , you get to know of the countless number of sound cards out there  in the market…. A complete official HowTo is available here. In short for a Dell Laptop with this problem, you gotta do this,

cat /proc/asound/card0/codec#* | grep Codec

This would give the model of your  audio card on your system  mine was

Codec: IDT 92HD73C1X5

Next you need to find the suitable audio model for this card  from  /usr/share/doc/alsa-base/driver/HD-Audio-Models.txt.gz.

zless /usr/share/doc/alsa-base/driver/HD-Audio-Models.txt.gz

Search for your card in that doc .. zless reads compressed text files 🙂 .. So search for 92HD73 in that text file.. sometimes the cards are wild carded so incase you dont find, keep searching with lesser number of characters from beginning . The text I found was ,

ref           Reference board
no-jd         BIOS setup but without jack-detection
intel         Intel DG45* mobos
dell-m6-amic  Dell desktops/laptops with analog mics
dell-m6-dmic  Dell desktops/laptops with digital mics
dell-m6       Dell desktops/laptops with both type of mics
dell-eq       Dell desktops/laptops
alienware     Alienware M17x
auto          BIOS setup (default)

So the one matching my case was dell-m6 . Find the appropriate one for your laptop and do the following  :

echo “options  snd-hda-intel model=dell-m6″ | sudo  tee /etc/modprobe.d/alsa-base.conf

A restart should now get your laptop headphone jacks work normal 🙂 ..

The Low resolution in the splash screen and Grub during Boot up :

This is a traditional problem when your hard ware ( in this case your monitor) doesnt report the proper specs of display it can support. So specifying it manually solves the problem. Earlier,  it was GRUB legacy and usplash until recently things begin to progress towards GRUB2 and plymouth. So here is a quick process to do it. My laptop supports 1920×1080 resolution. So replace that with your resolution wherever applicable. This mostly would happen also if you use 4GB ram + 1GB graphics card on windows XP or any other 32bit OSThis should solve the problem and you can see a very HD grub menu on your boot up 🙂 ..

  1. First we need to install the v86d package  which gives the backend for kernel drivers that execute  the x86 BIOS code. So run sudo apt-get install v86d
  2. The screen resolution settings for grub2 are in /etc/grub/default . The previous grub legacy would have it in /boot/grub/menu.list. Here you need to change the GRUB_CMDLINE_LINUX_DEFAULT= “quiet splash” to GRUB_CMDLINE_LINUX_DEFAULT=”quiet splash nomodeset video=uvesafb:mode_option=1920×1080-24,mtrr=3,scroll=ywrap”.
  3. Add GRUB_GFXMODE=1920×1080 to the end . A similar line with resolution as 640×480 would be commented with an # .. You can as well remove the # and change it over there instead.
  4. Now run echo “uvesafb mode_option=1280×1024-24 mtrr=3 scroll=ywrap” | sudo tee /etc/initramfs-tools/modules. This  takes care of custom made resolution for GRUB and needs the v86d package.. For more details, check out here.
  5. And again run echo “FRAMEBUFFER=y” | sudo tee /etc/initramfs-tools/conf.d/splash.
  6. sudo update-grub2 to  generate the grub.cfg and sudo  update-initramfs -u to  generate the new splash screen.

Enabling your 32 bit OS to address your 4GB Ram + 1GB Graphics Card :

Now this is an interesting problem that I discovered when I opened my system monitor. To my surprise, I found that my OS only could address 3GB of memory. Theoretically a 32-bit OS can address 4GB of memory but since I had a 1GB graphics card, it could only address 3GB of RAM and 1GB of Graphics card. So I need to change my OS to 64 bit or  have a PAE (Physical Address Extension) supported  kernel. The Desktop version kernel doesnt support PAE, so you need to install a PAE supported kernel. Do the following

sudo apt-get install linux-headers-server linux-image-server linux-server

Reboot and do a free -m to confirm all your 4GB is addressable :).

Will  be making a couple more tweaks on my machine, thanks to google :). So as they say, “Where is the fun when everything works out of the box… Think Linux!”

PS : @suren : I knw now u will try to say something abt  mac … bring it on 😉

Written by Sathya Narayanan

June 24, 2010 at 1:38 pm