While I love my shiny new unlocked iPhone, I was wondering whether leaving SSH on the phone really drains the battery, like some people say?

Does it really matter, or is it totally insignificant?