Aaron Marino of alpha m. discusses when wearing a fitted button-down dress shirt how many buttons is appropriate to leave undone. The answer depends on you, your shirt, your comfort level, and the comfort level of those around you.
The more buttons you have open the more casual and loose you are actually feeling and flowing. Some shirts have a natural tendency to lay open and some do not. You want your shirt opening just a little bit. If your shirt is wide open, you have unbuttoned too far. If you are wearing a shirt without a tie, one button looks uptight and uncomfortable. Go with a minimum of two buttons undone. Three buttons is for a sexy look. Four is too far! You will look like Aaron's Uncle Vito, sweaty and smelling like garlic. If you are around people that are uptight and you don't know as well, unbutton so that you are staying relatively closed. Bottom-line, rock it however you feel comfortable doin' it.