I work at home for my "day job". It wears me out more than the days when I have to go into the office. I actually catch myself calling a few times a month to see if I CAN come in.
What annoys me the most about working at home, is I have to keep my doors locked, and curtains/blinds drawn, as people (friends & family) think just because I am home, I can drop everything and entertain them. When I first started, it took me six months to get people to realize that I could not be their child's emergency contact for school, as I am not always in a position where I can drop everything and attend to their needs. A few even thought I should be willing to have their kids stay with me until the bus comes in the morning, and again after school.
Then, you have the mother (and the aunt/uncle combo) who thinks that just because I'm home, I should be able to drop everything (and catch up on it later), to go with them to the casino in the next state. Mainly, so I can babysit my uncle (52), as he has a habit of wandering off...and sweet talking the pretty young thing next to him, out of her cigarettes that hes NOT supposed to have.
TLDR: Working at home opens up another world of problems, and people (with their own problems).