Peace and quiet is so that you, candidate, can focus on the answers. I personally don't care if your kids organize the party in the same room, but I care if it causes you, candidate, to not be able to fully focus on the interview and then me, the interviewer, to have trouble understanding whether your behavior was due to disturbance.

Interviewing is an expensive investment. I spend a lot of time doing the interview and I want to use what little time we have to the fullest extent.