Whenever the game window loses focus (or is iconified), whether full screen or windowed mode, the game should stop all sound. If this isn't the default behavior, there should be a checkbox option to enable this behavior in settings.
The music is really nice, but the above is pretty standard behavior for most applications, since there may be other things running that also want to play sounds when they have focus.
I didn't search your bug/feature db, so +1 if this is already on your list.
I appreciate the opportunity to give feedback that has meaningful chance of being acted upon, unlike many games developed by large teams, which, counterintuitively, tend to be less responsive.