Kevin, it's because the pitch data (vocals have pitches, right?) doesn't change. You're not altering that data. You're simply duplicating some of the data (to make the song slow down) or deleting it (to make the song speed up).
The data you're repeating or removing doesn't change, so the pitch doesn't change.
Let's say that you sample the incoming music stream, and the actual pitch data value is 10. Let's also say for the sake of this example that the data value is constant over a one-second time frame. For our purposes, we'll sample that one-second frame five times. We get:
10 | 10 | 10 | 10 | 10 <- five samples of the value 10.
Now we want to slow down the music. To do that, we simply duplicate the value:
10 | 10 | 10 | 10 | 10 | 10 | 10 | 10 <- eight samples of the value 10.
The pitch data value, 10, didn't change, but we have more samples to output into the stream. We haven't changed the rate of samples at the output (five samples, remember?), so we have now stretched the song. We have slowed down the song without changing the pitch.
Now let's speed the song up. We take our original value:
10 | 10 | 10 | 10 | 10 <- five samples of the value 10.
And we remove some samples:
10 | 10 | 10 <- three samples of the value 10.
The pitch data value, 10, didn't change, but now we have fewer samples to output into the stream. We haven't changed the rate of samples at the output (five samples, remember?), so we have now shrunk the song. We have effectively sped up the song without changing the pitch.