I'm sure someone in the time-travel-fiction community has come up with a proper term for this... I'd call it P.S.G. (Paratemporal Substitution Guilt).
[1969; Yasgur's Farm!] (MG) "This looks like a nice spot!" (BG) "You got a good view of the stage, Nightingale?" (ND) "Yes. I should be able to get some high-quality footage from this vantage point." [CW has a look of trepidation on her face even as MG and company get the tent out] (MG) "Um... You okay, Sandy?" (CW) "I'm beginning to think that we shouldn't have brought a tent. I just know we cheated someone else out of their camping spot." (MG) "It's nothing to worry about! There's no quota on tents here, one more in the fold shouldn't matter at all!" (CW) "But Ma-- uh, Moonbat, what if someone else was supposed to camp or sit here and they were meant to fall in love with someone next to them? And then they won't possibly get married and have kids... They'll get with different people... LOTS of families could be rearranged... Oh shit..." (BF) "What's wrong with Sandstone?" (MG) "Butterfly... Let's just say it's the 'You' Effect. Not your fault, by the way."