Answer: Based on the sentence provided, it appears that "us" is referring to a group of people that the subject was a part of. He left the group shortly after an interaction, though the exact reason for his departure is unknown. It is possible that the interaction caused him to become uncomfortable or dissatisfied with the group, prompting him to leave. Alternatively, it could have been a matter of timing and he simply had other commitments that required his attention. Whatever the case may be, it is clear that the interaction was significant enough to cause him to leave the group.
