Is it true that you'll never get over the guy you lost your virginity to?

My ex was so secure and confident he wanted a relationship with me. He was very caring, respectful, and his family and friends (whom i met right away) said he was "head over heels". He started changing when he was with me - more responsible, dressed nicer, and in a way, I feel as if I helped him to find himself more. He and his friends would constantly tell me how "opposite" I was of his exes and he would always say i was everything he wanted in a girl. I'm 23 and i've had my "fun" (not promiscuous) and for me to comitt is a BIG deal...that person has to be special and i have to really care about them for me to have a relationship with them. I got to know him and learned he's a genuine person w. a big heart. I saw potential in him (as an individual) and someone I could potentially be with for a long time. I cared about him; NEVER pressured me for ...was the type to wait (only slept w. 3 girls in his life; all his long term exes)...i was aware that losing your virginity isn't "magical"...was aware of the realistic aspect but I risked it bc i had faith in me, him...us. We dated for 3 months and he broke up w. me because he "wasn't ready for a relationship". (He's 23 as well) and turns out it's not in him to be in a relationship now and for the first time wants to be alone and do his own thing. There's no other girl...he's not ready...wants to be "selfish"...also he's been feeling guilty bc he was drunk when i lost it to him. And since then he's been different...esp given his track record and mine. I felt dupped bc I'm always gonna remember him as the guy who wanted me, was ready to commit, i lost it to him, then 3 months later had a change of heart...oh yeah, he happened to be drunk too. I'm actually proud of myself w. how i handled this break up. The next day after, he texted me apologizing, full of remorse...he said he still cares about me but isn't the right time now. I'm usually the type of person to be okay with "exes" or past guys and still be cordial with them but w. this guy...i cut it off completely. Never texted him back or called him. For the most part, I really am okay. I usually dwell on every detail, find explainations months and months after things don't work out...very obsessive but w. this guy, i'm very okay. I normally miss him and grieve fr. time to time but i'm doing great, keeping busy and coming back to the same person I was before I met him :) It's been over a month and I hate that my girl friends tell me i'm feeling the way I am because i lost it to him. I never wanted that to be the reason of why i can't move on! I can't seem to understand where they're coming from. Why can't the reason why i grive about him is because i committed to him...and that i cared about him as a person? I don't want the virginity thing to be the reason...Besides, we only did it once and it wasn't as "pleasureable" (of course, since it was the first time). I'm really pro active in moving on and i'm proud of where I am....i just dont' understand why people are saying, "oh, it's because he's your first"