Is this normal or am I falling for him?

I’m in my first casual hookup “relationship” and trying to navigate my feelings.

Up until now, I’ve been the main person he’s been seeing. We see each other about 4/5 times a week. He hangs out with other girls in the day sometimes but then has me over at night where we chat and have dinner and sleep together. I’m emotionally fine with him seeing other people I guess because he’ll always kick them out for me.

But last night he asked me to leave so he could make a call to someone. Then today he called me and I heard his car beep so I asked where he was going and he just said “I’m going out”… meeting up with the girl from last night I’m sure. Before he hung up he said “I love you!” Which is something he’s never said before. When I didn’t return the sentiment he played it off like he was joking. Not sure if he was or not.

The whole rest of the night I haven’t heard from him at all so I’m sure she spent the night. And now I’m bothered. I’ve been thinking about it the whole time and I think I’m territorial. I want to stay his favorite. I know that’s silly but it’s how I feel.

What I’m worried about is if I’m thinking this but actually catching feelings for him. Is this normal or am I getting into the territory of wanting to be exclusive with this guy?