Why my boyfriend gets mad when I just imply how I feel about him?

Is this a red flag?
One example: he was often talking about his exes. I told him one day “seems you miss your exes all the time”. And he was like what do you mean? Why did you start talking about this out of nowhere. And I said because I have to say it. I want to know the reason why you talk about them and just said I wonder if you miss them. Like why are getting mad. Then he said that he’s getting mad cause I bring sth totally absurd and it’s a passive agression. That I assume things. And said “if you feel uncomfortable you just should say it openly, not implying things”. And said that he will never talk about them anymore.

But the things is it makes me upset that he doesn’t try to confront me but saying like this:(

Updates
8 d
Also he said it doesn’t have any sense that he doesn’t give an f about his past and why he should hide it. When we are suppoudly to me mutual confident we can talk about everything openly with each other
Why my boyfriend gets mad when I just imply how I feel about him?
Post Opinion