Caveat: Yes, the way I'm explaining this is very hetero. However, the advise applies to all.
Situation (generalized): Boyfriend has friends of both genders. Boyfriend is a social person. Girlfriend gets really insecure whenever the Boyfriend hangs out with his female friends. What should the Girlfriend do?
First and foremost the Girlfriend should look within and figure out why she feels insecure. Has she been cheated on in the past? Has the Boyfriend showed signs of untrustworthiness? Is she insecure about her self-worth? Is the female friend showing signs of flirting or attraction to the Boyfriend? Then she needs to ask herself, what is the feeling that she wants to feel or what is the desired outcome?
Next is COMMUNICATE! Communicate to the Boyfriend why there are feelings of insecurity. This could just be a matter of misinterpretation from one or both parties. The Boyfriend might come up with some ways to help ease the insecurity. The girlfriend needs to be careful in the way that this communication is done. No BLAMING! Use I statements (I feel ______ when this situation happens).
If the Girlfriend has been cheated on in the past, then of course it's going to affect future relationships. There's always going to be that lack of trust. The Boyfriend will have to try harder to earn that trust. At the same time, the Girlfriend needs to combat her own thought process to learn to trust people a bit easier.
If the Boyfriend had showed signs of being flirty or untrustworthy, then bring it to his attention. He might not even realize that that's what he's doing. Some people are just oblivious to that (damn you people with natural charisma). If he is fully aware of what he's doing and doesn't care, theeeeeen it might be time to walk away. You don't need that type of anxiety in your life.
If the female friend is showing signs of flirting or attraction to the Boyfriend, bring it up to the Boyfriend. "Hey, do you realize this girl is flirting with you? You need to address boundaries with her." A rational boyfriend will address boundaries. An untrustworthy boyfriend will shrug it off. Walk away from the untrustworthy ones.
Lastly, the Girlfriend needs to work on her own Self-Worth. Work on self care. Figure out what happened when she started to doubt herself, and why it started? Learn about stepping into her own power. Learn to be trusting. If she needs to step away from the relationship to work on herself, that's totally ok. If she needs her hand held by the boyfriend as she's working on herself, that's ok too. Every one of us is amazing and unique. We just need to see it for ourselves.