"Rob has always been in the shadows of his older sisters. It's been hard and he tried so hard to make a name for himself away from all of that," a source tells PEOPLE. "It's been embarrassing for him to not be as successful as his sisters."
MONEY$HOT / Splash News Online
Rob has shied away from the public eye for two years after battling weight gain and depression. He lost his father Robert Kardashian, who died of cancer in 2003, and the source says it was difficult for the star not to have a male role model around.
"He never had a solid, strong male role model in his life to help him. It's really sad, because he is such a good guy. Everyone wants him to be happy," says the source.
Rob's new flame Blac Chyna has a complicated relationship with the Kardashian-Jenner family. She is a former close friend of Kim Kardashian West and shares a son with Kylie Jenner's boyfriend Tyga – and source previously told PEOPLE that the family has conflicting feelings about the relationship.
Although it's "definitely very annoying that Chyna is the one that could get to him, at least Rob seems happier," said the source. "It's still frustrating for his family that they tried to help him for so long and he wanted nothing to do with their help, but at least his family gets a bit of a break."
Sources told PEOPLE earlier this week that Kylie was "livid" and that Khloé Kardashian felt "very betrayed."