Though the Keeping Up with the Kardashians has been living with his sister Khloé Kardashian, the source says the new romance with Blac Chyna means "he won't be welcomed back to Khloé's now," given the model's complicated history with the Kardashian-Jenner sisters.
"They are dating and have been for a few weeks. Who knows how serious it is," the source tells PEOPLE. "Rob has been lonely for so long, so of course it's great for him to spend time with someone other than his sisters. Rob has also found an escape from his family nagging him about his heath. He has ignored their attempts to contact him."
Over the past two years, Rob, 28, has battled depression and experienced significant weight gain and a health scare that saw him hospitalized and diagnosed with diabetes.
"Rob likes her and seems to think it's mutual. It's just hard to take things seriously since obviously both share a lot of anger toward the [Kardashian-Jenner] girls," says the source. "They have both had a lot of drama with Kim. Rob really seems to get a kick out of all the drama that he is causing."
Blac Chyna, 27, sparked speculation of a romance between her and Rob Monday when she posted a picture of herself with what looked like the reality star's arm wrapped around her. Rob added fuel to the fire when he posted a meme to Instagram Tuesday suggesting Blac Chyna was ready to give birth to a baby Kardashian.
Blac Chyna has a complicated history with the Kardashian-Jenner sisters. Her close friendship with Kim Kardashian West ended after Kim's sister Kylie Jenner started dating Tyga. Blac Chyna and Tyga have one child together, son King Cairo, 3.
A source told PEOPLE Monday that Jenner, 18, is "livid" about the romance between Rob and Blac Chyna.
"She feels super betrayed and can't understand why someone in her own family would stab her in the back like that," the source said.