"There was no one turning point that caused this," a friend tells PEOPLE, adding that the stress of the cameras was not a factor for the couple, who wed in 2010 and are parents of daughter Bryn, 2½.
The couple had argued over the years about everything from the frequency of their visits to his parents' home to birthday plans, but "they tried to make it work," says the friend. "It just got too hard to be husband and wife."
In the last year, Frankel, 42, had even taken time off from her various businesses to focus on her marriage. But one source says she found it hard to leave her work ethic at the office.
"With Bethenny, it was her way or the highway," says a source. "If you are a person who surrounds yourself with people telling you, 'You are right; we will bend over backwards to make things happen for you,' you will have a tough time in marriage. Marriage doesn't work that way."
And now, though a source says a reconciliation is out of the question, both parties are having a difficult time moving on.
"Bethenny is devastated," says the friend.
For Hoppy, 41, who was spotted out recently wearing his wedding ring, "The split has been tough," a source close to the couple adds. "You go into a marriage thinking it's going to be forever. When you realize it's not, it's really hard – especially when a child is involved."