"Aspen didn't go as well as planned ... there were more lows than highs," one source says. "Aspen was the final straw."
The couple, who were often apart due to work commitments, had hoped for quality time in the mountains.
"When they came together for Christmas as a family, things had changed and they fought a lot," another source says. "Their Aspen trip was a bit of a mess. Instead of enjoying being back together as a family, it was difficult for them to get along."
While on vacation, Klum hit a point where she was "done with the arguing," the second source says. "She doesn't want to fight anymore and thinks it's better to separate before their arguing starts to affect their kids."
Despite what was going on behind closed doors, people who are used to seeing Klum, 38, and Seal, 48, in the celeb-friendly ski town didn't necessarily notice signs of trouble.
"They seemed very happy together," one photographer tells PEOPLE of the pair, who are now grieving their relationship. "They couldn't stop kissing and cuddling. They spent a lot of time with their kids, but also managed to sneak in several solo ski runs together. If they were having problems, they were definitely not showing during the day."
Another source, who described the vacationing couple as "playful," says they were "talking and seemingly very much in love."
Still, when they failed to show up at the Regal Watering Hole, where they had reserved a table, a source says it raised eyebrows.
Reporting by Pernilla Cedenheim, Troy Hooper and Johnny Dodd