04/05/2013 AT 05:00 PM EDT
Being on Splash
was a matter of life or death to Kendra Wilkinson
The reality star, 27, who stepped down
from ABC's diving competition
on Tuesday, tells PEOPLE, "I feel like the world is off my shoulders now. Every morning I would wake up and be shaking. It's a really
risky show. I related jumping off the high board into the water to jumping to my death. That's how big of a deal it was. To me, it really meant risking my life."
A teary-eyed Wilkinson climbed down from the 23-foot platform on Tuesday's show, and said, "One thing I don't do is quit. This is the first time in my life I've quit something. This will haunt me for the rest of my life."
Wilkinson, a veteran in the water, thought she'd be doing aquatic activity when she signed up for the series.
"I love to be in the pool," she says. "I was always a swimmer, but this is an aerial sport. I didn't know that until I signed up. I thought it was swimming. But then I was like, 'Wow, this is acrobatics. This is what you see in the circus."
Despite some hate for quitting the show, Wilkinson says the positive has definitely outweighed the negative.
"People link quitting to failure," she explains. "But on the other hand, I think people understood. After I quit, I expected the world to fall down on me and it actually ended up being the opposite. It's one of those situations either you understand or you don't. If you don't, you hate me. And if you do, you love me."
But the former Girls Next Door
star understands what is behind the hate.
"People have this movie role fantasy," Wilkinson, whose show Kendra on Top
returns to WE tv this fall, says. "They think, 'Oh, she's scared, but she'll do it.' And everything will be just fine. People expect there to be a positive outcome and a success story, but sometimes people need to realize it's not always like that."
"The reality of the situation is I was truly afraid and could not fulfill my role on the show and had to let it go. There were other contestants who truly wanted to win. I just wanted to get over my fear of heights. I wasn't on there to be competitive."
The best support of all has come from not only Wilkinson's husband, Hank Baskett, but also her fellow contestants.
"We signed up for this not truly to compete with each other, but to go through our journeys," she says. "I'm not going to say names, but after I quit, I had a contestant call me and say they wanted to do the same thing I did but they didn't think they could do it. I praise the whole cast of Splash
with all I have."