03/25/2014 at 04:00 PM EDT
Cliff Robinson was not the first professional athlete to compete on Survivor
. Major league baseball player Jeff Kent competed in the Philippines, and NFL quarterback Gary Hogeboom strategized his way to the final seven in Guatemala. But few contestants were as recognizable as Robinson, a 6-ft., 10-inch basketball player who spent 18 seasons in the NBA.
After being blindsided on day 14
, Robinson became the fifth contestant to leave Survivor: Cagayan
. Following his elimination, Robinson spoke with PEOPLE about his tribe, his love of the game and why he should have made the finals. You're a recognizable guy. Did you tell everyone who you were?
I was upfront with who I was, because I didn't want to be caught in a lie. Everyone knew that I played in the NBA, and I didn't think it was really an issue. Besides, I'm too tall to blend in.
Wasn't that risky? People would want to take you out because you're an athlete.
But there's an argument that you can take me to the final three because no one is going to give the money to someone who has already made a lot of money. I was a perfect person to take to the end.
And yet they blindsided you. What went wrong?
I didn't realize that my tribe had been gunning for me. Tony had it out for me, but I didn't know how much. The reason why Sarah wanted me gone was because Tony was targeting me.
So do you blame Tony for your ouster?
I was in a four-way alliance with Sarah, Woo and Lindsey. Tony was whispering in Sarah's ear and had turned her against me, but I didn't know that out there. I was ready to stick with them to the end. I thought I was reading her really well, but I guess not.
I was surprised that your tribe lost the immunity challenge.
There was no reason why we should have lost the immunity challenge. We were stronger. There were four guys on our side, two on the other side. I still wonder if my tribe threw the challenge to get me out. I think about that a lot – did they want me out so bad that they threw the challenge? I don't know.
But you seem like a super likable guy. Why did they want you gone?
I don't think they didn't like me; they may have felt threatened or wanted to get me out because I was strong, but I don't really think I was a bad guy out there.
But your tribe did say you're lazy.
Trish says I'm lazy, but I'm not. Do you ever see her working hard around camp? I worked hard, too. Trish just had it out for me, but I want to make it clear that I am not a lazy person. I worked hard. I kept our tribe strong.
You and Trish didn't get along. Was that the hardest thing?
I told myself that I wasn't going to go out there and let myself get annoyed by people. You see it every year: People get annoyed by someone because of the way they talk, or the way they laugh, or the way they eat. I didn't want to do that. But then there was Trish, and she did get on my nerves. I didn't talk to her much, but I was never mean to her. I'm not a mean guy.
So straight up, which is harder: Survivor or the NBA?
. It's the hardest experience. I don't think you can fully appreciate how tough it is until you play it. It's tougher than the NBA without a doubt. But you know what? If I got the call to play it again, I'd be on a plane tomorrow. It's just that great of an experience!