04/28/2012 at 03:00 PM EDT
He escaped elimination for the past three weeks, but Troy "Troyzan" Robertson finally fell victim to the women of Survivor: One World.
The tribe split the vote
in case Robertson had a hidden immunity idol, but with no immunity necklace around his neck nor an idol in his pocket, the swimsuit photographer from Miami, Fla., was voted off Thursday and became the fourth member of a currently all-male jury.
Having watched his demise the night before, Robertson, 50, spoke to PEOPLE about how he got his nickname, why he doesn't understand why people hate fellow contestant Christina Cha and all about his cryptic goodbye message to Kat Edorsson.
Okay, explain the nickname "Troyzan."
I don't call myself Troyzan in my real life. I would look like a complete tool. It's a play on words. I raised a set of marmoset monkeys for the past 20 years. My friends know I work in a jungle area and a coworker compared me to [the character] Tarzan and it kind of just spread from there. [On the island] Tarzan and I started talking and my name came up and then [host Jeff Probst] asked Tarzan a question and he said, "Oh yeah, Troyzan." And Jeff said, "Is that what you're called?" And I said, "Yeah, if you want."
What did you mean when you told Kat "do it" as you left tribal council?
I know Kat is a highly competitive person and didn't like being the person being told what to do. Earlier I told her, "You can win this, you just have to make a move and take control of this game." It's my heart to root for the underdog and that was Kat.
Did you think about how that could make the women not trust her?
It didn't cross my mind until I was watching it [on TV]. Everyone out there was treated me like, "We have to gang up on the great Troyzan. There is no way we could take him to the end," so I wanted to give someone else a boost of confidence. I just didn't like the way some of the girls at the top were acting snooty like they were so safe.
You were a one-man alliance for a while out there.
The girls were acting so smug and cocky. I understand that they wanted me out but they were acting so noble and telling me how to act – "[to] take it like a man." That really irked me and that's why I went nuts on them. I made them paranoid as hell. They didn't know what I was going to do next.
Christina seems to be without friends as well.
I still don't understand to this day why Christina is so disliked. I don't know what she did to perturb anybody. She's a nice person and I got along with her. She wasn't coherent at times when I was trying to explain that she was on the bottom [of the women's alliance] but she didn't annoy me. There is something about when people are afraid that they don't want to make bold moves. They just want to stay around for the next day. That's what Christina was doing and that's no way to win. Most of those people are just playing to stay there.