11/10/2013 AT 01:25 PM EST
Kendall and Kylie Jenner share the spotlight with their famous older siblings, but only they know what it's like to grow up in the media glare – and they admit it's not easy.
"They got to live their teen years in a very different way than we did," Kendall, 18, told PEOPLE on Saturday at a Glendale, Calif., appearance with Kylie to promote their PacSun clothing line. Kim Kardashian
and her sisters, Kourtney
didn't become famous until they were in their 20s, she explains.
"They lived it normally and then once they were already grown up, they went through what we’re now going through."
"They even say to us: 'If I was in your position, I would not be able to handle it,' " Kendall added.
There have been some media reports detailing the pair's wild behavior
of late, but Kendall dismisses them, saying, "We’re growing up and no one seems to want to accept it, but whatever."
To cope, Kylie, 16, says she tries to focus on the good that can come with all that media scrutiny.
"You just have to come back and realize all of the great things that have come out of it. All of the good you can do with having the spotlight on you and just having your family around you," she says.
To that end, both girls say they relish their time with infant niece North
– Kim's daughter with Kanye West
– as well as their auntie time with Kourtney's children, Mason
, 3½, and Penelope
, 16 months.
"She’s amazing," Kendall says of North. "We love it. I mean we loved being an aunt to Mason and Penelope so the third one was a great, great thing for us. We love it."
"We’re like aunt times six now," Kylie adds. "I love all of my nieces and nephews."