"I face them every day," the songwriter, who recently opened up about battling an eating disorder, tells PEOPLE. "When I was trying to be an artist, people would say, 'Maybe you need to lose a little weight; your nose is too big.' When it's time to go on stage, I think, 'Shave [down] your nose!'"
DioGuardi, 38, has other body anxieties too: "I'm most insecure about my legs and my knees. They're kind of big," she says. "I'm not someone who wears a lot of shorts and short skirts."
Now that she's a grown woman, she feels more comfortable with herself. "If I look at pictures of me in my 20s, there are things that looked out there, my nose was too big, but I feel like I've grown into my looks," she says. "This is what I'm supposed to look like."
What's more, DioGuardi says she is no longer dealing with the eating disorder that caused her to overeat.
"With the amount of stress I'm under these days, the food just goes right through you," she says. "I'm running around, constantly challenging myself." To keep her body toned, she works out two to three times a week doing strength and resistance training.
But the Idol judge, who got engaged to general contractor Mike McCuddy in December, says the true source of her beauty is her state of mind.
"What makes me feel ugly is when I wake up and I'm not feeling great about the day," she says. "Beauty comes from the inside. Do you love what you're doing? Do you love who you're with? Do you have a life that you feel is balanced? That sort of happiness, you can exude it."