Good Charlotte rocker Joel Madden, 27, has been hooking up with The Simply Life's Nicole Richie, 25, for the last month, but does he still have his ex Hilary Duff on the brain?

A source tells Us he has been calling Duff, but she “doesn’t want to talk to him” (the pair split in late November after two and a half years).

Adds the source, “She can’t believe he’s dating Nicole and thinks she’s a skank.”

When Duff, 19, heard about Richie’s arrest, “she asked if Nicole had been coming from Joel’s,” says the source. “She kept saying she never knew Joel at all.”