Taking last year's nest out of a birdhouse?

Last year we hung a ceramic birdhouse outside our balcony. Within a few weeks some birds had made a nest inside and lived there for almost the whole summer (much to the pleasure of my window peering outside kitties!). This year not one bird has been itnerested in the birdhouse. I was wondering if I need to clear the nest out for any new birds to come (maybe they think it belongs to someone else?)