Yes, the Daily Show has suffered some significant dips in the past year or two, and when Colbert's show began he was briefly my favourite. But I find the Report unwatchable.
Other than “Better Know A District…” I find the recurring segments to all be weak. But mostly I'm turned off by Colbert himself. That caricature cannot support a nightly show without devolving into self-reference and self-aggrandizement.
The worst part is that BEHIND the caricature, the 'real' Colbert seems to reek of ego. Maybe it's the residue of role-playing, or the thrill of having a successful show, but he genuinely seems to shine with self-satisfaction. Because of this he winds up coming off as condescending to his guests (regardless of their political leanings) AND his audience.

Colbert is hilarious as a cluless straight man (as a correspondent for the Daily Show, and his own segments like “Better Know A District”) but I got tired very quickly of seeing him gleefully coast on his own 'charm.'