Following the release of its debut album, the band went into a self-imposed hiatus as Joe Pernice released a record in early 2000 on Sub Pop under the name Chappaquiddick Skyline. The record featured personnel made up of the Pernice Brothers' touring band: bassist Monahan, guitarist Pinkerton, new keyboardist Laura Stein, and drummer Mike Belitsky (both of whom were members of the Nova Scotia group Jale). The record was more sparse and downcast-sounding than the Pernice Brothers' debut. Both that record and Pernice's solo record from late 2000, Big Tobacco (which also featured Monahan, Pinkerton, and Stein), were, according to Pernice, made up of songs he considered not good enough to be released under the Pernice Brothers banner.

He finally came up with enough songs he deemed worthy of release by the band, and in 2001 the less orchestrated but still wonderful World Won't End was released to a ringing chorus of justified critical praise. The album featured the core band of both Pernice brothers, Pinkerton, Monahan, Stein, and Belitsky, and was released on Ashmont Records, a label that Pernice and business partner Joyce Linehan formed after Pernice parted ways somewhat acrimoniously with Sub Pop and found no other label that wanted to sign the band or that he wanted to sign with.

The band set off on a worldwide tour and returned to the studio in 2002 to craft their next release. Released in 2003, Yours, Mine & Ours had a less lush and arranged sound than the previous two records but showed no signs of artistic decline, as it was a strong batch of smart pop tunes by one of the finest bands anywhere in the early 2000s. Following the release of the album Belitsky and Stein left the group and were replaced by drummer Pat Berkery (also of the Bigger Lovers) and keyboardist James Walbourne (who played with Peter Bruntell). Nobody's Watching/Nobody's Listening, a live album/DVD set recorded in during their 2004 tour, was released in early 2005.

Meanwhile the group (which at this point consisted of Pernice, Monahan, Pinkerton, Berkery, and Walbourne) was recording their fifth album in Brooklyn, Massachusetts, Toronto, and Los Angeles. Titled Discover a Lovelier You, the album was released in June 2005 on Ashmont Records. 2006's Live a Little was the band's first record since Overcome by Happiness to be produced by Michael Deming and accordingly featured the return of that album's horns and strings, resulting in a sound that successfully melded the ornate approach of the group's debut and the more recent (relatively) stripped-down pop/rock sound. The group (now made up of Pernice, Walbourne and drummer Ric Menck with help from the returning Bob Pernice) successfully balanced both worlds on 2010's Goodbye, Killer.