The Kingsbury Manx was released by the label in 2000 to so little fanfare (failing to offer any information about the band or the recording) that it ended up creating a small amount of mystery. Managing to stay independent from any particular scene, the band cultivated a sound simultaneously derivative and original. The influences were timeless (early Pink Floyd, Simon & Garfunkel, the Beach Boys, and the Byrds have all been cited), but they were handled with such loving care and attention to detail that they were rendered largely insignificant. The album became one of the underground indie successes of 2000, landing in the year-end polls of NME (Top 50) and Magnet ("Ten Great Albums Buried in 2000"). A short tour of the U.S. followed in support of Elliott Smith. Let You Down followed in 2001. Its Japanese counterpart release included two bonus tracks, "Dirt and Grime" and "My Shaky Hand."

In support of the Afternoon Owls EP, which arrived in fall 2003, the Kingsbury Manx toured with the Sea and Cake. Additional shows with Gorky's Zygotic Mynci coincided with the release of the band's third album, Aztec Discipline (2003). In 2004, the bandmembers started working on tunes for their next album at their practice space (Pine Manor) in Chapel Hill. They traveled up to Michigan to record the tunes at the Key Club studios and, in early 2005, left longtime label Overcoat Recordings to sign with local North Carolina label Yep Roc Records. The band took the Key Club tapes to Chicago, where Wilco member Mikael Jorgensen mixed the album. The result was their 2005 release, The Fast Rise and Fall of the South.