Watch Conan, Jack White Play ‘We’re Going to Be Friends’ at Newport Folk Festival

The song has a long history with the comedian: It was the last song ever performed on O’Brien’s NBC late night show, and O’Brien uses the song for his podcast.

O’Brien opened the set by introducing himself as “folk music legend Conan O’Brien” before explaining that he’d showed up to close out the storied festival as an exercise in destruction: “I am here to prove I can ruin any genre,” he quipped.

White was the final guest during O’Brien’s set, which featured an original song from Triumph the Insult Comic Dog about how bad O’Brien was at telling music stories, like the one about O’Brien meeting Bob Dylan; classic rock covers from guests including Nathaniel Rateliff (“Let It Bleed”), Brittany Howard (“Come and Get Your Love”), and Taylor Goldsmith (“Ramblin’ Man”); and a rousing performance from Mavis Staples, who sang an extended rendition of her Staple Singers signature “I’ll Take You There.”

After the White Stripes ballad, White and O’Brien launched into the rockabilly rave “Twenty Flight Rock” by Eddie Cochran before O’Brien brought out a gaggle of singers from throughout the day for a Newport finale sing-along of the traditional “Midnight Special.”