In 1967, when Scott Walker was 24, he quit his band the Walker Brothers at the height of their fame–on their final tour the supporting acts were Cat Stevens, Engelbert Humperdinck, and the Jimi Hendrix Experience. Walker had always seemed uneasy with his role as a teen idol, and by tearing himself away from his adoring public to pursue an interior vision, he became a textbook example of the existential rock star.

Best of Chicago voting is live now. Vote for your favorites »

The new album is the product of seven years’ work. The BBC recently broadcast Walker’s first TV interview in more than a decade, playing studio footage where he’s showing musicians exactly how to bang a metal pipe or slap a side of raw pork to get just the right percussion sound. The Drift is unmistakably the product of a powerful urgency, but it’s nothing like a teenager’s urgent desire to be understood, which is easily frustrated and just as easily spent. Instead it’s like a monk’s desire for transcendence, expressed in a steadfast commitment to work patiently, a little each day, toward a goal that’s hardly understood.

Walker is 63, but neither yields to the pressure to sound superficially contemporary nor revisits the feel of his canonized late-60s material. The Drift is so idiosyncratic that only his previous record can provide a meaningful context for it. With its sinister undercurrents and occasional eruptions of metal and industrial noise, Tilt is one the most shocking and unsettling records I own, and its often lurid surface can make it hard to appreciate the songs themselves. Walker’s new disc redeploys the avant-garde collage approach of Tilt in the service of his classic albums’ emotional impact.

The accompaniment occasionally falls into an off-kilter burlesque bump reminiscent of the Get Hustle or Love Life, but in contrast to those bands’ discrete bursts, this is more a prepositional music–it’s always between states, on its way elsewhere, never settled. It’s as though Walker has written songs without verses or choruses, only long strings of bridges. When he loops a lopsided pattern for a few bars, the repetition is always a discharge of tension, a brief reprieve from the music’s unrelenting instability. It’s a sound that seems like a slightly sexy, slightly silly put-on when younger bands try it, but in Walker’s hands it’s truly heavy.

But Walker now refuses that simple connection with his audience. His recent music is gnostic and ecstatic–qualities that arise from its meticulously chaotic form, not from his performances. There’s no spontaneity or improvisation in his singing, but his songs demand audience engagement in a way that’s more like sparse free jazz than any variety of pop.