On one’s uppers

Reduced to poverty an American informal expression that dates from the late 18th/early 19th century from the allusion to being reduced to the upper parts of one’s shoes because the soles have been worn away.