Wide of the mark

Irrelevant or off the subject dates from the 15th century and makes use of one of the many meanings of mark, in this particular instance, a target or objective.