Soft spot can mean both a weakness and a source of affection, as in to attack a soft or weak spot or to have a soft spot for someone. Both usages are originally American and date from the early 19th century.