To drop a brick is to make a clumsy, indiscreet, embarrassing mistake or remark, from the obvious allusion of clumsily dropping a house brick that could cause some damage. The expression dates from the early 1900s.