Should you be panicking about the Google mobile search update?

Short answer: If your Google Analytics browser stats shows predominantly mobile traffic, yes.