Being able to rattle off even just a few set phrases in Mandarin will make living in Shanghai much easier. While English is widely spoken compared with pretty much every other city in China, the majority of Shanghai residents do not speak a great deal of English, and being able to communicate basic pleasantries and make basic requests can really help.
Even if you are only in Shanghai for a short time and will not be learning Mandarin long-term, it should be stressed that learning the very basics - greetings, asking for directions, ordering food, etc. - is extremely helpful. Not only will it ease and enrich your experience of the city in general, but it will also please the locals and possibly make them more inclined to help you, as they are generally equal parts surprised and delighted to hear a foreigner try to speak their language.
Don't be disheartened if your attempts at speaking Mandarin are at first met with a blank stare: many Chinese people will not be expecting a foreigner to be able to speak anything else but their native language, so your addressing them in Chinese may take them by surprise. Simply repeat the phrase a few times - if they're still not getting it after that, your pronunciation is probably off and you should probably switch back to gesticulating wildly to communicate your point.
You can find a pinyin pronunciation guide here.
Basic Mandarin phrases
Here is a post with some useful Chinese phrases. If you are going to boil it down to a handful of useful set phrases you'll use on a daily basis, however, the following phrases should be enough for you to get by.
Click the links for a page with an audio sample.
你好 - nǐ hǎo - "hello"
The most common greeting. You can amend it to fit the time of day (e.g. 早上好 - zǎoshang hǎo, "good morning"; or 晚上好 - wǎnshàng hǎo, "good evening"), but nǐ hǎo is good for any time of day or night.
再见 - zài jiàn - "goodbye"
The most common way to bid farewell. If it is somewhere you frequent, you can also use 下次见 - xià cì jiàn - "see you next time".
谢谢 - xiè xiè - "thank you"
The basic word for giving thanks. Combine with a smile to show your gratitude.
不要 - bú yào - "do not want"
Possibly the most useful phrase you can have at your disposal. You will be offered a lot of stuff you do not want in Shanghai, and bú yào will let them know immediately that you are not interested. A related phrase is 不用 - bú yòng, "do not need".
服务员 - fúwùyuán - "waiter"
In restaurants in China, if you want service it is not considered rude to locate the nearest waiter, raise your hand and shout fúwùyuán! Don't bother with subtly trying to make eye contact - just get their attention. Obviously there's no need to deafen the people on the table next to you, but don't be afraid to give it some volume if you're in a busy restaurant.
多少钱？ - duōshǎo qián? - "how much?"
You'll be asking this a lot in markets or anywhere where prices are not clearly marked. Pointing at something and asking 这个多少钱？ - zhège duōshǎo qián? - "how much is this one?" will generally yield the desired result.
你会(说)英文吗？ - nǐ huì (shuō) yīngwén ma? - "can you speak English?"
It's unlikely that somebody who by default addressed you in Mandarin would speak much English, but sometimes it's worth a go. Two related useful phrases are 听不懂 - tīng bù dǒng - "I don't understand", and 我不会(说)中文 - wǒ bú huì (shuō) zhōngwén - "I don't speak Chinese".
不好意思 - bù hǎoyìsi - "sorry"
Use this if you do something wrong (like bump into somebody in the street). For a more serious apology, use 对不起 - duìbùqǐ. If you just want to get somebody's attention to ask them a question you can preface your question with 请问 - qǐngwèn, "excuse me".
(place) 在哪里？ - (place) zài nǎli? - "where is (place)?"
The most common usage of this construction is probably going to be the phrase 厕所在哪里 - cèsuǒ zài nǎli? - "where is the bathroom?". Simply replace 厕所 (cèsuǒ) with the word for what you're looking for, e.g. 卫生间 (wèishēngjiān - "washroom" (as opposed to a WC)); 地铁站 (dìtiězhàn - "subway station"), etc.
The best way to learn how to say these phrases is not to do what many foreigners do and transcribe Mandarin phonemes into English phonetically (e.g. nǐ hǎo becomes "nee how"), but to listen to the phrases being spoken by a native speaker and repeat them enough times that you feel comfortable both saying and listening to them.
A quick note on the words for "yes" and "no" in Chinese - though it may seem strange, there are no concrete words for answering in the affirmative or negative in Chinese - what Chinese people usually do is restate the verb. For example, if asked 你要袋子吗？ (nǐ yào dàizi ma? - "do you want a bag?"), you would reply 要 (yào - "want"), or 不要 (bú yào - "do not want").
However, if your knowledge of Mandarin is sketchy at best and you simply want to tell somebody, if the item they are pointing at is the one you want, or if they are saying your name correctly, etc., you can use 对 (duì - "correct") and 不对 (bú duì - "wrong"), or 是 (shì - "is") and 不是 (bú shì - "is not").
Here's a slightly goofy but extremely useful video from YouTube user heylianne with most of the phrases above. Note that these guys are Taiwanese, but everything in the video is perfectly comprehensible to a mainlander. (If you're reading this in China, you'll need a VPN to watch it as it's on YouTube)
Chinese number gestures
For haggling at markets and the like, it's very worthwhile to learn Chinese hand gestures for numbers: these will not only show the other party that you're familiar enough with haggling in China that they may not try to rip you off quite so much, but it will also save you having to learn how to count (though you'll still need to know the words for 10 (十 - shí), 100 (百 - bǎi), and 1000 (千 - qiān)).
However, if you're not comfortable with this then don't worry - the standard way of overcoming the language barrier when haggling is for you and the shopkeeper to negotiate a fair price by using a nearby calculator.