I've been living here for close to 5 years, I've taught in an academic high school and I currently teach business English to executives in an international medical tech company.
German was hard for me to learn, and from what I've seen, hard for other native English speakers to learn in comparison to languages like Spanish or French.
The vocab can at times be very similar if not the same. The grammar and cases make it hard going, and 3 genders for every noun complicates things a lot more than 2 do for something like French. That said, if you can immerse yourself, you can get to a reasonable standard pretty quickly. The earlier you learn, the better. One other thing I notice I have huge issues with. If you can think in an ordered fashion, it makes things easier. German is a language where, in order to form a sentence correctly, you have to know how you're going to finish before you start. It all happens in microseconds on a subconscious level, but if you're like me and have no idea what you want to say until you start talking, you can have a hell of a time with articles, cases and word order in German.
One thing I've noticed here and nowhere else in Europe. If you try and speak German and fuck it up, the Germans will pretty much always correct you. If you go to France (not Paris), and fuck up their beautiful language, they will love you for at least trying to speak it (in my experience). Italy too. In Germany you have to prepare yourself for a language lesson from some grumpy fucker.
Something else I notice (at least where I am) - you can have the tiniest mispronounciation of a word and no-body will understand you. I was talking about iPads yesterday and was calling them 'tablet computers'. No-one had a clue what I was saying until someone said "Ach - Tepplet!". It's not so different, but it happens all the freaking time - and my German accent is usually very good.
Germans speak terrible English in comparison to their northern neighbours and to Switzerland. Much better than Italians and Spaniards, but the general level of English here is a lot worse than you would expect. I'm sure it has to do with having all their films and television shows dubbed into German, instead of having original language with subtitles like in Holland and Scandinavia.