OP, how cheap is "cheap"? Do you have any particular learning goals in mind? What sort of music do you like to listen to? Are you particularly keen on learning the keyboard, or have you considered other instruments?
>>6219 I was checking out ebay and came across roll-up keyboards that can plug into a USB slot. They cost about £35. I don't want to blow over £60ish on something I don't know how to play.
I can't pin down my own music taste, it is too varied. Classical, Bluegrass, film scores, Disco, House, Hip-hop, Rap, Pop, Rock from the 80s and 70s, obscure mashup (like Future Funk) producers on soundcloud and bandcamp, etc.
I thought about getting a second hand guitar, but I think the keyboard is probably the best.
>>6220 Any USB keyboards are likely to just be controllers, so you'd need to install software on your computer to actually produce the sound output for headphones. A quick google suggests that the following might help: http://www.synthfont.com/links_to_soundfonts.html.
If you'd rather just get a self-contained keyboard, then I'd suggest looking up some Casio ones. They tend to be cheaper than the other dominant brand, Yamaha, and though not typically as full-featured would be fine for basic learning/practice.
>>6220 >I was checking out ebay and came across roll-up keyboards that can plug into a USB slot. They cost about £35.
That'd be a waste of £35, the roll-up thing is a novelty. You should be able to find a passable keyboard with "expression" (responds accordingly to the pressure at which you hit the keys, like a real piano would) for around the same if you dig around some second-hand shops, over the coming months especially, as people do the post-Christmas "you need to get rid of that old thing"/spring clean toss-out. There are a lot of keyboards out there, and you'd be hard pressed to find one without a headphone socket. (Exceptions include the aforementioned MIDI keyboard, which is just an input device, or at the other end something like a full-size electric piano, which you won't find for £35). Assuming you've got the space to store one, it shouldn't be difficult to find something cheap to mess around with, and whatever you decide you don't like about it will probably be as valuable as what you learn on it.
USB keyboards don't work well unless you also buy an audio interface. The problem is that ordinary soundcards have quite a lot of latency, so there's a delay between pressing a key on the keyboard and hearing sound from the computer. More than about 40ms of latency is basically unplayable, but most built-in soundcards have about 250ms of latency. Macs are the exception to this.
The roll-up keyboards are absolutely hopeless. You can buy a perfectly playable USB controller keyboard for about £60 - an Alesis Q49 or Acorn Masterkey would fit the bill. The cheapest usable audio interface is the Behringer UCA222, which costs about £25.
Using a controller keyboard and an audio interface is a big faff if you just want to play the keyboard, but it does open up the world of audio production. If you're willing to invest the time in learning, this route would allow you to compose and record music using software synthesisers and samplers. I'm happy to advise on the basics of computer-based music production if this interests you.
If you just want to play the keyboard, have a look on eBay for a Casio or Yamaha keyboard. You'll find plenty around the £60 mark. If you're at all serious about learning piano then you'll outgrow one of these keyboards fairly quickly, but they're a decent starting point. Look for something with full-size keys; If you can find something with "velocity sensitivity" or "touch sensitivity" at that price, all the better.
>>6220 One thing I will say as you mentioned the possibility of getting a guitar: if you get an electric, you can buy headphone-only amps very cheaply from most music stores or online. Guitar itself will typically be a bit more of an investment but might be worth bearing in mind if you spot anything second hand you fancy trying.