A host is a server (computer with it's own software) different form your own (the client). You will also set up a server on your own computer to test things out before you FTP (upload) them to the server. There is client side code (HTML/CSS) and server side code (PHP, JavaScript), depending on which computer runs it.
The first issue to decide is which server type to use for hosting. The choices are Linux and Windows.
This is a fundamental issue but seems to be largely ignored in all introduction materials. Maybe you are just supposed to know this. Neither one is better than the other. It depends on your usage, background and budget.
Windows Hosting
Most people, as well as myself, use a Windows operating system, unless you are an Apple person. That is a separate issue. You may be familiar with Microsoft Office software from school or work. Microsoft produces some very fine software applications. The only problem is that they are very expensive and take huge amounts of processing power! I have extensive experience with Microsoft Visual Studio (Visual Basic, Visual C++, Visual Web Developer 2008 Express and .net languages like Visual Basic 2008 and Visual C#) . You would think I might lean that way, but I don't. There are a few reasons to choose this path. The first is that someone else is paying your bills. I hosted on a Widows host for a while, including databases. It worked fine but everything cost at least five times as much and all software is proprietary (It only works on Windows operating systems and you can't modify the code legally.) I did find a very good windows hosting company: Re-invent Technologies LLC. I no longer support Windows hosting so you won't find much of interest here. If you want to follow this route then go to the Visual Web Developer 2008 Express Edition website to download the free software. Windows hosted sites usually have file extensions like .aspx. About 15% of websites are hosted on Windows servers.
Linux Hosting
When I switched to Java programming I became more familiar with the Linux system even though it runs on a Windows operating system (and Apple) too. That's the beauty of it. It worked anywhere. It is also open source (The code is shared and can be modified.) Most software offers free editions which can be downloaded. 85% of websites are Linux hosted sites. You can tell by the file extensions like .php on the web pages. Both types use HTML and XHTML as well as others. I will use Unix hosting for all of the topics on this site. I started with a free site hosted on Yahoo/Geocities. Note: Yahoo/GeoCities is being discontinued later this year. I quickly upgraded to a pay site on their server. I stayed there for almost a year. I simultaneously had a duplicate Windows hosted site for 6 months. They used completely different technologies but functioned the same. I tried a major hosting service that advertises heavily using a female race car driver. I only stayed 2 weeks. I fortunately ended up at Host Monster where I currently reside and plan to stay. It is inexpensive, easy to use and the customer service is outstanding! Of course you are free to shop around but I spent a year sifting through thousands of hosting companies and pretty much found this by dumb luck. I was looking for a host that offered PHP and MySQL support at no additional cost or setup fee. Eventually you will need this.