To understand what LaTeX is, it's worth realizing that there are three ways that text is frequently handled, two of which you're very familiar with:
Since you may never have encountered markup in its raw form before, we're going to take a gander at some now. Most web-browsers have an option to view the source code for a page, so go ahead and view the source for this page. In most browsers, right-clicking on the page's background will call up a context menu with "View Source" (or something similar, using the word "Source" somewhere) as one of the options; select this option and it'll call up a window with the HTML used to render the page. Notice that what you see in the "Source" window is mostly the same text appearing when you look at the page in the browser, and the main difference is that there are a lot of codes in angle-brackets which correspond to special formatting: some for boldface, some for italics, some for the bulleted list above, etc. Most of these codes are more-or-less human-readable: <B> is "bold", <UL> is "unordered list", <P> is "paragraph", and <H1> is "header level 1", for instance.
LaTeX is a different sort of markup, but it's motivated by the same principles: that you can look at the raw, unformatted code, and edit it by hand, and know what changes you're making.
LaTeX has been the document-preparation standard in math and physics (and to a lesser degree other sciences) for decades because it does mathematical typography very well. Equations are easy to render in LaTeX and they have better spacing than in other systems, and once one is proficient, typing them in with a text editor is actually easier than hunting on dropdown menus for the right symbols and layouts and such. Math calls for unusual sizing, organization, and spacing: aligning multiple successive equations at an equals sign, laying out exponents and subscripts and matrices, and other rich, complicated layout requirements. LaTeX does these very well. It also does a very good job at typographical layout, and is inspired by typographical traditions regarding interletter and interword spacing to a far higher degree than mainstream word processors.
Some history: The TeX document-formatting system was written by computer-science luminary Donald Knuth in 1978. He called it "TeX" because it was technological, and that "X" at the end was a stand-in for the Greek letter χ which appears in the etymological root of the word "technology". It's also supposed to be pronounced like a χ, which is a hard "k" sound just like in the word "technology".
TeX was visionary (particularly in 1978, when getting computers to make text look even marginally better than typewriters seemed like magic), but it was very fussy to do anything even remotely complicated in, so Knuth's contemporary Leslie Lamport wrote a layer in the early 80s making the whole system easier to use while keeping the same core. "Lamport TeX" became "LaTeX", and it has kept that name through periodic updates since. That X is still supposed to be a χ, though, so it's not pronounced like the rubbery fabric whose name it shares, but is typically pronounced "LAH-tek" or "LAY-tek" (technically, the "a" should be phonetically identical to the "a" in "Lamport", but that's tricky to do, and nobody pronounces it that way).
LaTeX itself is the language used to write the markup which is eventually turned into documents. These files are typically called "LaTeX source" or "LaTeX files", and anyone can write them with no special tools: you could fire up Notepad and write up LaTeX source code, but it wouldn't be a very pleasant experience, and the code on its own isn't all that useful.
To get something useful out of LaTeX source code, you need the other half of the equation, the "LaTeX engine" or "LaTeX compiler". This is a program which will read LaTeX source code and output a formatted version of the text in it; it does to LaTeX files the same thing that your web browser does with HTML files. This is probably not already on your computer, and you'll need to install it. Historically, LaTeX produced an output file called a "DVI" (DeVice-Independent formatted text) which was also not widely supported; mercifully, recent versions of the LaTeX engine can (and often by default do) produce the Adobe PDF (Portable Document File) format.
Now that you know what LaTeX is, it's clear you'll need to install it. But even here there are some points to be considered.
As mentioned above, the LaTeX engine is just a program to convert one file type (the LaTeX source) into another (the PDF output). The engine itself doesn't have a lot of bells and whistles, just a single input and a single output. Most users don't just install the engine, but install some ways to make writing the code and communicating with the engine easier. The two options available to you are a general text editor or a LaTeX development environment. Most users will probably prefer a LaTeX development environment.
If you have a good development environment, you won't need to interact with the engine directly as a rule, but you do need to install it. The standard engine for Windows is MiKTeX, and the "Basic Installer" should suit your needs. The standard engine for OS X is MacTeX, which comes with some development environments as well, although you can also install your own development environment.
If you're going to be writing a lot of computer code, a good text editor is useful, and to simplify, you might just want to edit your LaTeX source code on the same editor you use for other purposes. Text editors range from absolutely bare-bones systems like Microsoft Notepad up to systems which will do a lot of your code-juggling and syntax-checking for you.
Note that Microsoft Word is not a text editor. It can save as plain text, but it doesn't do so by default, and most of its features aren't geared towards manipulating the underlying text, but rather applying formatting to it. You emphatically do not want to use Word to edit LaTeX code or any other computer code. You probably don't want to use Notepad either, because it's very primitive. Good features of a text editor include:
There are many general-purpose code editors out there, and choice depends much on preference, ranging from highly user-friendly ones like Geany to the inscrutible but streamlined vi. Many are free; some are commercial. An incredibly overdetailed comparison is on Wikipedia's list.
If you're only writing LaTeX, and you don't care about having a utility you can also use to write C++ or Lisp code, then you'd probably be happier having a dedicated LaTeX-oriented environment. These have the features of text editors, but tuned specifically towards writing LaTeX code (so the only syntax-highlighting or indentation modes they'll have are for LaTeX, not for other languages), and in addition they often have toolbars and quick menus for inserting common LaTeX code and for invoking the LaTeX engine. Most of them also have built-in PDF readers, so that you can view your source code and output side-by-side. Many of them even come prepackaged with the LaTeX engine, so that you don't have to download it separately.
There are almost as many LaTeX editors as there are more general text editors. I suggest, for purposes of having a common, fully-featured but not bloated environment, either of the two very similar projects Texmaker or TeXstudio. Installation instructions are on their websites.
For now, you're just going to download a source file and see how compiling it into a PDF works, to get a feel for your development environment. Go ahead and download this sample LaTeX file, keeping track of where you saved it, and open it up in your development environment.
If you push the "Quick build" button (circled in red below in both Texmaker and TeXstudio), it should turn the source file into a PDF, and display the result. The PDF file will have the same name and location as the LaTeX source file, but the extension .pdf instead of .tex.
If you get an error saying the command couldn't run, go back and make sure you actually installed a LaTeX engine; if you didn't already install MiKTeX, do so now.
One point which is a bit advanced but is worth mentioning now is that when you first compiled, the PDF produced probably had a lot of places where you would expect to find section numbers and the like, but instead see the symbol "??". LaTeX finds cross-references in the same pass through the source code as it builds the PDF, which means it can't use those cross-references until the next pass-through when you rebuild the PDF. Compile the code again and those symbols will go away to be replaced by numbers; generally speaking, it's a good idea to compile your code twice to get your final PDF for this reason.
Anyways, that's the basics of the development environment; hopefully you'll find the environment itself pretty usable, since it behaves like a text editor for the most part. If you want to know how to write source code, though, go on to Lesson #1!