Well, if we do away with aleph numbers and the like and just concentrate on this abstract concept ∞, then we have to think about what infinity is. This is perhaps not the most clear-cut question in the world, but - really - we can think of infinity by playing a counting game. As we count the numbers:

1, 2, 3, 4, 5, 6, 7, 8, 9...

We add one to each number to get the next number. We can go on counting:

...1,000, 1,001, 1,002, 1,003, 1,004, 1,005...

And on:

...1,000,000, 1,000,001, 1,000,002, 1,000,003...

If we kept on and on counting (although, if we counted one number per second, it would take us over 31.6 years of non-stop counting to reach one billion, so it would not be practically possible to do this, just theoretically) we can reach numbers of immense size, such as Graham's number, which far far outstrips the number of atoms in the universe. In fact, I don't think it would even be possible to imagine how much bigger Graham's number is than that. And we can carry on and what it's pretty easy to realise here is that it can go on forever, we can always keep adding one to make the next bigger number, with no end to the counting. This is where the concept of ∞ comes in.

In the theories of calculus and many other areas of mathematics, we use the symbol to denote unboundedness, really, doing it over and over without end. It is not really treated as a number. If we want to try to treat it as a number and do operations on it, we really need something like Cantor's cardinal arithmetic. But, without going into all of that, we can make it a little more intuitive and just say that ∞ is the end of counting. This is not all that precise, really, but if we just want to get our heads around what this symbol: ∞ means, we have to look back to counting. Essentially ∞ says that I can go on adding 1 for ever and ever and ever and it will not end (in-finite), so I will use this symbol ∞ to mean the point beyond which I cannot keep counting. For this reason 1 + ∞= ∞  and 1 - ∞ = -∞

noelevans makes an interesting and - of course - entirely correct point which also touches on the idea of infinitesimals. Naturally, however, in the same way that it is not possible (assuming one number per second) to count even as far as 3 billion, it is not possible to count all the numbers in 2 seconds, not even theoretically (well not in this universe anyway), since the Planck time is a finite period of time and no observable change is thought to be able to take place on a time-scale which is smaller than this.

There are a few paradoxes yes (potentially anyway). I think that makes it rather more fun

Oh yes, that's absolutely true and a very nice fact. In the universe of mathematics this is fine, the only problem would be if you tried to do it yourself!