If you know the character's code, you can enter it the same way. The double dagger () is unicode U+2021, so C-x RETURN 2021 RETURN will produce it. Wikipedia has a decent but incomplete list of unicode characters by number8 but the source9 is more complete, and affords you a sense of the awesome breadth of the unicode standard.
A much more cumbersome method is to change to the `ucs' input-method before inputting a string of characters by their unicode number. Type C-x RET C-\ and when prompted, choose the `ucs' (direct unicode) method. Now type u plus the number of the character (u2021 in this example of the double dagger); repeat for the next characters.
If you make use of some unusual symbols fairly regularly, it is worthwhile to bind those characters to a keystroke. See section 8.2 for details on how to do so.