The reasons that people collect certain types of books are varied and highly personal. A buyer may collect first editions from their favorite author, or their interest in collecting books may be incidental to the larger interest; such as an avid Beatles record collector acquiring the books of John Lennon as part of his collection. Ultimately, though, the value of a book as a collectible is directly related to the book's relative scarcity. When a lot of people want a book and there are few copies available, that book's value increases.
When we talk about rare books, we speak of books that have a limited supply. First editions tend to be considered "rare" as a generality because the first printing of a book represents only a small number of the total number of copies that are printed. Publishers, ever mindful of reducing their costs, will try to print the number of copies of that they project are likely to sell. The first printing for a first time author might be fairly limited since publishers don't necessarily expect an overwhelming demand. The hardcover printing of David Foster Wallace's first book, Broom of the System, was limited to a print-run of 1300 copies. On the other side of that example is the first edition of a best-selling author. The first US printing of JK Rowling's Harry Potter and the Deathly Hallows ran to 15 million copies.
Some entire categories of books have a greater chance of rarity. Poetry is a tiny portion of the total number of books sold in the world each year, so publishers don't tend to print that many copies of a book of poems. The value of poetry books is still dependent on collectors wanting that poet's work, of course, so the poems of the unknown, no matter how scarce, aren't likely to ever become valuable.
Children's books are famously scarce simply because children aren't the best library archivists! Torn pages, broken bindings and the addition of original works of art in crayon by the book's owner are common features of kid's books. Dust jackets are usually sacrificed early. That makes copies in collectible condition rare. First editions of many of Dr. Seuss's titles commonly sell for thousands of dollars.
Similar to children's books, cook books are often a casualty of their intended use. Favorite recipes are marked with samples of the ingredients, and wet hands leave damp pages. Cookbooks can provide a unique view into history by showing our relationship with food, so antiquarian cookbooks have a devoted collectors market, but are often difficult to come by in collectible condition.
Signed books are usually rare simply because most authors can't sign that many books, but rare is a relative term here. Some authors go on huge multi-continent promotional tours, signing books at bookshops around the world. In many cases, after the audience has left, an author will sign cartons of books for the store to sell in coming months. Some authors are more retiring and reticent. Thomas Pynchon and J.D. Salinger are both authors who famously avoided the limelight. Signed copies of both of their works exist, but are uncommon, and tend to command higher prices than many authors who travel the promotional tracks and meet their readers to sign books.
Some of the most rare signed books are association copies; books that have been signed and personalized by the author to another notable person, such as their editor, another author, or a politician.
An odd category of book collectible is ephemera. The name tells the story; these are ephemeral items. Book ephemera are printed documents and items that were never intended to last; a program from an inauguration, a signed and cashed check, a poster advertising a concert, things that most people discarded shortly after they came into existence. Their persistence is unexpected, and thus rare.
Age doesn't necessarily equate to scarcity or value. Books are fairly sturdy and resilient. It's hard to destroy a book, so unless a book is otherwise of interest to collectors, just being old isn't usually enough to make it valuable. Exceptions to that generality are the truly old book.
Incunabula, for example, are rare and valuable because of the place they hold in history. Incunabula are books printed in the 15th century in the infancy of moveable type. The Gutenberg Bible is one of the most famous and most valuable books in history because it is one of the first books ever printed using moveable type. The early works that followed are collectible regardless of their subject matter. As time went on, the scarcity of books decreased. New book production technologies were introduced throughout the 19th century, reducing the costs of book printing, making books increasingly affordable, and as a result, more common.
Professional booksellers and collectors typically look at two factors for determining the rarity of a book: the number of copies available, and the number of potential buyers that might exist for that volume. The number of copies of a book that are in existence is the most solid determinant of rarity, but other factors can come into play, such as the number of copies that are in private collections or libraries that aren't likely to come into the open market any time soon. Determining the rarity of a book can require research, and in many cases, first had experience with the book market. Contacting a reputable professional bookseller is always a useful tool.