Product Options and Catalogs are different methods for organizing your products online. It designates how the product will appear under Products on an online portal.
Product Option allows for a specific set of similar product items to be grouped online. The products would all basically have the same setup with the exception of a single variable, such as finish or size. This gives the appearance online of a single product with a drop-down for the individual products based on that variable.
Catalog allows for each product to be listed individually under the catalog drop-down. The catalog can be broad (ie. Marketing) or specific (ie. Banners). The catalogs can also be nested (ie. Marketing > Banners > Banner - 2 Sided - Gloss).
You are not restricted to using one method for organzing your products. A catalog and product option could both be applied to a product item, depending on how you would like it to appear online.