The antigens present on the cell surface membranes of organs are different from person to person due to genetic variation (leading to protein tertiary structure variation - remember antigens are proteins and that the genetic makeup of an individual determines the set of proteins that individual synthesises). The closer relatives 2 individuals are, the more similar their genetic makeup hence it is far less likely that an organ from say an identical twin would be rejected than that of an unrelated individual.
Rejection is when the specific immune response identifies the organ transplanted as foreign - i.e. treats it as it would a pathogen such as a virus or bacterium. The displayed antigens on the cell surface membranes of the cells on the transplanted organ are then identified by a specific B lymphocyte (recall that there is B cell diversity and B lymphocytes that circulate in the blood and lymph only identify non-self antigens) with a complementary receptor for the antigen. This activates the B cell, causing it to undergo clonal expansion whereby it produces many copies of B cells that are specific to the foreign antigen originally presented. The resulting cells then differentiate into plasma cells that produce antibodies that will attach to the antigens on the cell surface membrane and lead to the destruction of the organ and memory B cells. B lymphocytes also play a role in activating the corresponding T lymphocyte, also specific to the antigen originally presented through antigen presentation. The B cell presents the antigen to the T lymphocyte, causing T cell activation and the T cells undergo mitosis and differentiate into memory T cells, cytotoxic T cells, and T helper cells.