Many physical and chemical factors affect the rheological properties of crumb rubber modifier (CRM) binders. This study is aimed at understanding the influence of CRM microstructure on one of the rheological properties, the high temperature properties of the CRM binders. To this end, the high temperature properties of the binders that were modified by two types of CRM with different microstructures, three different sizes, two percentages as well as three different mixing times, were evaluated by dynamic shear rheometer (DSR) test on samples before and after RTFO aging. A series of scanning electron microscope (SEM) were taken on the two types of CRM of three different mesh sizes. Results from this study showed that: (1) ambient CRM has porous surfaces and cryogenic CRM has angular with smooth cracked surfaces; (2) an addition of 10–15% of CRM in the binder can increase one and two PG grades at high temperature for either ambient or cryogenic CRM regardless of the microstructure of CRM; (3) when a high percentage and coarser CRM is mixed, the microstructures of CRM have significant influence on the high failure temperature and (4) change in mixing time from 15 to 45 min did not alter much in the failure temperature, suggesting that a mixing time of 15 min interact completely for 15% CRM with the binder.