In papillary thyroid carcinoma (PTC), extrathyroid extension (Ex) and clinical lymph node metastasis (N) significantly affect the prognosis. We investigated the prognosis of patients with PTC 1 cm or less (1,220 patients), 1.1-2 cm (2,101 patients), 2.1-3 cm (1,249 patients), 3.1-4 cm (645 patients), and larger than 4 cm (563 patients). We classified N factor into three categories: N0, no clinical node metastasis: N1, clinical node metastasis smaller than 3 cm and without extranodal tumor extension requiring at least partial excision of adjacent organs for node dissection: and N2, clinical node metastasis 3 cm or larger or showing extranodal tumor extension. N2 markedly affected lymph node and distant recurrence-free survivals and cause-specific survival, regardless of the tumor size. N1 also adversely affected lymph node and distant recurrence-free survival but not cause-specific survival. Ex did not affect patients’ prognosis with PTC 1 cm or less. It became a prognostic factor with PTC larger than 1 cm, and worsened lymph node and distant recurrence-free survival not only for N0 but also for N1 PTC larger than 3 cm and larger than 2 cm, respectively. However, its influence is limited for N2 PTC patients. Furthermore, Ex worsened the CSS with PTC larger than 2 cm in combination with N2. We have to note that the prognostic significance for lymph node and distant recurrence-free and cause-specific survival of Ex and N varies according to the tumor size in order to accurately predict the clinical outcomes and establish therapeutic strategies for PTC patients.