We retrospectively investigated the effect of adding dipeptidyl peptidase-4 (DPP-4) inhibitor and tapering sulfonylurea on blood glucose fluctuation in Asian patients with type 2 diabetes mellitus under basal-supported oral therapy (BOT). We recruited twenty-two consecutive Japanese patients with type 2 diabetes mellitus who had blood glucose fluctuation under the combination therapy of insulin glargine and glimepiride and had sitagliptin initiated with glimepiride tapared. Their hemoglobin A1c levels and mean blood glucose profiles of seven points in self-monitoring blood glucose (SMBG) were 7.4 ± 0.6% and 8.6 ± 2.0 mmol/L, respectively. Sitagliptin was initiated with the dose of 50 mg per day and titrated up to 100 mg per day when necessary. Glimepiride was withdrawn if possible. Blood glucose fluctuation was evaluated with SMBG by calculating M-value, its range (the difference of maximum and minimum blood glucose levels), and its coefficient of variation (CV). Two months after sitagliptin add-on, M-value was decreased from 19 ± 13 to 13 ± 8 (p = 0.04). Blood glucose range and CV were also improved from 9.6 ± 2.9 mmol/L to 7.9 ± 2.6 mmol/L (p = 0.01), and from 33 ± 8% to 29 ± 8% (p < 0.01), respectively. Hemoglobin A1c levels and mean blood glucose profiles were unchanged (p = 0.93 and 0.47). In conclusion, blood glucose fluctuation was significantly improved two months after adding sitagliptin and tapering glimepiride in type 2 diabetic Japanese patients who were treated by BOT with insulin glargine and glimepiride.