The short answer is, you can't. We do not support modification of the stock dialogs (we recommend instead that you implement your own), so if you want to do so it is on an "at your own risk" basis. In this case, however, you should be able to do it fairly easily.

Example:
this.dayView.CalendarInfo.BeforeDisplayReminderDialog += new CancelableAppointmentEventHandler(CalendarInfo_BeforeDisplayReminderDialog);

void CalendarInfo_BeforeDisplayReminderDialog(object sender, CancelableAppointmentEventArgs e)
{
    e.Cancel = true;

    ReminderDialog dialog = new ReminderDialog( sender as UltraCalendarInfo );
    foreach( Control control in dialog.Controls )
    {
        if ( control.Name == "cmdDismissAll" )
            control.Visible = false;
    }

    dialog.AddAppointmentReminder( e.Appointment );
    dialog.Show();
}

That said, I highly recommend you test the code rigorously before deciding whether to use this solution (I didn't).