When you create your summary using Summaries.Add, the Add method returns a SummarySettings object.

SummarySettings summarySettings = band.Summaries.Add(...);

summarySettings.DisplayFormat = "{0:#,##0.000}";

thanks,  so on the event for SummaryValueChanged I want to set the display format to the format of that specific column,  do you know how I can get the column name or index? 

Right now I am setting it like this.

        private void ultraGrid1_SummaryValueChanged(object sender, SummaryValueChangedEventArgs e)
        {
            e.SummaryValue.SummarySettings.DisplayFormat = "{0:###,###,###,###,###.##}";
        }

I would not do this in SummaryValueChanged. I would do it when you add the summary, as I did in the code I posted above. Otherwise you will be continually setting and resetting the property to the same value, which is inefficient.

Well the summary is chosen to be used or not by the user at run time therefore I would like it done in an event.   

 You could make this a little more efficient like so: 

            foreach (SummarySettings summarySettings in e.Column.Band.Summaries)           
            {
                if (summarySettings.SourceColumn == e.Column)
                    summarySettings.DisplayFormat = summarySettings.SummaryType + "= {0:" + e.Column.Format + "}";
            }

This code also assume that Format is not empty. I'm not sure what will happen if the Format is blank. 

This will not work as you are assuming that the index of the Summaries matches the Column header value.

This is what I used to make it work:

for (int i = 0; i < e.Column.Band.Summaries.Count; i++)
{
    e.Column.Band.Summaries[ i ].DisplayFormat = e.Column.Band.Summaries[ i ].SummaryType + "= {0:" + e.Column.Band.Summaries[ i ].SourceColumn.Format + "}";
}

Its not very good as each time you add/remove any form of summary you end up resetting ALL summaries, but I have not come up with a better version ... yet

tried putting this in the AfterSummaryDialog event wont work [:'(]

 e.Column.Band.Summaries[e.Column.Key].DisplayFormat = e.Column.Format; 

In that case, you should use the AfterSummaryDialog event, so you set the DisplayFormat once when the summary is created, rather than every time the value changes.