I have created a subclass for ValueListItemUIElement to handle some rare exceptions that occur in a mixed WinForms/WPF environment (I have an override for InitAppearance that calls the base class and catches exceptions that occur in this call chain). This all works pretty well so far.
However, this seems to have broken UI automation and this is probably related to the way how I create and initialize the new instances. I used a CreationFilter (IUIElementCreationFilter) and in there I have the following section:
public void AfterCreateChildElements(UIElement parent) { switch (parent) { ... case ValueListItemContainerUIElement container: { // replace standard ValueListItemUIElement with our custom class that catches // null reference exceptions var newChildElements = parent.ChildElements.Select( c => { if (c is ValueListItemUIElement valueListItemUIElement) { var newValueListItemUIElement = new SafeValueListItemUIElement( container, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; return newValueListItemUIElement; } return null; }) .Where(i => i != null) .ToList(); parent.ChildElements.Clear(); parent.ChildElements.AddRange(newChildElements); parent.DirtyChildElements(true); break; }
I assume something is missing in my initialization of the new instances. In the debugger I noticed that the AccessibilityInstance of the new entities has an empty (0,0,0,0) BoundingRectangle and Bounds property.
Does anyone have any clue what might be missing here?
Hello,
I have created a small sample trying to reproduce the described behavior. I used the provided snippet, however since I'm not sure what is the implementation of the mentioned custom class and the data in the value list, I modified it a little bit in order to check whether the item value is null instead of the item itself. On my side, the only issue I'm seeing is that even after some child is removed the parent container size remains the same. If this is the " broken UI " you mentioned, I could say that the reason for such behavior is because in the AfterCreateChildElements method the container of the drop-down is already calculated. The best suggestion is not to add a null value or items to the ValueList when it is created and there to perform some kind of check if it is possible. This way there would be no need for CreationFilter.
Otherwise, attached you will find my sample for your reference. Please test it on your side and let me know how it behaves. If this is not an accurate demonstration of what you are trying to achieve please feel free to modify it and send it back to me along with steps to reproduce. Alternatively, if the behavior cannot be replicated please feel free to provide your own sample. Remove any external dependencies and code that is not directly related to the issue, zip your application and attach it in this case.
Having a working sample on my side, which I can debug, is going to be very helpful in finding the root cause of this behavior.
Thank you for your cooperation.
Looking forward to hearing from you.
Regards,
Teodosia Hristodorova
Associate Software Developer
3731.UltraGrid_ValueListItem_CreationFilter.zip
Hello Teodosia,
thanks for your quick response. In fact I can reproduce the issue with your attached project without having to introduce a true custom class.
The issue is actually not a broken UI but broken UI automation. UI Automation is what tools like TestComplete and others are using to run automated UI tests. I could verify the issue using the Inspect tool from the windows SDK (for example here: C:\Program Files (x86)\Windows Kits\10\bin\<version>\x64\inspect.exe).
The first screenshot shows the output of the Inspect tool when the creation filter is active:
My focus is on the Monday item of the drop down and the Inspect tool shows that this item is offscreen, thus it is not showing a highlight to indicate that the item is accessible for UI automation.
In a next step I commented out the creation filter and get this situation:
Here the monday item has a highlight and the offscreen property is set to false.
As I have mentioned above I assume the bounds of the Accessibility object of these ValueListItemUIElements are not properly initialized - but I have no clue how I would do this.
BTW, one modification I had to make to your project was to use Infragistics 11.2 because that's the version we are still on. Other than that your code is unmodified.
Thanks in advance for your help in this matter.
Best regards
Uwe
Puuh ... pretty complex sequence of events. I checked with 11.2 (had to uncomment the UIAutomationForCodedUITestingEnabled toggle because that does not exist in this version) - and it also works - great!
I currently do not think that this will re-introduce my WPF interaction problem. I basically solved this (kind of) by catching the NullReferenceException in the following overload:
protected override void InitAppearance(ref AppearanceData appearance, ref AppearancePropFlags requestedProps) { try { base.InitAppearance(ref appearance, ref requestedProps); } catch (NullReferenceException e) { TraceUI.Log.Error($"NullReferenceException in InitAppearance of {nameof(SafeValueListItemUIElement)}: {e}"); } }
I think this workaround should continue to work - I just avoid unnecessary replacement of UIElements using your suggestion. Unfortunately this issue cannot be reproduced easily - only one of our testers had a pretty good hit rate.
I will integrate your solution and ask him to try to reproduce.
Thanks a lot for your help in this matter!
Uwe Nassal said:I checked with 11.2 (had to uncomment the UIAutomationForCodedUITestingEnabled toggle because that does not exist in this version) - and it also works - great!
Yes, 11.2 was before we implemented UIA, so that property (which turns OFF UIA) does not exist in the older version. I needed that so I could reproduce the issue in the latest version.
I'd be interested to know what the original issue was. I know there's something going on here where you are hiding/removing the UIElements for ValueItems whose DataValue is null. Not sure why you don't just remove those items from the list instead, rather than removing them from the display via a CreationFilter. Or maybe use UltraDropDown instead of a ValueList and actually hide those rows.
At the time the original issue was reported I did not have a creation filter for the ValueListItemUIElements. I introduced this to allow me to inject a custom class that was catching this exception:
System.NullReferenceException: Object reference not set to an instance of an object. at Infragistics.Win.ValueListDropDownAppearanceManager.ResolveValueListItemAppearance(ValueListItemUIElement valueListItemElement, AppearanceData& appData, AppearancePropFlags requestedProps) at Infragistics.Win.ValueListItemUIElement.InitAppearance(AppearanceData& appearance, AppearancePropFlags& requestedProps) at Infragistics.Win.ValueListItemUIElement.PositionChildElements() at Infragistics.Win.UIElement.VerifyChildElements(ControlUIElementBase controlElement, Boolean recursive) at Infragistics.Win.UIElement.VerifyChildElements(ControlUIElementBase controlElement, Boolean recursive) at Infragistics.Win.UIElement.VerifyChildElements(ControlUIElementBase controlElement, Boolean recursive) at Infragistics.Win.UIElement.VerifyChildElements(Boolean recursive) at Infragistics.Win.ControlUIElementBase.VerifyIfElementsChanged(Boolean verify, Boolean syncMouseEntered) at Infragistics.Win.ControlUIElementBase.ProcessMouseHover(Object sender, EventArgs e) at Infragistics.Win.Utilities.ProcessEvent(Control control, ProcessEvent eventToProcess, EventArgs e) at Infragistics.Win.UltraControlBase.OnMouseHover(EventArgs e) at System.Windows.Forms.Control.WndProc(Message& m) at Infragistics.Win.ValueListDropDownUnsafe.WndProc(Message& message) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
I saw a video recorded by the submitter of the issue and the issue occurred after the following sequence of events:
While this sounds like a series of reproducable steps it was never reproducable on any machine that I was using. So there must be an additional condition (or probably a whole set) that is necessary to reproduce this issue. Since you guys (and I understand this) are always asking for reproducible samples I didn't even dare to submit this issue here.
I took a look at the ResolveValueListItemAppearance method and it has not changed a lot since 11.2. There are a number of places in this code where it might be conceivable that there would be a NullReferenceException. We could probably tighten it up a bit, but of course that would only happen in the latest release, so that wouldn't help you in v11.2. Sounds like maybe you are responding to the selection of a ValueList item and doing something (showing a dialog) that is forcing the ValueList to close while it's in the middle of painting or something like that. That's a pretty common scenario for exception like this to occur.But if that was the issue, then is seems like replacing the UIElement ONCE and re-using it should be a good solution and there's really no reason for you to keep replacing it every time like the sample here was doing. This will actually be more efficient, anyway.
Hi Mike - while I confirmed that your code fixes the automation it will not fix my problem because it effectively NEVER replaces the UIItems with the subclassed ones because of an error in the logic in your code. With your code my debugger never breaks in the section where you create a new instance. So the behavior is like the creation filter didn't exist. I think the code should look like this instead:
... // Try to find an existing MyValueListItemUIElement and re-use it if possible. MyValueListItemUIElement newValueListItemUIElement = parent.GetDescendant(typeof(MyValueListItemUIElement), valueListItemUIElement.ValueListItem) as MyValueListItemUIElement; // only create new UI item if we didn't find a subclassed one if (newValueListItemUIElement == null) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; } // replace with existing or new ui item parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); ...
However, once I come back to actually replacing the items IU automation no longer works (i.e. Inspect no longer highlights the items in the drop down). So it looks like the journey is not yet over ...
Yes - makes sense. And it's easy to do. I just added it to the sample program and it works as expected.
One thing you might consider is what to do if the cache gets too big. That could happen, in theory, if you created a lot of new ValueListItems, either in a bunch of different ValueLists, or maybe because you ended up re-creating them for some reason. It might not be a terrible idea to trap the grid's CellListDropDown event and call a method on the CreationFilter that clears the cached every time the ValueList drops down - just to be safe.
I don't think you did - I added tracepoints in my test program and can see that items are created only once and then retrieved when the dropdown is expanded again.
Since the dictionary takes a ValueListItem as a key there shouldn't be an issue with multiple drop downs in the same grid. And the creation filter is per grid instance i.e. multiple grids in an application should not be an issue either.
Looks like this time it's really solved. This issue took me through some ups and downs. I hope this time we can stay on the up level (for a while at least).
Thanks again for your continued help and perseverance in this matter!
Yes, I see my mistake there. Sorry about that.
This doesn't work because by the time AfterCreateChildElements has fired, the parent element is throwing away the existing MyValueListItemUIElement, and putting a regular ValueListItemUIElement in it's place. So your creation filter ends up swapping them out again. The code I put in to find an existing one never finds it, because it's already been removed. So I think we can get around this by caching the MyValueListItemUIElements in the CreationFilter.
public class ValueItemsCreationFilter : IUIElementCreationFilter { private Dictionary<ValueListItem, MyValueListItemUIElement> myValueListItemUIElements = new Dictionary<ValueListItem, MyValueListItemUIElement>(); public void AfterCreateChildElements(UIElement parent) { if (parent is ValueListItemContainerUIElement) { for (int i = parent.ChildElements.Count - 1; i >= 0; i--) { var valueListItemUIElement = parent.ChildElements[i] as ValueListItemUIElement; if (valueListItemUIElement != null && !(valueListItemUIElement is MyValueListItemUIElement)) { // Try to find an existing MyValueListItemUIElement and re-use it if possible. var valueListItem = valueListItemUIElement.ValueListItem; MyValueListItemUIElement newValueListItemUIElement; bool success = this.myValueListItemUIElements.TryGetValue(valueListItem, out newValueListItemUIElement); if (!success) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); myValueListItemUIElements[valueListItem] = newValueListItemUIElement; } else { newValueListItemUIElement.Initialize( valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex ); } } } //parent.DirtyChildElements(true); var height = parent.ChildElements.Count * parent.ChildElements[0].Rect.Height; parent.Rect = new Rectangle(parent.Rect.X, parent.Rect.Y, parent.Rect.Width, height); } } public bool BeforeCreateChildElements(UIElement parent) { return false; } }
This seems to work - it only replaces the elements the first time you drop down the list and then doesn't replace them again while the dropdown is open. Unless I made some other logical error.