I have created a subclass for ValueListItemUIElement to handle some rare exceptions that occur in a mixed WinForms/WPF environment (I have an override for InitAppearance that calls the base class and catches exceptions that occur in this call chain). This all works pretty well so far.
However, this seems to have broken UI automation and this is probably related to the way how I create and initialize the new instances. I used a CreationFilter (IUIElementCreationFilter) and in there I have the following section:
public void AfterCreateChildElements(UIElement parent) { switch (parent) { ... case ValueListItemContainerUIElement container: { // replace standard ValueListItemUIElement with our custom class that catches // null reference exceptions var newChildElements = parent.ChildElements.Select( c => { if (c is ValueListItemUIElement valueListItemUIElement) { var newValueListItemUIElement = new SafeValueListItemUIElement( container, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; return newValueListItemUIElement; } return null; }) .Where(i => i != null) .ToList(); parent.ChildElements.Clear(); parent.ChildElements.AddRange(newChildElements); parent.DirtyChildElements(true); break; }
I assume something is missing in my initialization of the new instances. In the debugger I noticed that the AccessibilityInstance of the new entities has an empty (0,0,0,0) BoundingRectangle and Bounds property.
Does anyone have any clue what might be missing here?
Hello,
I have created a small sample trying to reproduce the described behavior. I used the provided snippet, however since I'm not sure what is the implementation of the mentioned custom class and the data in the value list, I modified it a little bit in order to check whether the item value is null instead of the item itself. On my side, the only issue I'm seeing is that even after some child is removed the parent container size remains the same. If this is the " broken UI " you mentioned, I could say that the reason for such behavior is because in the AfterCreateChildElements method the container of the drop-down is already calculated. The best suggestion is not to add a null value or items to the ValueList when it is created and there to perform some kind of check if it is possible. This way there would be no need for CreationFilter.
Otherwise, attached you will find my sample for your reference. Please test it on your side and let me know how it behaves. If this is not an accurate demonstration of what you are trying to achieve please feel free to modify it and send it back to me along with steps to reproduce. Alternatively, if the behavior cannot be replicated please feel free to provide your own sample. Remove any external dependencies and code that is not directly related to the issue, zip your application and attach it in this case.
Having a working sample on my side, which I can debug, is going to be very helpful in finding the root cause of this behavior.
Thank you for your cooperation.
Looking forward to hearing from you.
Regards,
Teodosia Hristodorova
Associate Software Developer
3731.UltraGrid_ValueListItem_CreationFilter.zip
Hello Teodosia,
thanks for your quick response. In fact I can reproduce the issue with your attached project without having to introduce a true custom class.
The issue is actually not a broken UI but broken UI automation. UI Automation is what tools like TestComplete and others are using to run automated UI tests. I could verify the issue using the Inspect tool from the windows SDK (for example here: C:\Program Files (x86)\Windows Kits\10\bin\<version>\x64\inspect.exe).
The first screenshot shows the output of the Inspect tool when the creation filter is active:
My focus is on the Monday item of the drop down and the Inspect tool shows that this item is offscreen, thus it is not showing a highlight to indicate that the item is accessible for UI automation.
In a next step I commented out the creation filter and get this situation:
Here the monday item has a highlight and the offscreen property is set to false.
As I have mentioned above I assume the bounds of the Accessibility object of these ValueListItemUIElements are not properly initialized - but I have no clue how I would do this.
BTW, one modification I had to make to your project was to use Infragistics 11.2 because that's the version we are still on. Other than that your code is unmodified.
Thanks in advance for your help in this matter.
Best regards
Uwe
Hi Mike - while I confirmed that your code fixes the automation it will not fix my problem because it effectively NEVER replaces the UIItems with the subclassed ones because of an error in the logic in your code. With your code my debugger never breaks in the section where you create a new instance. So the behavior is like the creation filter didn't exist. I think the code should look like this instead:
... // Try to find an existing MyValueListItemUIElement and re-use it if possible. MyValueListItemUIElement newValueListItemUIElement = parent.GetDescendant(typeof(MyValueListItemUIElement), valueListItemUIElement.ValueListItem) as MyValueListItemUIElement; // only create new UI item if we didn't find a subclassed one if (newValueListItemUIElement == null) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; } // replace with existing or new ui item parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); ...
However, once I come back to actually replacing the items IU automation no longer works (i.e. Inspect no longer highlights the items in the drop down). So it looks like the journey is not yet over ...
Yes, I see my mistake there. Sorry about that.
This doesn't work because by the time AfterCreateChildElements has fired, the parent element is throwing away the existing MyValueListItemUIElement, and putting a regular ValueListItemUIElement in it's place. So your creation filter ends up swapping them out again. The code I put in to find an existing one never finds it, because it's already been removed. So I think we can get around this by caching the MyValueListItemUIElements in the CreationFilter.
public class ValueItemsCreationFilter : IUIElementCreationFilter { private Dictionary<ValueListItem, MyValueListItemUIElement> myValueListItemUIElements = new Dictionary<ValueListItem, MyValueListItemUIElement>(); public void AfterCreateChildElements(UIElement parent) { if (parent is ValueListItemContainerUIElement) { for (int i = parent.ChildElements.Count - 1; i >= 0; i--) { var valueListItemUIElement = parent.ChildElements[i] as ValueListItemUIElement; if (valueListItemUIElement != null && !(valueListItemUIElement is MyValueListItemUIElement)) { // Try to find an existing MyValueListItemUIElement and re-use it if possible. var valueListItem = valueListItemUIElement.ValueListItem; MyValueListItemUIElement newValueListItemUIElement; bool success = this.myValueListItemUIElements.TryGetValue(valueListItem, out newValueListItemUIElement); if (!success) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); myValueListItemUIElements[valueListItem] = newValueListItemUIElement; } else { newValueListItemUIElement.Initialize( valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex ); } } } //parent.DirtyChildElements(true); var height = parent.ChildElements.Count * parent.ChildElements[0].Rect.Height; parent.Rect = new Rectangle(parent.Rect.X, parent.Rect.Y, parent.Rect.Width, height); } } public bool BeforeCreateChildElements(UIElement parent) { return false; } }
This seems to work - it only replaces the elements the first time you drop down the list and then doesn't replace them again while the dropdown is open. Unless I made some other logical error.
I don't think you did - I added tracepoints in my test program and can see that items are created only once and then retrieved when the dropdown is expanded again.
Since the dictionary takes a ValueListItem as a key there shouldn't be an issue with multiple drop downs in the same grid. And the creation filter is per grid instance i.e. multiple grids in an application should not be an issue either.
Looks like this time it's really solved. This issue took me through some ups and downs. I hope this time we can stay on the up level (for a while at least).
Thanks again for your continued help and perseverance in this matter!
One thing you might consider is what to do if the cache gets too big. That could happen, in theory, if you created a lot of new ValueListItems, either in a bunch of different ValueLists, or maybe because you ended up re-creating them for some reason. It might not be a terrible idea to trap the grid's CellListDropDown event and call a method on the CreationFilter that clears the cached every time the ValueList drops down - just to be safe.
Yes - makes sense. And it's easy to do. I just added it to the sample program and it works as expected.