I have created a subclass for ValueListItemUIElement to handle some rare exceptions that occur in a mixed WinForms/WPF environment (I have an override for InitAppearance that calls the base class and catches exceptions that occur in this call chain). This all works pretty well so far.
However, this seems to have broken UI automation and this is probably related to the way how I create and initialize the new instances. I used a CreationFilter (IUIElementCreationFilter) and in there I have the following section:
public void AfterCreateChildElements(UIElement parent) { switch (parent) { ... case ValueListItemContainerUIElement container: { // replace standard ValueListItemUIElement with our custom class that catches // null reference exceptions var newChildElements = parent.ChildElements.Select( c => { if (c is ValueListItemUIElement valueListItemUIElement) { var newValueListItemUIElement = new SafeValueListItemUIElement( container, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; return newValueListItemUIElement; } return null; }) .Where(i => i != null) .ToList(); parent.ChildElements.Clear(); parent.ChildElements.AddRange(newChildElements); parent.DirtyChildElements(true); break; }
I assume something is missing in my initialization of the new instances. In the debugger I noticed that the AccessibilityInstance of the new entities has an empty (0,0,0,0) BoundingRectangle and Bounds property.
Does anyone have any clue what might be missing here?
Hello,
I have created a small sample trying to reproduce the described behavior. I used the provided snippet, however since I'm not sure what is the implementation of the mentioned custom class and the data in the value list, I modified it a little bit in order to check whether the item value is null instead of the item itself. On my side, the only issue I'm seeing is that even after some child is removed the parent container size remains the same. If this is the " broken UI " you mentioned, I could say that the reason for such behavior is because in the AfterCreateChildElements method the container of the drop-down is already calculated. The best suggestion is not to add a null value or items to the ValueList when it is created and there to perform some kind of check if it is possible. This way there would be no need for CreationFilter.
Otherwise, attached you will find my sample for your reference. Please test it on your side and let me know how it behaves. If this is not an accurate demonstration of what you are trying to achieve please feel free to modify it and send it back to me along with steps to reproduce. Alternatively, if the behavior cannot be replicated please feel free to provide your own sample. Remove any external dependencies and code that is not directly related to the issue, zip your application and attach it in this case.
Having a working sample on my side, which I can debug, is going to be very helpful in finding the root cause of this behavior.
Thank you for your cooperation.
Looking forward to hearing from you.
Regards,
Teodosia Hristodorova
Associate Software Developer
3731.UltraGrid_ValueListItem_CreationFilter.zip
Hello Teodosia,
thanks for your quick response. In fact I can reproduce the issue with your attached project without having to introduce a true custom class.
The issue is actually not a broken UI but broken UI automation. UI Automation is what tools like TestComplete and others are using to run automated UI tests. I could verify the issue using the Inspect tool from the windows SDK (for example here: C:\Program Files (x86)\Windows Kits\10\bin\<version>\x64\inspect.exe).
The first screenshot shows the output of the Inspect tool when the creation filter is active:
My focus is on the Monday item of the drop down and the Inspect tool shows that this item is offscreen, thus it is not showing a highlight to indicate that the item is accessible for UI automation.
In a next step I commented out the creation filter and get this situation:
Here the monday item has a highlight and the offscreen property is set to false.
As I have mentioned above I assume the bounds of the Accessibility object of these ValueListItemUIElements are not properly initialized - but I have no clue how I would do this.
BTW, one modification I had to make to your project was to use Infragistics 11.2 because that's the version we are still on. Other than that your code is unmodified.
Thanks in advance for your help in this matter.
Best regards
Uwe
Same version here - in the drop down in the toolbar on the left: did you use MSAA or UIAutomation (I am using UIAutomation). Again, see my screenshots above.
Yes, as you can see here, it's UIAutomation. And the element shows a highlight and the BoundingRectangle is correct. But IsOffScreen is not even displayed. And actually... I just figured out why. If you go into the Options menu under "Settings..." there's a list of properties to display and a CheckBox for "Display unsupported properties."If I select IsOffScreen and also check the checkbox for "Display unsupported properties." then I see IsOffScreen on the list, but it always returns "[Not Supported]". In fact... like I said, we don't actually support UIA for accessibility. Our implementation is extremely limited and is only for use with CodedUITests. So when I turn this option on, I actually see a huge number of "[Not Supported]" properties. Not sure why you are seeing a value there, though. It should not be working as far as I can see. I suppose it's possible that on your machine, the fallback to MSAA mechanism is somehow trying to interpret the MSAA properties to determine if the item is on-screen or not. I can't see how that could be or why it would fail, since the UIElement you are creating is virtually identical to the one we create by default.
I see that you have the yellow highlight around the list item - I think this indicates that this item is accessible through UIAutomation. Is this the sample with or without creation filter? Because on my end the list items don't get a highlight once I create them through the creation filter.
The IsOffScreen property was really just a property that sticked out (it was shown by default on my machine) to be different between working and non-working automation. The essential requirement is that the items are accessible to UIAutomation. The yellow highlight seems to express this better than the IsOffScreen property - but there is a correlation (at least it was for me).
Something that might have got lost in the thread: I'm still on Infragistics 11.2 - not sure of the behavior with respect to automation changed a lot in the last couple of releases.
BTW - this is how my settings screen looks like for the Inspect Tool:
I have also unsupported properties unchecked - but all properties are marked for the "Display in Main Window" section.
Still I get this property for the ValueListItemUIElement. But maybe this is really something that changed on your end between versions.
One thing that is probably relevant:
When the ValueListItemUIElement is not replaced in the creation filter (i.e. when I have working automation) the HitTest method is invoked for this element.
When I replace it with an override of the ValueListItemUIElement and create an override of the HitTest method this method is never hit. So for some reason the HitTest of the replaced UIElement is never called.
Yes - makes sense. And it's easy to do. I just added it to the sample program and it works as expected.
One thing you might consider is what to do if the cache gets too big. That could happen, in theory, if you created a lot of new ValueListItems, either in a bunch of different ValueLists, or maybe because you ended up re-creating them for some reason. It might not be a terrible idea to trap the grid's CellListDropDown event and call a method on the CreationFilter that clears the cached every time the ValueList drops down - just to be safe.
I don't think you did - I added tracepoints in my test program and can see that items are created only once and then retrieved when the dropdown is expanded again.
Since the dictionary takes a ValueListItem as a key there shouldn't be an issue with multiple drop downs in the same grid. And the creation filter is per grid instance i.e. multiple grids in an application should not be an issue either.
Looks like this time it's really solved. This issue took me through some ups and downs. I hope this time we can stay on the up level (for a while at least).
Thanks again for your continued help and perseverance in this matter!
Yes, I see my mistake there. Sorry about that.
This doesn't work because by the time AfterCreateChildElements has fired, the parent element is throwing away the existing MyValueListItemUIElement, and putting a regular ValueListItemUIElement in it's place. So your creation filter ends up swapping them out again. The code I put in to find an existing one never finds it, because it's already been removed. So I think we can get around this by caching the MyValueListItemUIElements in the CreationFilter.
public class ValueItemsCreationFilter : IUIElementCreationFilter { private Dictionary<ValueListItem, MyValueListItemUIElement> myValueListItemUIElements = new Dictionary<ValueListItem, MyValueListItemUIElement>(); public void AfterCreateChildElements(UIElement parent) { if (parent is ValueListItemContainerUIElement) { for (int i = parent.ChildElements.Count - 1; i >= 0; i--) { var valueListItemUIElement = parent.ChildElements[i] as ValueListItemUIElement; if (valueListItemUIElement != null && !(valueListItemUIElement is MyValueListItemUIElement)) { // Try to find an existing MyValueListItemUIElement and re-use it if possible. var valueListItem = valueListItemUIElement.ValueListItem; MyValueListItemUIElement newValueListItemUIElement; bool success = this.myValueListItemUIElements.TryGetValue(valueListItem, out newValueListItemUIElement); if (!success) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); myValueListItemUIElements[valueListItem] = newValueListItemUIElement; } else { newValueListItemUIElement.Initialize( valueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex ); } } } //parent.DirtyChildElements(true); var height = parent.ChildElements.Count * parent.ChildElements[0].Rect.Height; parent.Rect = new Rectangle(parent.Rect.X, parent.Rect.Y, parent.Rect.Width, height); } } public bool BeforeCreateChildElements(UIElement parent) { return false; } }
This seems to work - it only replaces the elements the first time you drop down the list and then doesn't replace them again while the dropdown is open. Unless I made some other logical error.
Hi Mike - while I confirmed that your code fixes the automation it will not fix my problem because it effectively NEVER replaces the UIItems with the subclassed ones because of an error in the logic in your code. With your code my debugger never breaks in the section where you create a new instance. So the behavior is like the creation filter didn't exist. I think the code should look like this instead:
... // Try to find an existing MyValueListItemUIElement and re-use it if possible. MyValueListItemUIElement newValueListItemUIElement = parent.GetDescendant(typeof(MyValueListItemUIElement), valueListItemUIElement.ValueListItem) as MyValueListItemUIElement; // only create new UI item if we didn't find a subclassed one if (newValueListItemUIElement == null) { newValueListItemUIElement = new MyValueListItemUIElement( parent as ValueListItemContainerUIElement, valueListItemUIElement.ValueListItem, valueListItemUIElement.IsMruItem, valueListItemUIElement.ListIndex) { Rect = valueListItemUIElement.Rect }; } // replace with existing or new ui item parent.ChildElements.RemoveAt(i); parent.ChildElements.Insert(i, newValueListItemUIElement); ...
However, once I come back to actually replacing the items IU automation no longer works (i.e. Inspect no longer highlights the items in the drop down). So it looks like the journey is not yet over ...